vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #346
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.iGzb6dKW5r', fmt=raw size=107374182414worker1 # Disk image does not exist, creating the virtualisation disk image...15client # mke2fs 1.47.4 (6-Mar-2025)16client: QEMU running (pid 51)17worker2 # Disk image does not exist, creating the virtualisation disk image...18worker2: QEMU running (pid 50)19worker1: QEMU running (pid 52)20lb # Disk image does not exist, creating the virtualisation disk image...21worker2 # Formatting '/build/vm-state-worker2/tmp.w7OelWcnYP', fmt=raw size=107374182422client # Discarding device blocks: 0/262144 done23lb # Formatting '/build/vm-state-lb/tmp.c6s0mKF66g', fmt=raw size=107374182424worker2 # mke2fs 1.47.4 (6-Mar-2025)25client # Creating filesystem with 262144 4k blocks and 65536 inodes26lb # mke2fs 1.47.4 (6-Mar-2025)27worker2 # Discarding device blocks: 0/262144 done28client # Filesystem UUID: 112807a0-6577-416e-bbb1-4b8788bb94f229lb # Discarding device blocks: 0/262144 done30worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes31client # Superblock backups stored on blocks:32lb # Creating filesystem with 262144 4k blocks and 65536 inodes33worker2 # Filesystem UUID: cfd7f4f4-4147-4be4-a1ab-49c6973fa37134client # 32768, 98304, 163840, 22937635lb # Filesystem UUID: 4151e13b-0aee-489f-86a7-16af63ca646036worker2 # Superblock backups stored on blocks:37client # 38lb # Superblock backups stored on blocks:39worker2 # 32768, 98304, 163840, 22937640client # Allocating group tables: 0/8 done41lb # 32768, 98304, 163840, 22937642worker2 # 43client # Writing inode tables: 0/8 done44lb # 45worker2 # Allocating group tables: 0/8 done46client # Creating journal (8192 blocks): done47lb # Allocating group tables: 0/8 done48worker2 # Writing inode tables: 0/8 done49client # Writing superblocks and filesystem accounting information: 0/8 done50lb # Writing inode tables: 0/8 done51worker2 # Creating journal (8192 blocks): done52client # 53lb # Creating journal (8192 blocks): done54worker2 # Writing superblocks and filesystem accounting information: 0/8 done55client # Virtualisation disk image created.56lb # Writing superblocks and filesystem accounting information: 0/8 done57worker2 # 58worker1 # Formatting '/build/vm-state-worker1/tmp.NQY9K4IbN2', fmt=raw size=107374182459lb: QEMU running (pid 49)60lb # 61worker2 # Virtualisation disk image created.62lb # Virtualisation disk image created.63(finished: start all VMs, in 0.68 seconds)64worker1 # mke2fs 1.47.4 (6-Mar-2025)65client: waiting for unit network-addresses-eth1.service66client: waiting for the VM to finish booting67worker1 # Discarding device blocks: 0/262144 done68worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes69worker1 # Filesystem UUID: 5810dfc5-862c-40e4-b6a0-0fc393131a2770worker1 # Superblock backups stored on blocks:71worker1 # 32768, 98304, 163840, 22937672worker1 # 73worker1 # Allocating group tables: 0/8 done74worker1 # Writing inode tables: 0/8 done75worker1 # Creating journal (8192 blocks): done76worker1 # Writing superblocks and filesystem accounting information: 0/8 done77worker1 # 78worker1 # Virtualisation disk image created.79client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]82client # [ 0.000000] KASLR enabled83client # [ 0.000000] random: crng init done84client # [ 0.000000] Machine model: linux,dummy-virt85client # [ 0.000000] efi: UEFI not found.86worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202687worker2 # [ 0.000000] KASLR enabled88worker2 # [ 0.000000] random: crng init done89client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT90worker2 # [ 0.000000] Machine model: linux,dummy-virt91worker2 # [ 0.000000] efi: UEFI not found.92client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]93client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]94worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT95client # [ 0.000000] Zone ranges:96worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]97client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]98client # [ 0.000000] DMA32 empty99worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]100client # [ 0.000000] Normal empty101worker2 # [ 0.000000] Zone ranges:102client # [ 0.000000] Device empty103client # [ 0.000000] Movable zone start for each node104worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]105worker2 # [ 0.000000] DMA32 empty106client # [ 0.000000] Early memory node ranges107worker2 # [ 0.000000] Normal empty108worker2 # [ 0.000000] Device empty109client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]110worker2 # [ 0.000000] Movable zone start for each node111worker2 # [ 0.000000] Early memory node ranges112client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]113client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000114worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]115client # [ 0.000000] psci: probing for conduit method from DT.116worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]117client # [ 0.000000] psci: PSCIv1.3 detected in firmware.118worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000119client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs120worker2 # [ 0.000000] psci: probing for conduit method from DT.121client # [ 0.000000] psci: Trusted OS migration not required122worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.123client # [ 0.000000] psci: SMC Calling Convention v1.1124worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs125worker2 # [ 0.000000] psci: Trusted OS migration not required126client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)127worker2 # [ 0.000000] psci: SMC Calling Convention v1.1128client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296129client # [ 0.000000] Detected PIPT I-cache on CPU0130worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)131client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)132worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296133client # [ 0.000000] CPU features: detected: GICv3 CPU interface134worker2 # [ 0.000000] Detected PIPT I-cache on CPU0135client # [ 0.000000] CPU features: detected: Spectre-v4136worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)137client # [ 0.000000] CPU features: detected: Spectre-BHB138worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface139client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38140worker2 # [ 0.000000] CPU features: detected: Spectre-v4141client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23142worker2 # [ 0.000000] CPU features: detected: Spectre-BHB143client # [ 0.000000] alternatives: applying boot alternatives144worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38145worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23146worker2 # [ 0.000000] alternatives: applying boot alternatives147client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test/init regInfo=/nix/store/f1h6zzz9lalh4vd08pfs1d8rys1abxi0-closure-info/registration console=ttyAMA0,115200n8 console=tty0148client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/f1h6zzz9lalh4vd08pfs1d8rys1abxi0-closure-info/registration", will be passed to user space.149worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lwjwr69nvgic4id2a0liqvrxsl7xi5q5-nixos-system-worker2-test/init regInfo=/nix/store/40cgfsvljmvjcgvggnq2nfs8gid2ii8f-closure-info/registration console=ttyAMA0,115200n8 console=tty0150client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes151client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)152worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/40cgfsvljmvjcgvggnq2nfs8gid2ii8f-closure-info/registration", will be passed to user space.153client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)154worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes155client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB156worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)157client # [ 0.000000] software IO TLB: area num 1.158worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)159client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB160worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB161client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)162worker2 # [ 0.000000] software IO TLB: area num 1.163client # [ 0.000000] Fallback order for Node 0: 0164worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB165client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216166client # [ 0.000000] Policy zone: DMA167worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)168worker2 # [ 0.000000] Fallback order for Node 0: 0169client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off170client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1171worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216172worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]173worker2 # [ 0.000000] Policy zone: DMA174client # [ 0.000000] allocated 3145728 bytes of page_ext175worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off176client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages177worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026178client # [ 0.000000] ftrace: allocated 294 pages with 4 groups179worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1180worker1 # [ 0.000000] KASLR enabled181client # [ 0.000000] rcu: Hierarchical RCU implementation.182worker1 # [ 0.000000] random: crng init done183worker2 # [ 0.000000] allocated 3145728 bytes of page_ext184client # [ 0.000000] rcu: RCU event tracing is enabled.185worker1 # [ 0.000000] Machine model: linux,dummy-virt186worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages187worker1 # [ 0.000000] efi: UEFI not found.188client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.189worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups190worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT191client # [ 0.000000] Trampoline variant of Tasks RCU enabled.192worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.193client # [ 0.000000] Rude variant of Tasks RCU enabled.194worker2 # [ 0.000000] rcu: RCU event tracing is enabled.195worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]196client # [ 0.000000] Tracing variant of Tasks RCU enabled.197worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.198worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]199worker1 # [ 0.000000] Zone ranges:200client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.201worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.202worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]203worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.204client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1205worker1 # [ 0.000000] DMA32 empty206worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.207worker1 # [ 0.000000] Normal empty208worker1 # [ 0.000000] Device empty209client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.210worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.211worker1 # [ 0.000000] Movable zone start for each node212worker1 # [ 0.000000] Early memory node ranges213client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.214worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1215worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]216lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]217client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.218worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.219worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]220client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0221worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000222worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.223client # [ 0.000000] GICv3: 256 SPIs implemented224worker1 # [ 0.000000] psci: probing for conduit method from DT.225lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026226client # [ 0.000000] GICv3: 0 Extended SPIs implemented227lb # [ 0.000000] KASLR enabled228worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.229lb # [ 0.000000] random: crng init done230worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.231client # [ 0.000000] Root IRQ handler: gic_handle_irq232worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs233lb # [ 0.000000] Machine model: linux,dummy-virt234client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI235worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0236lb # [ 0.000000] efi: UEFI not found.237worker1 # [ 0.000000] psci: Trusted OS migration not required238client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0239worker2 # [ 0.000000] GICv3: 256 SPIs implemented240worker1 # [ 0.000000] psci: SMC Calling Convention v1.1241worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented242lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT243client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000244worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq245client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]246worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)247lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]248worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI249lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]250worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296251worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0252client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)253lb # [ 0.000000] Zone ranges:254worker1 # [ 0.000000] Detected PIPT I-cache on CPU0255worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000256lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]257worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]258lb # [ 0.000000] DMA32 empty259worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)260client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)261lb # [ 0.000000] Normal empty262lb # [ 0.000000] Device empty263client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000264worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface265worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)266worker1 # [ 0.000000] CPU features: detected: Spectre-v4267lb # [ 0.000000] Movable zone start for each node268client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000269lb # [ 0.000000] Early memory node ranges270worker1 # [ 0.000000] CPU features: detected: Spectre-BHB271worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)272client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.273lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]274worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38275worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000276worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23277lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]278client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns279worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000280worker1 # [ 0.000000] alternatives: applying boot alternatives281lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000282client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).283lb # [ 0.000000] psci: probing for conduit method from DT.284worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.285lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.286client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns287lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs288worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns289lb # [ 0.000000] psci: Trusted OS migration not required290client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns291worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).292lb # [ 0.000000] psci: SMC Calling Convention v1.1293client # [ 0.000048] arm-pv: using stolen time PV294worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rbrvibrmg6njs73xdbl40qs6xvcpzckr-nixos-system-worker1-test/init regInfo=/nix/store/wz2yndnwkwz68zbyqp12dqpnc5i811j8-closure-info/registration console=ttyAMA0,115200n8 console=tty0295worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns296lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)297client # [ 0.000631] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)298lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296299client # [ 0.000844] Console: colour dummy device 80x25300worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns301lb # [ 0.000000] Detected PIPT I-cache on CPU0302client # [ 0.000852] printk: legacy console [tty0] enabled303worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/wz2yndnwkwz68zbyqp12dqpnc5i811j8-closure-info/registration", will be passed to user space.304worker2 # [ 0.000037] arm-pv: using stolen time PV305lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)306worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes307worker2 # [ 0.000653] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)308client # [ 0.001069] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)309lb # [ 0.000000] CPU features: detected: GICv3 CPU interface310worker2 # [ 0.000864] Console: colour dummy device 80x25311worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)312client # [ 0.001076] pid_max: default: 32768 minimum: 301313lb # [ 0.000000] CPU features: detected: Spectre-v4314worker2 # [ 0.000872] printk: legacy console [tty0] enabled315client # [ 0.001148] LSM: initializing lsm=capability,landlock,yama,bpf,ima316lb # [ 0.000000] CPU features: detected: Spectre-BHB317worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)318client # [ 0.001315] landlock: Up and running.319lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38320client # [ 0.001317] Yama: becoming mindful.321worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB322worker2 # [ 0.001068] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)323client # [ 0.002003] LSM support for eBPF active324worker1 # [ 0.000000] software IO TLB: area num 1.325lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23326worker2 # [ 0.001075] pid_max: default: 32768 minimum: 301327lb # [ 0.000000] alternatives: applying boot alternatives328worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB329client # [ 0.002163] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)330worker2 # [ 0.001159] LSM: initializing lsm=capability,landlock,yama,bpf,ima331worker2 # [ 0.001321] landlock: Up and running.332client # [ 0.002207] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)333worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)334worker2 # [ 0.001323] Yama: becoming mindful.335worker1 # [ 0.000000] Fallback order for Node 0: 0336worker2 # [ 0.001956] LSM support for eBPF active337client # [ 0.003459] cacheinfo: Unable to detect cache hierarchy for CPU 0338client # [ 0.004381] rcu: Hierarchical SRCU implementation.339worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216340worker2 # [ 0.002170] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)341worker1 # [ 0.000000] Policy zone: DMA342client # [ 0.004386] rcu: Max phase no-delay instances is 1000.343worker2 # [ 0.002205] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)344worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off345client # [ 0.005687] fsl-mc MSI: its@8080000 domain created346lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lai4n7chb0cnvnygfis27sipihspygia-nixos-system-lb-test/init regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration console=ttyAMA0,115200n8 console=tty0347client # [ 0.005781] EFI services will not be available.348worker2 # [ 0.003476] cacheinfo: Unable to detect cache hierarchy for CPU 0349worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1350client # [ 0.005854] smp: Bringing up secondary CPUs ...351worker2 # [ 0.004377] rcu: Hierarchical SRCU implementation.352worker1 # [ 0.000000] allocated 3145728 bytes of page_ext353client # [ 0.005863] smp: Brought up 1 node, 1 CPU354worker2 # [ 0.004382] rcu: Max phase no-delay instances is 1000.355worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages356lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration", will be passed to user space.357client # [ 0.005866] SMP: Total of 1 processors activated.358worker2 # [ 0.005623] fsl-mc MSI: its@8080000 domain created359worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups360client # [ 0.005872] CPU: All CPU(s) started at EL1361lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes362worker2 # [ 0.005716] EFI services will not be available.363worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.364client # [ 0.005884] CPU features: detected: Branch Target Identification365worker2 # [ 0.005816] smp: Bringing up secondary CPUs ...366worker1 # [ 0.000000] rcu: RCU event tracing is enabled.367lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)368worker2 # [ 0.005825] smp: Brought up 1 node, 1 CPU369client # [ 0.005889] CPU features: detected: ARMv8.4 Translation Table Level370worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.371worker2 # [ 0.005829] SMP: Total of 1 processors activated.372lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)373worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.374worker2 # [ 0.005836] CPU: All CPU(s) started at EL1375client # [ 0.005893] CPU features: detected: Instruction cache invalidation not required for I/D coherence376lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB377worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.378worker2 # [ 0.005847] CPU features: detected: Branch Target Identification379lb # [ 0.000000] software IO TLB: area num 1.380worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.381client # [ 0.005896] CPU features: detected: Data cache clean to the PoU not required for I/D coherence382lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB383worker2 # [ 0.005853] CPU features: detected: ARMv8.4 Translation Table Level384client # [ 0.005900] CPU features: detected: Common not Private translations385worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.386client # [ 0.005903] CPU features: detected: CRC32 instructions387lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)388worker2 # [ 0.005856] CPU features: detected: Instruction cache invalidation not required for I/D coherence389worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1390lb # [ 0.000000] Fallback order for Node 0: 0391client # [ 0.005906] CPU features: detected: Data cache clean to Point of Deep Persistence392worker2 # [ 0.005860] CPU features: detected: Data cache clean to the PoU not required for I/D coherence393lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216394worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.395lb # [ 0.000000] Policy zone: DMA396client # [ 0.005910] CPU features: detected: Data cache clean to Point of Persistence397worker2 # [ 0.005864] CPU features: detected: Common not Private translations398worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.399lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off400worker2 # [ 0.005867] CPU features: detected: CRC32 instructions401client # [ 0.005913] CPU features: detected: Data independent timing control (DIT)402client # [ 0.005916] CPU features: detected: E0PD403lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1404worker2 # [ 0.005870] CPU features: detected: Data cache clean to Point of Deep Persistence405worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.406lb # [ 0.000000] allocated 3145728 bytes of page_ext407client # [ 0.005919] CPU features: detected: Enhanced Counter Virtualization408worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0409worker2 # [ 0.005873] CPU features: detected: Data cache clean to Point of Persistence410lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages411worker1 # [ 0.000000] GICv3: 256 SPIs implemented412client # [ 0.005922] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)413worker2 # [ 0.005877] CPU features: detected: Data independent timing control (DIT)414lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups415worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented416client # [ 0.005925] CPU features: detected: Enhanced Virtualization Traps417worker2 # [ 0.005880] CPU features: detected: E0PD418lb # [ 0.000000] rcu: Hierarchical RCU implementation.419worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq420client # [ 0.005928] CPU features: detected: Fine Grained Traps421lb # [ 0.000000] rcu: RCU event tracing is enabled.422worker2 # [ 0.005882] CPU features: detected: Enhanced Counter Virtualization423worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI424worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0425client # [ 0.005932] CPU features: detected: Generic authentication (architected QARMA5 algorithm)426lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.427worker2 # [ 0.005885] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)428client # [ 0.005938] CPU features: detected: RCpc load-acquire (LDAPR)429lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.430worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000431worker2 # [ 0.005889] CPU features: detected: Enhanced Virtualization Traps432worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]433lb # [ 0.000000] Rude variant of Tasks RCU enabled.434client # [ 0.005941] CPU features: detected: LSE atomic instructions435worker2 # [ 0.005892] CPU features: detected: Fine Grained Traps436lb # [ 0.000000] Tracing variant of Tasks RCU enabled.437client # [ 0.005944] CPU features: detected: Privileged Access Never438worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)439worker2 # [ 0.005896] CPU features: detected: Generic authentication (architected QARMA5 algorithm)440client # [ 0.005946] CPU features: detected: PMUv3441lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.442worker2 # [ 0.005901] CPU features: detected: RCpc load-acquire (LDAPR)443client # [ 0.005949] CPU features: detected: RAS Extension Support444lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1445worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)446worker2 # [ 0.005904] CPU features: detected: LSE atomic instructions447worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000448worker2 # [ 0.005908] CPU features: detected: Privileged Access Never449lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.450client # [ 0.005952] CPU features: detected: RASv1p1 Extension Support451worker2 # [ 0.005910] CPU features: detected: PMUv3452worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000453client # [ 0.005955] CPU features: detected: Random Number Generator454lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.455worker2 # [ 0.005914] CPU features: detected: RAS Extension Support456worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.457client # [ 0.005957] CPU features: detected: Speculation barrier (SB)458lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.459client # [ 0.005961] CPU features: detected: Stage-2 Force Write-Back460worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns461worker2 # [ 0.005916] CPU features: detected: RASv1p1 Extension Support462lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0463client # [ 0.005964] CPU features: detected: TLB range maintenance instructions464lb # [ 0.000000] GICv3: 256 SPIs implemented465worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).466worker2 # [ 0.005919] CPU features: detected: Random Number Generator467lb # [ 0.000000] GICv3: 0 Extended SPIs implemented468client # [ 0.005969] CPU features: detected: Speculative Store Bypassing Safe (SSBS)469worker2 # [ 0.005922] CPU features: detected: Speculation barrier (SB)470lb # [ 0.000000] Root IRQ handler: gic_handle_irq471client # [ 0.006013] alternatives: applying system-wide alternatives472worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns473worker2 # [ 0.005924] CPU features: detected: Stage-2 Force Write-Back474lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI475client # [ 0.009143] CPU features: detected: BBM Level 2 without TLB conflict abort476worker2 # [ 0.005928] CPU features: detected: TLB range maintenance instructions477lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0478worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns479worker1 # [ 0.000034] arm-pv: using stolen time PV480lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000481worker2 # [ 0.005933] CPU features: detected: Speculative Store Bypassing Safe (SSBS)482lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]483worker2 # [ 0.006003] alternatives: applying system-wide alternatives484client # [ 0.009371] Memory: 1406504K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)485worker1 # [ 0.000487] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)486client # [ 0.009820] devtmpfs: initialized487worker1 # [ 0.000667] Console: colour dummy device 80x25488worker2 # [ 0.009081] CPU features: detected: BBM Level 2 without TLB conflict abort489lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)490client # [ 0.011768] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)491worker1 # [ 0.000676] printk: legacy console [tty0] enabled492lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)493client # [ 0.011793] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).494worker2 # [ 0.009310] Memory: 1406524K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)495worker1 # [ 0.000864] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)496lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000497worker2 # [ 0.009730] devtmpfs: initialized498client # [ 0.012020] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL499worker1 # [ 0.000871] pid_max: default: 32768 minimum: 301500lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000501client # [ 0.012025] 0 pages in range for non-PLT usage502worker1 # [ 0.000941] LSM: initializing lsm=capability,landlock,yama,bpf,ima503worker2 # [ 0.011701] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)504worker1 # [ 0.001089] landlock: Up and running.505client # [ 0.012026] 508288 pages in range for PLT usage506lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.507worker1 # [ 0.001092] Yama: becoming mindful.508worker2 # [ 0.011725] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).509client # [ 0.012129] pinctrl core: initialized pinctrl subsystem510worker1 # [ 0.001576] LSM support for eBPF active511client # [ 0.012900] DMI not present or invalid.512worker2 # [ 0.011915] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL513lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns514worker1 # [ 0.001728] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)515client # [ 0.016778] NET: Registered PF_NETLINK/PF_ROUTE protocol family516worker2 # [ 0.011919] 0 pages in range for non-PLT usage517lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).518worker2 # [ 0.011920] 508288 pages in range for PLT usage519worker1 # [ 0.001762] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)520client # [ 0.019208] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations521worker2 # [ 0.012029] pinctrl core: initialized pinctrl subsystem522worker2 # [ 0.012863] DMI not present or invalid.523worker1 # [ 0.002949] cacheinfo: Unable to detect cache hierarchy for CPU 0524client # [ 0.019400] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations525lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns526worker1 # [ 0.003729] rcu: Hierarchical SRCU implementation.527worker2 # [ 0.016801] NET: Registered PF_NETLINK/PF_ROUTE protocol family528lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns529client # [ 0.019617] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations530worker1 # [ 0.003734] rcu: Max phase no-delay instances is 1000.531lb # [ 0.000031] arm-pv: using stolen time PV532worker2 # [ 0.019278] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations533worker1 # [ 0.004950] fsl-mc MSI: its@8080000 domain created534client # [ 0.019646] audit: initializing netlink subsys (disabled)535worker1 # [ 0.005041] EFI services will not be available.536worker2 # [ 0.019465] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations537client # [ 0.020282] thermal_sys: Registered thermal governor 'fair_share'538lb # [ 0.000410] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)539worker1 # [ 0.005125] smp: Bringing up secondary CPUs ...540lb # [ 0.000577] Console: colour dummy device 80x25541client # [ 0.020285] thermal_sys: Registered thermal governor 'bang_bang'542worker1 # [ 0.005134] smp: Brought up 1 node, 1 CPU543worker2 # [ 0.019674] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations544lb # [ 0.000586] printk: legacy console [tty0] enabled545worker1 # [ 0.005137] SMP: Total of 1 processors activated.546client # [ 0.020289] thermal_sys: Registered thermal governor 'step_wise'547worker2 # [ 0.019701] audit: initializing netlink subsys (disabled)548worker1 # [ 0.005143] CPU: All CPU(s) started at EL1549client # [ 0.020294] thermal_sys: Registered thermal governor 'user_space'550worker2 # [ 0.020343] thermal_sys: Registered thermal governor 'fair_share'551worker1 # [ 0.005153] CPU features: detected: Branch Target Identification552lb # [ 0.000785] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)553client # [ 0.020297] thermal_sys: Registered thermal governor 'power_allocator'554worker2 # [ 0.020345] thermal_sys: Registered thermal governor 'bang_bang'555lb # [ 0.000792] pid_max: default: 32768 minimum: 301556worker1 # [ 0.005158] CPU features: detected: ARMv8.4 Translation Table Level557lb # [ 0.000864] LSM: initializing lsm=capability,landlock,yama,bpf,ima558worker2 # [ 0.020349] thermal_sys: Registered thermal governor 'step_wise'559client # [ 0.020325] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1560lb # [ 0.001004] landlock: Up and running.561worker1 # [ 0.005162] CPU features: detected: Instruction cache invalidation not required for I/D coherence562client # [ 0.020334] cpuidle: using governor ladder563worker2 # [ 0.020355] thermal_sys: Registered thermal governor 'user_space'564lb # [ 0.001007] Yama: becoming mindful.565client # [ 0.020339] cpuidle: using governor menu566lb # [ 0.001495] LSM support for eBPF active567worker2 # [ 0.020358] thermal_sys: Registered thermal governor 'power_allocator'568worker1 # [ 0.005166] CPU features: detected: Data cache clean to the PoU not required for I/D coherence569client # [ 0.020540] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.570worker1 # [ 0.005169] CPU features: detected: Common not Private translations571lb # [ 0.001643] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)572worker2 # [ 0.020387] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1573client # [ 0.020557] ASID allocator initialised with 65536 entries574worker1 # [ 0.005173] CPU features: detected: CRC32 instructions575worker2 # [ 0.020396] cpuidle: using governor ladder576client # [ 0.021867] Serial: AMBA PL011 UART driver577lb # [ 0.001679] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)578worker2 # [ 0.020402] cpuidle: using governor menu579worker1 # [ 0.005175] CPU features: detected: Data cache clean to Point of Deep Persistence580lb # [ 0.002843] cacheinfo: Unable to detect cache hierarchy for CPU 0581client # [ 0.027336] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1582worker2 # [ 0.020614] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.583lb # [ 0.003663] rcu: Hierarchical SRCU implementation.584worker1 # [ 0.005179] CPU features: detected: Data cache clean to Point of Persistence585client # [ 0.027558] printk: console [ttyAMA0] enabled586worker2 # [ 0.020632] ASID allocator initialised with 65536 entries587lb # [ 0.003668] rcu: Max phase no-delay instances is 1000.588worker1 # [ 0.005183] CPU features: detected: Data independent timing control (DIT)589worker2 # [ 0.021946] Serial: AMBA PL011 UART driver590lb # [ 0.004911] fsl-mc MSI: its@8080000 domain created591worker1 # [ 0.005186] CPU features: detected: E0PD592lb # [ 0.005008] EFI services will not be available.593worker2 # [ 0.027422] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1594worker1 # [ 0.005188] CPU features: detected: Enhanced Counter Virtualization595lb # [ 0.005086] smp: Bringing up secondary CPUs ...596worker2 # [ 0.027570] printk: console [ttyAMA0] enabled597lb # [ 0.005095] smp: Brought up 1 node, 1 CPU598worker1 # [ 0.005191] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)599lb # [ 0.005098] SMP: Total of 1 processors activated.600worker1 # [ 0.005194] CPU features: detected: Enhanced Virtualization Traps601lb # [ 0.005105] CPU: All CPU(s) started at EL1602worker1 # [ 0.005197] CPU features: detected: Fine Grained Traps603lb # [ 0.005116] CPU features: detected: Branch Target Identification604worker1 # [ 0.005200] CPU features: detected: Generic authentication (architected QARMA5 algorithm)605lb # [ 0.005122] CPU features: detected: ARMv8.4 Translation Table Level606worker1 # [ 0.005205] CPU features: detected: RCpc load-acquire (LDAPR)607lb # [ 0.005125] CPU features: detected: Instruction cache invalidation not required for I/D coherence608worker1 # [ 0.005208] CPU features: detected: LSE atomic instructions609worker1 # [ 0.005211] CPU features: detected: Privileged Access Never610lb # [ 0.005128] CPU features: detected: Data cache clean to the PoU not required for I/D coherence611worker1 # [ 0.005214] CPU features: detected: PMUv3612lb # [ 0.005132] CPU features: detected: Common not Private translations613worker1 # [ 0.005217] CPU features: detected: RAS Extension Support614lb # [ 0.005135] CPU features: detected: CRC32 instructions615worker1 # [ 0.005220] CPU features: detected: RASv1p1 Extension Support616worker1 # [ 0.005222] CPU features: detected: Random Number Generator617client # [ 0.151096] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages618lb # [ 0.005138] CPU features: detected: Data cache clean to Point of Deep Persistence619worker1 # [ 0.005225] CPU features: detected: Speculation barrier (SB)620client # [ 0.151119] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page621lb # [ 0.005142] CPU features: detected: Data cache clean to Point of Persistence622worker1 # [ 0.005228] CPU features: detected: Stage-2 Force Write-Back623client # [ 0.151125] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages624lb # [ 0.005145] CPU features: detected: Data independent timing control (DIT)625worker1 # [ 0.005231] CPU features: detected: TLB range maintenance instructions626lb # [ 0.005148] CPU features: detected: E0PD627worker2 # [ 0.151579] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages628client # [ 0.151129] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page629worker1 # [ 0.005236] CPU features: detected: Speculative Store Bypassing Safe (SSBS)630lb # [ 0.005150] CPU features: detected: Enhanced Counter Virtualization631worker2 # [ 0.151602] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page632client # [ 0.151133] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages633worker1 # [ 0.005275] alternatives: applying system-wide alternatives634lb # [ 0.005153] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)635client # [ 0.151137] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page636worker2 # [ 0.151608] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages637worker1 # [ 0.008432] CPU features: detected: BBM Level 2 without TLB conflict abort638lb # [ 0.005156] CPU features: detected: Enhanced Virtualization Traps639worker2 # [ 0.151612] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page640client # [ 0.151142] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages641lb # [ 0.005159] CPU features: detected: Fine Grained Traps642worker2 # [ 0.151617] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages643client # [ 0.151146] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page644worker1 # [ 0.008674] Memory: 1406560K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)645lb # [ 0.005163] CPU features: detected: Generic authentication (architected QARMA5 algorithm)646worker2 # [ 0.151621] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page647worker1 # [ 0.009085] devtmpfs: initialized648lb # [ 0.005168] CPU features: detected: RCpc load-acquire (LDAPR)649worker2 # [ 0.151626] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages650worker1 # [ 0.011040] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)651lb # [ 0.005171] CPU features: detected: LSE atomic instructions652client # [ 0.158709] fbcon: Taking over console653worker2 # [ 0.151630] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page654lb # [ 0.005174] CPU features: detected: Privileged Access Never655client # [ 0.158728] ACPI: Interpreter disabled.656worker1 # [ 0.011065] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).657lb # [ 0.005177] CPU features: detected: PMUv3658worker1 # [ 0.011239] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL659client # [ 0.160674] iommu: Default domain type: Translated660worker1 # [ 0.011244] 0 pages in range for non-PLT usage661worker2 # [ 0.159088] fbcon: Taking over console662client # [ 0.160684] iommu: DMA domain TLB invalidation policy: strict mode663lb # [ 0.005180] CPU features: detected: RAS Extension Support664worker1 # [ 0.011245] 508288 pages in range for PLT usage665worker2 # [ 0.159104] ACPI: Interpreter disabled.666worker1 # [ 0.011371] pinctrl core: initialized pinctrl subsystem667lb # [ 0.005182] CPU features: detected: RASv1p1 Extension Support668client # [ 0.162351] SCSI subsystem initialized669worker1 # [ 0.012215] DMI not present or invalid.670worker2 # [ 0.160989] iommu: Default domain type: Translated671lb # [ 0.005185] CPU features: detected: Random Number Generator672worker1 # [ 0.015864] NET: Registered PF_NETLINK/PF_ROUTE protocol family673lb # [ 0.005187] CPU features: detected: Speculation barrier (SB)674worker2 # [ 0.161001] iommu: DMA domain TLB invalidation policy: strict mode675worker1 # [ 0.018242] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations676lb # [ 0.005190] CPU features: detected: Stage-2 Force Write-Back677lb # [ 0.005193] CPU features: detected: TLB range maintenance instructions678worker1 # [ 0.018434] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations679lb # [ 0.005198] CPU features: detected: Speculative Store Bypassing Safe (SSBS)680worker1 # [ 0.018645] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations681lb # [ 0.005237] alternatives: applying system-wide alternatives682worker1 # [ 0.018670] audit: initializing netlink subsys (disabled)683lb # [ 0.008341] CPU features: detected: BBM Level 2 without TLB conflict abort684client # [ 0.167632] usbcore: registered new interface driver usbfs685worker1 # [ 0.019310] thermal_sys: Registered thermal governor 'fair_share'686client # [ 0.167666] usbcore: registered new interface driver hub687worker1 # [ 0.019311] thermal_sys: Registered thermal governor 'bang_bang'688lb # [ 0.008591] Memory: 1406524K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)689client # [ 0.167685] usbcore: registered new device driver usb690lb # [ 0.009015] devtmpfs: initialized691worker1 # [ 0.019315] thermal_sys: Registered thermal governor 'step_wise'692worker2 # [ 0.168105] SCSI subsystem initialized693client # [ 0.167962] pps_core: LinuxPPS API ver. 1 registered694worker1 # [ 0.019321] thermal_sys: Registered thermal governor 'user_space'695lb # [ 0.010835] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)696client # [ 0.167968] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>697worker1 # [ 0.019324] thermal_sys: Registered thermal governor 'power_allocator'698worker2 # [ 0.168807] usbcore: registered new interface driver usbfs699lb # [ 0.010858] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).700client # [ 0.167979] PTP clock support registered701worker2 # [ 0.168839] usbcore: registered new interface driver hub702client # [ 0.168027] EDAC MC: Ver: 3.0.0703worker1 # [ 0.019350] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1704lb # [ 0.011044] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL705worker2 # [ 0.168857] usbcore: registered new device driver usb706lb # [ 0.011049] 0 pages in range for non-PLT usage707worker1 # [ 0.019358] cpuidle: using governor ladder708client # [ 0.172680] scmi_core: SCMI protocol bus registered709worker1 # [ 0.019364] cpuidle: using governor menu710worker2 # [ 0.169129] pps_core: LinuxPPS API ver. 1 registered711lb # [ 0.011050] 508288 pages in range for PLT usage712lb # [ 0.011150] pinctrl core: initialized pinctrl subsystem713worker1 # [ 0.019564] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.714client # [ 0.173625] FPGA manager framework715worker2 # [ 0.169135] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>716lb # [ 0.011977] DMI not present or invalid.717client # [ 0.174601] vgaarb: loaded718worker1 # [ 0.019581] ASID allocator initialised with 65536 entries719worker2 # [ 0.169145] PTP clock support registered720worker1 # [ 0.020851] Serial: AMBA PL011 UART driver721lb # [ 0.015362] NET: Registered PF_NETLINK/PF_ROUTE protocol family722worker2 # [ 0.169198] EDAC MC: Ver: 3.0.0723client # [ 0.175236] clocksource: Switched to clocksource arch_sys_counter724lb # [ 0.017670] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations725worker2 # [ 0.173795] scmi_core: SCMI protocol bus registered726worker1 # [ 0.026156] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1727worker1 # [ 0.026327] printk: console [ttyAMA0] enabled728lb # [ 0.017921] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations729worker2 # [ 0.174750] FPGA manager framework730worker2 # [ 0.175690] vgaarb: loaded731lb # [ 0.018129] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations732lb # [ 0.018157] audit: initializing netlink subsys (disabled)733worker2 # [ 0.176384] clocksource: Switched to clocksource arch_sys_counter734lb # [ 0.018854] thermal_sys: Registered thermal governor 'fair_share'735lb # [ 0.018856] thermal_sys: Registered thermal governor 'bang_bang'736lb # [ 0.018861] thermal_sys: Registered thermal governor 'step_wise'737lb # [ 0.018866] thermal_sys: Registered thermal governor 'user_space'738client # [ 0.181376] VFS: Disk quotas dquot_6.6.0739worker2 # [ 0.179593] VFS: Disk quotas dquot_6.6.0740lb # [ 0.018869] thermal_sys: Registered thermal governor 'power_allocator'741client # [ 0.181427] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)742worker2 # [ 0.179631] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)743lb # [ 0.018898] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1744client # [ 0.183137] netfs: FS-Cache loaded745lb # [ 0.018907] cpuidle: using governor ladder746client # [ 0.183720] pnp: PnP ACPI: disabled747lb # [ 0.018913] cpuidle: using governor menu748lb # [ 0.019108] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.749worker1 # [ 0.146851] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages750lb # [ 0.019126] ASID allocator initialised with 65536 entries751worker2 # [ 0.183498] netfs: FS-Cache loaded752lb # [ 0.020479] Serial: AMBA PL011 UART driver753worker2 # [ 0.183648] pnp: PnP ACPI: disabled754worker1 # [ 0.146873] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page755worker1 # [ 0.146878] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages756lb # [ 0.025945] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1757lb # [ 0.026097] printk: console [ttyAMA0] enabled758worker1 # [ 0.146882] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page759worker1 # [ 0.146887] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages760worker1 # [ 0.146891] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page761worker1 # [ 0.146895] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages762worker2 # [ 0.187710] NET: Registered PF_INET protocol family763worker1 # [ 0.146900] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page764worker2 # [ 0.188012] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)765client # [ 0.187314] NET: Registered PF_INET protocol family766client # [ 0.187611] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)767worker1 # [ 0.154440] fbcon: Taking over console768worker1 # [ 0.154457] ACPI: Interpreter disabled.769lb # [ 0.148038] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages770lb # [ 0.148061] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page771lb # [ 0.148067] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages772lb # [ 0.148071] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page773lb # [ 0.148076] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages774lb # [ 0.148080] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page775worker1 # [ 0.163031] iommu: Default domain type: Translated776lb # [ 0.148085] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages777worker1 # [ 0.163043] iommu: DMA domain TLB invalidation policy: strict mode778lb # [ 0.148089] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page779worker1 # [ 0.163411] SCSI subsystem initialized780worker1 # [ 0.165397] usbcore: registered new interface driver usbfs781lb # [ 0.155715] fbcon: Taking over console782worker1 # [ 0.165430] usbcore: registered new interface driver hub783lb # [ 0.155732] ACPI: Interpreter disabled.784worker1 # [ 0.165448] usbcore: registered new device driver usb785worker1 # [ 0.165743] pps_core: LinuxPPS API ver. 1 registered786worker1 # [ 0.165749] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>787worker1 # [ 0.165759] PTP clock support registered788worker1 # [ 0.165837] EDAC MC: Ver: 3.0.0789worker1 # [ 0.170479] scmi_core: SCMI protocol bus registered790worker1 # [ 0.171504] FPGA manager framework791worker1 # [ 0.172536] vgaarb: loaded792worker1 # [ 0.173197] clocksource: Switched to clocksource arch_sys_counter793lb # [ 0.164358] iommu: Default domain type: Translated794lb # [ 0.164372] iommu: DMA domain TLB invalidation policy: strict mode795lb # [ 0.164781] SCSI subsystem initialized796lb # [ 0.166903] usbcore: registered new interface driver usbfs797worker1 # [ 0.177921] VFS: Disk quotas dquot_6.6.0798lb # [ 0.166937] usbcore: registered new interface driver hub799worker1 # [ 0.177982] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)800lb # [ 0.166957] usbcore: registered new device driver usb801worker1 # [ 0.179692] netfs: FS-Cache loaded802lb # [ 0.167298] pps_core: LinuxPPS API ver. 1 registered803worker1 # [ 0.179811] pnp: PnP ACPI: disabled804lb # [ 0.167305] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>805lb # [ 0.167316] PTP clock support registered806lb # [ 0.167381] EDAC MC: Ver: 3.0.0807lb # [ 0.172158] scmi_core: SCMI protocol bus registered808lb # [ 0.173182] FPGA manager framework809lb # [ 0.174222] vgaarb: loaded810worker1 # [ 0.183833] NET: Registered PF_INET protocol family811lb # [ 0.174878] clocksource: Switched to clocksource arch_sys_counter812worker1 # [ 0.184124] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)813worker2 # [ 0.218377] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)814worker2 # [ 0.218437] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)815client # [ 0.218008] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)816lb # [ 0.179275] VFS: Disk quotas dquot_6.6.0817lb # [ 0.179325] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)818worker2 # [ 0.218473] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)819client # [ 0.218065] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)820lb # [ 0.180971] netfs: FS-Cache loaded821worker2 # [ 0.218561] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)822client # [ 0.218098] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)823lb # [ 0.181100] pnp: PnP ACPI: disabled824worker2 # [ 0.218705] TCP: Hash tables configured (established 16384 bind 16384)825client # [ 0.218184] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)826worker2 # [ 0.218801] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)827client # [ 0.218330] TCP: Hash tables configured (established 16384 bind 16384)828worker2 # [ 0.218839] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)829client # [ 0.218434] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)830worker2 # [ 0.218877] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)831client # [ 0.218476] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)832worker2 # [ 0.218986] NET: Registered PF_UNIX/PF_LOCAL protocol family833client # [ 0.218516] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)834worker2 # [ 0.219045] NET: Registered PF_XDP protocol family835client # [ 0.218611] NET: Registered PF_UNIX/PF_LOCAL protocol family836worker2 # [ 0.219065] PCI: CLS 0 bytes, default 64837lb # [ 0.185135] NET: Registered PF_INET protocol family838client # [ 0.218659] NET: Registered PF_XDP protocol family839worker2 # [ 0.219327] Trying to unpack rootfs image as initramfs...840client # [ 0.218680] PCI: CLS 0 bytes, default 64841lb # [ 0.185425] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)842client # [ 0.218932] Trying to unpack rootfs image as initramfs...843worker2 # [ 0.234275] kvm [1]: HYP mode not available844client # [ 0.237239] kvm [1]: HYP mode not available845worker1 # [ 0.215269] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)846worker1 # [ 0.215332] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)847worker1 # [ 0.215366] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)848worker1 # [ 0.215458] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849worker1 # [ 0.215601] TCP: Hash tables configured (established 16384 bind 16384)850worker1 # [ 0.215700] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851worker1 # [ 0.215740] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)852worker1 # [ 0.215791] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)853worker1 # [ 0.215883] NET: Registered PF_UNIX/PF_LOCAL protocol family854worker1 # [ 0.215942] NET: Registered PF_XDP protocol family855worker1 # [ 0.215961] PCI: CLS 0 bytes, default 64856worker1 # [ 0.216235] Trying to unpack rootfs image as initramfs...857worker1 # [ 0.231137] kvm [1]: HYP mode not available858lb # [ 0.216857] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)859lb # [ 0.216917] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)860lb # [ 0.216952] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)861lb # [ 0.217042] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)862lb # [ 0.217187] TCP: Hash tables configured (established 16384 bind 16384)863lb # [ 0.217286] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)864lb # [ 0.217349] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865lb # [ 0.217389] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866lb # [ 0.217494] NET: Registered PF_UNIX/PF_LOCAL protocol family867lb # [ 0.217526] NET: Registered PF_XDP protocol family868lb # [ 0.217546] PCI: CLS 0 bytes, default 64869lb # [ 0.217811] Trying to unpack rootfs image as initramfs...870lb # [ 0.232805] kvm [1]: HYP mode not available871client # [ 0.364326] Initialise system trusted keyrings872client # [ 0.365072] workingset: timestamp_bits=42 max_order=19 bucket_order=0873client # [ 0.366357] squashfs: version 4.0 (2009/01/31) Phillip Lougher874client # [ 0.367142] 9p: Installing v9fs 9p2000 file system support875worker2 # [ 0.369024] Initialise system trusted keyrings876worker2 # [ 0.369773] workingset: timestamp_bits=42 max_order=19 bucket_order=0877worker2 # [ 0.371125] squashfs: version 4.0 (2009/01/31) Phillip Lougher878worker2 # [ 0.371909] 9p: Installing v9fs 9p2000 file system support879worker1 # [ 0.357862] Initialise system trusted keyrings880client # [ 0.395848] Key type asymmetric registered881client # [ 0.395879] Asymmetric key parser 'x509' registered882worker1 # [ 0.358655] workingset: timestamp_bits=42 max_order=19 bucket_order=0883client # [ 0.395962] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)884client # [ 0.398136] io scheduler mq-deadline registered885worker1 # [ 0.360047] squashfs: version 4.0 (2009/01/31) Phillip Lougher886client # [ 0.398148] io scheduler kyber registered887worker1 # [ 0.360841] 9p: Installing v9fs 9p2000 file system support888worker2 # [ 0.400559] Key type asymmetric registered889worker2 # [ 0.400590] Asymmetric key parser 'x509' registered890worker2 # [ 0.400667] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)891worker2 # [ 0.402803] io scheduler mq-deadline registered892worker2 # [ 0.402815] io scheduler kyber registered893lb # [ 0.357073] Initialise system trusted keyrings894lb # [ 0.357848] workingset: timestamp_bits=42 max_order=19 bucket_order=0895lb # [ 0.359190] squashfs: version 4.0 (2009/01/31) Phillip Lougher896client # [ 0.407432] pl061_gpio 9030000.pl061: PL061 GPIO chip registered897lb # [ 0.359983] 9p: Installing v9fs 9p2000 file system support898worker2 # [ 0.412583] pl061_gpio 9030000.pl061: PL061 GPIO chip registered899client # [ 0.408877] ledtrig-cpu: registered to indicate activity on CPUs900client # [ 0.409265] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:901client # [ 0.409284] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000902worker2 # [ 0.413986] ledtrig-cpu: registered to indicate activity on CPUs903client # [ 0.409295] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000904worker2 # [ 0.414361] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:905client # [ 0.409304] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000906worker2 # [ 0.414379] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000907client # [ 0.409323] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits908worker2 # [ 0.414392] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000909client # [ 0.409347] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]910client # [ 0.409432] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00911worker2 # [ 0.414401] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000912client # [ 0.409441] pci_bus 0000:00: root bus resource [bus 00-ff]913worker2 # [ 0.414429] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits914client # [ 0.409448] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]915worker2 # [ 0.414453] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]916client # [ 0.409453] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]917worker2 # [ 0.414535] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00918client # [ 0.409458] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]919worker2 # [ 0.414544] pci_bus 0000:00: root bus resource [bus 00-ff]920client # [ 0.409518] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint921worker2 # [ 0.414550] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]922worker1 # [ 0.389529] Key type asymmetric registered923client # [ 0.409945] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint924worker2 # [ 0.414556] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]925worker1 # [ 0.389560] Asymmetric key parser 'x509' registered926client # [ 0.410127] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]927worker2 # [ 0.414561] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]928worker1 # [ 0.389634] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)929client # [ 0.410143] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]930worker1 # [ 0.391828] io scheduler mq-deadline registered931worker2 # [ 0.414621] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint932client # [ 0.410172] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]933worker1 # [ 0.391840] io scheduler kyber registered934client # [ 0.410188] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]935worker2 # [ 0.415064] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint936worker2 # [ 0.415252] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]937client # [ 0.410631] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint938lb # [ 0.383949] Key type asymmetric registered939worker2 # [ 0.415269] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]940client # [ 0.410810] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]941worker2 # [ 0.415299] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]942lb # [ 0.383979] Asymmetric key parser 'x509' registered943client # [ 0.410826] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]944worker2 # [ 0.415316] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]945lb # [ 0.384058] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)946client # [ 0.410855] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]947lb # [ 0.386222] io scheduler mq-deadline registered948worker2 # [ 0.415765] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint949client # [ 0.411340] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint950lb # [ 0.386233] io scheduler kyber registered951worker2 # [ 0.415950] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]952client # [ 0.411545] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]953worker2 # [ 0.415966] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]954client # [ 0.411568] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]955worker2 # [ 0.415999] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]956client # [ 0.411598] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]957client # [ 0.412071] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint958worker2 # [ 0.435900] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint959client # [ 0.412259] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]960worker2 # [ 0.436088] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]961client # [ 0.412276] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]962worker2 # [ 0.436105] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]963worker1 # [ 0.401321] pl061_gpio 9030000.pl061: PL061 GPIO chip registered964client # [ 0.412311] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]965worker2 # [ 0.436136] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]966client # [ 0.412766] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint967client # [ 0.412944] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]968worker2 # [ 0.439771] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint969client # [ 0.412960] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]970worker2 # [ 0.439961] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]971worker1 # [ 0.402725] ledtrig-cpu: registered to indicate activity on CPUs972client # [ 0.412989] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]973worker2 # [ 0.439978] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]974worker1 # [ 0.403134] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:975worker2 # [ 0.440010] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]976client # [ 0.413432] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint977worker1 # [ 0.403153] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000978client # [ 0.413612] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]979client # [ 0.413628] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]980worker1 # [ 0.403166] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000981client # [ 0.413657] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]982worker1 # [ 0.403174] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000983client # [ 0.414104] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint984worker1 # [ 0.403198] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits985client # [ 0.414283] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]986lb # [ 0.399758] pl061_gpio 9030000.pl061: PL061 GPIO chip registered987client # [ 0.414299] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]988worker1 # [ 0.403223] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]989client # [ 0.414327] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990worker1 # [ 0.403302] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00991client # [ 0.414343] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]992worker1 # [ 0.403311] pci_bus 0000:00: root bus resource [bus 00-ff]993worker1 # [ 0.403318] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]994client # [ 0.414822] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint995client # [ 0.415006] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]996worker1 # [ 0.403323] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]997client # [ 0.415036] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998worker1 # [ 0.403328] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]999worker2 # [ 0.448549] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1000worker1 # [ 0.403390] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1001worker2 # [ 0.448769] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1002lb # [ 0.401118] ledtrig-cpu: registered to indicate activity on CPUs1003worker1 # [ 0.403838] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1004worker2 # [ 0.448786] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1005lb # [ 0.401541] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1006worker1 # [ 0.404030] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1007worker2 # [ 0.448817] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1008worker1 # [ 0.404046] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1009lb # [ 0.401559] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001010worker2 # [ 0.449341] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1011worker1 # [ 0.404076] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1012lb # [ 0.401572] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001013worker2 # [ 0.449531] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1014worker1 # [ 0.404093] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1015worker2 # [ 0.449548] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1016lb # [ 0.401581] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001017worker1 # [ 0.404555] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1018worker2 # [ 0.449580] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019worker1 # [ 0.404746] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1020lb # [ 0.401601] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1021worker2 # [ 0.450107] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1022worker1 # [ 0.404762] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1023worker2 # [ 0.450297] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1024lb # [ 0.401623] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1025worker1 # [ 0.404792] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1026worker2 # [ 0.450315] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1027lb # [ 0.401702] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001028worker2 # [ 0.450344] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1029lb # [ 0.401711] pci_bus 0000:00: root bus resource [bus 00-ff]1030client # [ 0.459653] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1031worker2 # [ 0.450362] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1032lb # [ 0.401717] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1033client # [ 0.459897] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1034lb # [ 0.401722] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1035worker2 # [ 0.450839] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1036client # [ 0.459927] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037worker2 # [ 0.451033] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1038lb # [ 0.401728] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1039client # [ 0.460337] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1040worker2 # [ 0.451063] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1041lb # [ 0.401791] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1042client # [ 0.460520] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1043worker2 # [ 0.451537] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1044lb # [ 0.402241] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1045client # [ 0.460783] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1046worker2 # [ 0.451735] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1047lb # [ 0.402439] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1048client # [ 0.461096] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1049worker2 # [ 0.451765] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1050lb # [ 0.402455] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1051client # [ 0.461114] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1052lb # [ 0.402485] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1053worker2 # [ 0.452162] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1054client # [ 0.461143] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1055lb # [ 0.402502] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1056worker2 # [ 0.452348] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1057client # [ 0.461609] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1058lb # [ 0.403012] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1059client # [ 0.461797] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1060lb # [ 0.403206] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1061client # [ 0.461813] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1062lb # [ 0.403222] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1063client # [ 0.461843] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1064lb # [ 0.403253] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065client # [ 0.462429] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1066worker2 # [ 0.469504] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1067lb # [ 0.403755] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1068client # [ 0.462440] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1069lb # [ 0.403958] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1070worker2 # [ 0.469806] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1071worker1 # [ 0.429289] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1072worker2 # [ 0.469824] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1073client # [ 0.462446] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1074lb # [ 0.403974] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1075worker1 # [ 0.429502] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1076worker2 # [ 0.469855] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077lb # [ 0.404005] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078worker1 # [ 0.429519] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1079client # [ 0.462490] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1080worker1 # [ 0.429550] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081worker2 # [ 0.470360] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1082lb # [ 0.404471] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1083client # [ 0.462537] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1084worker2 # [ 0.470551] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1085lb # [ 0.404661] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1086worker1 # [ 0.430062] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1087client # [ 0.462584] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1088worker2 # [ 0.470568] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1089lb # [ 0.404677] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1090worker1 # [ 0.430253] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1091worker2 # [ 0.470597] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1092client # [ 0.462631] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1093lb # [ 0.404713] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1094worker1 # [ 0.430269] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1095worker2 # [ 0.471185] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1096client # [ 0.462681] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1097lb # [ 0.405200] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1098worker1 # [ 0.430303] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1099lb # [ 0.405390] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1100worker2 # [ 0.471196] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1101client # [ 0.462729] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1102worker1 # [ 0.430764] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1103lb # [ 0.405406] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1104worker1 # [ 0.430946] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1105worker2 # [ 0.471202] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1106client # [ 0.462778] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1107lb # [ 0.405436] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108worker1 # [ 0.430963] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1109worker2 # [ 0.471249] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1110client # [ 0.462825] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1111lb # [ 0.405933] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1112worker1 # [ 0.430993] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1113worker2 # [ 0.471298] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1114lb # [ 0.406127] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1115client # [ 0.462873] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1116worker1 # [ 0.431459] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1117lb # [ 0.406144] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1118worker2 # [ 0.471346] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1119worker1 # [ 0.431642] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1120client # [ 0.462953] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1121lb # [ 0.406176] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122worker1 # [ 0.431657] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1123worker2 # [ 0.471394] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1124client # [ 0.462999] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1125worker1 # [ 0.431689] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1126lb # [ 0.406642] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1127worker2 # [ 0.471441] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1128client # [ 0.463022] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1129lb # [ 0.406836] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1130worker1 # [ 0.432144] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1131client # [ 0.463043] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1132worker2 # [ 0.471488] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1133lb # [ 0.406852] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1134worker1 # [ 0.432329] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1135client # [ 0.463065] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1136worker2 # [ 0.471536] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1137worker1 # [ 0.432347] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1138client # [ 0.463090] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1139worker2 # [ 0.471585] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1140worker1 # [ 0.432377] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1141client # [ 0.463112] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1142worker1 # [ 0.432394] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1143worker2 # [ 0.471632] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1144client # [ 0.463134] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1145worker1 # [ 0.432866] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1146worker2 # [ 0.471708] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1147client # [ 0.463156] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1148worker1 # [ 0.433053] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1149worker2 # [ 0.471755] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1150client # [ 0.463179] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1151worker1 # [ 0.433083] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1152worker2 # [ 0.471777] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1153client # [ 0.463201] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1154worker1 # [ 0.433559] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1155worker2 # [ 0.471799] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1156client # [ 0.463224] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1157worker1 # [ 0.433748] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1158worker2 # [ 0.471821] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1159worker1 # [ 0.433802] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1160worker2 # [ 0.471846] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1161worker2 # [ 0.471868] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1162worker1 # [ 0.434205] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1163worker1 # [ 0.434391] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1164worker2 # [ 0.471890] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1165worker2 # [ 0.471912] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1166worker1 # [ 0.434640] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1167worker1 # [ 0.434913] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1168worker2 # [ 0.471934] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1169lb # [ 0.446916] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1170worker1 # [ 0.434932] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1171worker2 # [ 0.471956] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1172lb # [ 0.446946] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1173worker1 # [ 0.434962] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1174worker2 # [ 0.471980] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1175lb # [ 0.447465] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1176worker1 # [ 0.435433] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1177worker2 # [ 0.472002] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1178lb # [ 0.447666] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1179worker1 # [ 0.435618] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1180worker2 # [ 0.472024] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1181lb # [ 0.447697] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1182worker1 # [ 0.435634] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1183worker2 # [ 0.472045] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1184worker1 # [ 0.435665] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1185worker2 # [ 0.472067] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1186lb # [ 0.448173] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1187worker2 # [ 0.472089] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1188lb # [ 0.448368] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1189worker1 # [ 0.436260] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1190worker2 # [ 0.472110] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1191lb # [ 0.448398] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1192worker1 # [ 0.436272] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1193worker2 # [ 0.472132] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1194lb # [ 0.448797] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1195worker1 # [ 0.436278] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1196worker2 # [ 0.472154] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1197client # [ 0.503309] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1198lb # [ 0.448989] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1199worker1 # [ 0.436325] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1200worker2 # [ 0.472179] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1201client # [ 0.503358] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1202lb # [ 0.449239] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1203worker2 # [ 0.472202] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1204worker1 # [ 0.436373] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1205client # [ 0.503381] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1206lb # [ 0.449533] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1207worker2 # [ 0.472229] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1208client # [ 0.503404] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1209worker1 # [ 0.436420] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1210lb # [ 0.449550] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1211worker2 # [ 0.472239] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1212client # [ 0.503427] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1213worker1 # [ 0.436467] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1214lb # [ 0.449581] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1215worker2 # [ 0.472244] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1216client # [ 0.503450] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1217worker1 # [ 0.436514] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1218lb # [ 0.450066] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1219client # [ 0.503473] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1220lb # [ 0.450256] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1221worker1 # [ 0.436560] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1222client # [ 0.503497] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1223lb # [ 0.450272] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1224client # [ 0.503520] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1225worker1 # [ 0.436607] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1226lb # [ 0.450302] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1227client # [ 0.503543] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1228worker1 # [ 0.436655] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1229client # [ 0.503577] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1230worker1 # [ 0.436702] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1231client # [ 0.503587] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1232client # [ 0.503591] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1233worker1 # [ 0.436770] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1234client # [ 0.504789] pci 0000:00:0a.0: enabling device (0000 -> 0002)1235worker1 # [ 0.436853] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1236worker1 # [ 0.436876] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1237worker1 # [ 0.436897] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1238worker1 # [ 0.436919] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1239worker1 # [ 0.436944] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1240worker1 # [ 0.436969] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1241worker2 # [ 0.517543] pci 0000:00:0a.0: enabling device (0000 -> 0002)1242worker1 # [ 0.436992] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1243worker1 # [ 0.437016] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1244worker1 # [ 0.437039] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1245lb # [ 0.470966] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1246worker1 # [ 0.437061] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1247lb # [ 0.470990] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1248worker1 # [ 0.437084] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1249lb # [ 0.470996] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1250worker1 # [ 0.437106] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1251worker1 # [ 0.437128] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1252lb # [ 0.471082] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1253worker1 # [ 0.437150] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1254lb # [ 0.471132] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1255worker1 # [ 0.437171] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1256lb # [ 0.471179] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1257lb # [ 0.471228] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1258lb # [ 0.471276] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1259lb # [ 0.471325] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1260lb # [ 0.471374] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1261lb # [ 0.471445] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1262lb # [ 0.471494] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1263lb # [ 0.471640] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1264lb # [ 0.471689] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1265lb # [ 0.471711] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1266lb # [ 0.471733] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1267lb # [ 0.471755] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1268client # [ 0.535307] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1269lb # [ 0.471781] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1270client # [ 0.537590] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1271lb # [ 0.471803] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1272lb # [ 0.471826] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1273lb # [ 0.471848] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1274worker2 # [ 0.538607] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1275worker1 # [ 0.501254] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1276lb # [ 0.471871] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1277worker1 # [ 0.501356] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1278lb # [ 0.471893] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1279worker1 # [ 0.501383] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1280lb # [ 0.471917] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1281worker1 # [ 0.501407] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1282worker1 # [ 0.501435] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1283lb # [ 0.471940] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1284worker1 # [ 0.501458] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1285lb # [ 0.471963] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1286worker1 # [ 0.501499] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1287lb # [ 0.471984] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1288worker1 # [ 0.501509] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1289lb # [ 0.472007] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1290worker1 # [ 0.501514] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1291lb # [ 0.472029] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1292worker1 # [ 0.502770] pci 0000:00:0a.0: enabling device (0000 -> 0002)1293lb # [ 0.472051] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1294lb # [ 0.472073] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1295lb # [ 0.472095] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1296client # [ 0.547531] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1297lb # [ 0.472117] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1298client # [ 0.550266] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1299lb # [ 0.472139] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1300lb # [ 0.472168] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1301lb # [ 0.472178] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1302worker2 # [ 0.549300] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1303lb # [ 0.472182] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1304worker2 # [ 0.551609] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1305lb # [ 0.473374] pci 0000:00:0a.0: enabling device (0000 -> 0002)1306worker2 # [ 0.555636] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1307client # [ 0.561632] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1308worker2 # [ 0.564437] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1309worker2 # [ 0.566796] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1310worker1 # [ 0.530994] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1311client # [ 0.567962] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1312client # [ 0.570140] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1313lb # [ 0.526653] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1314client # [ 0.580935] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1315worker1 # [ 0.541387] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1316client # [ 0.584267] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1317worker1 # [ 0.544723] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1318client # [ 0.586424] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1319lb # [ 0.536485] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1320worker1 # [ 0.547035] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1321worker2 # [ 0.585097] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1322worker2 # [ 0.587575] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1323worker2 # [ 0.590131] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1324worker1 # [ 0.557280] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1325lb # [ 0.547702] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1326client # [ 0.597124] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1327worker2 # [ 0.592235] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1328worker1 # [ 0.560277] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1329lb # [ 0.550039] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1330lb # [ 0.553851] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1331worker1 # [ 0.572143] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1332lb # [ 0.561332] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1333client # [ 0.612357] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1334worker2 # [ 0.610419] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1335client # [ 0.615137] msm_serial: driver initialized1336worker1 # [ 0.575657] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1337client # [ 0.615913] SuperH (H)SCI(F) driver initialized1338worker2 # [ 0.616896] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1339client # [ 0.615970] STM32 USART driver initialized1340lb # [ 0.571661] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1341worker1 # [ 0.585508] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1342worker1 # [ 0.587616] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1343worker2 # [ 0.625954] msm_serial: driver initialized1344worker2 # [ 0.626148] SuperH (H)SCI(F) driver initialized1345worker2 # [ 0.626211] STM32 USART driver initialized1346worker1 # [ 0.591647] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1347lb # [ 0.579647] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1348lb # [ 0.581953] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1349lb # [ 0.592338] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1350lb # [ 0.595800] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1351worker1 # [ 0.614535] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1352client # [ 0.655542] loop: module loaded1353client # [ 0.655768] virtio_blk virtio5: 1/0/0 default/read/poll queues1354lb # [ 0.609257] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1355client # [ 0.656622] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1356worker1 # [ 0.617301] msm_serial: driver initialized1357worker1 # [ 0.617448] SuperH (H)SCI(F) driver initialized1358worker1 # [ 0.617503] STM32 USART driver initialized1359lb # [ 0.611927] msm_serial: driver initialized1360worker2 # [ 0.659646] loop: module loaded1361lb # [ 0.612088] SuperH (H)SCI(F) driver initialized1362worker2 # [ 0.659886] virtio_blk virtio5: 1/0/0 default/read/poll queues1363lb # [ 0.612145] STM32 USART driver initialized1364worker2 # [ 0.660901] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1365client # [ 0.664149] megasas: 07.734.00.00-rc11366client # [ 0.664906] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1367client # [ 0.667014] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001368client # [ 0.667042] Intel/Sharp Extended Query Table at 0x00311369worker2 # [ 0.669181] megasas: 07.734.00.00-rc11370worker2 # [ 0.670016] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1371client # [ 0.668867] Using buffer write method1372client # [ 0.668933] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1373worker2 # [ 0.672134] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001374worker2 # [ 0.672185] Intel/Sharp Extended Query Table at 0x00311375client # [ 0.670923] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001376client # [ 0.670980] Intel/Sharp Extended Query Table at 0x00311377client # [ 0.680312] Using buffer write method1378client # [ 0.680373] Concatenating MTD devices:1379client # [ 0.680377] (0): "0.flash"1380client # [ 0.680381] (1): "0.flash"1381client # [ 0.680384] into device "0.flash"1382worker1 # [ 0.648318] loop: module loaded1383worker1 # [ 0.648537] virtio_blk virtio5: 1/0/0 default/read/poll queues1384worker1 # [ 0.650533] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1385worker2 # [ 0.685684] Using buffer write method1386worker2 # [ 0.685809] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1387worker2 # [ 0.687697] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001388worker2 # [ 0.687747] Intel/Sharp Extended Query Table at 0x00311389worker2 # [ 0.693078] Using buffer write method1390worker2 # [ 0.693126] Concatenating MTD devices:1391worker2 # [ 0.693129] (0): "0.flash"1392worker2 # [ 0.693133] (1): "0.flash"1393worker2 # [ 0.693137] into device "0.flash"1394lb # [ 0.648118] loop: module loaded1395lb # [ 0.648315] virtio_blk virtio5: 1/0/0 default/read/poll queues1396lb # [ 0.649106] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1397worker1 # [ 0.662015] megasas: 07.734.00.00-rc11398lb # [ 0.655709] megasas: 07.734.00.00-rc11399worker1 # [ 0.662754] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1400lb # [ 0.656475] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1401worker1 # [ 0.665175] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402lb # [ 0.658833] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001403worker1 # [ 0.665263] Intel/Sharp Extended Query Table at 0x00311404lb # [ 0.658860] Intel/Sharp Extended Query Table at 0x00311405worker1 # [ 0.666875] Using buffer write method1406worker1 # [ 0.666946] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1407worker1 # [ 0.668722] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408worker1 # [ 0.668755] Intel/Sharp Extended Query Table at 0x00311409lb # [ 0.667997] Using buffer write method1410lb # [ 0.668082] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1411worker1 # [ 0.687488] Using buffer write method1412worker1 # [ 0.687535] Concatenating MTD devices:1413worker1 # [ 0.687540] (0): "0.flash"1414worker1 # [ 0.687546] (1): "0.flash"1415worker1 # [ 0.687550] into device "0.flash"1416lb # [ 0.679478] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417lb # [ 0.679552] Intel/Sharp Extended Query Table at 0x00311418lb # [ 0.681077] Using buffer write method1419lb # [ 0.681110] Concatenating MTD devices:1420lb # [ 0.681115] (0): "0.flash"1421lb # [ 0.681119] (1): "0.flash"1422lb # [ 0.681122] into device "0.flash"1423client # [ 0.950928] Freeing initrd memory: 26160K1424client # [ 0.957055] tun: Universal TUN/TAP device driver, 1.61425client # [ 0.960926] thunder_xcv, ver 1.01426client # [ 0.960976] thunder_bgx, ver 1.01427client # [ 0.960998] nicpf, ver 1.01428client # [ 0.961537] e1000: Intel(R) PRO/1000 Network Driver1429client # [ 0.961544] e1000: Copyright (c) 1999-2006 Intel Corporation.1430client # [ 0.961568] e1000e: Intel(R) PRO/1000 Network Driver1431client # [ 0.961577] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432client # [ 0.961606] igb: Intel(R) Gigabit Ethernet Network Driver1433client # [ 0.961612] igb: Copyright (c) 2007-2014 Intel Corporation.1434client # [ 0.961635] igbvf: Intel(R) Gigabit Virtual Function Network Driver1435client # [ 0.961641] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1436client # [ 0.961771] sky2: driver version 1.301437client # [ 0.964069] ehci-pci 0000:00:0a.0: EHCI Host Controller1438client # [ 0.964096] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11439client # [ 0.964352] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001440worker2 # [ 0.971790] Freeing initrd memory: 26148K1441client # [ 0.974064] usbcore: registered new interface driver usb-storage1442client # [ 0.974216] usbcore: registered new interface driver usbserial_generic1443client # [ 0.974232] usbserial: USB Serial support registered for generic1444client # [ 0.974867] hv_vmbus: registering driver hyperv_keyboard1445client # [ 0.975302] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001446client # [ 0.975639] hub 1-0:1.0: USB hub found1447worker2 # [ 0.977855] tun: Universal TUN/TAP device driver, 1.61448client # [ 0.975661] hub 1-0:1.0: 6 ports detected1449client # [ 0.981131] rtc-pl031 9010000.pl031: registered as rtc01450client # [ 0.981166] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:46:31 UTC (1789908391)1451client # [ 0.981508] i2c_dev: i2c /dev entries driver1452worker2 # [ 0.982010] thunder_xcv, ver 1.01453client # [ 0.986560] sdhci: Secure Digital Host Controller Interface driver1454worker2 # [ 0.982063] thunder_bgx, ver 1.01455client # [ 0.986570] sdhci: Copyright(c) Pierre Ossman1456worker2 # [ 0.982088] nicpf, ver 1.01457worker2 # [ 0.982654] e1000: Intel(R) PRO/1000 Network Driver1458client # [ 0.986835] Synopsys Designware Multimedia Card Interface Driver1459client # [ 0.987197] sdhci-pltfm: SDHCI platform and OF driver helper1460worker2 # [ 0.982663] e1000: Copyright (c) 1999-2006 Intel Corporation.1461worker2 # [ 0.982689] e1000e: Intel(R) PRO/1000 Network Driver1462worker2 # [ 0.982700] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1463worker2 # [ 0.982730] igb: Intel(R) Gigabit Ethernet Network Driver1464worker2 # [ 0.982736] igb: Copyright (c) 2007-2014 Intel Corporation.1465worker2 # [ 0.982758] igbvf: Intel(R) Gigabit Virtual Function Network Driver1466client # [ 0.991720] hid: raw HID events driver (C) Jiri Kosina1467worker2 # [ 0.982764] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1468client # [ 0.991973] usbcore: registered new interface driver usbhid1469worker2 # [ 0.982895] sky2: driver version 1.301470client # [ 0.991980] usbhid: USB HID core driver1471worker2 # [ 0.985206] ehci-pci 0000:00:0a.0: EHCI Host Controller1472worker2 # [ 0.985239] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11473client # [ 0.994902] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1474worker2 # [ 0.985517] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001475client # [ 0.997559] drop_monitor: Initializing network drop monitor service1476client # [ 0.997766] NET: Registered PF_INET6 protocol family1477worker2 # [ 0.995332] usbcore: registered new interface driver usb-storage1478client # [ 0.999946] Segment Routing with IPv61479worker2 # [ 0.995393] usbcore: registered new interface driver usbserial_generic1480worker1 # [ 0.960464] Freeing initrd memory: 26148K1481client # [ 0.999967] In-situ OAM (IOAM) with IPv61482worker2 # [ 0.995407] usbserial: USB Serial support registered for generic1483client # [ 0.999998] NET: Registered PF_PACKET protocol family1484worker2 # [ 0.996005] hv_vmbus: registering driver hyperv_keyboard1485worker2 # [ 0.996452] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001486worker2 # [ 0.996865] hub 1-0:1.0: USB hub found1487worker2 # [ 0.996888] hub 1-0:1.0: 6 ports detected1488client # [ 1.001613] 9pnet: Installing 9P2000 support1489worker1 # [ 0.966693] tun: Universal TUN/TAP device driver, 1.61490worker2 # [ 1.002493] rtc-pl031 9010000.pl031: registered as rtc01491worker2 # [ 1.002530] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:46:31 UTC (1789908391)1492worker2 # [ 1.002867] i2c_dev: i2c /dev entries driver1493client # [ 1.004540] Key type dns_resolver registered1494worker2 # [ 1.007880] sdhci: Secure Digital Host Controller Interface driver1495lb # [ 0.962482] Freeing initrd memory: 26140K1496worker2 # [ 1.007895] sdhci: Copyright(c) Pierre Ossman1497client # [ 1.010959] registered taskstats version 11498worker2 # [ 1.008158] Synopsys Designware Multimedia Card Interface Driver1499client # [ 1.011131] Loading compiled-in X.509 certificates1500worker1 # [ 0.970600] thunder_xcv, ver 1.01501worker1 # [ 0.970643] thunder_bgx, ver 1.01502worker1 # [ 0.970671] nicpf, ver 1.01503worker1 # [ 0.971223] e1000: Intel(R) PRO/1000 Network Driver1504worker1 # [ 0.971230] e1000: Copyright (c) 1999-2006 Intel Corporation.1505worker1 # [ 0.971254] e1000e: Intel(R) PRO/1000 Network Driver1506worker2 # [ 1.010648] sdhci-pltfm: SDHCI platform and OF driver helper1507worker1 # [ 0.971265] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1508worker2 # [ 1.012363] hid: raw HID events driver (C) Jiri Kosina1509worker1 # [ 0.971298] igb: Intel(R) Gigabit Ethernet Network Driver1510worker2 # [ 1.012617] usbcore: registered new interface driver usbhid1511lb # [ 0.968719] tun: Universal TUN/TAP device driver, 1.61512worker1 # [ 0.971304] igb: Copyright (c) 2007-2014 Intel Corporation.1513worker2 # [ 1.012625] usbhid: USB HID core driver1514worker1 # [ 0.971329] igbvf: Intel(R) Gigabit Virtual Function Network Driver1515worker1 # [ 0.971334] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1516worker2 # [ 1.016263] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1517worker1 # [ 0.971466] sky2: driver version 1.301518worker1 # [ 0.973079] usbcore: registered new interface driver usb-storage1519client # [ 1.019954] Demotion targets for Node 0: null1520worker1 # [ 0.973146] usbcore: registered new interface driver usbserial_generic1521client # [ 1.020069] Key type .fscrypt registered1522worker2 # [ 1.018932] drop_monitor: Initializing network drop monitor service1523worker1 # [ 0.973160] usbserial: USB Serial support registered for generic1524client # [ 1.020076] Key type fscrypt-provisioning registered1525worker2 # [ 1.019104] NET: Registered PF_INET6 protocol family1526worker1 # [ 0.974015] ehci-pci 0000:00:0a.0: EHCI Host Controller1527client # [ 1.020172] ima: No TPM chip found, activating TPM-bypass!1528client # [ 1.020192] ima: Allocated hash algorithm: sha11529worker1 # [ 0.974042] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11530worker2 # [ 1.021335] Segment Routing with IPv61531client # [ 1.020216] ima: No architecture policies found1532worker2 # [ 1.021359] In-situ OAM (IOAM) with IPv61533lb # [ 0.972562] thunder_xcv, ver 1.01534worker2 # [ 1.021395] NET: Registered PF_PACKET protocol family1535lb # [ 0.972610] thunder_bgx, ver 1.01536worker1 # [ 0.974357] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001537lb # [ 0.972634] nicpf, ver 1.01538client # [ 1.024704] input: gpio-keys as /devices/platform/gpio-keys/input/input01539worker1 # [ 0.985696] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001540lb # [ 0.973198] e1000: Intel(R) PRO/1000 Network Driver1541worker1 # [ 0.986106] hub 1-0:1.0: USB hub found1542lb # [ 0.973207] e1000: Copyright (c) 1999-2006 Intel Corporation.1543worker2 # [ 1.023034] 9pnet: Installing 9P2000 support1544worker1 # [ 0.986123] hub 1-0:1.0: 6 ports detected1545lb # [ 0.973231] e1000e: Intel(R) PRO/1000 Network Driver1546lb # [ 0.973240] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1547lb # [ 0.973270] igb: Intel(R) Gigabit Ethernet Network Driver1548worker1 # [ 0.989486] hv_vmbus: registering driver hyperv_keyboard1549lb # [ 0.973276] igb: Copyright (c) 2007-2014 Intel Corporation.1550lb # [ 0.973300] igbvf: Intel(R) Gigabit Virtual Function Network Driver1551worker2 # [ 1.026016] Key type dns_resolver registered1552worker1 # [ 0.991100] rtc-pl031 9010000.pl031: registered as rtc01553lb # [ 0.973306] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1554lb # [ 0.973447] sky2: driver version 1.301555worker1 # [ 0.991130] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:46:31 UTC (1789908391)1556lb # [ 0.975633] ehci-pci 0000:00:0a.0: EHCI Host Controller1557worker1 # [ 0.991435] i2c_dev: i2c /dev entries driver1558lb # [ 0.975660] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11559lb # [ 0.975865] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001560worker1 # [ 0.996437] sdhci: Secure Digital Host Controller Interface driver1561worker1 # [ 0.996448] sdhci: Copyright(c) Pierre Ossman1562lb # [ 0.985513] usbcore: registered new interface driver usb-storage1563worker1 # [ 0.996713] Synopsys Designware Multimedia Card Interface Driver1564lb # [ 0.985627] usbcore: registered new interface driver usbserial_generic1565worker1 # [ 0.997090] sdhci-pltfm: SDHCI platform and OF driver helper1566lb # [ 0.985644] usbserial: USB Serial support registered for generic1567worker2 # [ 1.032394] registered taskstats version 11568lb # [ 0.986256] hv_vmbus: registering driver hyperv_keyboard1569worker2 # [ 1.032585] Loading compiled-in X.509 certificates1570lb # [ 0.986949] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001571lb # [ 0.987322] hub 1-0:1.0: USB hub found1572lb # [ 0.987403] hub 1-0:1.0: 6 ports detected1573worker1 # [ 1.001514] hid: raw HID events driver (C) Jiri Kosina1574worker1 # [ 1.001786] usbcore: registered new interface driver usbhid1575worker1 # [ 1.001793] usbhid: USB HID core driver1576lb # [ 0.992724] rtc-pl031 9010000.pl031: registered as rtc01577client # [ 1.042304] clk: Disabling unused clocks1578lb # [ 0.992759] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:46:31 UTC (1789908391)1579worker1 # [ 1.004781] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1580worker2 # [ 1.041377] Demotion targets for Node 0: null1581client # [ 1.042337] PM: genpd: Disabling unused power domains1582lb # [ 0.993120] i2c_dev: i2c /dev entries driver1583worker2 # [ 1.041501] Key type .fscrypt registered1584worker2 # [ 1.041509] Key type fscrypt-provisioning registered1585worker2 # [ 1.041610] ima: No TPM chip found, activating TPM-bypass!1586worker1 # [ 1.007411] drop_monitor: Initializing network drop monitor service1587worker2 # [ 1.041630] ima: Allocated hash algorithm: sha11588worker1 # [ 1.007593] NET: Registered PF_INET6 protocol family1589worker2 # [ 1.041655] ima: No architecture policies found1590client # [ 1.046669] Freeing unused kernel memory: 4736K1591lb # [ 0.998119] sdhci: Secure Digital Host Controller Interface driver1592client # [ 1.046857] Run /init as init process1593worker1 # [ 1.009904] Segment Routing with IPv61594lb # [ 0.998131] sdhci: Copyright(c) Pierre Ossman1595worker1 # [ 1.009924] In-situ OAM (IOAM) with IPv61596worker2 # [ 1.046278] input: gpio-keys as /devices/platform/gpio-keys/input/input01597lb # [ 0.998401] Synopsys Designware Multimedia Card Interface Driver1598worker1 # [ 1.009960] NET: Registered PF_PACKET protocol family1599lb # [ 0.998773] sdhci-pltfm: SDHCI platform and OF driver helper1600worker1 # [ 1.011574] 9pnet: Installing 9P2000 support1601lb # [ 1.003207] hid: raw HID events driver (C) Jiri Kosina1602lb # [ 1.003476] usbcore: registered new interface driver usbhid1603lb # [ 1.003483] usbhid: USB HID core driver1604worker1 # [ 1.014397] Key type dns_resolver registered1605lb # [ 1.006407] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1606lb # [ 1.009129] drop_monitor: Initializing network drop monitor service1607lb # [ 1.009323] NET: Registered PF_INET6 protocol family1608worker1 # [ 1.020827] registered taskstats version 11609worker1 # [ 1.020985] Loading compiled-in X.509 certificates1610lb # [ 1.011591] Segment Routing with IPv61611lb # [ 1.011616] In-situ OAM (IOAM) with IPv61612lb # [ 1.011655] NET: Registered PF_PACKET protocol family1613client # [ 1.061598] systemd[1]: Successfully made /usr/ read-only.1614lb # [ 1.013320] 9pnet: Installing 9P2000 support1615lb # [ 1.016184] Key type dns_resolver registered1616worker1 # [ 1.029878] Demotion targets for Node 0: null1617worker1 # [ 1.030002] Key type .fscrypt registered1618worker1 # [ 1.030008] Key type fscrypt-provisioning registered1619worker2 # [ 1.065594] clk: Disabling unused clocks1620worker1 # [ 1.030111] ima: No TPM chip found, activating TPM-bypass!1621worker2 # [ 1.065635] PM: genpd: Disabling unused power domains1622worker1 # [ 1.030130] ima: Allocated hash algorithm: sha11623lb # [ 1.022599] registered taskstats version 11624worker1 # [ 1.030156] ima: No architecture policies found1625lb # [ 1.022762] Loading compiled-in X.509 certificates1626worker2 # [ 1.070035] Freeing unused kernel memory: 4736K1627worker2 # [ 1.070215] Run /init as init process1628worker1 # [ 1.034647] input: gpio-keys as /devices/platform/gpio-keys/input/input01629lb # [ 1.031467] Demotion targets for Node 0: null1630lb # [ 1.031587] Key type .fscrypt registered1631lb # [ 1.031594] Key type fscrypt-provisioning registered1632lb # [ 1.031697] ima: No TPM chip found, activating TPM-bypass!1633lb # [ 1.031718] ima: Allocated hash algorithm: sha11634lb # [ 1.031743] ima: No architecture policies found1635lb # [ 1.036270] input: gpio-keys as /devices/platform/gpio-keys/input/input01636worker2 # [ 1.087274] systemd[1]: Successfully made /usr/ read-only.1637worker1 # [ 1.053680] clk: Disabling unused clocks1638worker1 # [ 1.053713] PM: genpd: Disabling unused power domains1639worker1 # [ 1.058147] Freeing unused kernel memory: 4736K1640worker1 # [ 1.058328] Run /init as init process1641lb # [ 1.054946] clk: Disabling unused clocks1642lb # [ 1.054979] PM: genpd: Disabling unused power domains1643lb # [ 1.059335] Freeing unused kernel memory: 4736K1644lb # [ 1.059546] Run /init as init process1645worker1 # [ 1.075604] systemd[1]: Successfully made /usr/ read-only.1646lb # [ 1.077397] systemd[1]: Successfully made /usr/ read-only.1647client # [ 1.223330] usb 1-1: new high-speed USB device number 2 using ehci-pci1648worker2 # [ 1.244471] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker1 # [ 1.233282] usb 1-1: new high-speed USB device number 2 using ehci-pci1650lb # [ 1.235014] usb 1-1: new high-speed USB device number 2 using ehci-pci1651client # [ 1.375277] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652worker2 # [ 1.398933] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653client # [ 1.396567] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654client # [ 1.408805] systemd[1]: Detected virtualization qemu.1655client # [ 1.410884] systemd[1]: Detected architecture arm64.1656client # [ 1.412953] systemd[1]: Running in initrd.1657client # [ 1.415668] systemd[1]: Initializing machine ID from random generator.1658client # [ 1.418608] systemd[1]: Hostname set to <client>.1659worker1 # [ 1.391724] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660worker2 # [ 1.422365] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1661worker2 # [ 1.434784] systemd[1]: Detected virtualization qemu.1662worker2 # [ 1.436998] systemd[1]: Detected architecture arm64.1663lb # [ 1.387327] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11664worker2 # [ 1.438983] systemd[1]: Running in initrd.1665worker2 # [ 1.441673] systemd[1]: Initializing machine ID from random generator.1666worker2 # [ 1.444599] systemd[1]: Hostname set to <worker2>.1667worker1 # [ 1.410708] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668worker1 # [ 1.422932] systemd[1]: Detected virtualization qemu.1669worker1 # [ 1.424962] systemd[1]: Detected architecture arm64.1670worker1 # [ 1.426992] systemd[1]: Running in initrd.1671client # [ 1.463588] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01672worker1 # [ 1.429724] systemd[1]: Initializing machine ID from random generator.1673worker1 # [ 1.432654] systemd[1]: Hostname set to <worker1>.1674lb # [ 1.412407] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1675lb # [ 1.424836] systemd[1]: Detected virtualization qemu.1676lb # [ 1.427019] systemd[1]: Detected architecture arm64.1677lb # [ 1.428966] systemd[1]: Running in initrd.1678lb # [ 1.431732] systemd[1]: Initializing machine ID from random generator.1679lb # [ 1.434708] systemd[1]: Hostname set to <lb>.1680worker2 # [ 1.496702] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681worker1 # [ 1.477542] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.479300] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683client # [ 1.587334] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker2 # [ 1.620549] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker1 # [ 1.601245] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.602932] usb 1-2: new high-speed USB device number 3 using ehci-pci1687client # [ 1.709604] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688worker2 # [ 1.735515] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689client # [ 1.743682] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21690client # [ 1.746359] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01691worker1 # [ 1.729100] systemd[1]: bpf-restrict-fs: LSM BPF program attached1692worker2 # [ 1.774187] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21693worker2 # [ 1.777802] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694client # [ 1.790179] systemd[1]: Queued start job for default target Initrd Default Target.1695lb # [ 1.750728] systemd[1]: bpf-restrict-fs: LSM BPF program attached1696worker1 # [ 1.761509] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21697client # [ 1.802040] systemd[1]: Created slice Slice /system/modprobe.1698client # [ 1.803114] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1699client # [ 1.804372] systemd[1]: Expecting device /dev/disk/by-label/nixos...1700worker1 # [ 1.764755] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01701client # [ 1.805350] systemd[1]: Reached target Path Units.1702client # [ 1.806049] systemd[1]: Reached target Slice Units.1703client # [ 1.806759] systemd[1]: Reached target Swaps.1704client # [ 1.807433] systemd[1]: Reached target Timer Units.1705client # [ 1.808359] systemd[1]: Listening on D-Bus System Message Bus Socket.1706client # [ 1.809434] systemd[1]: Listening on Journal Socket (/dev/log).1707client # [ 1.810392] systemd[1]: Listening on Journal Sockets.1708client # [ 1.811223] systemd[1]: Listening on udev Control Socket.1709client # [ 1.812187] systemd[1]: Listening on udev Kernel Socket.1710client # [ 1.812213] systemd[1]: Reached target Socket Units.1711client # [ 1.815295] systemd[1]: Starting Create List of Static Device Nodes...1712lb # [ 1.763629] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21713worker2 # [ 1.819072] systemd[1]: Queued start job for default target Initrd Default Target.1714lb # [ 1.769373] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01715client # [ 1.823561] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1716client # [ 1.824744] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1717client # [ 1.830403] systemd[1]: Mounting Kernel Configuration File System...1718worker2 # [ 1.830970] systemd[1]: Created slice Slice /system/modprobe.1719worker2 # [ 1.832096] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1720worker2 # [ 1.833362] systemd[1]: Expecting device /dev/disk/by-label/nixos...1721worker2 # [ 1.833401] systemd[1]: Reached target Path Units.1722worker2 # [ 1.833421] systemd[1]: Reached target Slice Units.1723worker2 # [ 1.833441] systemd[1]: Reached target Swaps.1724worker2 # [ 1.833461] systemd[1]: Reached target Timer Units.1725worker2 # [ 1.833659] systemd[1]: Listening on D-Bus System Message Bus Socket.1726worker2 # [ 1.833803] systemd[1]: Listening on Journal Socket (/dev/log).1727worker2 # [ 1.833930] systemd[1]: Listening on Journal Sockets.1728worker2 # [ 1.834093] systemd[1]: Listening on udev Control Socket.1729worker2 # [ 1.834201] systemd[1]: Listening on udev Kernel Socket.1730worker2 # [ 1.834220] systemd[1]: Reached target Socket Units.1731worker2 # [ 1.844187] systemd[1]: Starting Create List of Static Device Nodes...1732client # [ 1.846767] systemd[1]: Starting Journal Service...1733worker2 # [ 1.852617] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1734worker2 # [ 1.853689] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1735worker1 # [ 1.822347] systemd[1]: Queued start job for default target Initrd Default Target.1736worker2 # [ 1.859869] systemd[1]: Mounting Kernel Configuration File System...1737client # [ 1.867753] systemd[1]: Starting Load Kernel Modules...1738worker1 # [ 1.831516] systemd[1]: Created slice Slice /system/modprobe.1739client # [ 1.868160] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1740worker1 # [ 1.832682] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1741worker1 # [ 1.834059] systemd[1]: Expecting device /dev/disk/by-label/nixos...1742worker1 # [ 1.835017] systemd[1]: Reached target Path Units.1743worker1 # [ 1.835837] systemd[1]: Reached target Slice Units.1744worker1 # [ 1.836579] systemd[1]: Reached target Swaps.1745worker1 # [ 1.837302] systemd[1]: Reached target Timer Units.1746worker1 # [ 1.838271] systemd[1]: Listening on D-Bus System Message Bus Socket.1747worker1 # [ 1.839383] systemd[1]: Listening on Journal Socket (/dev/log).1748worker1 # [ 1.840427] systemd[1]: Listening on Journal Sockets.1749worker1 # [ 1.841357] systemd[1]: Listening on udev Control Socket.1750worker1 # [ 1.842311] systemd[1]: Listening on udev Kernel Socket.1751worker1 # [ 1.843137] systemd[1]: Reached target Socket Units.1752worker2 # [ 1.875578] systemd[1]: Starting Journal Service...1753worker1 # [ 1.849452] systemd[1]: Starting Create List of Static Device Nodes...1754worker1 # [ 1.852318] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1755worker1 # [ 1.853432] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1756worker2 # [ 1.896938] systemd[1]: Starting Load Kernel Modules...1757worker1 # [ 1.861394] systemd[1]: Mounting Kernel Configuration File System...1758client # [ 1.899572] systemd[1]: Starting Coldplug All udev Devices...1759worker2 # [ 1.900593] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1760client # [ 1.905635] systemd-journald[73]: Collecting audit messages is disabled.1761lb # [ 1.865874] systemd[1]: Queued start job for default target Initrd Default Target.1762worker1 # [ 1.882130] systemd[1]: Starting Journal Service...1763worker2 # [ 1.921056] systemd-journald[73]: Collecting audit messages is disabled.1764worker2 # [ 1.922175] systemd[1]: Starting Coldplug All udev Devices...1765lb # [ 1.877355] systemd[1]: Created slice Slice /system/modprobe.1766worker1 # [ 1.889493] systemd[1]: Starting Load Kernel Modules...1767lb # [ 1.878577] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1768lb # [ 1.879972] systemd[1]: Expecting device /dev/disk/by-label/nixos...1769worker1 # [ 1.890388] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1770lb # [ 1.881010] systemd[1]: Reached target Path Units.1771lb # [ 1.881802] systemd[1]: Reached target Slice Units.1772client # [ 1.927465] systemd[1]: Finished Create List of Static Device Nodes.1773lb # [ 1.882633] systemd[1]: Reached target Swaps.1774client # [ 1.928662] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1775lb # [ 1.883525] systemd[1]: Reached target Timer Units.1776client # [ 1.929012] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1777lb # [ 1.884541] systemd[1]: Listening on D-Bus System Message Bus Socket.1778lb # [ 1.885744] systemd[1]: Listening on Journal Socket (/dev/log).1779worker2 # [ 1.932810] systemd[1]: Finished Create List of Static Device Nodes.1780lb # [ 1.886924] systemd[1]: Listening on Journal Sockets.1781lb # [ 1.887946] systemd[1]: Listening on udev Control Socket.1782worker2 # [ 1.934723] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1783lb # [ 1.888973] systemd[1]: Listening on udev Kernel Socket.1784lb # [ 1.889852] systemd[1]: Reached target Socket Units.1785client # [ 1.938068] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1786lb # [ 1.892464] systemd[1]: Starting Create List of Static Device Nodes...1787worker1 # [ 1.905554] systemd[1]: Starting Coldplug All udev Devices...1788client # [ 1.947751] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1789lb # [ 1.903166] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1790lb # [ 1.904431] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1791client # [ 1.955426] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1792worker2 # [ 1.952737] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1793worker2 # [ 1.953461] systemd[1]: Mounted Kernel Configuration File System.1794worker1 # [ 1.921387] systemd[1]: Finished Create List of Static Device Nodes.1795lb # [ 1.911863] systemd[1]: Mounting Kernel Configuration File System...1796worker1 # [ 1.923351] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1797worker2 # [ 1.960821] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1798client # [ 1.963499] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01799client # [ 1.963758] [drm] features: -virgl +edid -resource_blob -host_visible1800client # [ 1.963770] [drm] features: -context_init1801client # [ 1.964569] [drm] number of scanouts: 11802client # [ 1.964591] [drm] number of cap sets: 01803worker1 # [ 1.933733] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1804worker1 # [ 1.934201] systemd[1]: Mounted Kernel Configuration File System.1805lb # [ 1.927354] systemd[1]: Starting Journal Service...1806client # [ 1.977662] systemd[1]: Mounted Kernel Configuration File System.1807worker1 # [ 1.941650] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1808client # [ 1.987653] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1809client # [ 1.987677] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01810worker2 # [ 1.996204] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1811lb # [ 1.951101] systemd[1]: Starting Load Kernel Modules...1812lb # [ 1.951244] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1813worker1 # [ 1.962884] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1814worker1 # [ 1.968447] systemd-journald[73]: Collecting audit messages is disabled.1815lb # [ 1.963498] systemd[1]: Starting Coldplug All udev Devices...1816worker2 # [ 2.012512] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1817worker2 # [ 2.017291] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01818worker2 # [ 2.017540] [drm] features: -virgl +edid -resource_blob -host_visible1819worker2 # [ 2.017550] [drm] features: -context_init1820worker2 # [ 2.018293] [drm] number of scanouts: 11821worker2 # [ 2.018314] [drm] number of cap sets: 01822worker1 # [ 1.985579] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1823client # [ 2.023756] Console: switching to colour frame buffer device 160x501824client # [ 2.028524] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1825worker1 # [ 1.993749] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01826worker1 # [ 1.994026] [drm] features: -virgl +edid -resource_blob -host_visible1827worker1 # [ 1.994035] [drm] features: -context_init1828client # [ 2.032028] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1829worker1 # [ 1.994762] [drm] number of scanouts: 11830worker1 # [ 1.994781] [drm] number of cap sets: 01831worker2 # [ 2.033122] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1832client # [ 2.035710] systemd[1]: Starting Create Static Device Nodes in /dev...1833worker2 # [ 2.035837] systemd[1]: Starting Create Static Device Nodes in /dev...1834lb # [ 1.987049] systemd[1]: Finished Create List of Static Device Nodes.1835lb # [ 1.988370] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1836lb # [ 1.988709] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1837lb # [ 1.988963] systemd[1]: Mounted Kernel Configuration File System.1838worker2 # [ 2.040812] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1839worker2 # [ 2.040839] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01840lb # [ 1.999320] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1841worker1 # [ 2.009994] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1842worker1 # [ 2.012653] systemd[1]: Starting Create Static Device Nodes in /dev...1843lb # [ 2.012161] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1844client # [ 2.063597] systemd[1]: Finished Load Kernel Modules.1845client # [ 2.067460] systemd[1]: Starting Apply Kernel Variables...1846worker1 # [ 2.021530] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1847lb # [ 2.023402] systemd-journald[73]: Collecting audit messages is disabled.1848worker1 # [ 2.021557] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01849worker2 # [ 2.072745] Console: switching to colour frame buffer device 160x501850lb # [ 2.035269] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1851worker1 # [ 2.045531] Console: switching to colour frame buffer device 160x501852lb # [ 2.043008] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01853worker1 # [ 2.052576] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1854lb # [ 2.043263] [drm] features: -virgl +edid -resource_blob -host_visible1855lb # [ 2.043272] [drm] features: -context_init1856lb # [ 2.044051] [drm] number of scanouts: 11857lb # [ 2.044070] [drm] number of cap sets: 01858worker2 # [ 2.095024] systemd[1]: Finished Create Static Device Nodes in /dev.1859worker2 # [ 2.095227] systemd[1]: Reached target Preparation for Local File Systems.1860worker2 # [ 2.095254] systemd[1]: Reached target Local File Systems.1861worker2 # [ 2.099315] systemd[1]: Starting Rule-based Manager for Device Events and Files...1862client # [ 2.103663] systemd[1]: Finished Create Static Device Nodes in /dev.1863client # [ 2.103955] systemd[1]: Reached target Preparation for Local File Systems.1864client # [ 2.103994] systemd[1]: Reached target Local File Systems.1865client # [ 2.109482] systemd[1]: Starting Rule-based Manager for Device Events and Files...1866lb # [ 2.063711] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1867worker2 # [ 2.109118] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1868lb # [ 2.066516] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1869lb # [ 2.066544] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01870worker1 # [ 2.077966] systemd[1]: Finished Create Static Device Nodes in /dev.1871worker2 # [ 2.100514] systemd-modules-load[74]: Inserted module 'dm_mod'1872worker1 # [ 2.078210] systemd[1]: Reached target Preparation for Local File Systems.1873client # [ 2.100478] systemd-modules-load[75]: Inserted module 'dm_mod'1874worker2 # [ 2.102219] systemd-modules-load[74]: Module 'virtio_balloon' is built in1875worker1 # [ 2.078246] systemd[1]: Reached target Local File Systems.1876client # [ 2.101832] systemd-modules-load[75]: Module 'virtio_balloon' is built in1877worker2 # [ 2.103310] systemd-modules-load[74]: Module 'virtio_console' is built in1878client # [ 2.103173] systemd-modules-load[75]: Module 'virtio_console' is built in1879worker2 # [ 2.121524] systemd[1]: Started Journal Service.1880worker1 # [ 2.085872] systemd[1]: Starting Rule-based Manager for Device Events and Files...1881worker2 # [ 2.109288] systemd-modules-load[74]: Inserted module 'virtio_gpu'1882client # [ 2.127514] systemd[1]: Started Journal Service.1883worker2 # [ 2.112325] systemd-modules-load[74]: Module 'virtio_rng' is built in1884lb # [ 2.079554] systemd[1]: Starting Create Static Device Nodes in /dev...1885client # [ 2.116593] systemd-modules-load[75]: Inserted module 'virtio_gpu'1886client # [ 2.117688] systemd-modules-load[75]: Module 'virtio_rng' is built in1887worker2 # [ 2.120121] systemd[1]: Finished Load Kernel Modules.1888client # [ 2.121600] systemd[1]: Finished Apply Kernel Variables.1889worker1 # [ 2.105901] systemd[1]: Finished Load Kernel Modules.1890lb # [ 2.096069] Console: switching to colour frame buffer device 160x501891worker2 # [ 2.137504] systemd[1]: Starting Apply Kernel Variables...1892client # [ 2.140938] systemd[1]: Starting Create System Files and Directories...1893worker1 # [ 2.117796] systemd[1]: Starting Apply Kernel Variables...1894lb # [ 2.131620] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1895worker2 # [ 2.163208] systemd[1]: Starting Create System Files and Directories...1896lb # [ 2.133949] systemd[1]: Finished Create Static Device Nodes in /dev.1897lb # [ 2.136986] systemd[1]: Reached target Preparation for Local File Systems.1898lb # [ 2.137039] systemd[1]: Reached target Local File Systems.1899lb # [ 2.142055] systemd[1]: Starting Rule-based Manager for Device Events and Files...1900worker1 # [ 2.161451] systemd[1]: Finished Apply Kernel Variables.1901worker1 # [ 2.163403] systemd[1]: Started Journal Service.1902worker1 # [ 2.149414] systemd-modules-load[74]: Inserted module 'dm_mod'1903worker1 # [ 2.155133] systemd-modules-load[74]: Module 'virtio_balloon' is built in1904client # [ 2.187808] systemd-udevd[81]: Using default interface naming scheme 'v261'.1905worker2 # [ 2.189710] systemd-udevd[79]: Using default interface naming scheme 'v261'.1906client # [ 2.191096] systemd[1]: Finished Create System Files and Directories.1907lb # [ 2.163039] systemd[1]: Finished Load Kernel Modules.1908worker1 # [ 2.164466] systemd-modules-load[74]: Module 'virtio_console' is built in1909worker1 # [ 2.165622] systemd-modules-load[74]: Inserted module 'virtio_gpu'1910worker1 # [ 2.166608] systemd-modules-load[74]: Module 'virtio_rng' is built in1911worker1 # [ 2.167605] systemd[1]: Starting Create System Files and Directories...1912worker2 # [ 2.204581] systemd[1]: Finished Create System Files and Directories.1913lb # [ 2.171311] systemd[1]: Starting Apply Kernel Variables...1914worker2 # [ 2.207239] systemd[1]: Finished Apply Kernel Variables.1915client # [ 2.222017] systemd[1]: Started Rule-based Manager for Device Events and Files.1916worker2 # [ 2.232395] systemd[1]: Started Rule-based Manager for Device Events and Files.1917worker1 # [ 2.205690] systemd-udevd[80]: Using default interface naming scheme 'v261'.1918lb # [ 2.204565] systemd-modules-load[74]: Inserted module 'dm_mod'1919worker1 # [ 2.217756] systemd[1]: Finished Create System Files and Directories.1920lb # [ 2.220665] systemd[1]: Started Journal Service.1921lb # [ 2.209293] systemd-modules-load[74]: Module 'virtio_balloon' is built in1922lb # [ 2.210449] systemd-modules-load[74]: Module 'virtio_console' is built in1923lb # [ 2.211649] systemd-modules-load[74]: Inserted module 'virtio_gpu'1924worker1 # [ 2.239957] systemd[1]: Started Rule-based Manager for Device Events and Files.1925lb # [ 2.228443] systemd-modules-load[74]: Module 'virtio_rng' is built in1926lb # [ 2.229600] systemd[1]: Starting Create System Files and Directories...1927client # [ 2.285857] systemd[1]: Starting Virtual Console Setup...1928lb # [ 2.242832] systemd[1]: Finished Apply Kernel Variables.1929worker2 # [ 2.296147] systemd[1]: Starting Virtual Console Setup...1930lb # [ 2.263013] systemd-udevd[79]: Using default interface naming scheme 'v261'.1931lb # [ 2.277848] systemd[1]: Finished Create System Files and Directories.1932worker1 # [ 2.304770] systemd[1]: Starting Virtual Console Setup...1933client # [ 2.336590] systemd-vconsole-setup[105]: Configuration of first virtual console was skipped, ignoring remaining ones.1934client # [ 2.340232] systemd[1]: Finished Virtual Console Setup.1935worker2 # [ 2.344621] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1936lb # [ 2.302520] systemd[1]: Started Rule-based Manager for Device Events and Files.1937worker2 # [ 2.348292] systemd[1]: Finished Virtual Console Setup.1938worker1 # [ 2.360617] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1939worker1 # [ 2.364252] systemd[1]: Finished Virtual Console Setup.1940lb # [ 2.372154] systemd[1]: Starting Virtual Console Setup...1941lb # [ 2.420650] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942lb # [ 2.424193] systemd[1]: Finished Virtual Console Setup.1943worker2 # [ 2.945459] systemd[1]: Finished Coldplug All udev Devices.1944worker2 # [ 2.947092] systemd[1]: Reached target System Initialization.1945worker2 # [ 2.952177] systemd[1]: Reached target Basic System.1946worker1 # [ 2.948058] systemd[1]: Finished Coldplug All udev Devices.1947client # [ 2.981112] systemd[1]: Finished Coldplug All udev Devices.1948client # [ 2.983007] systemd[1]: Reached target System Initialization.1949worker1 # [ 2.952160] systemd[1]: Reached target System Initialization.1950client # [ 2.984893] systemd[1]: Reached target Basic System.1951worker1 # [ 2.953046] systemd[1]: Reached target Basic System.1952worker2 # [ 3.087928] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1953lb # [ 3.068544] systemd[1]: Finished Coldplug All udev Devices.1954lb # [ 3.072090] systemd[1]: Reached target System Initialization.1955lb # [ 3.073017] systemd[1]: Reached target Basic System.1956worker2 # [ 3.119155] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1957worker1 # [ 3.087764] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1958worker2 # [ 3.130393] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1959client # [ 3.151065] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1960worker1 # [ 3.120814] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1961worker1 # [ 3.128924] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1962client # [ 3.166313] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1963client # [ 3.176776] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1964worker2 # [ 3.193115] systemd[1]: Found device /dev/disk/by-label/nixos.1965worker2 # [ 3.199393] systemd[1]: Reached target Initrd Root Device.1966worker2 # [ 3.203946] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1967client # [ 3.233559] systemd[1]: Found device /dev/disk/by-label/nixos.1968client # [ 3.235868] systemd[1]: Reached target Initrd Root Device.1969client # [ 3.241286] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1970lb # [ 3.207594] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 3.221995] systemd[1]: Found device /dev/disk/by-label/nixos.1972worker1 # [ 3.226082] systemd[1]: Reached target Initrd Root Device.1973worker2 # [ 3.259182] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1974worker1 # [ 3.228254] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1975worker2 # [ 3.268169] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1976worker2 # [ 3.270830] systemd[1]: Mounting /sysroot...1977client # [ 3.287777] systemd-fsck[114]: nixos: clean, 12/65536 files, 13019/262144 blocks1978client # [ 3.299315] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1979lb # [ 3.261066] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1980worker1 # [ 3.277695] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1981client # [ 3.310695] systemd[1]: Mounting /sysroot...1982lb # [ 3.267082] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1983worker2 # [ 3.328147] EXT4-fs (vda): mounted filesystem cfd7f4f4-4147-4be4-a1ab-49c6973fa371 r/w with ordered data mode. Quota mode: none.1984worker1 # [ 3.283709] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1985worker2 # [ 3.317772] systemd[1]: Mounted /sysroot.1986worker2 # [ 3.321346] systemd[1]: Reached target Initrd Root File System.1987worker1 # [ 3.288754] systemd[1]: Mounting /sysroot...1988worker2 # [ 3.325646] systemd[1]: Starting Mountpoints Configured in the Real Root...1989client # [ 3.365415] EXT4-fs (vda): mounted filesystem 112807a0-6577-416e-bbb1-4b8788bb94f2 r/w with ordered data mode. Quota mode: none.1990worker2 # [ 3.352243] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload.1991client # [ 3.353599] systemd[1]: Mounted /sysroot.1992client # [ 3.355998] systemd[1]: Reached target Initrd Root File System.1993client # [ 3.361165] systemd[1]: Starting Mountpoints Configured in the Real Root...1994worker2 # [ 3.362102] systemd[1]: Reload requested from client PID 119 ('systemd-sysroot') (unit initrd-parse-etc.service)...1995worker1 # [ 3.342246] EXT4-fs (vda): mounted filesystem 5810dfc5-862c-40e4-b6a0-0fc393131a27 r/w with ordered data mode. Quota mode: none.1996worker2 # [ 3.364655] systemd[1]: Reloading...1997worker1 # [ 3.334636] systemd[1]: Mounted /sysroot.1998worker1 # [ 3.337368] systemd[1]: Reached target Initrd Root File System.1999worker1 # [ 3.342288] systemd[1]: Starting Mountpoints Configured in the Real Root...2000client # [ 3.388302] systemd-sysroot-fstab-check[122]: /sysroot should be mounted in the initrd, will request daemon-reload.
loading 1,521 hidden lines…
3522lb # [ 20.778783] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3523lb # [ 20.784764] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.matching.generic_proxy_request_input.input: envoy.matching.generic_proxy.input.host, envoy.matching.generic_proxy.input.method, envoy.matching.generic_proxy.input.path, envoy.matching.generic_proxy.input.property, envoy.matching.generic_proxy.input.request, envoy.matching.generic_proxy.input.service3524lb # [ 20.789768] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.clusters: envoy.cluster.dns, envoy.cluster.eds, envoy.cluster.logical_dns, envoy.cluster.original_dst, envoy.cluster.static, envoy.cluster.strict_dns, envoy.clusters.aggregate, envoy.clusters.composite, envoy.clusters.dynamic_forward_proxy, envoy.clusters.dynamic_modules, envoy.clusters.mcp_multicluster, envoy.clusters.redis, envoy.clusters.reverse_connection3525lb # [ 20.800222] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.filters.http: envoy.bandwidth_limit, envoy.buffer, envoy.cors, envoy.csrf, envoy.ext_authz, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.fault, envoy.filters.http.a2a, envoy.filters.http.adaptive_concurrency, envoy.filters.http.admission_control, envoy.filters.http.ai_protocol_manager, envoy.filters.http.alternate_protocols_cache, envoy.filters.http.api_key_auth, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.bandwidth_limit, envoy.filters.http.bandwidth_share, envoy.filters.http.basic_auth, envoy.filters.http.buffer, envoy.filters.http.cache, envoy.filters.http.cache_v2, envoy.filters.http.cdn_loop, envoy.filters.http.composite, envoy.filters.http.compressor, envoy.filters.http.connect_grpc_bridge, envoy.filters.http.cors, envoy.filters.http.credential_injector, envoy.filters.http.csrf, envoy.filters.http.custom_response, envoy.filters.http.decompressor, envoy.filters.http.dynamic_forward_proxy, envoy.filters.http.ext_authz, envoy.filters.http.ext_proc, envoy.filters.http.fault, envoy.filters.http.file_server, envoy.filters.http.file_system_buffer, envoy.filters.http.filter_chain, envoy.filters.http.gcp_authn, envoy.filters.http.geoip, envoy.filters.http.grpc_field_extraction, envoy.filters.http.grpc_http1_bridge, envoy.filters.http.grpc_http1_reverse_bridge, envoy.filters.http.grpc_json_reverse_transcoder, envoy.filters.http.grpc_json_transcoder, envoy.filters.http.grpc_stats, envoy.filters.http.grpc_web, envoy.filters.http.header_mutation, envoy.filters.http.header_to_metadata, envoy.filters.http.health_check, envoy.filters.http.ip_tagging, envoy.filters.http.json_to_metadata, envoy.filters.http.jwt_authn, envoy.filters.http.local_ratelimit, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.mcp, envoy.filters.http.mcp_json_rest_bridge, envoy.filters.http.mcp_router, envoy.filters.http.oauth2, envoy.filters.http.on_demand, envoy.filters.http.original_src, envoy.filters.http.proto_api_scrubber, envoy.filters.http.proto_message_extraction, envoy.filters.http.rate_limit_quota, envoy.filters.http.ratelimit, envoy.filters.http.rbac, envoy.filters.http.router, envoy.filters.http.set_filter_state, envoy.filters.http.set_metadata, envoy.filters.http.sse_to_metadata, envoy.filters.http.stateful_session, envoy.filters.http.tap, envoy.filters.http.thrift_to_metadata, envoy.filters.http.transform, envoy.filters.http.wasm, envoy.geoip, envoy.grpc_http1_bridge, envoy.grpc_json_transcoder, envoy.grpc_web, envoy.health_check, envoy.ip_tagging, envoy.local_rate_limit, envoy.lua, envoy.rate_limit, envoy.router3526lb # [ 20.835144] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3527lb # [ 20.837595] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.regex_engines: envoy.regex_engines.google_re23528lb # [ 20.839915] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83529lb # [ 20.842422] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.resource_detectors: envoy.tracers.opentelemetry.resource_detectors.dynatrace, envoy.tracers.opentelemetry.resource_detectors.environment, envoy.tracers.opentelemetry.resource_detectors.static_config3530lb # [ 20.848970] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.path.match: envoy.path.match.uri_template.uri_template_matcher3531lb # [ 20.851882] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3532lb # [ 20.858135] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.matching.stat_tag_matching_data.input: stat_tag_value_input3533lb # [ 20.860959] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.path.rewrite: envoy.path.rewrite.uri_template.uri_template_rewriter3534lb # [ 20.863745] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.matching.action: envoy.matching.action.transport_socket.name, envoy.matching.actions.format_string, filter-chain-name3535lb # [ 20.866934] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.load_balancing_policies: envoy.load_balancing_policies.client_side_weighted_round_robin, envoy.load_balancing_policies.cluster_provided, envoy.load_balancing_policies.dynamic_modules, envoy.load_balancing_policies.least_request, envoy.load_balancing_policies.load_aware_locality, envoy.load_balancing_policies.maglev, envoy.load_balancing_policies.override_host, envoy.load_balancing_policies.random, envoy.load_balancing_policies.ring_hash, envoy.load_balancing_policies.round_robin, envoy.load_balancing_policies.subset, envoy.load_balancing_policies.wrr_locality3536lb # [ 20.875571] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3537lb # [ 20.884118] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.filters.udp.session: envoy.filters.udp.session.dynamic_forward_proxy, envoy.filters.udp.session.ext_authz, envoy.filters.udp.session.http_capsule3538lb # [ 20.888829] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.http.ext_proc.response_processors: envoy.extensions.http.ext_proc.save_processing_response3539lb # [ 20.894340] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.http.cache: envoy.extensions.http.cache.file_system_http_cache, envoy.extensions.http.cache.simple3540lb # [ 20.901295] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.grpc_credentials: envoy.grpc_credentials.default, envoy.grpc_credentials.file_based_metadata3541lb # [ 20.903809] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.config_mux: envoy.config_mux.delta_grpc_mux_factory, envoy.config_mux.grpc_mux_factory, envoy.config_mux.new_grpc_mux_factory, envoy.config_mux.sotw_grpc_mux_factory3542lb # [ 20.909337] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23543lb # [ 20.913218] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3544lb # [ 20.916460] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.geoip_providers: envoy.geoip_providers.maxmind3545lb # [ 20.920273] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3546lb # [ 20.924508] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3547lb # [ 20.927768] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.upstream.local_address_selector: envoy.upstream.local_address_selector.default_local_address_selector, envoy.upstream.local_address_selector.filter_state_override3548lb # [ 20.931758] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3549lb # [ 20.939853] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.quic.client_packet_writer: envoy.quic.packet_writer.default3550lb # [ 20.942488] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.transport_sockets.upstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.http_11_proxy, envoy.transport_sockets.internal_upstream, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, envoy.transport_sockets.upstream_proxy_protocol, raw_buffer, starttls, tls3551lb # [ 20.949900] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.rate_limit_descriptors: envoy.rate_limit_descriptors.expr3552lb # [ 20.953385] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3553lb # [ 20.956503] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.ssl.server_context_factory: envoy.ssl.server_context_factory.default3554lb # [ 20.958587] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.udp_packet_writer: envoy.udp_packet_writer.default, envoy.udp_packet_writer.gso3555lb # [ 20.960963] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.resource_monitors: envoy.resource_monitors.cgroup_memory, envoy.resource_monitors.cpu_utilization, envoy.resource_monitors.fixed_heap, envoy.resource_monitors.injected_resource3556lb # [ 20.964801] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector3557lb # [ 20.972241] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.serializers: dubbo.hessian23558lb # [ 20.974328] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.route_config_update_requester: envoy.route_config_update_requester.default3559lb # [ 20.976770] envoy[913]: [2026-09-20 12:46:51.022][913][info][main] [source/server/server.cc:471] envoy.built_in_formatters: envoy.built_in_formatters.cel, envoy.built_in_formatters.http.default, envoy.built_in_formatters.metadata, envoy.built_in_formatters.stream_info.default, envoy.built_in_formatters.xfcc_value3560lb # [ 20.980992] envoy[913]: [2026-09-20 12:46:51.062][913][info][main] [source/server/server.cc:531] HTTP header map info:3561lb # [ 20.982527] envoy[913]: [2026-09-20 12:46:51.064][913][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3562lb # [ 20.999635] envoy[913]: [2026-09-20 12:46:51.064][913][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3563lb # [ 21.002436] envoy[913]: [2026-09-20 12:46:51.064][913][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3564lb # [ 21.012586] envoy[913]: [2026-09-20 12:46:51.064][913][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3565lb # [ 21.014510] envoy[913]: [2026-09-20 12:46:51.282][913][info][main] [source/server/server.cc:968] runtime: {}3566lb # [ 21.016424] envoy[913]: [2026-09-20 12:46:51.306][913][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3567worker1 # [ 21.028856] systemd[1]: Starting Nix worker-protocol over gRPC...3568lb # [ 21.018314] envoy[913]: [2026-09-20 12:46:51.306][913][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013569lb # [ 21.020398] envoy[913]: [2026-09-20 12:46:51.306][913][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013570lb # [ 21.022262] envoy[913]: [2026-09-20 12:46:51.317][913][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3571lb # [ 21.024188] envoy[913]: [2026-09-20 12:46:51.317][913][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3572lb # [ 21.029299] envoy[913]: [2026-09-20 12:46:51.317][913][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3573lb # [ 21.031622] envoy[913]: [2026-09-20 12:46:51.549][913][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3574worker2 # [ 21.077795] systemd[1]: Reached target Network is Online.3575lb # [ 21.043379] envoy[913]: [2026-09-20 12:46:51.565][913][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3576worker2 # [ 21.090812] systemd[1]: Starting scheduler-token.service...3577lb # [ 21.046399] envoy[913]: [2026-09-20 12:46:51.567][913][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3578lb # [ 21.050796] envoy[913]: [2026-09-20 12:46:51.568][913][info][main] [source/server/server.cc:1077] starting main dispatch loop3579worker2 # [ 21.221820] systemd[1]: scheduler-token.service: Deactivated successfully.3580worker2 # [ 21.225951] systemd[1]: Finished scheduler-token.service.3581worker2 # [ 21.227336] systemd[1]: scheduler-token.service: Consumed 61ms CPU time over 134ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3582worker2 # [ 21.238489] systemd[1]: Starting Nix worker-protocol over gRPC...3583worker1 # [ 21.278130] nix-grpc-daemon[777]: ts=2026-09-20T12:46:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3584worker1 # [ 21.291067] nix-grpc-daemon[777]: ts=2026-09-20T12:46:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3585worker1 # [ 21.293938] nix-grpc-daemon[777]: ts=2026-09-20T12:46:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13586worker1 # [ 21.348311] nix-grpc-daemon[777]: ts=2026-09-20T12:46:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3587worker1 # [ 21.350239] systemd[1]: Started Nix worker-protocol over gRPC.3588worker1 # [ 21.382643] nix-grpc-daemon[777]: ts=2026-09-20T12:46:51Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3589lb # [ 21.380276] envoy[913]: [2026-09-20 12:46:51.901][913][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3590lb # [ 21.383212] envoy[913]: [2026-09-20 12:46:51.901][913][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3591lb # [ 21.388343] envoy[913]: [2026-09-20 12:46:51.901][913][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3592lb # [ 21.391079] envoy[913]: [2026-09-20 12:46:51.901][913][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3593worker2 # [ 21.464307] nix-grpc-daemon[779]: ts=2026-09-20T12:46:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3594worker2 # [ 21.478056] nix-grpc-daemon[779]: ts=2026-09-20T12:46:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3595worker2 # [ 21.480708] nix-grpc-daemon[779]: ts=2026-09-20T12:46:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13596worker2 # [ 21.518320] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3597worker2 # [ 21.533863] systemd[1]: Started Nix worker-protocol over gRPC.3598worker2 # [ 21.568941] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_connected addr=lb:500513599worker2 # [ 21.574616] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3600worker1 # [ 21.584325] nix-grpc-daemon[777]: ts=2026-09-20T12:46:52Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3601worker2 # [ 21.776040] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_connected addr=lb:500513602worker2 # [ 21.779002] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3603worker2 # [ 21.980994] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_connected addr=lb:500513604worker2 # [ 21.986949] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3605worker1 # [ 21.994673] nix-grpc-daemon[777]: ts=2026-09-20T12:46:52Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3606worker2 # [ 22.188780] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_connected addr=lb:500513607worker2 # [ 22.195087] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3608worker2 # [ 22.396682] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_connected addr=lb:500513609worker2 # [ 22.402365] nix-grpc-daemon[779]: ts=2026-09-20T12:46:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3610worker2 # [ 22.607562] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513611worker2 # [ 22.614733] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3612worker2 # [ 22.808624] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513613worker2 # [ 22.814355] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3614worker1 # [ 22.796538] nix-grpc-daemon[777]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513615worker1 # [ 22.797390] nix-grpc-daemon[777]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3616worker2 # [ 23.015957] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513617worker2 # [ 23.021905] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3618worker1 # [ 22.998807] nix-grpc-daemon[777]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513619worker1 # [ 23.004827] nix-grpc-daemon[777]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3620worker2 # [ 23.223466] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513621worker2 # [ 23.229334] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3622worker1 # [ 23.205769] nix-grpc-daemon[777]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513623worker1 # [ 23.211498] nix-grpc-daemon[777]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3624worker2 # [ 23.430874] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513625worker2 # [ 23.436825] nix-grpc-daemon[779]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3626worker1 # [ 23.412388] nix-grpc-daemon[777]: ts=2026-09-20T12:46:53Z level=info event=scheduler_connected addr=lb:500513627worker1 # [ 23.416411] nix-grpc-daemon[777]: ts=2026-09-20T12:46:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3628worker2 # [ 23.640487] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513629worker2 # [ 23.644831] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3630worker1 # [ 23.617391] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513631worker1 # [ 23.622138] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3632worker2 # [ 23.842394] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513633worker2 # [ 23.848172] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3634worker1 # [ 23.825518] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513635worker1 # [ 23.829827] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker2 # [ 24.049631] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513637worker1 # [ 24.025556] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513638worker2 # [ 24.055386] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3639worker1 # [ 24.027967] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker2 # [ 24.257048] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513641worker1 # [ 24.229278] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513642worker2 # [ 24.262895] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3643worker1 # [ 24.235113] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker2 # [ 24.464483] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513645worker1 # [ 24.435661] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_connected addr=lb:500513646worker1 # [ 24.438961] nix-grpc-daemon[777]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3647worker2 # [ 24.470133] nix-grpc-daemon[779]: ts=2026-09-20T12:46:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker2 # [ 24.673377] nix-grpc-daemon[779]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 24.640219] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513650worker2 # [ 24.678454] nix-grpc-daemon[779]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3651worker1 # [ 24.645772] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker2 # [ 24.879933] nix-grpc-daemon[779]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 24.848743] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513654worker1 # [ 24.852095] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3655worker2 # [ 24.885489] nix-grpc-daemon[779]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker1 # [ 25.050024] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513657worker1 # [ 25.053311] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3658worker2 # [ 25.087299] nix-grpc-daemon[779]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513659worker2 # [ 25.093174] nix-grpc-daemon[779]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker1 # [ 25.254261] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 25.257772] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3662worker2 # [ 25.294700] nix-grpc-daemon[779]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513663worker2 # [ 25.300464] nix-grpc-daemon[779]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 25.458748] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 25.464761] nix-grpc-daemon[777]: ts=2026-09-20T12:46:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3666worker2 # [ 25.502337] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513667worker2 # [ 25.508265] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker1 # [ 25.665322] nix-grpc-daemon[777]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 25.668664] nix-grpc-daemon[777]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3670worker2 # [ 25.711789] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513671worker2 # [ 25.715098] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker1 # [ 25.871879] nix-grpc-daemon[777]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513673worker2 # [ 25.912718] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513674worker1 # [ 25.876682] nix-grpc-daemon[777]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3675worker2 # [ 25.914900] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker1 # [ 26.073404] nix-grpc-daemon[777]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513677worker2 # [ 26.115931] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513678worker1 # [ 26.079749] nix-grpc-daemon[777]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3679worker2 # [ 26.118398] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker1 # [ 26.280812] nix-grpc-daemon[777]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513681worker2 # [ 26.319594] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_connected addr=lb:500513682worker2 # [ 26.322308] nix-grpc-daemon[779]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3683worker1 # [ 26.286810] nix-grpc-daemon[777]: ts=2026-09-20T12:46:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 26.487890] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 26.493619] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3686worker2 # [ 26.524099] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513687worker2 # [ 26.530255] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 26.695041] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513689worker2 # [ 26.733377] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513690worker1 # [ 26.704389] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3691worker2 # [ 26.738655] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker1 # [ 26.903997] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513693worker2 # [ 26.940292] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513694worker1 # [ 26.908896] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3695worker2 # [ 26.945969] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker1 # [ 27.104894] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 27.109230] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3698worker2 # [ 27.147436] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513699worker2 # [ 27.154037] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 27.309847] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513701worker1 # [ 27.313612] nix-grpc-daemon[777]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3702worker2 # [ 27.355055] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_connected addr=lb:500513703worker2 # [ 27.361257] nix-grpc-daemon[779]: ts=2026-09-20T12:46:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 27.514583] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513705worker1 # [ 27.520856] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3706worker2 # [ 27.563887] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513707worker2 # [ 27.570724] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker1 # [ 27.721349] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 27.723903] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3710worker2 # [ 27.774565] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513711worker2 # [ 27.779087] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker1 # [ 27.929116] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 27.933599] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3714worker2 # [ 27.975932] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513715worker2 # [ 27.979865] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker1 # [ 28.128576] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 28.131891] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3718worker2 # [ 28.181636] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513719worker2 # [ 28.185989] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker1 # [ 28.334073] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513721worker1 # [ 28.340553] nix-grpc-daemon[777]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3722worker2 # [ 28.387812] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_connected addr=lb:500513723worker2 # [ 28.394362] nix-grpc-daemon[779]: ts=2026-09-20T12:46:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker1 # [ 28.541051] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 28.544281] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3726worker2 # [ 28.594829] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513727worker2 # [ 28.597147] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker1 # [ 28.745305] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513729worker1 # [ 28.749054] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3730worker2 # [ 28.799862] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513731worker2 # [ 28.803336] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker1 # [ 28.950149] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 28.954839] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3734worker2 # [ 29.005491] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513735worker2 # [ 29.011619] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker1 # [ 29.156683] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 29.163080] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3738worker2 # [ 29.212981] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513739worker2 # [ 29.220754] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker1 # [ 29.364382] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 29.370561] nix-grpc-daemon[777]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3742worker2 # [ 29.420170] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_connected addr=lb:500513743worker2 # [ 29.425849] nix-grpc-daemon[779]: ts=2026-09-20T12:46:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker1 # [ 29.572098] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 29.578182] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3746worker2 # [ 29.623212] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513747worker2 # [ 29.629172] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker1 # [ 29.779067] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 29.782902] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3750worker2 # [ 29.833968] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513751worker2 # [ 29.838594] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker1 # [ 29.990172] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513753worker1 # [ 30.000635] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3754worker2 # [ 30.035359] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513755worker2 # [ 30.041648] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756worker1 # [ 30.188697] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 30.195024] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758worker2 # [ 30.242153] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513759worker2 # [ 30.244514] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3760worker1 # [ 30.395525] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513761worker1 # [ 30.398333] nix-grpc-daemon[777]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3762worker2 # [ 30.446615] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_connected addr=lb:500513763worker2 # [ 30.452892] nix-grpc-daemon[779]: ts=2026-09-20T12:47:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3764worker1 # [ 30.599837] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513765worker1 # [ 30.603082] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker2 # [ 30.654647] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513767worker2 # [ 30.660587] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768worker1 # [ 30.805293] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513769worker1 # [ 30.811828] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3770worker2 # [ 30.863468] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513771worker2 # [ 30.865968] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3772worker1 # [ 31.015790] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513773worker1 # [ 31.020506] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774worker2 # [ 31.064855] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513775worker2 # [ 31.067740] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3776worker1 # [ 31.216657] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513777worker1 # [ 31.219718] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3778worker2 # [ 31.270375] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513779worker2 # [ 31.276623] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3780worker1 # [ 31.421212] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513781worker1 # [ 31.428979] nix-grpc-daemon[777]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3782worker2 # [ 31.478764] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_connected addr=lb:500513783worker2 # [ 31.484932] nix-grpc-daemon[779]: ts=2026-09-20T12:47:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3784worker1 # [ 31.630989] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513785worker1 # [ 31.637245] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3786worker2 # [ 31.685992] nix-grpc-daemon[779]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513787worker2 # [ 31.689992] nix-grpc-daemon[779]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3788worker1 # [ 31.838435] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513789worker1 # [ 31.844464] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3790worker2 # [ 31.894948] nix-grpc-daemon[779]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513791worker2 # [ 31.899568] nix-grpc-daemon[779]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3792worker1 # [ 32.047267] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513793worker1 # [ 32.053426] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3794worker2 # [ 32.096574] nix-grpc-daemon[779]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513795worker2 # [ 32.102547] nix-grpc-daemon[779]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3796worker1 # [ 32.254743] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513797worker1 # [ 32.261034] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3798worker2 # [ 32.304312] nix-grpc-daemon[779]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513799worker2 # [ 32.310344] nix-grpc-daemon[779]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3800worker1 # [ 32.462387] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_connected addr=lb:500513801worker1 # [ 32.468836] nix-grpc-daemon[777]: ts=2026-09-20T12:47:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3802worker2 # [ 32.511514] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513803worker2 # [ 32.515463] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3804worker1 # [ 32.669317] nix-grpc-daemon[777]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513805worker1 # [ 32.672077] nix-grpc-daemon[777]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3806worker2 # [ 32.716819] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513807worker2 # [ 32.721031] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3808worker1 # [ 32.873858] nix-grpc-daemon[777]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513809worker1 # [ 32.880144] nix-grpc-daemon[777]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3810worker2 # [ 32.925729] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513811worker2 # [ 32.930421] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3812worker1 # [ 33.084489] nix-grpc-daemon[777]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513813worker1 # [ 33.088895] nix-grpc-daemon[777]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3814worker2 # [ 33.125659] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513815worker2 # [ 33.128367] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3816lb # [ 33.252894] rustfs-bucket-start[1042]: mb s3://farm3817lb # [ 33.262608] systemd[1]: Finished rustfs-bucket.service.3818lb # [ 33.274337] systemd[1]: Starting niks3 server...3819worker1 # [ 33.285171] nix-grpc-daemon[777]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513820worker1 # [ 33.289053] nix-grpc-daemon[777]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3821worker2 # [ 33.330793] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_connected addr=lb:500513822worker2 # [ 33.336956] nix-grpc-daemon[779]: ts=2026-09-20T12:47:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3823worker1 # [ 33.489399] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513824worker1 # [ 33.493266] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3825worker2 # [ 33.538540] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513826worker2 # [ 33.544841] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3827lb # [ 33.539469] postgres[1058]: [1058] ERROR: relation "goose_db_version" does not exist at character 363828lb # [ 33.544577] postgres[1058]: [1058] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3829lb # [ 33.602148] niks3-server[1051]: 2026/09/20 12:47:04 OK 20241026095416_initial_model.sql (28.87ms)3830lb # [ 33.607329] niks3-server[1051]: 2026/09/20 12:47:04 OK 20251210153512_drop_unused_gin_index.sql (3.3ms)3831lb # [ 33.614552] niks3-server[1051]: 2026/09/20 12:47:04 OK 20251218171726_add_pins.sql (7.18ms)3832lb # [ 33.621110] niks3-server[1051]: 2026/09/20 12:47:04 OK 20260628120000_add_object_size_and_stats.sql (6.5ms)3833lb # [ 33.627765] niks3-server[1051]: 2026/09/20 12:47:04 OK 20260905000000_add_claims.sql (6.59ms)3834lb # [ 33.633730] niks3-server[1051]: 2026/09/20 12:47:04 OK 20260920000000_drop_claims.sql (5.78ms)3835lb # [ 33.635036] niks3-server[1051]: 2026/09/20 12:47:04 goose: successfully migrated database to version: 202609200000003836lb # [ 33.641524] niks3-server[1051]: 2026/09/20 12:47:04 OK 1_commit_pending_closure.sql (7.8ms)3837lb # [ 33.645258] niks3-server[1051]: 2026/09/20 12:47:04 OK 2_object_stats_trigger.sql (3.56ms)3838lb # [ 33.646760] niks3-server[1051]: 2026/09/20 12:47:04 goose: up to current file version: 23839lb # [ 33.651796] niks3-server[1051]: 2026/09/20 12:47:04 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3840worker1 # [ 33.693805] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513841worker1 # [ 33.696439] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3842worker2 # [ 33.747185] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513843worker2 # [ 33.750140] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3844lb # [ 33.706272] niks3-server[1051]: 2026/09/20 12:47:04 INFO Read proxy enabled — serving cache objects from S33845lb # [ 33.709207] niks3-server[1051]: 2026/09/20 12:47:04 INFO Using socket-activated listener address=0.0.0.0:57513846lb # [ 33.717501] niks3-server[1051]: 2026/09/20 12:47:04 INFO systemd watchdog enabled interval=15s3847lb # [ 33.719424] niks3-server[1051]: 2026/09/20 12:47:04 INFO Starting HTTP server address=0.0.0.0:57513848lb # [ 33.722572] systemd[1]: Started niks3 server.3849lb # [ 33.726237] systemd[1]: Reached target Multi-User System.3850worker2 # [ 33.776366] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_take_over3851lb # [ 33.731456] niks3-server[1051]: 2026/09/20 12:47:04 INFO lead: acquired remote=192.168.1.4:385623852lb # [ 33.737795] systemd[1]: Startup finished in 1.049s (kernel) + 4.732s (initrd) + 27.934s (userspace) = 33.716s.3853worker1 # [ 33.898205] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513854lb: (finished: waiting for unit niks3.service, in 18.38 seconds)3855worker1 # [ 33.904508] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3856lb: waiting for TCP port 5751 on localhost3857worker2 # [ 33.955401] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513858worker2 # [ 33.960434] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3859lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3860lb: (finished: waiting for TCP port 5751 on localhost, in 0.08 seconds)3861lb: waiting for unit envoy.service3862lb: (finished: waiting for unit envoy.service, in 0.09 seconds)3863lb: waiting for TCP port 50051 on localhost3864worker1 # [ 34.106884] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513865worker1 # [ 34.109834] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3866worker2 # [ 34.157865] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513867worker2 # [ 34.162663] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3868lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3869lb # [ 34.134296] envoy[913]: conn peer=[::1]:56854 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=53870lb: (finished: waiting for TCP port 50051 on localhost, in 0.07 seconds)3871worker1 # [ 34.307854] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513872worker1 # [ 34.310910] nix-grpc-daemon[777]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3873worker2 # [ 34.359634] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_connected addr=lb:500513874worker2 # [ 34.365627] nix-grpc-daemon[779]: ts=2026-09-20T12:47:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3875worker2 # [ 34.406603] systemd[1]: Starting scheduler-token.service...3876worker2 # [ 34.507765] systemd[1]: scheduler-token.service: Deactivated successfully.3877worker2 # [ 34.510219] systemd[1]: Finished scheduler-token.service.3878worker2 # [ 34.514573] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 105ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3879lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3880??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.3881 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1283882lb: must succeed: curl -sf localhost:9901/clusters3883worker1 # [ 34.512211] nix-grpc-daemon[777]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500513884worker1 # [ 34.518959] nix-grpc-daemon[777]: ts=2026-09-20T12:47:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3885worker2 # [ 34.566214] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500513886worker2 # [ 34.568569] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3887lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.13 seconds)3888lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.14 seconds)3889subtest: one node schedules and both builders hold a WorkerSession on it3890lb: must succeed: curl -sf localhost:9901/clusters3891lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3892worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3893worker1 # [ 34.719582] nix-grpc-daemon[777]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500513894worker2 # [ 34.758989] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:545363895worker2 # [ 34.762022] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03896worker2 # [ 34.769521] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=info event=scheduler_connected addr=lb:500513897worker2 # [ 34.782527] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=info event=oidc_jwks provider=mock keys=13898worker2 # [ 34.784414] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:545363899worker2 # [ 34.787012] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03900worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3901lb: must succeed: curl -sf localhost:9901/clusters3902lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)3903lb: must succeed: curl -sf localhost:9901/clusters3904lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3905worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3906worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)3907(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.32 seconds)3908subtest: balancer auth: client cert, bearer token, nothing, unknown CN3909worker2 # [ 35.357918] nix-grpc-daemon[779]: ts=2026-09-20T12:47:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545403910worker2 # [ 35.381956] systemd[1]: Started Nix Daemon.3911worker2 # [ 35.692714] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)3912worker2 # [ 35.743424] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)3913worker2 # [ 35.769663] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 127 ms (attempt 2/5)3914worker2 # [ 35.899737] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 277 ms (attempt 3/5)3915worker2 # [ 36.180870] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 313 ms (attempt 4/5)3916worker2 # [ 36.506902] nix-grpc-daemon[779]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3917worker1 # [ 36.525730] nix-grpc-daemon[777]: ts=2026-09-20T12:47:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537863918worker1 # [ 36.547605] systemd[1]: Started Nix Daemon.3919worker1 # [ 36.945399] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3920worker1 # [ 36.979707] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)3921worker1 # [ 37.031842] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 2/5)3922worker1 # [ 37.190221] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 3/5)3923worker1 # [ 37.232364] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 260 ms (attempt 4/5)3924worker1 # [ 37.494645] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3925worker1 # [ 37.505694] nix-grpc-daemon[777]: ts=2026-09-20T12:47:08Z level=debug event=path_info_miss path=mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test error="error: path '/nix/store/mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test' is required, but there is no substituter that can build it"3926worker2 # [ 37.550521] nix-grpc-daemon[779]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:545403927worker2 # [ 37.570484] nix-grpc-daemon[779]: ts=2026-09-20T12:47:08Z level=debug event=path_info_miss path=mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test error="error: path '/nix/store/mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test' is required, but there is no substituter that can build it"3928client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3929client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.12 seconds)3930client: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3931client: (finished: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.06 seconds)3932worker2 # [ 38.098692] nix-grpc-daemon[779]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:545403933worker1 # [ 38.107265] nix-grpc-daemon[777]: ts=2026-09-20T12:47:08Z level=info event=oidc_jwks provider=mock keys=13934worker1 # [ 38.113454] nix-grpc-daemon[777]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:537863935worker1 # [ 38.123980] nix-grpc-daemon[777]: ts=2026-09-20T12:47:08Z level=debug event=path_info_miss path=mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test error="error: path '/nix/store/mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test' is required, but there is no substituter that can build it"3936worker2 # [ 38.164988] nix-grpc-daemon[779]: ts=2026-09-20T12:47:08Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:545403937worker2 # [ 38.182370] nix-grpc-daemon[779]: ts=2026-09-20T12:47:08Z level=debug event=path_info_miss path=mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test error="error: path '/nix/store/mxjjcdp3piiw7m6ki3ycv2085q5nmrdz-nixos-system-client-test' is required, but there is no substituter that can build it"3938client: must succeed: rm -r /var/lib/nix-grpc-store3939client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.06 seconds)3940worker1 # [ 40.554331] nix-grpc-daemon[777]: ts=2026-09-20T12:47:11Z level=info event=denied method=QueryMissing cn=- role=none3941worker2 # [ 40.901434] nix-grpc-daemon[779]: ts=2026-09-20T12:47:11Z level=info event=denied method=QueryMissing cn=stranger role=none3942(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 5.94 seconds)3943subtest: DAG build is scheduled across workers and lands in the cache3944client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23945lb # [ 41.031951] envoy[913]: conn peer=[2001:db8:1::1]:54642 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=22633946lb # [ 41.033109] envoy[913]: conn peer=[2001:db8:1::1]:54652 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=1033947lb # [ 41.033360] envoy[913]: conn peer=[2001:db8:1::1]:54668 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=199 ms=253948lb # [ 41.033561] envoy[913]: conn peer=[2001:db8:1::1]:49076 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=133949worker2 # [ 41.192546] nix-grpc-daemon[779]: ts=2026-09-20T12:47:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545403950client # don't know how to build these paths:3951client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3952worker2 # [ 41.209514] nix-grpc-daemon[779]: ts=2026-09-20T12:47:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545403953worker2 # [ 41.270675] nix-grpc-daemon[779]: ts=2026-09-20T12:47:11Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03954worker2 # [ 41.277563] nix-grpc-daemon[779]: ts=2026-09-20T12:47:11Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=13955worker1 # [ 41.293911] nix-grpc-daemon[777]: ts=2026-09-20T12:47:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537863956worker1 # [ 41.326958] nix-grpc-daemon[777]: ts=2026-09-20T12:47:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537863957client # copying 1 paths to 'lb:50051'...3958worker1 # [ 41.350233] nix-grpc-daemon[777]: ts=2026-09-20T12:47:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537863959client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3960worker1 # [ 41.371356] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3961worker1 # [ 41.396127] nix-grpc-daemon[777]: ts=2026-09-20T12:47:11Z level=info event=niks3_push_started pid=8463962worker1 # [ 41.729507] nix-grpc-daemon[846]: time=2026-09-20T12:47:12.249Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""3963worker1 # [ 41.951297] nix-daemon[835]: accepted connection from pid 859, user nix-grpc-daemon (trusted)3964worker1 # [ 41.979944] nix-daemon[835]: reaped child process 862, status = succeeded3965worker1 # [ 42.198050] nix-daemon[835]: accepted connection from pid 864, user nix-grpc-daemon (trusted)3966worker1 # [ 42.221380] nix-daemon[835]: reaped child process 867, status = succeeded3967lb # [ 42.225718] niks3-server[1051]: 2026/09/20 12:47:12 INFO Received uploads request method=POST path=/api/pending_closures3968worker1 # [ 42.277354] nix-grpc-daemon[846]: time=2026-09-20T12:47:12.797Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3969worker1 # [ 42.279305] nix-grpc-daemon[846]: time=2026-09-20T12:47:12.798Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3970lb # [ 42.346079] niks3-server[1051]: 2026/09/20 12:47:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3971lb # [ 42.359864] niks3-server[1051]: 2026/09/20 12:47:12 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3972lb # [ 42.402510] niks3-server[1051]: 2026/09/20 12:47:12 INFO Signed narinfos id=1 count=13973worker1 # [ 42.420296] nix-grpc-daemon[846]: time=2026-09-20T12:47:12.941Z level=INFO msg="Uploading 1 narinfos"3974lb # [ 42.426921] niks3-server[1051]: 2026/09/20 12:47:12 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3975lb # [ 42.437842] niks3-server[1051]: 2026/09/20 12:47:12 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3976lb # [ 42.449506] niks3-server[1051]: 2026/09/20 12:47:12 INFO Completed upload id=13977worker1 # [ 42.462521] nix-grpc-daemon[846]: time=2026-09-20T12:47:12.984Z level=INFO msg="Upload complete. (723ms)"3978lb # [ 42.453449] niks3-server[1051]: 2026/09/20 12:47:12 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3979worker1 # [ 42.478697] nix-grpc-daemon[777]: ts=2026-09-20T12:47:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 paths=1 nar_bytes_in=5043980worker1 # [ 42.501290] nix-grpc-daemon[777]: ts=2026-09-20T12:47:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537863981worker1 # [ 42.513419] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3982worker1 # [ 42.523012] nix-daemon[835]: reaped child process 844, status = succeeded3983client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3984worker1 # [ 42.538988] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3985client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3986client # farm-a> LOG-farm-a-t13987worker1 # [ 43.369423] nix-daemon[835]: reaped child process 871, status = succeeded3988worker1 # [ 43.568558] nix-daemon[835]: accepted connection from pid 887, user nix-grpc-daemon (trusted)3989worker1 # [ 43.591214] nix-daemon[835]: reaped child process 890, status = succeeded3990lb # [ 43.593731] niks3-server[1051]: 2026/09/20 12:47:14 INFO Received uploads request method=POST path=/api/pending_closures3991worker1 # [ 43.624894] nix-grpc-daemon[846]: time=2026-09-20T12:47:14.144Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3992worker1 # [ 43.626875] nix-grpc-daemon[846]: time=2026-09-20T12:47:14.144Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3993lb # [ 43.662272] niks3-server[1051]: 2026/09/20 12:47:14 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3994lb # [ 43.669775] niks3-server[1051]: 2026/09/20 12:47:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3995lb # [ 43.671915] niks3-server[1051]: 2026/09/20 12:47:14 INFO Signed narinfos id=2 count=13996worker1 # [ 43.683516] nix-grpc-daemon[846]: time=2026-09-20T12:47:14.204Z level=INFO msg="Uploading 1 narinfos"3997lb # [ 43.681769] niks3-server[1051]: 2026/09/20 12:47:14 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3998lb # [ 43.697722] niks3-server[1051]: 2026/09/20 12:47:14 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3999lb # [ 43.704000] niks3-server[1051]: 2026/09/20 12:47:14 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4000worker1 # [ 43.714717] nix-grpc-daemon[846]: time=2026-09-20T12:47:14.236Z level=INFO msg="Upload complete. (358ms)"4001lb # [ 43.706116] niks3-server[1051]: 2026/09/20 12:47:14 INFO Completed upload id=24002lb # [ 43.709100] niks3-server[1051]: 2026/09/20 12:47:14 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4003worker1 # [ 43.729965] nix-grpc-daemon[777]: ts=2026-09-20T12:47:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14004worker1 # [ 43.748269] nix-daemon[835]: reaped child process 869, status = succeeded4005worker2 # [ 43.781408] nix-grpc-daemon[779]: ts=2026-09-20T12:47:14Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04006worker2 # [ 43.788088] nix-grpc-daemon[779]: ts=2026-09-20T12:47:14Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14007worker1 # [ 43.804265] nix-grpc-daemon[777]: ts=2026-09-20T12:47:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864008worker1 # [ 43.830835] nix-grpc-daemon[777]: ts=2026-09-20T12:47:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864009worker1 # [ 43.856410] nix-grpc-daemon[846]: time=2026-09-20T12:47:14.378Z level=INFO msg="All 1 paths already cached"4010client # copying 1 paths to 'lb:50051'...4011worker1 # [ 43.873381] nix-grpc-daemon[777]: ts=2026-09-20T12:47:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537864012client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4013worker1 # [ 43.898987] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4014worker1 # [ 44.175746] nix-daemon[835]: accepted connection from pid 894, user nix-grpc-daemon (trusted)4015worker1 # [ 44.201065] nix-daemon[835]: reaped child process 897, status = succeeded4016worker1 # [ 44.419678] nix-daemon[835]: accepted connection from pid 899, user nix-grpc-daemon (trusted)4017worker1 # [ 44.443365] nix-daemon[835]: reaped child process 902, status = succeeded4018lb # [ 44.444162] niks3-server[1051]: 2026/09/20 12:47:14 INFO Received uploads request method=POST path=/api/pending_closures4019worker1 # [ 44.472362] nix-grpc-daemon[846]: time=2026-09-20T12:47:14.993Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4020worker1 # [ 44.475462] nix-grpc-daemon[846]: time=2026-09-20T12:47:14.993Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4021lb # [ 44.498367] niks3-server[1051]: 2026/09/20 12:47:15 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4022lb # [ 44.512465] niks3-server[1051]: 2026/09/20 12:47:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4023worker1 # [ 44.532007] nix-grpc-daemon[846]: time=2026-09-20T12:47:15.053Z level=INFO msg="Uploading 1 narinfos"4024lb # [ 44.523579] niks3-server[1051]: 2026/09/20 12:47:15 INFO Signed narinfos id=3 count=14025lb # [ 44.532206] niks3-server[1051]: 2026/09/20 12:47:15 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4026lb # [ 44.545585] niks3-server[1051]: 2026/09/20 12:47:15 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4027lb # [ 44.563111] niks3-server[1051]: 2026/09/20 12:47:15 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4028lb # [ 44.572305] niks3-server[1051]: 2026/09/20 12:47:15 INFO Completed upload id=34029worker1 # [ 44.585224] nix-grpc-daemon[846]: time=2026-09-20T12:47:15.106Z level=INFO msg="Upload complete. (654ms)"4030worker1 # [ 44.597747] nix-grpc-daemon[777]: ts=2026-09-20T12:47:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_in=6244031worker1 # [ 44.612778] nix-grpc-daemon[777]: ts=2026-09-20T12:47:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864032worker1 # [ 44.632748] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4033worker1 # [ 44.641725] nix-daemon[835]: reaped child process 892, status = succeeded4034client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4035client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4036worker1 # [ 44.656996] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4037client # farm-b> LOG-farm-b-t14038worker1 # [ 45.887727] nix-daemon[835]: reaped child process 906, status = succeeded4039worker1 # [ 46.072399] nix-daemon[835]: accepted connection from pid 921, user nix-grpc-daemon (trusted)4040worker1 # [ 46.096433] nix-daemon[835]: reaped child process 924, status = succeeded4041lb # [ 46.099358] niks3-server[1051]: 2026/09/20 12:47:16 INFO Received uploads request method=POST path=/api/pending_closures4042worker1 # [ 46.130102] nix-grpc-daemon[846]: time=2026-09-20T12:47:16.650Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4043worker1 # [ 46.131937] nix-grpc-daemon[846]: time=2026-09-20T12:47:16.650Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4044lb # [ 46.152526] niks3-server[1051]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4045lb # [ 46.162110] niks3-server[1051]: 2026/09/20 12:47:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4046worker1 # [ 46.178610] nix-grpc-daemon[846]: time=2026-09-20T12:47:16.699Z level=INFO msg="Uploading 1 narinfos"4047lb # [ 46.169827] niks3-server[1051]: 2026/09/20 12:47:16 INFO Signed narinfos id=4 count=14048lb # [ 46.174258] niks3-server[1051]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4049lb # [ 46.181667] niks3-server[1051]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4050lb # [ 46.187417] niks3-server[1051]: 2026/09/20 12:47:16 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4051lb # [ 46.193183] niks3-server[1051]: 2026/09/20 12:47:16 INFO Completed upload id=44052worker1 # [ 46.204159] nix-grpc-daemon[846]: time=2026-09-20T12:47:16.725Z level=INFO msg="Upload complete. (330ms)"4053lb # [ 46.194833] niks3-server[1051]: 2026/09/20 12:47:16 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4054worker1 # [ 46.212800] nix-grpc-daemon[777]: ts=2026-09-20T12:47:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14055worker1 # [ 46.221587] nix-daemon[835]: reaped child process 904, status = succeeded4056worker2 # [ 46.257854] nix-grpc-daemon[779]: ts=2026-09-20T12:47:16Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04057worker2 # [ 46.258519] nix-grpc-daemon[779]: ts=2026-09-20T12:47:16Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14058worker1 # [ 46.270123] nix-grpc-daemon[777]: ts=2026-09-20T12:47:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864059worker1 # [ 46.288343] nix-grpc-daemon[777]: ts=2026-09-20T12:47:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864060worker1 # [ 46.305739] nix-grpc-daemon[846]: time=2026-09-20T12:47:16.827Z level=INFO msg="All 2 paths already cached"4061client # copying 1 paths to 'lb:50051'...4062worker1 # [ 46.319262] nix-grpc-daemon[777]: ts=2026-09-20T12:47:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537864063client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4064worker1 # [ 46.345244] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4065worker1 # [ 46.600297] nix-daemon[835]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4066worker1 # [ 46.637857] nix-daemon[835]: reaped child process 931, status = succeeded4067worker1 # [ 46.868965] nix-daemon[835]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4068worker1 # [ 46.892134] nix-daemon[835]: reaped child process 937, status = succeeded4069lb # [ 46.893317] niks3-server[1051]: 2026/09/20 12:47:17 INFO Received uploads request method=POST path=/api/pending_closures4070worker1 # [ 46.918597] nix-grpc-daemon[846]: time=2026-09-20T12:47:17.440Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4071worker1 # [ 46.921154] nix-grpc-daemon[846]: time=2026-09-20T12:47:17.442Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4072lb # [ 46.922002] niks3-server[1051]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4073lb # [ 46.928968] niks3-server[1051]: 2026/09/20 12:47:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4074worker1 # [ 46.943961] nix-grpc-daemon[846]: time=2026-09-20T12:47:17.465Z level=INFO msg="Uploading 1 narinfos"4075lb # [ 46.934471] niks3-server[1051]: 2026/09/20 12:47:17 INFO Signed narinfos id=5 count=14076lb # [ 46.935993] niks3-server[1051]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4077lb # [ 46.944137] niks3-server[1051]: 2026/09/20 12:47:17 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4078worker1 # [ 46.960064] nix-grpc-daemon[846]: time=2026-09-20T12:47:17.482Z level=INFO msg="Upload complete. (593ms)"4079lb # [ 46.949183] niks3-server[1051]: 2026/09/20 12:47:17 INFO Completed upload id=54080lb # [ 46.951970] niks3-server[1051]: 2026/09/20 12:47:17 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4081worker1 # [ 46.967783] nix-grpc-daemon[777]: ts=2026-09-20T12:47:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_in=7524082worker1 # [ 46.979189] nix-grpc-daemon[777]: ts=2026-09-20T12:47:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864083worker1 # [ 46.984527] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4084worker1 # [ 46.989576] nix-daemon[835]: reaped child process 926, status = succeeded4085client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4086worker1 # [ 47.004328] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4087client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4088client # farm-top> LOG-farm-top-t14089worker1 # [ 48.634458] nix-daemon[835]: reaped child process 941, status = succeeded4090worker1 # [ 48.822345] nix-daemon[835]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4091worker1 # [ 48.849626] nix-daemon[835]: reaped child process 959, status = succeeded4092lb # [ 48.847244] niks3-server[1051]: 2026/09/20 12:47:19 INFO Received uploads request method=POST path=/api/pending_closures4093worker1 # [ 48.872903] nix-grpc-daemon[846]: time=2026-09-20T12:47:19.394Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4094worker1 # [ 48.876812] nix-grpc-daemon[846]: time=2026-09-20T12:47:19.396Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4095lb # [ 48.879330] niks3-server[1051]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4096lb # [ 48.887498] niks3-server[1051]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4097lb # [ 48.897509] niks3-server[1051]: 2026/09/20 12:47:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4098worker1 # [ 48.912894] nix-grpc-daemon[846]: time=2026-09-20T12:47:19.434Z level=INFO msg="Uploading 1 narinfos"4099lb # [ 48.901787] niks3-server[1051]: 2026/09/20 12:47:19 INFO Signed narinfos id=6 count=14100lb # [ 48.904996] niks3-server[1051]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4101lb # [ 48.911761] niks3-server[1051]: 2026/09/20 12:47:19 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4102lb # [ 48.917110] niks3-server[1051]: 2026/09/20 12:47:19 INFO Completed upload id=64103worker1 # [ 48.929464] nix-grpc-daemon[846]: time=2026-09-20T12:47:19.451Z level=INFO msg="Upload complete. (314ms)"4104lb # [ 48.919040] niks3-server[1051]: 2026/09/20 12:47:19 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4105worker1 # [ 48.936726] nix-grpc-daemon[777]: ts=2026-09-20T12:47:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14106worker1 # [ 48.945780] nix-daemon[835]: reaped child process 939, status = succeeded4107client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 8.08 seconds)4108client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4109client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.23 seconds)4110client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4111client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.04 seconds)4112client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4113client # copying 3 paths...4114client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4115client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4116client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4117client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4118(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.76 seconds)4119subtest: repeat is answered Cached by the scheduler without building4120worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4121worker1 # finding garbage collector roots...4122worker1 # deleting specified paths...4123worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4124worker1 # deleting unused links...4125worker1 # note: hard linking is currently saving 0.0 KiB4126worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4127worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4128worker2 # finding garbage collector roots...4129worker2 # deleting specified paths...4130worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4131worker2 # deleting unused links...4132worker2 # note: hard linking is currently saving 0.0 KiB4133worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.32 seconds)4134client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24135worker2 # [ 50.550444] nix-grpc-daemon[779]: ts=2026-09-20T12:47:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545404136client # don't know how to build these paths:4137client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4138worker2 # [ 50.563326] nix-grpc-daemon[779]: ts=2026-09-20T12:47:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545404139worker2 # [ 50.601355] nix-grpc-daemon[779]: ts=2026-09-20T12:47:21Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14140client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.32 seconds)4141client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4142client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.23 seconds)4143lb: must succeed: curl -sf localhost:9901/clusters4144lb # [ 51.030637] envoy[913]: conn peer=[2001:db8:1::1]:49090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9566 tx=3881 ms=78114145lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4146lb: must succeed: curl -sf localhost:9901/clusters4147lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4148worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4149worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4150(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.54 seconds)4151subtest: inputs already in the cache are not sent to the scheduler4152lb: must succeed: curl -sf localhost:9901/clusters4153lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4154lb: must succeed: curl -sf localhost:9901/clusters4155lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4156worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4157worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4158client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24159lb # [ 51.034867] envoy[913]: conn peer=[2001:db8:1::1]:49098 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=481 ms=784160client # don't know how to build these paths:4161worker1 # [ 51.765166] nix-grpc-daemon[777]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537864162client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4163worker1 # [ 51.774905] nix-grpc-daemon[777]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864164worker1 # [ 51.789717] nix-grpc-daemon[846]: time=2026-09-20T12:47:22.310Z level=INFO msg="All 2 paths already cached"4165worker2 # [ 51.843640] nix-grpc-daemon[779]: ts=2026-09-20T12:47:22Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04166worker2 # [ 51.843939] nix-grpc-daemon[779]: ts=2026-09-20T12:47:22Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14167worker1 # [ 51.859168] nix-grpc-daemon[777]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864168worker1 # [ 51.878425] nix-grpc-daemon[777]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864169worker1 # [ 51.894697] nix-grpc-daemon[846]: time=2026-09-20T12:47:22.416Z level=INFO msg="All 2 paths already cached"4170client # copying 1 paths to 'lb:50051'...4171worker1 # [ 51.903692] nix-grpc-daemon[777]: ts=2026-09-20T12:47:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537864172client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4173worker1 # [ 51.919253] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4174worker1 # [ 52.134017] nix-daemon[835]: accepted connection from pid 976, user nix-grpc-daemon (trusted)4175worker1 # [ 52.155103] nix-daemon[835]: reaped child process 979, status = succeeded4176worker1 # [ 52.343423] nix-daemon[835]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4177worker1 # [ 52.366789] nix-daemon[835]: reaped child process 984, status = succeeded4178lb # [ 52.370291] niks3-server[1051]: 2026/09/20 12:47:22 INFO Received uploads request method=POST path=/api/pending_closures4179worker1 # [ 52.391569] nix-grpc-daemon[846]: time=2026-09-20T12:47:22.913Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4180worker1 # [ 52.393402] nix-grpc-daemon[846]: time=2026-09-20T12:47:22.913Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4181lb # [ 52.400539] niks3-server[1051]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4182lb # [ 52.419494] niks3-server[1051]: 2026/09/20 12:47:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4183worker1 # [ 52.439472] nix-grpc-daemon[846]: time=2026-09-20T12:47:22.961Z level=INFO msg="Uploading 1 narinfos"4184lb # [ 52.434489] niks3-server[1051]: 2026/09/20 12:47:22 INFO Signed narinfos id=7 count=14185lb # [ 52.441143] niks3-server[1051]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4186lb # [ 52.450900] niks3-server[1051]: 2026/09/20 12:47:22 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4187worker1 # [ 52.471498] nix-grpc-daemon[846]: time=2026-09-20T12:47:22.993Z level=INFO msg="Upload complete. (534ms)"4188lb # [ 52.459776] niks3-server[1051]: 2026/09/20 12:47:22 INFO Completed upload id=74189worker1 # [ 52.476352] nix-grpc-daemon[777]: ts=2026-09-20T12:47:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_in=7524190lb # [ 52.464137] niks3-server[1051]: 2026/09/20 12:47:22 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4191worker1 # [ 52.482247] nix-grpc-daemon[777]: ts=2026-09-20T12:47:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864192worker1 # [ 52.491333] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4193worker1 # [ 52.494912] nix-daemon[835]: reaped child process 974, status = succeeded4194client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4195worker1 # [ 52.507394] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4196client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4197worker1 # [ 53.331376] nix-daemon[835]: reaped child process 988, status = succeeded4198client # farm-top> LOG-farm-top-t1b4199worker1 # [ 53.482729] nix-daemon[835]: accepted connection from pid 1003, user nix-grpc-daemon (trusted)4200worker1 # [ 53.507141] nix-daemon[835]: reaped child process 1006, status = succeeded4201lb # [ 53.507361] niks3-server[1051]: 2026/09/20 12:47:24 INFO Received uploads request method=POST path=/api/pending_closures4202worker1 # [ 53.633214] nix-grpc-daemon[846]: time=2026-09-20T12:47:24.152Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4203worker1 # [ 53.645800] nix-grpc-daemon[846]: time=2026-09-20T12:47:24.152Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4204lb # [ 53.664859] niks3-server[1051]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4205worker1 # [ 53.688645] nix-grpc-daemon[846]: time=2026-09-20T12:47:24.210Z level=INFO msg="Uploading 1 narinfos"4206lb # [ 53.679722] niks3-server[1051]: 2026/09/20 12:47:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4207lb # [ 53.689683] niks3-server[1051]: 2026/09/20 12:47:24 INFO Signed narinfos id=8 count=14208lb # [ 53.696783] niks3-server[1051]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4209lb # [ 53.711043] niks3-server[1051]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4210lb # [ 53.718489] niks3-server[1051]: 2026/09/20 12:47:24 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4211lb # [ 53.725063] niks3-server[1051]: 2026/09/20 12:47:24 INFO Completed upload id=84212worker1 # [ 53.736106] nix-grpc-daemon[846]: time=2026-09-20T12:47:24.257Z level=INFO msg="Upload complete. (420ms)"4213lb # [ 53.734091] niks3-server[1051]: 2026/09/20 12:47:24 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4214worker1 # [ 53.747542] nix-grpc-daemon[777]: ts=2026-09-20T12:47:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14215worker1 # [ 53.764180] nix-daemon[835]: reaped child process 986, status = succeeded4216client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.38 seconds)4217client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4218client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.25 seconds)4219lb: must succeed: curl -sf localhost:9901/clusters4220lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4221lb: must succeed: curl -sf localhost:9901/clusters4222lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4223worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4224worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4225(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.06 seconds)4226subtest: two clients wanting the same drv build it once4227client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4228client # [ 54.373861] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4229client # Running as unit: dedup1.service; invocation ID: 5785f0228dec41649651beae59d2697e4230client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4231client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4232worker2 # [ 54.643500] nix-grpc-daemon[779]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:545404233client # [ 54.652691] nix[882]: don't know how to build these paths:4234client # [ 54.654004] nix[882]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4235worker2 # [ 54.670138] nix-grpc-daemon[779]: ts=2026-09-20T12:47:25Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04236worker2 # [ 54.679305] nix-grpc-daemon[779]: ts=2026-09-20T12:47:25Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14237worker2 # [ 54.719488] nix-grpc-daemon[779]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545404238worker2 # [ 54.744749] nix-grpc-daemon[779]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545404239client # [ 54.760306] nix[882]: copying 1 paths to 'lb:50051'...4240worker2 # [ 54.761975] nix-grpc-daemon[779]: ts=2026-09-20T12:47:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545404241client # [ 54.763690] nix[882]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4242worker2 # [ 54.778962] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4243worker2 # [ 54.798829] nix-grpc-daemon[779]: ts=2026-09-20T12:47:25Z level=info event=niks3_push_started pid=9004244worker2 # [ 55.048292] nix-grpc-daemon[900]: time=2026-09-20T12:47:25.560Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4245worker2 # [ 55.251638] nix-daemon[848]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4246worker2 # [ 55.275619] nix-daemon[848]: reaped child process 916, status = succeeded4247worker2 # [ 55.469745] nix-daemon[848]: accepted connection from pid 918, user nix-grpc-daemon (trusted)4248worker2 # [ 55.491254] nix-daemon[848]: reaped child process 921, status = succeeded4249lb # [ 55.457872] niks3-server[1051]: 2026/09/20 12:47:25 INFO Received uploads request method=POST path=/api/pending_closures4250client # [ 55.502647] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4251worker2 # [ 55.523034] nix-grpc-daemon[900]: time=2026-09-20T12:47:26.036Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4252worker2 # [ 55.526759] nix-grpc-daemon[900]: time=2026-09-20T12:47:26.038Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4253client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.15 seconds)4254lb # [ 55.507869] niks3-server[1051]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4255lb # [ 55.523090] niks3-server[1051]: 2026/09/20 12:47:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4256worker2 # [ 55.577182] nix-grpc-daemon[900]: time=2026-09-20T12:47:26.090Z level=INFO msg="Uploading 1 narinfos"4257lb # [ 55.534017] niks3-server[1051]: 2026/09/20 12:47:26 INFO Signed narinfos id=9 count=14258lb # [ 55.545297] niks3-server[1051]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4259lb # [ 55.559104] niks3-server[1051]: 2026/09/20 12:47:26 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4260lb # [ 55.568837] niks3-server[1051]: 2026/09/20 12:47:26 INFO Completed upload id=94261worker2 # [ 55.614864] nix-grpc-daemon[900]: time=2026-09-20T12:47:26.127Z level=INFO msg="Upload complete. (556ms)"4262lb # [ 55.573337] niks3-server[1051]: 2026/09/20 12:47:26 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4263worker2 # [ 55.620373] nix-grpc-daemon[779]: ts=2026-09-20T12:47:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54540 duration_s=0 paths=1 nar_bytes_in=4324264worker2 # [ 55.632176] nix-grpc-daemon[779]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545404265worker2 # [ 55.644250] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4266worker2 # [ 55.646626] nix-daemon[848]: reaped child process 898, status = succeeded4267client # Running as unit: dedup2.service; invocation ID: b163d482d12e472883a18abcb6c8c95f4268client # [ 55.656987] nix[882]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4269worker2 # [ 55.663272] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4270client # [ 55.854380] nix[897]: don't know how to build these paths:4271worker1 # [ 55.820743] nix-grpc-daemon[777]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537864272client # [ 55.855474] nix[897]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4273worker2 # [ 55.867670] nix-grpc-daemon[779]: ts=2026-09-20T12:47:26Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04274client # [ 55.878402] nix[897]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4275worker2 # [ 55.878321] nix-grpc-daemon[779]: ts=2026-09-20T12:47:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545404276client: must succeed: sleep 24277client: (finished: must succeed: sleep 2, in 2.06 seconds)4278??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4279 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394280client: waiting for success: ! systemctl is-active dedup1 dedup24281??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4282 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394283worker2 # [ 60.289871] nix-daemon[848]: reaped child process 929, status = succeeded4284worker2 # [ 60.463956] nix-daemon[848]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4285worker2 # [ 60.484113] nix-daemon[848]: reaped child process 963, status = succeeded4286lb # [ 60.453498] niks3-server[1051]: 2026/09/20 12:47:30 INFO Received uploads request method=POST path=/api/pending_closures4287worker2 # [ 60.508490] nix-grpc-daemon[900]: time=2026-09-20T12:47:31.021Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4288worker2 # [ 60.510064] nix-grpc-daemon[900]: time=2026-09-20T12:47:31.021Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4289lb # [ 60.501100] niks3-server[1051]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4290lb # [ 60.516407] niks3-server[1051]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4291lb # [ 60.526016] niks3-server[1051]: 2026/09/20 12:47:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4292worker2 # [ 60.576253] nix-grpc-daemon[900]: time=2026-09-20T12:47:31.089Z level=INFO msg="Uploading 1 narinfos"4293lb # [ 60.531587] niks3-server[1051]: 2026/09/20 12:47:31 INFO Signed narinfos id=10 count=14294lb # [ 60.534944] niks3-server[1051]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4295lb # [ 60.548396] niks3-server[1051]: 2026/09/20 12:47:31 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4296lb # [ 60.557425] niks3-server[1051]: 2026/09/20 12:47:31 INFO Completed upload id=104297worker2 # [ 60.603000] nix-grpc-daemon[900]: time=2026-09-20T12:47:31.116Z level=INFO msg="Upload complete. (326ms)"4298lb # [ 60.561196] niks3-server[1051]: 2026/09/20 12:47:31 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4299worker2 # [ 60.616319] nix-grpc-daemon[779]: ts=2026-09-20T12:47:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54540 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14300worker2 # [ 60.632516] nix-grpc-daemon[779]: ts=2026-09-20T12:47:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54540 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14301worker2 # [ 60.639032] nix-daemon[848]: reaped child process 927, status = succeeded4302client # [ 60.662026] systemd[1]: dedup1.service: Deactivated successfully.4303client # [ 60.674594] systemd[1]: dedup1.service: Consumed 170ms CPU time over 6.293s wall clock time, 37.1M memory peak, 8.3K incoming IP traffic, 8.7K outgoing IP traffic.4304client # [ 60.712289] systemd[1]: dedup2.service: Deactivated successfully.4305client # [ 60.714775] systemd[1]: dedup2.service: Consumed 194ms CPU time over 5.212s wall clock time, 33M memory peak, 7.5K incoming IP traffic, 6.4K outgoing IP traffic.4306lb # [ 61.032732] envoy[913]: conn peer=[2001:db8:1::1]:57342 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4172 tx=1701 ms=20874307lb # [ 61.035265] envoy[913]: conn peer=[2001:db8:1::1]:57348 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1357 ms=59904308lb # [ 61.037653] envoy[913]: conn peer=[2001:db8:1::1]:57362 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1082 ms=48354309client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.15 seconds)4310client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4311client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4312worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4313worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4314worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4315worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4316worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4317worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4318worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4319worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4320(finished: subtest: two clients wanting the same drv build it once, in 7.49 seconds)4321subtest: build hook: nix-daemon with builders = grpc://lb4322client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4323client # [ 62.027869] systemd[1]: Started Nix Daemon.4324client # [ 62.231925] nix-daemon[938]: accepted connection from pid 935, user root (trusted)4325worker1 # [ 62.417350] nix-grpc-daemon[777]: ts=2026-09-20T12:47:32Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:537864326worker1 # [ 62.424137] nix-grpc-daemon[777]: ts=2026-09-20T12:47:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864327worker2 # [ 62.481642] nix-grpc-daemon[779]: ts=2026-09-20T12:47:32Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04328worker2 # [ 62.487304] nix-grpc-daemon[779]: ts=2026-09-20T12:47:32Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14329worker1 # [ 62.494813] nix-grpc-daemon[777]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864330worker1 # [ 62.513795] nix-grpc-daemon[777]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864331worker1 # [ 62.533683] nix-grpc-daemon[777]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537864332worker1 # [ 62.549970] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4333worker1 # [ 62.768203] nix-daemon[835]: accepted connection from pid 1035, user nix-grpc-daemon (trusted)4334worker1 # [ 62.792332] nix-daemon[835]: reaped child process 1038, status = succeeded4335worker1 # [ 62.966480] nix-daemon[835]: accepted connection from pid 1040, user nix-grpc-daemon (trusted)4336worker1 # [ 62.986704] nix-daemon[835]: reaped child process 1043, status = succeeded4337lb # [ 62.985649] niks3-server[1051]: 2026/09/20 12:47:33 INFO Received uploads request method=POST path=/api/pending_closures4338worker1 # [ 63.007837] nix-grpc-daemon[846]: time=2026-09-20T12:47:33.529Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4339worker1 # [ 63.009737] nix-grpc-daemon[846]: time=2026-09-20T12:47:33.529Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4340lb # [ 63.015279] niks3-server[1051]: 2026/09/20 12:47:33 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4341worker1 # [ 63.037071] nix-grpc-daemon[846]: time=2026-09-20T12:47:33.558Z level=INFO msg="Uploading 1 narinfos"4342lb # [ 63.024507] niks3-server[1051]: 2026/09/20 12:47:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4343lb # [ 63.032821] niks3-server[1051]: 2026/09/20 12:47:33 INFO Signed narinfos id=11 count=14344lb # [ 63.038162] niks3-server[1051]: 2026/09/20 12:47:33 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4345lb # [ 63.046124] niks3-server[1051]: 2026/09/20 12:47:33 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4346worker1 # [ 63.065097] nix-grpc-daemon[846]: time=2026-09-20T12:47:33.587Z level=INFO msg="Upload complete. (497ms)"4347lb # [ 63.053393] niks3-server[1051]: 2026/09/20 12:47:33 INFO Completed upload id=114348lb # [ 63.055832] niks3-server[1051]: 2026/09/20 12:47:33 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4349worker1 # [ 63.072199] nix-grpc-daemon[777]: ts=2026-09-20T12:47:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_in=5124350worker1 # [ 63.080800] nix-grpc-daemon[777]: ts=2026-09-20T12:47:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864351worker1 # [ 63.094489] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4352worker1 # [ 63.097568] nix-daemon[835]: reaped child process 1033, status = succeeded4353worker1 # [ 63.111289] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4354worker1 # [ 63.955612] nix-daemon[835]: reaped child process 1047, status = succeeded4355worker1 # [ 64.150618] nix-daemon[835]: accepted connection from pid 1062, user nix-grpc-daemon (trusted)4356worker1 # [ 64.174635] nix-daemon[835]: reaped child process 1065, status = succeeded4357lb # [ 64.174764] niks3-server[1051]: 2026/09/20 12:47:34 INFO Received uploads request method=POST path=/api/pending_closures4358worker1 # [ 64.195885] nix-grpc-daemon[846]: time=2026-09-20T12:47:34.717Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4359worker1 # [ 64.201696] nix-grpc-daemon[846]: time=2026-09-20T12:47:34.719Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4360lb # [ 64.212779] niks3-server[1051]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4361lb # [ 64.230168] niks3-server[1051]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4362lb # [ 64.240776] niks3-server[1051]: 2026/09/20 12:47:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4363worker1 # [ 64.259033] nix-grpc-daemon[846]: time=2026-09-20T12:47:34.780Z level=INFO msg="Uploading 1 narinfos"4364lb # [ 64.249818] niks3-server[1051]: 2026/09/20 12:47:34 INFO Signed narinfos id=12 count=14365lb # [ 64.255277] niks3-server[1051]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4366lb # [ 64.267460] niks3-server[1051]: 2026/09/20 12:47:34 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4367worker1 # [ 64.284944] nix-grpc-daemon[846]: time=2026-09-20T12:47:34.806Z level=INFO msg="Upload complete. (363ms)"4368lb # [ 64.273163] niks3-server[1051]: 2026/09/20 12:47:34 INFO Completed upload id=124369lb # [ 64.277120] niks3-server[1051]: 2026/09/20 12:47:34 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4370worker1 # [ 64.292693] nix-grpc-daemon[777]: ts=2026-09-20T12:47:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14371worker1 # [ 64.302356] nix-daemon[835]: reaped child process 1045, status = succeeded4372worker2 # [ 64.334473] nix-grpc-daemon[779]: ts=2026-09-20T12:47:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:545404373worker2 # [ 64.352702] nix-grpc-daemon[779]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4374worker1 # [ 64.371231] nix-grpc-daemon[777]: ts=2026-09-20T12:47:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537864375worker1 # [ 64.380150] nix-grpc-daemon[777]: ts=2026-09-20T12:47:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_out=1284376worker1 # [ 64.623749] nix-grpc-daemon[777]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:537864377worker1 # [ 64.632079] nix-grpc-daemon[777]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864378worker1 # [ 64.639365] nix-grpc-daemon[846]: time=2026-09-20T12:47:35.159Z level=INFO msg="All 1 paths already cached"4379worker2 # [ 64.685637] nix-grpc-daemon[779]: ts=2026-09-20T12:47:35Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04380worker2 # [ 64.693113] nix-grpc-daemon[779]: ts=2026-09-20T12:47:35Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14381worker1 # [ 64.664452] nix-grpc-daemon[777]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864382worker1 # [ 64.680599] nix-grpc-daemon[777]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864383worker1 # [ 64.696383] nix-grpc-daemon[846]: time=2026-09-20T12:47:35.218Z level=INFO msg="All 1 paths already cached"4384worker1 # [ 64.706744] nix-grpc-daemon[777]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537864385worker1 # [ 64.721936] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4386worker1 # [ 64.943084] nix-daemon[835]: accepted connection from pid 1069, user nix-grpc-daemon (trusted)4387worker1 # [ 64.965621] nix-daemon[835]: reaped child process 1072, status = succeeded4388worker1 # [ 65.152644] nix-daemon[835]: accepted connection from pid 1074, user nix-grpc-daemon (trusted)4389worker1 # [ 65.172980] nix-daemon[835]: reaped child process 1077, status = succeeded4390lb # [ 65.171277] niks3-server[1051]: 2026/09/20 12:47:35 INFO Received uploads request method=POST path=/api/pending_closures4391worker1 # [ 65.240547] nix-grpc-daemon[846]: time=2026-09-20T12:47:35.761Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4392worker1 # [ 65.242592] nix-grpc-daemon[846]: time=2026-09-20T12:47:35.761Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4393lb # [ 65.249212] niks3-server[1051]: 2026/09/20 12:47:35 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4394lb # [ 65.260886] niks3-server[1051]: 2026/09/20 12:47:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4395lb # [ 65.268121] niks3-server[1051]: 2026/09/20 12:47:35 INFO Signed narinfos id=13 count=14396worker1 # [ 65.279634] nix-grpc-daemon[846]: time=2026-09-20T12:47:35.801Z level=INFO msg="Uploading 1 narinfos"4397lb # [ 65.275014] niks3-server[1051]: 2026/09/20 12:47:35 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4398lb # [ 65.289035] niks3-server[1051]: 2026/09/20 12:47:35 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4399lb # [ 65.296191] niks3-server[1051]: 2026/09/20 12:47:35 INFO Completed upload id=134400worker1 # [ 65.308119] nix-grpc-daemon[846]: time=2026-09-20T12:47:35.829Z level=INFO msg="Upload complete. (568ms)"4401lb # [ 65.299022] niks3-server[1051]: 2026/09/20 12:47:35 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4402worker1 # [ 65.321752] nix-grpc-daemon[777]: ts=2026-09-20T12:47:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_in=6244403worker1 # [ 65.336643] nix-grpc-daemon[777]: ts=2026-09-20T12:47:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864404worker1 # [ 65.347632] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4405worker1 # [ 65.355547] nix-daemon[835]: reaped child process 1067, status = succeeded4406worker1 # [ 65.373501] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4407worker1 # [ 66.628213] nix-daemon[835]: reaped child process 1081, status = succeeded4408worker1 # [ 66.813307] nix-daemon[835]: accepted connection from pid 1096, user nix-grpc-daemon (trusted)4409worker1 # [ 66.834615] nix-daemon[835]: reaped child process 1099, status = succeeded4410lb # [ 66.835022] niks3-server[1051]: 2026/09/20 12:47:37 INFO Received uploads request method=POST path=/api/pending_closures4411worker1 # [ 66.860073] nix-grpc-daemon[846]: time=2026-09-20T12:47:37.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4412worker1 # [ 66.861624] nix-grpc-daemon[846]: time=2026-09-20T12:47:37.378Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4413lb # [ 66.878837] niks3-server[1051]: 2026/09/20 12:47:37 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4414worker1 # [ 66.903170] nix-grpc-daemon[846]: time=2026-09-20T12:47:37.424Z level=INFO msg="Uploading 1 narinfos"4415lb # [ 66.892459] niks3-server[1051]: 2026/09/20 12:47:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4416lb # [ 66.902427] niks3-server[1051]: 2026/09/20 12:47:37 INFO Signed narinfos id=14 count=14417lb # [ 66.909702] niks3-server[1051]: 2026/09/20 12:47:37 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4418lb # [ 66.914761] niks3-server[1051]: 2026/09/20 12:47:37 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4419lb # [ 66.923165] niks3-server[1051]: 2026/09/20 12:47:37 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4420worker1 # [ 66.935487] nix-grpc-daemon[846]: time=2026-09-20T12:47:37.457Z level=INFO msg="Upload complete. (348ms)"4421lb # [ 66.926385] niks3-server[1051]: 2026/09/20 12:47:37 INFO Completed upload id=144422lb # [ 66.930218] niks3-server[1051]: 2026/09/20 12:47:37 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4423worker1 # [ 66.949239] nix-grpc-daemon[777]: ts=2026-09-20T12:47:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14424worker2 # [ 66.985907] nix-grpc-daemon[779]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:545404425worker1 # [ 66.964581] nix-daemon[835]: reaped child process 1079, status = succeeded4426worker2 # [ 67.000224] nix-grpc-daemon[779]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4427worker1 # [ 67.011592] nix-grpc-daemon[777]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537864428worker1 # [ 67.018780] nix-grpc-daemon[777]: ts=2026-09-20T12:47:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_out=1764429worker2 # [ 67.334003] nix-grpc-daemon[779]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:545404430worker1 # [ 67.305208] nix-grpc-daemon[777]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864431worker1 # [ 67.316407] nix-grpc-daemon[846]: time=2026-09-20T12:47:37.838Z level=INFO msg="All 2 paths already cached"4432worker2 # [ 67.370510] nix-grpc-daemon[779]: ts=2026-09-20T12:47:37Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04433worker2 # [ 67.376936] nix-grpc-daemon[779]: ts=2026-09-20T12:47:37Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14434worker1 # [ 67.348627] nix-grpc-daemon[777]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864435worker1 # [ 67.366343] nix-grpc-daemon[777]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864436worker1 # [ 67.383392] nix-grpc-daemon[846]: time=2026-09-20T12:47:37.905Z level=INFO msg="All 2 paths already cached"4437worker1 # [ 67.394321] nix-grpc-daemon[777]: ts=2026-09-20T12:47:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537864438worker1 # [ 67.409510] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4439worker1 # [ 67.625396] nix-daemon[835]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)4440worker1 # [ 67.651268] nix-daemon[835]: reaped child process 1106, status = succeeded4441worker1 # [ 67.840635] nix-daemon[835]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)4442worker1 # [ 67.861454] nix-daemon[835]: reaped child process 1111, status = succeeded4443lb # [ 67.860718] niks3-server[1051]: 2026/09/20 12:47:38 INFO Received uploads request method=POST path=/api/pending_closures4444worker1 # [ 68.008534] nix-grpc-daemon[846]: time=2026-09-20T12:47:38.528Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4445worker1 # [ 68.013120] nix-grpc-daemon[846]: time=2026-09-20T12:47:38.528Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4446lb # [ 68.020764] niks3-server[1051]: 2026/09/20 12:47:38 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4447lb # [ 68.031617] niks3-server[1051]: 2026/09/20 12:47:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4448worker1 # [ 68.051362] nix-grpc-daemon[846]: time=2026-09-20T12:47:38.572Z level=INFO msg="Uploading 1 narinfos"4449lb # [ 68.042511] niks3-server[1051]: 2026/09/20 12:47:38 INFO Signed narinfos id=15 count=14450lb # [ 68.050744] niks3-server[1051]: 2026/09/20 12:47:38 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4451lb # [ 68.062969] niks3-server[1051]: 2026/09/20 12:47:38 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4452lb # [ 68.072652] niks3-server[1051]: 2026/09/20 12:47:38 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4453lb # [ 68.077997] niks3-server[1051]: 2026/09/20 12:47:38 INFO Completed upload id=154454worker1 # [ 68.090208] nix-grpc-daemon[846]: time=2026-09-20T12:47:38.612Z level=INFO msg="Upload complete. (663ms)"4455worker1 # [ 68.102698] nix-grpc-daemon[777]: ts=2026-09-20T12:47:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_in=7524456worker1 # [ 68.120928] nix-grpc-daemon[777]: ts=2026-09-20T12:47:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864457worker1 # [ 68.131854] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4458worker1 # [ 68.144371] nix-daemon[835]: reaped child process 1101, status = succeeded4459worker1 # [ 68.158549] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4460worker1 # [ 69.605960] nix-daemon[835]: reaped child process 1115, status = succeeded4461worker1 # [ 69.782310] nix-daemon[835]: accepted connection from pid 1130, user nix-grpc-daemon (trusted)4462worker1 # [ 69.802899] nix-daemon[835]: reaped child process 1133, status = succeeded4463lb # [ 69.807623] niks3-server[1051]: 2026/09/20 12:47:40 INFO Received uploads request method=POST path=/api/pending_closures4464worker1 # [ 69.832521] nix-grpc-daemon[846]: time=2026-09-20T12:47:40.352Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4465worker1 # [ 69.834196] nix-grpc-daemon[846]: time=2026-09-20T12:47:40.352Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4466lb # [ 69.855876] niks3-server[1051]: 2026/09/20 12:47:40 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4467lb # [ 69.869438] niks3-server[1051]: 2026/09/20 12:47:40 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4468lb # [ 69.879370] niks3-server[1051]: 2026/09/20 12:47:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4469worker1 # [ 69.898257] nix-grpc-daemon[846]: time=2026-09-20T12:47:40.419Z level=INFO msg="Uploading 1 narinfos"4470lb # [ 69.885517] niks3-server[1051]: 2026/09/20 12:47:40 INFO Signed narinfos id=16 count=14471lb # [ 69.892232] niks3-server[1051]: 2026/09/20 12:47:40 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4472lb # [ 69.902263] niks3-server[1051]: 2026/09/20 12:47:40 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4473lb # [ 69.911878] niks3-server[1051]: 2026/09/20 12:47:40 INFO Completed upload id=164474worker1 # [ 69.924548] nix-grpc-daemon[846]: time=2026-09-20T12:47:40.446Z level=INFO msg="Upload complete. (351ms)"4475lb # [ 69.915256] niks3-server[1051]: 2026/09/20 12:47:40 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4476worker1 # [ 69.937316] nix-grpc-daemon[777]: ts=2026-09-20T12:47:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14477worker1 # [ 69.958609] nix-grpc-daemon[777]: ts=2026-09-20T12:47:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537864478worker1 # [ 69.964684] nix-daemon[835]: reaped child process 1113, status = succeeded4479worker1 # [ 69.977839] nix-grpc-daemon[777]: ts=2026-09-20T12:47:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537864480worker1 # [ 69.984502] nix-grpc-daemon[777]: ts=2026-09-20T12:47:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_out=2324481client # [ 70.151041] nix-daemon[938]: reaped child process 940, status = succeeded4482client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.38 seconds)4483client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4484client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4485client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4486client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4487client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4488client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4489(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.51 seconds)4490subtest: an input only one worker has still reaches the builder4491client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4492client # this derivation will be built:4493client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4494client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4495client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.48 seconds)4496client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4497client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)4498worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4499lb # [ 71.029680] envoy[913]: conn peer=[2001:db8:1::1]:33120 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3448 tx=1792 ms=19834500lb # [ 71.034934] envoy[913]: conn peer=[2001:db8:1::1]:33128 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=334 ms=334501lb # [ 71.039706] envoy[913]: conn peer=[2001:db8:1::1]:33150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=284502lb # [ 71.044746] envoy[913]: conn peer=[2001:db8:1::1]:33138 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3901 tx=1965 ms=24144503lb # [ 71.050793] envoy[913]: conn peer=[2001:db8:1::1]:33152 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4352 tx=2110 ms=27434504lb # [ 71.055689] envoy[913]: conn peer=[2001:db8:1::1]:33158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=714505worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4506worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4507worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4508client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24509client # [ 71.520902] nix-daemon[938]: accepted connection from pid 1018, user root (trusted)4510client # this derivation will be built:4511client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4512worker1 # [ 71.699830] nix-grpc-daemon[777]: ts=2026-09-20T12:47:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:537864513client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4514worker1 # [ 71.708088] nix-grpc-daemon[777]: ts=2026-09-20T12:47:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864515worker1 # [ 71.913383] nix-daemon[835]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)4516worker1 # [ 71.933144] nix-daemon[835]: reaped child process 1143, status = succeeded4517lb # [ 71.932940] niks3-server[1051]: 2026/09/20 12:47:42 INFO Received uploads request method=POST path=/api/pending_closures4518worker1 # [ 71.955634] nix-grpc-daemon[846]: time=2026-09-20T12:47:42.476Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4519worker1 # [ 71.957426] nix-grpc-daemon[846]: time=2026-09-20T12:47:42.476Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4520lb # [ 71.962503] niks3-server[1051]: 2026/09/20 12:47:42 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4521lb # [ 71.977664] niks3-server[1051]: 2026/09/20 12:47:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4522worker1 # [ 71.995526] nix-grpc-daemon[846]: time=2026-09-20T12:47:42.517Z level=INFO msg="Uploading 1 narinfos"4523lb # [ 71.985766] niks3-server[1051]: 2026/09/20 12:47:42 INFO Signed narinfos id=17 count=14524lb # [ 71.993762] niks3-server[1051]: 2026/09/20 12:47:42 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4525lb # [ 72.002689] niks3-server[1051]: 2026/09/20 12:47:42 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4526worker1 # [ 72.020179] nix-grpc-daemon[846]: time=2026-09-20T12:47:42.542Z level=INFO msg="Upload complete. (313ms)"4527lb # [ 72.009797] niks3-server[1051]: 2026/09/20 12:47:42 INFO Completed upload id=174528lb # [ 72.013161] niks3-server[1051]: 2026/09/20 12:47:42 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4529worker2 # [ 72.079590] nix-grpc-daemon[779]: ts=2026-09-20T12:47:42Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04530worker2 # [ 72.085685] nix-grpc-daemon[779]: ts=2026-09-20T12:47:42Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14531worker2 # [ 72.123324] nix-grpc-daemon[779]: ts=2026-09-20T12:47:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545404532worker2 # [ 72.139922] nix-grpc-daemon[779]: ts=2026-09-20T12:47:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545404533client # copying 2 paths to 'lb:50051'...4534worker2 # [ 72.160222] nix-grpc-daemon[779]: ts=2026-09-20T12:47:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545404535client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4536client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4537worker2 # [ 72.171167] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4538worker2 # [ 72.358876] nix-daemon[848]: accepted connection from pid 989, user nix-grpc-daemon (trusted)4539worker2 # [ 72.380666] nix-daemon[848]: reaped child process 992, status = succeeded4540worker2 # [ 72.558805] nix-daemon[848]: accepted connection from pid 994, user nix-grpc-daemon (trusted)4541worker2 # [ 72.577788] nix-daemon[848]: reaped child process 997, status = succeeded4542lb # [ 72.543637] niks3-server[1051]: 2026/09/20 12:47:43 INFO Received uploads request method=POST path=/api/pending_closures4543lb # [ 72.553928] niks3-server[1051]: 2026/09/20 12:47:43 INFO Received uploads request method=POST path=/api/pending_closures4544worker2 # [ 72.609271] nix-grpc-daemon[900]: time=2026-09-20T12:47:43.122Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4545worker2 # [ 72.610799] nix-grpc-daemon[900]: time=2026-09-20T12:47:43.122Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4546worker2 # [ 72.612586] nix-grpc-daemon[900]: time=2026-09-20T12:47:43.122Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4547lb # [ 72.586989] niks3-server[1051]: 2026/09/20 12:47:43 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4548lb # [ 72.605994] niks3-server[1051]: 2026/09/20 12:47:43 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4549lb # [ 72.617093] niks3-server[1051]: 2026/09/20 12:47:43 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4550lb # [ 72.625140] niks3-server[1051]: 2026/09/20 12:47:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4551lb # [ 72.629908] niks3-server[1051]: 2026/09/20 12:47:43 INFO Signed narinfos id=18 count=14552lb # [ 72.633288] niks3-server[1051]: 2026/09/20 12:47:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4553worker2 # [ 72.680334] nix-grpc-daemon[900]: time=2026-09-20T12:47:43.192Z level=INFO msg="Uploading 3 narinfos"4554lb # [ 72.637222] niks3-server[1051]: 2026/09/20 12:47:43 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4555lb # [ 72.639912] niks3-server[1051]: 2026/09/20 12:47:43 INFO Signed narinfos id=19 count=24556lb # [ 72.665225] niks3-server[1051]: 2026/09/20 12:47:43 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4557lb # [ 72.675442] niks3-server[1051]: 2026/09/20 12:47:43 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4558lb # [ 72.678599] niks3-server[1051]: 2026/09/20 12:47:43 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4559lb # [ 72.683545] niks3-server[1051]: 2026/09/20 12:47:43 INFO Completed upload id=194560lb # [ 72.685782] niks3-server[1051]: 2026/09/20 12:47:43 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4561worker2 # [ 72.731040] nix-grpc-daemon[900]: time=2026-09-20T12:47:43.244Z level=INFO msg="Upload complete. (548ms)"4562lb # [ 72.688476] niks3-server[1051]: 2026/09/20 12:47:43 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4563lb # [ 72.691097] niks3-server[1051]: 2026/09/20 12:47:43 INFO Completed upload id=184564worker2 # [ 72.742950] nix-grpc-daemon[779]: ts=2026-09-20T12:47:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54540 duration_s=0 paths=2 nar_bytes_in=9844565worker2 # [ 72.760789] nix-grpc-daemon[779]: ts=2026-09-20T12:47:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545404566worker2 # [ 72.770280] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4567worker2 # [ 72.778166] nix-daemon[848]: reaped child process 987, status = succeeded4568worker2 # [ 72.795951] nix-grpc-daemon[779]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4569client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4570client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4571worker2 # [ 72.818937] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4572worker2 # [ 74.265234] nix-daemon[848]: reaped child process 1002, status = succeeded4573worker2 # [ 74.455602] nix-daemon[848]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)4574worker2 # [ 74.473289] nix-daemon[848]: reaped child process 1020, status = succeeded4575lb # [ 74.446130] niks3-server[1051]: 2026/09/20 12:47:44 INFO Received uploads request method=POST path=/api/pending_closures4576worker2 # [ 74.500197] nix-grpc-daemon[900]: time=2026-09-20T12:47:45.012Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4577lb # [ 74.478462] niks3-server[1051]: 2026/09/20 12:47:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4578worker2 # [ 74.534269] nix-grpc-daemon[900]: time=2026-09-20T12:47:45.047Z level=INFO msg="Uploading 1 narinfos"4579lb # [ 74.495895] niks3-server[1051]: 2026/09/20 12:47:45 INFO Signed narinfos id=20 count=14580lb # [ 74.502647] niks3-server[1051]: 2026/09/20 12:47:45 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4581lb # [ 74.508770] niks3-server[1051]: 2026/09/20 12:47:45 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4582lb # [ 74.522443] niks3-server[1051]: 2026/09/20 12:47:45 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4583lb # [ 74.526144] niks3-server[1051]: 2026/09/20 12:47:45 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4584lb # [ 74.530253] niks3-server[1051]: 2026/09/20 12:47:45 INFO Completed upload id=204585worker2 # [ 74.575459] nix-grpc-daemon[900]: time=2026-09-20T12:47:45.089Z level=INFO msg="Upload complete. (338ms)"4586worker2 # [ 74.589291] nix-grpc-daemon[779]: ts=2026-09-20T12:47:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54540 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14587worker2 # [ 74.605301] nix-daemon[848]: reaped child process 999, status = succeeded4588client # copying 1 paths...4589worker1 # [ 74.599244] nix-grpc-daemon[777]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537864590worker1 # [ 74.616473] nix-grpc-daemon[777]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4591client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4592worker2 # [ 74.701225] nix-grpc-daemon[779]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:545404593worker2 # [ 74.709394] nix-grpc-daemon[779]: ts=2026-09-20T12:47:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:54540 duration_s=0 paths=1 nar_bytes_out=1204594client # [ 74.839654] nix-daemon[938]: reaped child process 1021, status = succeeded4595client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.57 seconds)4596client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24597client # [ 75.063593] nix-daemon[938]: accepted connection from pid 1039, user root (trusted)4598client # this derivation will be built:4599client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4600worker2 # [ 75.267687] nix-grpc-daemon[779]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:545404601client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4602worker1 # [ 75.238646] nix-grpc-daemon[777]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864603worker1 # [ 75.250999] nix-grpc-daemon[846]: time=2026-09-20T12:47:45.772Z level=INFO msg="All 1 paths already cached"4604worker2 # [ 75.304279] nix-grpc-daemon[779]: ts=2026-09-20T12:47:45Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04605worker2 # [ 75.311197] nix-grpc-daemon[779]: ts=2026-09-20T12:47:45Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14606worker1 # [ 75.281700] nix-grpc-daemon[777]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864607worker1 # [ 75.297831] nix-grpc-daemon[777]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537864608client # copying 1 paths to 'lb:50051'...4609client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4610worker1 # [ 75.320528] nix-grpc-daemon[777]: ts=2026-09-20T12:47:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537864611worker1 # [ 75.335603] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4612worker1 # [ 75.358414] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 1/5)4613worker1 # [ 75.425342] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 2/5)4614worker1 # [ 75.501198] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 212 ms (attempt 3/5)4615worker1 # [ 75.714626] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 4/5)4616worker1 # [ 75.766425] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4617worker1 # [ 75.776784] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4618worker1 # [ 76.016831] nix-daemon[835]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)4619worker1 # [ 76.036488] nix-daemon[835]: reaped child process 1155, status = succeeded4620worker1 # [ 76.214325] nix-daemon[835]: accepted connection from pid 1157, user nix-grpc-daemon (trusted)4621worker1 # [ 76.232783] nix-daemon[835]: reaped child process 1160, status = succeeded4622lb # [ 76.233055] niks3-server[1051]: 2026/09/20 12:47:46 INFO Received uploads request method=POST path=/api/pending_closures4623worker1 # [ 76.258515] nix-grpc-daemon[846]: time=2026-09-20T12:47:46.779Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4624worker1 # [ 76.260103] nix-grpc-daemon[846]: time=2026-09-20T12:47:46.779Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4625lb # [ 76.268767] niks3-server[1051]: 2026/09/20 12:47:46 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4626worker1 # [ 76.292569] nix-grpc-daemon[846]: time=2026-09-20T12:47:46.814Z level=INFO msg="Uploading 1 narinfos"4627lb # [ 76.283396] niks3-server[1051]: 2026/09/20 12:47:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4628lb # [ 76.294032] niks3-server[1051]: 2026/09/20 12:47:46 INFO Signed narinfos id=21 count=14629worker1 # [ 76.318653] nix-grpc-daemon[846]: time=2026-09-20T12:47:46.840Z level=INFO msg="Upload complete. (509ms)"4630lb # [ 76.307012] niks3-server[1051]: 2026/09/20 12:47:46 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4631lb # [ 76.312986] niks3-server[1051]: 2026/09/20 12:47:46 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4632worker1 # [ 76.328282] nix-grpc-daemon[777]: ts=2026-09-20T12:47:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 paths=1 nar_bytes_in=5684633lb # [ 76.318591] niks3-server[1051]: 2026/09/20 12:47:46 INFO Completed upload id=214634lb # [ 76.321483] niks3-server[1051]: 2026/09/20 12:47:46 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4635worker1 # [ 76.334687] nix-grpc-daemon[777]: ts=2026-09-20T12:47:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537864636worker1 # [ 76.347491] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4637worker1 # [ 76.353806] nix-daemon[835]: reaped child process 1146, status = succeeded4638client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4639worker1 # [ 76.367184] nix-daemon[835]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4640client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4641worker1 # [ 77.792336] nix-daemon[835]: reaped child process 1164, status = succeeded4642worker1 # [ 77.957892] nix-daemon[835]: accepted connection from pid 1179, user nix-grpc-daemon (trusted)4643worker1 # [ 77.979766] nix-daemon[835]: reaped child process 1182, status = succeeded4644lb # [ 77.981260] niks3-server[1051]: 2026/09/20 12:47:48 INFO Received uploads request method=POST path=/api/pending_closures4645worker1 # [ 78.004471] nix-grpc-daemon[846]: time=2026-09-20T12:47:48.525Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4646lb # [ 78.029975] niks3-server[1051]: 2026/09/20 12:47:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4647worker1 # [ 78.054968] nix-grpc-daemon[846]: time=2026-09-20T12:47:48.576Z level=INFO msg="Uploading 1 narinfos"4648lb # [ 78.046927] niks3-server[1051]: 2026/09/20 12:47:48 INFO Signed narinfos id=22 count=14649lb # [ 78.056389] niks3-server[1051]: 2026/09/20 12:47:48 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4650lb # [ 78.062193] niks3-server[1051]: 2026/09/20 12:47:48 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4651worker1 # [ 78.081359] nix-grpc-daemon[846]: time=2026-09-20T12:47:48.603Z level=INFO msg="Upload complete. (304ms)"4652lb # [ 78.070227] niks3-server[1051]: 2026/09/20 12:47:48 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4653lb # [ 78.073199] niks3-server[1051]: 2026/09/20 12:47:48 INFO Completed upload id=224654lb # [ 78.076548] niks3-server[1051]: 2026/09/20 12:47:48 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4655client # copying 1 paths...4656worker1 # [ 78.093202] nix-grpc-daemon[777]: ts=2026-09-20T12:47:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14657worker1 # [ 78.109594] nix-grpc-daemon[777]: ts=2026-09-20T12:47:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537864658client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4659worker1 # [ 78.114260] nix-daemon[835]: reaped child process 1162, status = succeeded4660worker1 # [ 78.139440] nix-grpc-daemon[777]: ts=2026-09-20T12:47:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537864661worker1 # [ 78.145167] nix-grpc-daemon[777]: ts=2026-09-20T12:47:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_out=1204662client # [ 78.309389] nix-daemon[938]: reaped child process 1042, status = succeeded4663client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.47 seconds)4664client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24665client # [ 78.528133] nix-daemon[938]: accepted connection from pid 1060, user root (trusted)4666client # this derivation will be built:4667client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4668worker2 # [ 78.734438] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:545404669client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4670worker2 # [ 78.742617] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545404671worker2 # [ 78.754003] nix-grpc-daemon[900]: time=2026-09-20T12:47:49.266Z level=INFO msg="All 1 paths already cached"4672worker2 # [ 78.769766] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04673worker2 # [ 78.777645] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14674worker2 # [ 78.786873] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545404675worker2 # [ 78.798744] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545404676worker2 # [ 78.813535] nix-grpc-daemon[900]: time=2026-09-20T12:47:49.327Z level=INFO msg="All 1 paths already cached"4677client # copying 1 paths to 'lb:50051'...4678client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4679worker2 # [ 78.824113] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545404680worker2 # [ 78.839037] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4681worker2 # [ 79.048655] nix-daemon[848]: accepted connection from pid 1028, user nix-grpc-daemon (trusted)4682worker2 # [ 79.070025] nix-daemon[848]: reaped child process 1031, status = succeeded4683worker2 # [ 79.245321] nix-daemon[848]: accepted connection from pid 1033, user nix-grpc-daemon (trusted)4684worker2 # [ 79.265109] nix-daemon[848]: reaped child process 1036, status = succeeded4685lb # [ 79.235221] niks3-server[1051]: 2026/09/20 12:47:49 INFO Received uploads request method=POST path=/api/pending_closures4686worker2 # [ 79.290095] nix-grpc-daemon[900]: time=2026-09-20T12:47:49.803Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4687worker2 # [ 79.291655] nix-grpc-daemon[900]: time=2026-09-20T12:47:49.803Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4688lb # [ 79.267267] niks3-server[1051]: 2026/09/20 12:47:49 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4689worker2 # [ 79.325823] nix-grpc-daemon[900]: time=2026-09-20T12:47:49.839Z level=INFO msg="Uploading 1 narinfos"4690lb # [ 79.280565] niks3-server[1051]: 2026/09/20 12:47:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4691lb # [ 79.293025] niks3-server[1051]: 2026/09/20 12:47:49 INFO Signed narinfos id=23 count=14692lb # [ 79.297140] niks3-server[1051]: 2026/09/20 12:47:49 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4693lb # [ 79.306206] niks3-server[1051]: 2026/09/20 12:47:49 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4694worker2 # [ 79.356384] nix-grpc-daemon[900]: time=2026-09-20T12:47:49.870Z level=INFO msg="Upload complete. (501ms)"4695lb # [ 79.311478] niks3-server[1051]: 2026/09/20 12:47:49 INFO Completed upload id=234696lb # [ 79.315485] niks3-server[1051]: 2026/09/20 12:47:49 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4697worker2 # [ 79.364351] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54540 duration_s=0 paths=1 nar_bytes_in=5684698worker2 # [ 79.372991] nix-grpc-daemon[779]: ts=2026-09-20T12:47:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545404699worker2 # [ 79.384210] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4700worker2 # [ 79.387799] nix-daemon[848]: reaped child process 1026, status = succeeded4701worker2 # [ 79.398829] nix-daemon[848]: accepted connection from pid 779, user nix-grpc-daemon (trusted)4702client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4703client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4704worker2 # [ 80.846930] nix-daemon[848]: reaped child process 1040, status = succeeded4705worker2 # [ 81.023501] nix-daemon[848]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4706worker2 # [ 81.044897] nix-daemon[848]: reaped child process 1058, status = succeeded4707lb # [ 81.013924] niks3-server[1051]: 2026/09/20 12:47:51 INFO Received uploads request method=POST path=/api/pending_closures4708worker2 # [ 81.067336] nix-grpc-daemon[900]: time=2026-09-20T12:47:51.580Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4709lb # [ 81.031944] envoy[913]: conn peer=[2001:db8:1::1]:54066 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3865 tx=1877 ms=30374710lb # [ 81.042775] envoy[913]: conn peer=[2001:db8:1::1]:54072 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=744711lb # [ 81.055385] envoy[913]: conn peer=[2001:db8:1::1]:54080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3713 tx=1805 ms=29714712worker2 # [ 81.114062] nix-grpc-daemon[900]: time=2026-09-20T12:47:51.627Z level=INFO msg="Uploading 1 narinfos"4713lb # [ 81.070591] envoy[913]: conn peer=[2001:db8:1::1]:54082 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=714714lb # [ 81.080561] niks3-server[1051]: 2026/09/20 12:47:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4715lb # [ 81.083865] niks3-server[1051]: 2026/09/20 12:47:51 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4716lb # [ 81.089814] niks3-server[1051]: 2026/09/20 12:47:51 INFO Signed narinfos id=24 count=14717lb # [ 81.095788] niks3-server[1051]: 2026/09/20 12:47:51 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4718lb # [ 81.100390] niks3-server[1051]: 2026/09/20 12:47:51 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4719worker2 # [ 81.145473] nix-grpc-daemon[900]: time=2026-09-20T12:47:51.659Z level=INFO msg="Upload complete. (333ms)"4720lb # [ 81.103730] niks3-server[1051]: 2026/09/20 12:47:51 INFO Completed upload id=244721lb # [ 81.105579] niks3-server[1051]: 2026/09/20 12:47:51 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4722worker2 # [ 81.157387] nix-grpc-daemon[779]: ts=2026-09-20T12:47:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54540 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14723worker2 # [ 81.177801] nix-daemon[848]: reaped child process 1038, status = succeeded4724client # copying 1 paths...4725worker1 # [ 81.174931] nix-grpc-daemon[777]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537864726worker1 # [ 81.189222] nix-grpc-daemon[777]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4727client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4728worker1 # [ 81.239673] nix-grpc-daemon[777]: ts=2026-09-20T12:47:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:537864729worker1 # [ 81.245732] nix-grpc-daemon[777]: ts=2026-09-20T12:47:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53786 duration_s=0 paths=1 nar_bytes_out=1204730client # [ 81.401765] nix-daemon[938]: reaped child process 1063, status = succeeded4731client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.09 seconds)4732(finished: subtest: an input only one worker has still reaches the builder, in 11.11 seconds)4733subtest: upload whose reference the worker lacks is completed from the cache4734client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4735client # this derivation will be built:4736client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4737client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4738client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.45 seconds)4739client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4740client # this derivation will be built:4741client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4742client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4743client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.45 seconds)4744client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4745client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.21 seconds)4746worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4747worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4748worker2 # [ 82.871305] systemd[1]: Stopping Nix worker-protocol over gRPC...4749worker2 # [ 83.566270] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=signal_exit4750worker2 # [ 83.765175] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=scheduler_yield4751worker2 # [ 83.775717] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=scheduler_restarting addr=lb:500514752lb # [ 83.731285] niks3-server[1051]: 2026/09/20 12:47:54 INFO lead: released remote=192.168.1.4:385624753worker1 # [ 83.747092] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_take_over4754worker2 # [ 83.781897] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:545364755worker1 # [ 83.752743] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_restarting addr=lb:500514756worker2 # [ 83.789039] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=worker_gone addr=192.168.1.4:500514757worker2 # [ 83.793334] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:545364758worker2 # [ 83.797981] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=worker_gone addr=192.168.1.3:500514759worker2 # [ 83.977598] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500514760worker2 # [ 83.983370] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4761worker1 # [ 83.954181] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500514762worker1 # [ 83.959857] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4763worker2 # [ 84.184720] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500514764worker2 # [ 84.190423] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4765worker1 # [ 84.160842] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500514766worker1 # [ 84.166523] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4767worker2 # [ 84.391894] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500514768worker2 # [ 84.397589] nix-grpc-daemon[779]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4769worker1 # [ 84.367260] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500514770lb # [ 83.738163] niks3-server[1051]: 2026/09/20 12:47:54 INFO lead: acquired remote=192.168.1.3:579224771worker1 # [ 84.372929] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4772worker2 # [ 84.600989] nix-grpc-daemon[779]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500514773worker2 # [ 84.605448] nix-grpc-daemon[779]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4774worker1 # [ 84.574108] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500514775worker1 # [ 84.580544] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4776worker2 # [ 84.770361] nix-grpc-daemon[779]: ts=2026-09-20T12:47:55Z level=info event=scheduler_restarting4777worker1 # [ 84.783418] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500514778worker1 # [ 84.787838] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4779worker2 # [ 84.827757] nix-grpc-daemon[779]: ts=2026-09-20T12:47:55Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4780worker2 # [ 84.833921] nix-daemon[848]: reaped child process 850, status = succeeded4781worker1 # [ 84.984110] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500514782worker1 # [ 84.989463] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4783worker1 # [ 85.190311] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500514784worker1 # [ 85.195763] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4785worker1 # [ 85.396850] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500514786worker1 # [ 85.403137] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4787worker1 # [ 85.603613] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500514788worker1 # [ 85.624738] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:401444789worker1 # [ 85.630121] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04790worker2 # [ 85.714485] nix-grpc-daemon[779]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4791worker2 # [ 85.719311] nix-grpc-daemon[779]: E0000 00:00:1789908476.226544 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4792worker2 # [ 85.726227] nix-grpc-daemon[779]: E0000 00:00:1789908476.226871 822 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4793worker2 # [ 85.853112] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4794worker2 # [ 85.860958] systemd[1]: Stopped Nix worker-protocol over gRPC.4795worker2 # [ 85.863220] systemd[1]: nix-grpc-daemon.service: Consumed 2.023s CPU time over 1min 4.610s wall clock time, 51.7M memory peak, 56.5K incoming IP traffic, 148.4K outgoing IP traffic.4796worker2 # [ 85.880684] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4797worker2 # [ 85.883621] systemd[1]: Closed Nix worker-protocol over gRPC.4798worker2 # [ 85.886811] systemd[1]: nix-grpc-daemon.socket: Consumed 644us CPU time over 1min 14.868s wall clock time, 256K memory peak, 76.5K incoming IP traffic, 61.2K outgoing IP traffic.4799lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4800lb: must succeed: curl -sf localhost:9901/clusters4801lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4802lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)4803client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24804worker1 # [ 86.204891] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537864805client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4806worker1 # [ 86.212778] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:537864807client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.30 seconds)4808worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4809worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4810worker2 # [ 86.394875] systemd[1]: Listening on Nix worker-protocol over gRPC.4811worker1 # [ 86.447486] systemd[1]: Stopping Nix worker-protocol over gRPC...4812worker2 # [ 86.844450] systemd[1]: Starting scheduler-token.service...4813worker2 # [ 86.958962] systemd[1]: scheduler-token.service: Deactivated successfully.4814worker2 # [ 86.964357] systemd[1]: Finished scheduler-token.service.4815worker2 # [ 86.966100] systemd[1]: scheduler-token.service: Consumed 57ms CPU time over 118ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4816worker2 # [ 86.976546] systemd[1]: Starting Nix worker-protocol over gRPC...4817worker2 # [ 87.170795] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:57Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4818worker2 # [ 87.173454] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:57Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4819worker2 # [ 87.177843] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:57Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14820worker2 # [ 87.186931] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:57Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4821worker2 # [ 87.198449] systemd[1]: Started Nix worker-protocol over gRPC.4822worker2 # [ 87.218155] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500514823worker1 # [ 87.188561] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:401444824worker1 # [ 87.194585] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04825worker1 # [ 87.404382] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=signal_exit4826worker1 # [ 87.669160] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_yield4827lb # [ 87.663903] niks3-server[1051]: 2026/09/20 12:47:58 INFO lead: released remote=192.168.1.3:579224828worker2 # [ 87.708974] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:58Z level=info event=scheduler_restarting addr=lb:500514829worker1 # [ 87.677470] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401444830worker1 # [ 87.686615] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=worker_gone addr=192.168.1.4:500514831worker1 # [ 87.691659] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_restarting addr=lb:500514832worker1 # [ 87.696164] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:401444833worker1 # [ 87.700737] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=worker_gone addr=192.168.1.3:500514834worker2 # [ 87.910771] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500514835worker2 # [ 87.916778] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4836worker1 # [ 87.885625] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500514837worker1 # [ 87.891359] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4838worker2 # [ 88.118275] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500514839worker2 # [ 88.123976] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4840worker1 # [ 88.092411] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500514841worker1 # [ 88.098118] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4842worker2 # [ 88.327432] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500514843worker1 # [ 88.298865] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500514844worker2 # [ 88.331977] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4845worker1 # [ 88.304515] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4846worker2 # [ 88.529065] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500514847worker2 # [ 88.534796] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4848worker1 # [ 88.507387] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500514849worker1 # [ 88.513328] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4850worker1 # [ 88.675101] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_restarting4851worker2 # [ 88.736258] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500514852worker2 # [ 88.741988] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4853worker1 # [ 88.734057] nix-daemon[835]: reaped child process 838, status = succeeded4854worker1 # [ 88.736936] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4855worker2 # [ 88.943484] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500514856worker2 # [ 88.949406] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4857worker2 # [ 89.150816] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500514858worker2 # [ 89.156378] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4859worker2 # [ 89.359568] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500514860worker2 # [ 89.364114] nix-grpc-daemon[1082]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4861worker1 # [ 89.517124] nix-grpc-daemon[777]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4862worker1 # [ 89.521832] nix-grpc-daemon[777]: E0000 00:00:1789908480.037417 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4863worker2 # [ 89.561106] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500514864worker2 # [ 89.566754] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4865worker1 # [ 89.585338] nix-grpc-daemon[777]: E0000 00:00:1789908480.107107 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4866worker2 # [ 89.768196] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500514867worker2 # [ 89.773880] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4868worker1 # [ 89.745212] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4869worker1 # [ 89.755983] systemd[1]: Stopped Nix worker-protocol over gRPC.4870worker1 # [ 89.759911] systemd[1]: nix-grpc-daemon.service: Consumed 4.136s CPU time over 1min 8.715s wall clock time, 59.4M memory peak, 88.3K incoming IP traffic, 161.4K outgoing IP traffic.4871worker1 # [ 89.779369] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4872worker1 # [ 89.781817] systemd[1]: Closed Nix worker-protocol over gRPC.4873worker1 # [ 89.785706] systemd[1]: nix-grpc-daemon.socket: Consumed 656us CPU time over 1min 17.753s wall clock time, 256K memory peak, 87K incoming IP traffic, 72.8K outgoing IP traffic.4874lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4875lb: must succeed: curl -sf localhost:9901/clusters4876lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4877lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4878client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24879worker2 # [ 89.975364] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500514880worker2 # [ 89.981164] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4881worker2 # [ 90.180284] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:607164882worker2 # [ 90.195290] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500514883worker2 # [ 90.206530] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4884worker2 # [ 90.221513] nix-daemon[848]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)4885worker2 # [ 90.239954] nix-grpc-daemon[1082]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)4886worker2 # [ 90.302545] nix-grpc-daemon[1082]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)4887worker2 # [ 90.394329] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500514888worker2 # [ 90.398899] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4889worker2 # [ 90.466435] nix-grpc-daemon[1082]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 154 ms (attempt 3/5)4890worker2 # [ 90.595611] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500514891worker2 # [ 90.602031] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4892worker2 # [ 90.628527] nix-grpc-daemon[1082]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 533 ms (attempt 4/5)4893worker2 # [ 90.802825] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500514894worker2 # [ 90.808888] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4895worker2 # [ 91.010082] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500514896worker2 # [ 91.015586] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4897lb # [ 91.028726] envoy[913]: conn peer=[2001:db8:1::1]:54096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3727 tx=1894 ms=25974898lb # [ 91.029021] envoy[913]: conn peer=[2001:db8:1::1]:33374 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=644899lb # [ 91.029136] envoy[913]: conn peer=[2001:db8:1::4]:57422 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=53533 tx=2754 ms=632504900lb # [ 91.029371] envoy[913]: conn peer=[2001:db8:1::1]:33380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=254901lb # [ 91.029497] envoy[913]: conn peer=192.168.1.3:53646 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=11693 tx=3242 ms=661544902worker2 # [ 91.170344] nix-grpc-daemon[1082]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4903client # copying 2 paths...4904client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4905worker2 # [ 91.202193] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:607164906client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4907worker2 # [ 91.217234] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500514908worker2 # [ 91.232733] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4909worker2 # [ 91.247533] nix-daemon[848]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)4910worker2 # [ 91.268711] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=niks3_push_started pid=11384911worker2 # [ 91.424956] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500514912worker2 # [ 91.444282] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4913worker2 # [ 91.494774] nix-grpc-daemon[1138]: time=2026-09-20T12:48:02.007Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4914worker2 # [ 91.629934] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500514915worker2 # [ 91.632562] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4916worker2 # [ 91.681584] nix-daemon[848]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)4917worker2 # [ 91.703144] nix-daemon[848]: reaped child process 1154, status = succeeded4918lb # [ 91.672760] niks3-server[1051]: 2026/09/20 12:48:02 INFO Received uploads request method=POST path=/api/pending_closures4919lb # [ 91.682376] niks3-server[1051]: 2026/09/20 12:48:02 INFO Received uploads request method=POST path=/api/pending_closures4920worker2 # [ 91.739082] nix-grpc-daemon[1138]: time=2026-09-20T12:48:02.250Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4921worker2 # [ 91.741805] nix-grpc-daemon[1138]: time=2026-09-20T12:48:02.250Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4922worker2 # [ 91.743628] nix-grpc-daemon[1138]: time=2026-09-20T12:48:02.250Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4923lb # [ 91.750832] niks3-server[1051]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4924lb # [ 91.767222] niks3-server[1051]: 2026/09/20 12:48:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4925lb # [ 91.780342] niks3-server[1051]: 2026/09/20 12:48:02 INFO Signed narinfos id=26 count=24926worker2 # [ 91.824467] nix-grpc-daemon[1138]: time=2026-09-20T12:48:02.336Z level=INFO msg="Uploading 3 narinfos"4927lb # [ 91.786687] niks3-server[1051]: 2026/09/20 12:48:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4928worker2 # [ 91.835538] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500514929lb # [ 91.795942] niks3-server[1051]: 2026/09/20 12:48:02 INFO Signed narinfos id=25 count=14930worker2 # [ 91.841779] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4931lb # [ 91.800958] niks3-server[1051]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4932lb # [ 91.811507] niks3-server[1051]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4933lb # [ 91.819478] niks3-server[1051]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4934lb # [ 91.824977] niks3-server[1051]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4935lb # [ 91.827847] niks3-server[1051]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4936lb # [ 91.836233] niks3-server[1051]: 2026/09/20 12:48:02 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4937lb # [ 91.843028] niks3-server[1051]: 2026/09/20 12:48:02 INFO Completed upload id=254938lb # [ 91.845591] niks3-server[1051]: 2026/09/20 12:48:02 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4939lb # [ 91.847930] niks3-server[1051]: 2026/09/20 12:48:02 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4940lb # [ 91.851283] niks3-server[1051]: 2026/09/20 12:48:02 INFO Completed upload id=264941worker2 # [ 91.896629] nix-grpc-daemon[1138]: time=2026-09-20T12:48:02.410Z level=INFO msg="Upload complete. (397ms)"4942worker2 # [ 91.908753] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60716 duration_s=0 paths=2 nar_bytes_in=3124943worker2 # [ 91.926634] nix-daemon[848]: reaped child process 1136, status = succeeded4944client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.04 seconds)4945worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4946worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)4947worker2 # [ 92.042511] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500514948worker2 # [ 92.048271] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4949worker1 # [ 92.051446] systemd[1]: Listening on Nix worker-protocol over gRPC.4950worker1 # [ 92.062635] systemd[1]: Starting Nix worker-protocol over gRPC...4951lb # [ 92.155358] niks3-server[1051]: 2026/09/20 12:48:02 INFO lead: acquired remote=192.168.1.4:507464952worker2 # [ 92.205340] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=scheduler_take_over4953worker2 # [ 92.249306] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500514954worker2 # [ 92.268162] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=oidc_jwks provider=mock keys=14955worker2 # [ 92.273794] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:349444956worker2 # [ 92.279299] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04957worker1 # [ 92.266409] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:02Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4958worker1 # [ 92.269005] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:02Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4959worker1 # [ 92.273332] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:02Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14960worker1 # [ 92.281739] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:02Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4961worker1 # [ 92.283486] systemd[1]: Started Nix worker-protocol over gRPC.4962lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4963lb: must succeed: curl -sf localhost:9901/clusters4964worker1 # [ 92.312165] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500514965worker2 # [ 92.348202] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:349444966worker2 # [ 92.353309] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:02Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04967lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4968lb: must succeed: curl -sf localhost:9901/clusters4969lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4970lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.18 seconds)4971(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.08 seconds)4972subtest: scheduler down: the other node takes the lock and keeps it4973lb: must succeed: curl -sf localhost:9901/clusters4974lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4975lb: must succeed: curl -sf localhost:9901/clusters4976lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4977lb: must succeed: curl -sf localhost:9901/clusters4978lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4979lb: must succeed: curl -sf localhost:9901/clusters4980lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4981worker2 # [ 93.775091] systemd[1]: Stopping Nix worker-protocol over gRPC...4982worker2 # [ 94.193189] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:04Z level=info event=signal_exit4983worker2 # [ 94.409168] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:04Z level=info event=scheduler_yield4984lb # [ 94.373463] niks3-server[1051]: 2026/09/20 12:48:04 INFO lead: released remote=192.168.1.4:507464985worker1 # [ 94.389694] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:04Z level=info event=scheduler_restarting addr=lb:500514986worker2 # [ 94.426762] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:349444987worker2 # [ 94.433839] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:04Z level=info event=worker_gone addr=192.168.1.3:500514988worker2 # [ 94.438139] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:04Z level=info event=scheduler_restarting addr=lb:500514989worker2 # [ 94.442449] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:349444990worker2 # [ 94.447087] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:04Z level=info event=worker_gone addr=192.168.1.4:500514991worker1 # [ 94.589961] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500514992worker2 # [ 94.628399] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500514993worker1 # [ 94.595906] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4994worker2 # [ 94.634018] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4995worker1 # [ 94.796698] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500514996worker1 # [ 94.802156] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4997worker2 # [ 94.835681] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500514998worker2 # [ 94.841375] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4999worker1 # [ 95.003433] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500515000worker2 # [ 95.042319] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500515001worker1 # [ 95.009262] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5002worker2 # [ 95.047964] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5003worker1 # [ 95.210223] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500515004worker1 # [ 95.216153] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5005worker2 # [ 95.251407] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500515006worker2 # [ 95.255976] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5007worker2 # [ 95.415033] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=scheduler_restarting5008worker1 # [ 95.419195] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_connected addr=lb:500515009worker1 # [ 95.423871] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5010worker2 # [ 95.449675] nix-daemon[848]: reaped child process 1131, status = succeeded5011worker2 # [ 95.473735] nix-grpc-daemon[1082]: ts=2026-09-20T12:48:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5012worker1 # [ 95.620103] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500515013worker1 # [ 95.625836] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5014worker1 # [ 95.826923] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500515015worker1 # [ 95.832776] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5016worker1 # [ 96.033731] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500515017worker1 # [ 96.039491] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5018worker2 # [ 96.237232] nix-grpc-daemon[1082]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5019worker2 # [ 96.242260] nix-grpc-daemon[1082]: E0000 00:00:1789908486.750671 1125 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5020worker2 # [ 96.251120] nix-grpc-daemon[1082]: E0000 00:00:1789908486.763147 1125 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5021worker1 # [ 96.240618] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500515022worker1 # [ 96.246355] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5023worker1 # [ 96.449138] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_connected addr=lb:500515024worker1 # [ 96.454183] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5025worker1 # [ 96.655208] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500515026worker1 # [ 96.661090] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5027worker1 # [ 96.862526] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500515028worker1 # [ 96.868349] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5029worker1 # [ 97.069284] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500515030worker1 # [ 97.074910] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5031worker2 # [ 97.236812] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5032worker2 # [ 97.240844] systemd[1]: Stopped Nix worker-protocol over gRPC.5033worker2 # [ 97.243403] systemd[1]: nix-grpc-daemon.service: Consumed 531ms CPU time over 10.255s wall clock time, 30.8M memory peak, 23.7K incoming IP traffic, 48.4K outgoing IP traffic.5034worker2 # [ 97.260504] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5035worker2 # [ 97.262717] systemd[1]: Closed Nix worker-protocol over gRPC.5036worker2 # [ 97.267646] systemd[1]: nix-grpc-daemon.socket: Consumed 964us CPU time over 10.877s wall clock time, 256K memory peak, 20.7K incoming IP traffic, 15.4K outgoing IP traffic.5037worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5038worker1 # [ 97.275986] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_connected addr=lb:500515039worker1 # [ 97.288242] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5040lb # [ 97.283224] niks3-server[1051]: 2026/09/20 12:48:07 INFO lead: acquired remote=192.168.1.3:602105041worker1 # [ 97.296233] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:07Z level=info event=scheduler_take_over5042worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.12 seconds)5043worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5044worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5045worker1 # [ 97.486186] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:08Z level=info event=scheduler_connected addr=lb:500515046worker1 # [ 97.490961] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5047worker1 # [ 97.692041] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:08Z level=info event=scheduler_connected addr=lb:500515048worker1 # [ 97.697700] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5049worker1 # [ 97.898675] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:08Z level=info event=scheduler_connected addr=lb:500515050worker1 # [ 97.909285] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:08Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:484305051worker1 # [ 97.915033] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:08Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05052worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5053worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5054client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25055worker1 # [ 98.757107] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:484425056worker1 # [ 98.769270] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5057client # don't know how to build these paths:5058client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5059worker1 # [ 98.790040] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:484425060worker1 # [ 98.807652] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 83 ms (attempt 1/5)5061worker1 # [ 98.898557] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 2/5)5062worker1 # [ 99.080245] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 321 ms (attempt 3/5)5063worker1 # [ 99.402175] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 440 ms (attempt 4/5)5064worker1 # [ 99.845343] nix-grpc-daemon[1206]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5065worker1 # [ 99.882186] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:10Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05066worker1 # [ 99.889643] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:10Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15067worker1 # [ 99.929886] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:484425068worker1 # [ 99.947093] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:484425069client # copying 1 paths to 'lb:50051'...5070client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5071worker1 # [ 99.963112] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:484425072worker1 # [ 99.980262] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5073worker1 # [ 100.009546] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:10Z level=info event=niks3_push_started pid=12765074worker1 # [ 100.230209] nix-grpc-daemon[1276]: time=2026-09-20T12:48:10.751Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5075worker1 # [ 100.407862] nix-daemon[835]: accepted connection from pid 1289, user nix-grpc-daemon (trusted)5076worker1 # [ 100.426253] nix-daemon[835]: reaped child process 1292, status = succeeded5077worker1 # [ 100.604554] nix-daemon[835]: accepted connection from pid 1294, user nix-grpc-daemon (trusted)5078worker1 # [ 100.622269] nix-daemon[835]: reaped child process 1297, status = succeeded5079lb # [ 100.626120] niks3-server[1051]: 2026/09/20 12:48:11 INFO Received uploads request method=POST path=/api/pending_closures5080worker1 # [ 100.653465] nix-grpc-daemon[1276]: time=2026-09-20T12:48:11.173Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5081worker1 # [ 100.655427] nix-grpc-daemon[1276]: time=2026-09-20T12:48:11.174Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5082lb # [ 100.680948] niks3-server[1051]: 2026/09/20 12:48:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5083lb # [ 100.697191] niks3-server[1051]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5084worker1 # [ 100.728149] nix-grpc-daemon[1276]: time=2026-09-20T12:48:11.249Z level=INFO msg="Uploading 1 narinfos"5085lb # [ 100.715011] niks3-server[1051]: 2026/09/20 12:48:11 INFO Signed narinfos id=27 count=15086lb # [ 100.733878] niks3-server[1051]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5087lb # [ 100.743553] niks3-server[1051]: 2026/09/20 12:48:11 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5088lb # [ 100.748108] niks3-server[1051]: 2026/09/20 12:48:11 INFO Completed upload id=275089worker1 # [ 100.760508] nix-grpc-daemon[1276]: time=2026-09-20T12:48:11.282Z level=INFO msg="Upload complete. (526ms)"5090lb # [ 100.757397] niks3-server[1051]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5091worker1 # [ 100.772649] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48442 duration_s=0 paths=1 nar_bytes_in=5205092worker1 # [ 100.792557] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:484425093worker1 # [ 100.801393] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5094worker1 # [ 100.807459] nix-daemon[835]: reaped child process 1274, status = succeeded5095client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5096worker1 # [ 100.820274] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5097client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5098lb # [ 101.032401] envoy[913]: conn peer=[2001:db8:1::1]:33388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1010 tx=305 ms=17435099lb # [ 101.037761] envoy[913]: conn peer=[2001:db8:1::4]:36834 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=20956 tx=1101 ms=82495100client # farm-a> LOG-farm-a-on-standby5101worker1 # [ 101.849174] nix-daemon[835]: reaped child process 1301, status = succeeded5102worker1 # [ 102.012517] nix-daemon[835]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5103worker1 # [ 102.033465] nix-daemon[835]: reaped child process 1319, status = succeeded5104lb # [ 102.033026] niks3-server[1051]: 2026/09/20 12:48:12 INFO Received uploads request method=POST path=/api/pending_closures5105worker1 # [ 102.060450] nix-grpc-daemon[1276]: time=2026-09-20T12:48:12.580Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5106worker1 # [ 102.061969] nix-grpc-daemon[1276]: time=2026-09-20T12:48:12.580Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5107lb # [ 102.085212] niks3-server[1051]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5108worker1 # [ 102.113869] nix-grpc-daemon[1276]: time=2026-09-20T12:48:12.634Z level=INFO msg="Uploading 1 narinfos"5109lb # [ 102.103007] niks3-server[1051]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5110lb # [ 102.111065] niks3-server[1051]: 2026/09/20 12:48:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5111lb # [ 102.116691] niks3-server[1051]: 2026/09/20 12:48:12 INFO Signed narinfos id=28 count=15112lb # [ 102.124401] niks3-server[1051]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5113lb # [ 102.129386] niks3-server[1051]: 2026/09/20 12:48:12 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5114lb # [ 102.132842] niks3-server[1051]: 2026/09/20 12:48:12 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5115worker1 # [ 102.144970] nix-grpc-daemon[1276]: time=2026-09-20T12:48:12.666Z level=INFO msg="Upload complete. (310ms)"5116lb # [ 102.135537] niks3-server[1051]: 2026/09/20 12:48:12 INFO Completed upload id=285117worker1 # [ 102.168823] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48442 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15118worker1 # [ 102.180316] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:12Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05119worker1 # [ 102.183549] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:12Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15120worker1 # [ 102.188484] nix-daemon[835]: reaped child process 1299, status = succeeded5121worker1 # [ 102.209402] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:484425122worker1 # [ 102.224249] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:484425123worker1 # [ 102.238960] nix-grpc-daemon[1276]: time=2026-09-20T12:48:12.760Z level=INFO msg="All 1 paths already cached"5124client # copying 1 paths to 'lb:50051'...5125client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5126worker1 # [ 102.249578] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:484425127worker1 # [ 102.263952] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5128worker1 # [ 102.470287] nix-daemon[835]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5129worker1 # [ 102.489500] nix-daemon[835]: reaped child process 1328, status = succeeded5130worker1 # [ 102.666495] nix-daemon[835]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)5131worker1 # [ 102.684963] nix-daemon[835]: reaped child process 1333, status = succeeded5132lb # [ 102.683625] niks3-server[1051]: 2026/09/20 12:48:13 INFO Received uploads request method=POST path=/api/pending_closures5133worker1 # [ 102.708830] nix-grpc-daemon[1276]: time=2026-09-20T12:48:13.230Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5134worker1 # [ 102.710426] nix-grpc-daemon[1276]: time=2026-09-20T12:48:13.230Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5135worker1 # [ 102.741101] nix-grpc-daemon[1276]: time=2026-09-20T12:48:13.262Z level=INFO msg="Uploading 1 narinfos"5136lb # [ 102.718305] niks3-server[1051]: 2026/09/20 12:48:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5137lb # [ 102.734301] niks3-server[1051]: 2026/09/20 12:48:13 INFO Signed narinfos id=29 count=15138lb # [ 102.742758] niks3-server[1051]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5139lb # [ 102.752298] niks3-server[1051]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5140worker1 # [ 102.777050] nix-grpc-daemon[1276]: time=2026-09-20T12:48:13.298Z level=INFO msg="Upload complete. (497ms)"5141lb # [ 102.765622] niks3-server[1051]: 2026/09/20 12:48:13 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5142lb # [ 102.770775] niks3-server[1051]: 2026/09/20 12:48:13 INFO Completed upload id=295143worker1 # [ 102.784134] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48442 duration_s=0 paths=1 nar_bytes_in=6405144lb # [ 102.774261] niks3-server[1051]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5145worker1 # [ 102.799730] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:484425146worker1 # [ 102.806542] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5147worker1 # [ 102.811716] nix-daemon[835]: reaped child process 1323, status = succeeded5148client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5149worker1 # [ 102.824586] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5150client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5151client # farm-b> LOG-farm-b-on-standby5152worker1 # [ 103.654806] nix-daemon[835]: reaped child process 1337, status = succeeded5153worker1 # [ 103.819227] nix-daemon[835]: accepted connection from pid 1352, user nix-grpc-daemon (trusted)5154worker1 # [ 103.840513] nix-daemon[835]: reaped child process 1355, status = succeeded5155lb # [ 103.842585] niks3-server[1051]: 2026/09/20 12:48:14 INFO Received uploads request method=POST path=/api/pending_closures5156worker1 # [ 103.866343] nix-grpc-daemon[1276]: time=2026-09-20T12:48:14.387Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5157worker1 # [ 103.867992] nix-grpc-daemon[1276]: time=2026-09-20T12:48:14.387Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5158lb # [ 103.884321] niks3-server[1051]: 2026/09/20 12:48:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5159worker1 # [ 103.906930] nix-grpc-daemon[1276]: time=2026-09-20T12:48:14.428Z level=INFO msg="Uploading 1 narinfos"5160lb # [ 103.902512] niks3-server[1051]: 2026/09/20 12:48:14 INFO Signed narinfos id=30 count=15161lb # [ 103.911876] niks3-server[1051]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5162lb # [ 103.920379] niks3-server[1051]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5163lb # [ 103.929077] niks3-server[1051]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5164worker1 # [ 103.946193] nix-grpc-daemon[1276]: time=2026-09-20T12:48:14.468Z level=INFO msg="Upload complete. (307ms)"5165lb # [ 103.934401] niks3-server[1051]: 2026/09/20 12:48:14 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5166lb # [ 103.939717] niks3-server[1051]: 2026/09/20 12:48:14 INFO Completed upload id=305167lb # [ 103.942720] niks3-server[1051]: 2026/09/20 12:48:14 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5168worker1 # [ 103.957772] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48442 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15169worker1 # [ 103.973212] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:14Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05170worker1 # [ 103.979772] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:14Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15171worker1 # [ 103.984066] nix-daemon[835]: reaped child process 1335, status = succeeded5172worker1 # [ 104.021452] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:484425173worker1 # [ 104.037180] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:484425174worker1 # [ 104.053016] nix-grpc-daemon[1276]: time=2026-09-20T12:48:14.574Z level=INFO msg="All 2 paths already cached"5175client # copying 1 paths to 'lb:50051'...5176client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5177worker1 # [ 104.063932] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:484425178worker1 # [ 104.079119] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5179worker1 # [ 104.286631] nix-daemon[835]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5180worker1 # [ 104.307990] nix-daemon[835]: reaped child process 1364, status = succeeded5181worker1 # [ 104.481677] nix-daemon[835]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5182worker1 # [ 104.501735] nix-daemon[835]: reaped child process 1369, status = succeeded5183lb # [ 104.500888] niks3-server[1051]: 2026/09/20 12:48:15 INFO Received uploads request method=POST path=/api/pending_closures5184worker1 # [ 104.523755] nix-grpc-daemon[1276]: time=2026-09-20T12:48:15.045Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5185worker1 # [ 104.525443] nix-grpc-daemon[1276]: time=2026-09-20T12:48:15.045Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5186lb # [ 104.532465] niks3-server[1051]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5187lb # [ 104.546807] niks3-server[1051]: 2026/09/20 12:48:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5188worker1 # [ 104.567030] nix-grpc-daemon[1276]: time=2026-09-20T12:48:15.088Z level=INFO msg="Uploading 1 narinfos"5189lb # [ 104.560914] niks3-server[1051]: 2026/09/20 12:48:15 INFO Signed narinfos id=31 count=15190lb # [ 104.563460] niks3-server[1051]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5191lb # [ 104.577342] niks3-server[1051]: 2026/09/20 12:48:15 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5192lb # [ 104.582775] niks3-server[1051]: 2026/09/20 12:48:15 INFO Completed upload id=315193worker1 # [ 104.595104] nix-grpc-daemon[1276]: time=2026-09-20T12:48:15.117Z level=INFO msg="Upload complete. (498ms)"5194lb # [ 104.585288] niks3-server[1051]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5195worker1 # [ 104.604253] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48442 duration_s=0 paths=1 nar_bytes_in=7685196worker1 # [ 104.615858] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:484425197worker1 # [ 104.623299] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5198worker1 # [ 104.628621] nix-daemon[835]: reaped child process 1359, status = succeeded5199client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5200client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5201worker1 # [ 104.642571] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5202client # farm-top> LOG-farm-top-on-standby5203worker1 # [ 106.284934] nix-daemon[835]: reaped child process 1373, status = succeeded5204worker1 # [ 106.479430] nix-daemon[835]: accepted connection from pid 1388, user nix-grpc-daemon (trusted)5205worker1 # [ 106.498525] nix-daemon[835]: reaped child process 1391, status = succeeded5206lb # [ 106.499819] niks3-server[1051]: 2026/09/20 12:48:17 INFO Received uploads request method=POST path=/api/pending_closures5207worker1 # [ 106.528479] nix-grpc-daemon[1276]: time=2026-09-20T12:48:17.048Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5208worker1 # [ 106.530196] nix-grpc-daemon[1276]: time=2026-09-20T12:48:17.048Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5209lb # [ 106.544973] niks3-server[1051]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5210lb # [ 106.556565] niks3-server[1051]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5211lb # [ 106.566837] niks3-server[1051]: 2026/09/20 12:48:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5212worker1 # [ 106.584836] nix-grpc-daemon[1276]: time=2026-09-20T12:48:17.106Z level=INFO msg="Uploading 1 narinfos"5213lb # [ 106.575487] niks3-server[1051]: 2026/09/20 12:48:17 INFO Signed narinfos id=32 count=15214lb # [ 106.580128] niks3-server[1051]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5215lb # [ 106.587307] niks3-server[1051]: 2026/09/20 12:48:17 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5216lb # [ 106.594564] niks3-server[1051]: 2026/09/20 12:48:17 INFO Completed upload id=325217worker1 # [ 106.606262] nix-grpc-daemon[1276]: time=2026-09-20T12:48:17.128Z level=INFO msg="Upload complete. (350ms)"5218lb # [ 106.597295] niks3-server[1051]: 2026/09/20 12:48:17 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5219worker1 # [ 106.621245] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48442 duration_s=2 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15220worker1 # [ 106.638246] nix-daemon[835]: reaped child process 1371, status = succeeded5221client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 8.18 seconds)5222client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5223client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5224worker2 # [ 107.030845] systemd[1]: Listening on Nix worker-protocol over gRPC.5225worker2 # [ 107.040812] systemd[1]: Starting scheduler-token.service...5226worker2 # [ 107.151086] systemd[1]: scheduler-token.service: Deactivated successfully.5227worker2 # [ 107.155630] systemd[1]: Finished scheduler-token.service.5228worker2 # [ 107.160682] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 113ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5229worker2 # [ 107.166712] systemd[1]: Starting Nix worker-protocol over gRPC...5230worker2 # [ 107.351943] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:17Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5231worker2 # [ 107.354574] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5232worker2 # [ 107.358885] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15233worker2 # [ 107.367145] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5234worker2 # [ 107.382620] systemd[1]: Started Nix worker-protocol over gRPC.5235lb: must succeed: curl -sf localhost:9901/clusters5236worker2 # [ 107.400419] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:17Z level=info event=scheduler_connected addr=lb:500515237worker1 # [ 107.379481] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:17Z level=info event=oidc_jwks provider=mock keys=15238worker1 # [ 107.379917] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:17Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:484305239worker1 # [ 107.389123] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05240lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5241worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5242worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5243lb: must succeed: curl -sf localhost:9901/clusters5244lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5245lb: must succeed: curl -sf localhost:9901/clusters5246lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5247worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5248worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5249(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.28 seconds)5250subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5251client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5252client # [ 107.850872] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5253client # Running as unit: rs.service; invocation ID: 3515d650c51f41e0920b2953c6f34a105254client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds)5255worker1 # [ 108.069755] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:484425256client # [ 108.109367] nix[1162]: don't know how to build these paths:5257client # [ 108.110407] nix[1162]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5258worker1 # [ 108.091200] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:18Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05259worker1 # [ 108.098716] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:18Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15260worker1 # [ 108.137199] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:484425261worker1 # [ 108.154371] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:484425262client # [ 108.199631] nix[1162]: copying 1 paths to 'lb:50051'...5263client # [ 108.205996] nix[1162]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5264worker1 # [ 108.173647] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:484425265worker1 # [ 108.189309] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5266worker1 # [ 108.404197] nix-daemon[835]: accepted connection from pid 1404, user nix-grpc-daemon (trusted)5267worker1 # [ 108.423842] nix-daemon[835]: reaped child process 1407, status = succeeded5268worker1 # [ 108.600504] nix-daemon[835]: accepted connection from pid 1409, user nix-grpc-daemon (trusted)5269worker1 # [ 108.621934] nix-daemon[835]: reaped child process 1412, status = succeeded5270lb # [ 108.621476] niks3-server[1051]: 2026/09/20 12:48:19 INFO Received uploads request method=POST path=/api/pending_closures5271worker1 # [ 108.644674] nix-grpc-daemon[1276]: time=2026-09-20T12:48:19.166Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5272worker1 # [ 108.646613] nix-grpc-daemon[1276]: time=2026-09-20T12:48:19.168Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5273lb # [ 108.655212] niks3-server[1051]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5274worker1 # [ 108.682206] nix-grpc-daemon[1276]: time=2026-09-20T12:48:19.203Z level=INFO msg="Uploading 1 narinfos"5275lb # [ 108.672381] niks3-server[1051]: 2026/09/20 12:48:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5276lb # [ 108.683105] niks3-server[1051]: 2026/09/20 12:48:19 INFO Signed narinfos id=33 count=15277lb # [ 108.688829] niks3-server[1051]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5278lb # [ 108.697966] niks3-server[1051]: 2026/09/20 12:48:19 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5279lb # [ 108.702479] niks3-server[1051]: 2026/09/20 12:48:19 INFO Completed upload id=335280worker1 # [ 108.714719] nix-grpc-daemon[1276]: time=2026-09-20T12:48:19.236Z level=INFO msg="Upload complete. (508ms)"5281lb # [ 108.705364] niks3-server[1051]: 2026/09/20 12:48:19 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5282worker1 # [ 108.724395] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48442 duration_s=0 paths=1 nar_bytes_in=4165283worker1 # [ 108.733175] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:484425284worker1 # [ 108.747111] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5285worker1 # [ 108.753258] nix-daemon[835]: reaped child process 1402, status = succeeded5286client # [ 108.796354] nix[1162]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5287worker1 # [ 108.768455] nix-daemon[835]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5288lb: must succeed: curl -sf localhost:9901/clusters5289lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5290lb: must succeed: curl -sf localhost:9901/clusters5291lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5292lb: must succeed: curl -sf localhost:9901/clusters5293lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5294lb: must succeed: curl -sf localhost:9901/clusters5295lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5296worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5297worker2 # [ 110.835154] systemd-journald[267]: Received client request to rotate journal, rotating.5298worker2 # [ 110.856000] systemd-journald[267]: Vacuuming done, freed 0B of archived journals from /var/log/journal/379daaf79f3c404594fb77369a01d950.5299worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5300worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5301worker1 # [ 110.877700] systemd[1]: Stopping Nix worker-protocol over gRPC...5302worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5303??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5304 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735305client: must succeed: sleep 5.05306lb # [ 111.030588] envoy[913]: conn peer=[2001:db8:1::1]:47680 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9636 tx=3884 ms=78865307worker1 # [ 111.294857] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:21Z level=info event=draining5308client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5309worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5310worker1 # [ 116.424162] nix-daemon[835]: reaped child process 1416, status = succeeded5311worker1 # [ 116.609512] nix-daemon[835]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5312worker1 # [ 116.629143] nix-daemon[835]: reaped child process 1459, status = succeeded5313lb # [ 116.629085] niks3-server[1051]: 2026/09/20 12:48:27 INFO Received uploads request method=POST path=/api/pending_closures5314worker1 # [ 116.652488] nix-grpc-daemon[1276]: time=2026-09-20T12:48:27.173Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5315worker1 # [ 116.654172] nix-grpc-daemon[1276]: time=2026-09-20T12:48:27.173Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5316lb # [ 116.662794] niks3-server[1051]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5317lb # [ 116.688136] niks3-server[1051]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5318lb # [ 116.698192] niks3-server[1051]: 2026/09/20 12:48:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5319worker1 # [ 116.717154] nix-grpc-daemon[1276]: time=2026-09-20T12:48:27.238Z level=INFO msg="Uploading 1 narinfos"5320lb # [ 116.709003] niks3-server[1051]: 2026/09/20 12:48:27 INFO Signed narinfos id=34 count=15321lb # [ 116.711595] niks3-server[1051]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5322lb # [ 116.722335] niks3-server[1051]: 2026/09/20 12:48:27 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5323lb # [ 116.728470] niks3-server[1051]: 2026/09/20 12:48:27 INFO Completed upload id=345324worker1 # [ 116.740901] nix-grpc-daemon[1276]: time=2026-09-20T12:48:27.262Z level=INFO msg="Upload complete. (355ms)"5325lb # [ 116.730758] niks3-server[1051]: 2026/09/20 12:48:27 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5326worker1 # [ 116.757268] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48442 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15327worker1 # [ 116.774139] nix-daemon[835]: reaped child process 1414, status = succeeded5328client # [ 116.830195] systemd[1]: rs.service: Deactivated successfully.5329client # [ 116.834592] systemd[1]: rs.service: Consumed 165ms CPU time over 8.983s wall clock time, 38.9M memory peak, 8.5K incoming IP traffic, 9K outgoing IP traffic.5330worker1 # [ 117.296120] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:27Z level=info event=signal_exit5331worker1 # [ 117.517390] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_yield5332lb # [ 117.508259] niks3-server[1051]: 2026/09/20 12:48:28 INFO lead: released remote=192.168.1.3:602105333worker2 # [ 117.557505] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_restarting addr=lb:500515334worker1 # [ 117.528788] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:484305335worker1 # [ 117.535558] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=worker_gone addr=192.168.1.4:500515336worker1 # [ 117.539985] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_restarting addr=lb:500515337worker1 # [ 117.543972] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:484305338worker1 # [ 117.548380] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=worker_gone addr=192.168.1.3:500515339worker2 # [ 117.757924] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_connected addr=lb:500515340worker2 # [ 117.763564] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5341worker1 # [ 117.735477] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_connected addr=lb:500515342worker1 # [ 117.742877] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5343worker2 # [ 117.964856] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_connected addr=lb:500515344worker2 # [ 117.970196] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5345worker1 # [ 117.946179] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_connected addr=lb:500515346worker1 # [ 117.950429] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5347worker2 # [ 118.171470] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_connected addr=lb:500515348worker2 # [ 118.176888] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5349worker1 # [ 118.146610] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_connected addr=lb:500515350worker1 # [ 118.151980] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5351worker2 # [ 118.378203] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_connected addr=lb:500515352worker2 # [ 118.383475] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker1 # [ 118.352883] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_connected addr=lb:500515354worker1 # [ 118.358193] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.20 seconds)5356client: waiting for success: ! systemctl is-active rs5357worker1 # [ 118.523503] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:29Z level=info event=scheduler_restarting5358client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5359client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5360worker2 # [ 118.586021] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_connected addr=lb:500515361worker2 # [ 118.591308] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5362worker1 # [ 118.579227] nix-daemon[835]: reaped child process 1267, status = succeeded5363worker1 # [ 118.582285] nix-grpc-daemon[1206]: ts=2026-09-20T12:48:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5364client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5365client: must fail: journalctl -u rs | grep -q 'reconnecting'5366client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5367lb: must succeed: curl -sf localhost:9901/clusters5368lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5369worker2 # [ 118.792680] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_connected addr=lb:500515370worker2 # [ 118.798018] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5371worker1 # [ 118.799436] nix-grpc-daemon[1206]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5372worker1 # [ 118.804279] nix-grpc-daemon[1206]: E0000 00:00:1789908509.320902 1249 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5373worker1 # [ 118.810710] nix-grpc-daemon[1206]: E0000 00:00:1789908509.321174 1249 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5374worker2 # [ 118.999287] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_connected addr=lb:500515375worker2 # [ 119.004724] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376worker2 # [ 119.205860] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_connected addr=lb:500515377worker2 # [ 119.211182] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5378worker2 # [ 119.412549] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_connected addr=lb:500515379worker2 # [ 119.419060] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5380worker1 # [ 119.577027] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5381worker1 # [ 119.583544] systemd[1]: Stopped Nix worker-protocol over gRPC.5382worker2 # [ 119.622027] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515383worker1 # [ 119.589076] systemd[1]: nix-grpc-daemon.service: Consumed 1.855s CPU time over 27.512s wall clock time, 34.6M memory peak, 49.2K incoming IP traffic, 89.2K outgoing IP traffic.5384worker2 # [ 119.626243] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5385worker1 # [ 119.603637] systemd[1]: Starting Nix worker-protocol over gRPC...5386lb: must succeed: curl -sf localhost:9901/clusters5387worker2 # [ 119.823064] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515388worker2 # [ 119.828655] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5389lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5390worker1 # [ 119.809650] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5391worker1 # [ 119.812353] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5392worker1 # [ 119.816544] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15393worker1 # [ 119.824955] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5394worker1 # [ 119.826813] systemd[1]: Started Nix worker-protocol over gRPC.5395worker1 # [ 119.836901] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_take_over5396lb # [ 119.823599] niks3-server[1051]: 2026/09/20 12:48:30 INFO lead: acquired remote=192.168.1.3:598385397worker1 # [ 119.850921] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515398worker1 # [ 119.852904] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5399worker2 # [ 120.030046] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515400worker2 # [ 120.035930] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5401worker1 # [ 120.054181] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515402worker1 # [ 120.059804] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5403worker2 # [ 120.237210] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515404worker2 # [ 120.242652] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5405worker1 # [ 120.260759] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515406worker1 # [ 120.266411] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker2 # [ 120.443853] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515408worker2 # [ 120.449532] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5409worker1 # [ 120.467470] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_connected addr=lb:500515410worker1 # [ 120.473128] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker2 # [ 120.652153] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:31Z level=info event=scheduler_connected addr=lb:500515412worker1 # [ 120.632274] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=info event=oidc_jwks provider=mock keys=15413worker1 # [ 120.637416] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:329145414worker1 # [ 120.642823] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05415worker1 # [ 120.673887] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=info event=scheduler_connected addr=lb:500515416worker1 # [ 120.679453] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:329145417worker1 # [ 120.684517] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05418lb: must succeed: curl -sf localhost:9901/clusters5419lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5420worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5421worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5422(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.21 seconds)5423subtest: scheduler restart mid-build: build finishes, no double build5424client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5425client # [ 121.057940] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5426lb # [ 121.030685] envoy[913]: conn peer=[2001:db8:1::1]:36582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=1349 ms=86995427client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5428lb # [ 121.035878] envoy[913]: conn peer=[2001:db8:1::3]:60208 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4160 tx=1321 ms=262615429worker1 # [ 121.278743] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:602325430worker1 # [ 121.297092] nix-daemon[835]: accepted connection from pid 1466, user nix-grpc-daemon (trusted)5431client # Running as unit: mid.service; invocation ID: 9a313e19283545a3b9861c616a61a0175432client # [ 121.345414] nix[1197]: don't know how to build these paths:5433client # [ 121.349672] nix[1197]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5434worker1 # [ 121.326991] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05435worker1 # [ 121.333977] nix-grpc-daemon[1466]: ts=2026-09-20T12:48:31Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15436worker2 # [ 121.429318] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:515705437worker2 # [ 121.444641] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5438worker2 # [ 121.470933] nix-grpc-daemon[1178]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 1/5)5439worker2 # [ 121.537732] nix-grpc-daemon[1178]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 2/5)5440worker2 # [ 121.550250] nix-grpc-daemon[1178]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 236 ms (attempt 3/5)5441worker2 # [ 121.795133] nix-grpc-daemon[1178]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 4/5)5442worker2 # [ 121.942553] nix-grpc-daemon[1178]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5443worker2 # [ 121.965246] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:515705444client # [ 121.980204] nix[1197]: copying 1 paths to 'lb:50051'...5445worker2 # [ 121.982436] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:515705446client # [ 121.983344] nix[1197]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5447worker2 # [ 121.999720] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5448worker2 # [ 122.032809] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:32Z level=info event=niks3_push_started pid=12885449worker2 # [ 122.251544] nix-grpc-daemon[1288]: time=2026-09-20T12:48:32.764Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5450worker2 # [ 122.445433] nix-daemon[848]: accepted connection from pid 1301, user nix-grpc-daemon (trusted)5451worker2 # [ 122.464144] nix-daemon[848]: reaped child process 1308, status = succeeded5452worker2 # [ 122.641467] nix-daemon[848]: accepted connection from pid 1310, user nix-grpc-daemon (trusted)5453worker2 # [ 122.660573] nix-daemon[848]: reaped child process 1313, status = succeeded5454lb # [ 122.627373] niks3-server[1051]: 2026/09/20 12:48:33 INFO Received uploads request method=POST path=/api/pending_closures5455worker2 # [ 122.683935] nix-grpc-daemon[1288]: time=2026-09-20T12:48:33.196Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5456worker2 # [ 122.685862] nix-grpc-daemon[1288]: time=2026-09-20T12:48:33.196Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5457lb # [ 122.665550] niks3-server[1051]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5458worker2 # [ 122.729379] nix-grpc-daemon[1288]: time=2026-09-20T12:48:33.242Z level=INFO msg="Uploading 1 narinfos"5459lb # [ 122.683050] niks3-server[1051]: 2026/09/20 12:48:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5460lb # [ 122.694364] niks3-server[1051]: 2026/09/20 12:48:33 INFO Signed narinfos id=35 count=15461lb # [ 122.701201] niks3-server[1051]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5462lb # [ 122.714956] niks3-server[1051]: 2026/09/20 12:48:33 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5463lb # [ 122.720498] niks3-server[1051]: 2026/09/20 12:48:33 INFO Completed upload id=355464worker2 # [ 122.765324] nix-grpc-daemon[1288]: time=2026-09-20T12:48:33.279Z level=INFO msg="Upload complete. (509ms)"5465lb # [ 122.723457] niks3-server[1051]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5466worker2 # [ 122.777909] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51570 duration_s=0 paths=1 nar_bytes_in=4245467worker2 # [ 122.791525] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:515705468worker2 # [ 122.797789] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5469worker2 # [ 122.803073] nix-daemon[848]: reaped child process 1286, status = succeeded5470client # [ 122.813314] nix[1197]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5471worker2 # [ 122.814436] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5472lb: must succeed: curl -sf localhost:9901/clusters5473lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5474lb: must succeed: curl -sf localhost:9901/clusters5475lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5476worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5477worker1 # [ 125.049986] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1466 (nix-grpc-daemon) on client request.5478worker1 # [ 125.071329] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5479lb # [ 125.061900] niks3-server[1051]: 2026/09/20 12:48:35 INFO lead: released remote=192.168.1.3:598385480worker2 # [ 125.120163] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:35Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"5481client # [ 125.120651] nix[1197]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5482worker1 # [ 125.109361] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5483worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.13 seconds)5484client: must succeed: sleep 5.05485worker1 # [ 125.129189] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5486worker1 # [ 125.131067] systemd[1]: nix-grpc-daemon.service: Consumed 205ms CPU time over 5.510s wall clock time, 11.6M memory peak, 9.8K incoming IP traffic, 10K outgoing IP traffic.5487worker1 # [ 125.134639] nix-daemon[835]: reaped child process 1522, status = succeeded5488worker2 # [ 125.320530] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:35Z level=info event=scheduler_connected addr=lb:500515489worker1 # [ 125.292166] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5490worker1 # [ 125.306754] systemd[1]: Starting Nix worker-protocol over gRPC...5491worker1 # [ 125.522440] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5492worker1 # [ 125.525048] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5493worker1 # [ 125.529208] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15494worker1 # [ 125.537710] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5495worker1 # [ 125.539488] systemd[1]: Started Nix worker-protocol over gRPC.5496worker1 # [ 125.548939] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=scheduler_take_over5497lb # [ 125.536799] niks3-server[1051]: 2026/09/20 12:48:36 INFO lead: acquired remote=192.168.1.3:346425498worker1 # [ 125.566744] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=scheduler_connected addr=lb:500515499worker1 # [ 125.570793] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:602365500worker1 # [ 125.572493] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05501worker1 # [ 125.574187] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=oidc_jwks provider=mock keys=15502worker1 # [ 125.575435] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:602365503worker1 # [ 125.577351] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:36Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=15504client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5505client: waiting for success: ! systemctl is-active mid5506worker2 # [ 130.663005] nix-daemon[848]: reaped child process 1317, status = succeeded5507worker2 # [ 130.841410] nix-daemon[848]: accepted connection from pid 1353, user nix-grpc-daemon (trusted)5508worker2 # [ 130.860751] nix-daemon[848]: reaped child process 1356, status = succeeded5509lb # [ 130.827568] niks3-server[1051]: 2026/09/20 12:48:41 INFO Received uploads request method=POST path=/api/pending_closures5510worker2 # [ 130.887705] nix-grpc-daemon[1288]: time=2026-09-20T12:48:41.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5511worker2 # [ 130.889993] nix-grpc-daemon[1288]: time=2026-09-20T12:48:41.398Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5512lb # [ 130.869215] niks3-server[1051]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5513lb # [ 130.883078] niks3-server[1051]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5514lb # [ 130.896585] niks3-server[1051]: 2026/09/20 12:48:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5515worker2 # [ 130.947291] nix-grpc-daemon[1288]: time=2026-09-20T12:48:41.459Z level=INFO msg="Uploading 1 narinfos"5516lb # [ 130.906212] niks3-server[1051]: 2026/09/20 12:48:41 INFO Signed narinfos id=36 count=15517lb # [ 130.911105] niks3-server[1051]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5518lb # [ 130.919406] niks3-server[1051]: 2026/09/20 12:48:41 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5519lb # [ 130.928086] niks3-server[1051]: 2026/09/20 12:48:41 INFO Completed upload id=365520worker2 # [ 130.971955] nix-grpc-daemon[1288]: time=2026-09-20T12:48:41.485Z level=INFO msg="Upload complete. (337ms)"5521worker2 # [ 130.985149] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51570 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15522worker2 # [ 131.000325] nix-grpc-daemon[1288]: time=2026-09-20T12:48:41.506Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n"5523worker2 # [ 131.007014] nix-daemon[848]: reaped child process 1315, status = succeeded5524client # [ 131.026427] systemd[1]: mid.service: Deactivated successfully.5525client # [ 131.030768] systemd[1]: mid.service: Consumed 175ms CPU time over 9.972s wall clock time, 37M memory peak, 8.8K incoming IP traffic, 9.3K outgoing IP traffic.5526lb # [ 131.031653] envoy[913]: conn peer=[2001:db8:1::3]:39144 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=972 tx=362 ms=52445527lb # [ 131.037600] envoy[913]: conn peer=[2001:db8:1::1]:49302 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3035 tx=1424 ms=96925528client: (finished: waiting for success: ! systemctl is-active mid, in 1.15 seconds)5529client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5530client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5531(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.74 seconds)5532subtest: interrupting the client stops the build on the worker5533client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5534client # [ 131.772509] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5535client # Running as unit: intr.service; invocation ID: 1e0b92555f8f42dab2506ced0763320e5536client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5537client # [ 132.007720] nix[1232]: don't know how to build these paths:5538client # [ 132.008822] nix[1232]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5539worker2 # [ 132.008286] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:515705540worker1 # [ 131.998396] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:42Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05541worker1 # [ 132.004304] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:42Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15542worker1 # [ 132.046522] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005543worker1 # [ 132.062893] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5544worker1 # [ 132.089220] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)5545worker1 # [ 132.169765] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 105 ms (attempt 2/5)5546worker1 # [ 132.283023] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 3/5)5547worker1 # [ 132.449303] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 599 ms (attempt 4/5)5548worker1 # [ 133.058041] nix-grpc-daemon[1550]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5549worker1 # [ 133.081763] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005550client # [ 133.134163] nix[1232]: copying 1 paths to 'lb:50051'...5551worker1 # [ 133.105927] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421005552client # [ 133.139503] nix[1232]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5553worker1 # [ 133.125045] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5554worker1 # [ 133.156148] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:43Z level=info event=niks3_push_started pid=16245555worker1 # [ 133.387694] nix-grpc-daemon[1624]: time=2026-09-20T12:48:43.908Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5556worker1 # [ 133.569401] nix-daemon[835]: accepted connection from pid 1637, user nix-grpc-daemon (trusted)5557worker1 # [ 133.588835] nix-daemon[835]: reaped child process 1640, status = succeeded5558worker1 # [ 133.771769] nix-daemon[835]: accepted connection from pid 1642, user nix-grpc-daemon (trusted)5559worker1 # [ 133.793370] nix-daemon[835]: reaped child process 1645, status = succeeded5560lb # [ 133.794308] niks3-server[1051]: 2026/09/20 12:48:44 INFO Received uploads request method=POST path=/api/pending_closures5561worker1 # [ 133.829983] nix-grpc-daemon[1624]: time=2026-09-20T12:48:44.350Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5562worker1 # [ 133.831858] nix-grpc-daemon[1624]: time=2026-09-20T12:48:44.350Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5563lb # [ 133.864895] niks3-server[1051]: 2026/09/20 12:48:44 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5564worker1 # [ 133.887678] nix-grpc-daemon[1624]: time=2026-09-20T12:48:44.408Z level=INFO msg="Uploading 1 narinfos"5565lb # [ 133.875936] niks3-server[1051]: 2026/09/20 12:48:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5566lb # [ 133.887667] niks3-server[1051]: 2026/09/20 12:48:44 INFO Signed narinfos id=37 count=15567lb # [ 133.893936] niks3-server[1051]: 2026/09/20 12:48:44 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5568lb # [ 133.904469] niks3-server[1051]: 2026/09/20 12:48:44 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5569lb # [ 133.910223] niks3-server[1051]: 2026/09/20 12:48:44 INFO Completed upload id=375570worker1 # [ 133.922068] nix-grpc-daemon[1624]: time=2026-09-20T12:48:44.443Z level=INFO msg="Upload complete. (530ms)"5571lb # [ 133.913330] niks3-server[1051]: 2026/09/20 12:48:44 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5572worker1 # [ 133.932429] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=4245573worker1 # [ 133.952503] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005574worker1 # [ 133.962042] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5575worker1 # [ 133.968583] nix-daemon[835]: reaped child process 1622, status = succeeded5576client # [ 134.012808] nix[1232]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5577worker1 # [ 133.986077] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5578client: must succeed: systemctl kill -s INT intr5579client # [ 135.545078] systemd[1]: intr.service: Sent signal SIGINT to main process 1232 (nix) on client request.5580client # [ 135.552635] nix[1232]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5581client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5582client # [ 135.562599] nix[1232]: error: interrupted by the user5583client # [ 135.597698] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5584client # [ 135.602835] systemd[1]: intr.service: Failed with result 'exit-code'.5585client # [ 135.605171] systemd[1]: intr.service: Consumed 150ms CPU time over 3.828s wall clock time, 37.2M memory peak, 7.8K incoming IP traffic, 8.7K outgoing IP traffic.5586worker1 # [ 135.620583] nix-daemon[835]: reaped child process 1647, status = succeeded5587worker1 # [ 135.623866] nix-daemon[1649]: unexpected Nix daemon error: error: interrupted by the user5588worker1 # [ 135.632421] nix-daemon[835]: reaped child process 1649, status = failed with exit code 15589(finished: subtest: interrupting the client stops the build on the worker, in 5.19 seconds)5590subtest: a deploy mid-build drains: the build finishes, the new generation takes over5591client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5592client # [ 136.990172] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5593client # Running as unit: sw.service; invocation ID: 9bc7108dcdd848ad93ad448eb9f91f845594client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5595client # [ 137.251979] nix[1252]: don't know how to build these paths:5596client # [ 137.253009] nix[1252]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5597worker2 # [ 137.252977] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:515705598worker1 # [ 137.237081] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:47Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05599worker2 # [ 137.314343] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:515705600worker2 # [ 137.332684] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:515705601client # [ 137.348414] nix[1252]: copying 1 paths to 'lb:50051'...5602worker2 # [ 137.349985] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:515705603client # [ 137.350833] nix[1252]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5604worker2 # [ 137.368944] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5605worker2 # [ 137.580867] nix-daemon[848]: accepted connection from pid 1388, user nix-grpc-daemon (trusted)5606worker2 # [ 137.601269] nix-daemon[848]: reaped child process 1391, status = succeeded5607worker2 # [ 137.788121] nix-daemon[848]: accepted connection from pid 1393, user nix-grpc-daemon (trusted)5608worker2 # [ 137.808153] nix-daemon[848]: reaped child process 1396, status = succeeded5609lb # [ 137.775506] niks3-server[1051]: 2026/09/20 12:48:48 INFO Received uploads request method=POST path=/api/pending_closures5610worker2 # [ 137.832300] nix-grpc-daemon[1288]: time=2026-09-20T12:48:48.344Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5611worker2 # [ 137.833875] nix-grpc-daemon[1288]: time=2026-09-20T12:48:48.345Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5612lb # [ 137.810216] niks3-server[1051]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5613lb # [ 137.829132] niks3-server[1051]: 2026/09/20 12:48:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5614worker2 # [ 137.879097] nix-grpc-daemon[1288]: time=2026-09-20T12:48:48.392Z level=INFO msg="Uploading 1 narinfos"5615lb # [ 137.835059] niks3-server[1051]: 2026/09/20 12:48:48 INFO Signed narinfos id=38 count=15616lb # [ 137.842306] niks3-server[1051]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5617lb # [ 137.852937] niks3-server[1051]: 2026/09/20 12:48:48 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5618lb # [ 137.858416] niks3-server[1051]: 2026/09/20 12:48:48 INFO Completed upload id=385619worker2 # [ 137.903220] nix-grpc-daemon[1288]: time=2026-09-20T12:48:48.416Z level=INFO msg="Upload complete. (514ms)"5620worker1 # [ 137.243412] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:47Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15621lb # [ 137.861023] niks3-server[1051]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5622worker2 # [ 137.913523] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51570 duration_s=0 paths=1 nar_bytes_in=4165623worker2 # [ 137.928684] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:515705624worker2 # [ 137.936543] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5625worker2 # [ 137.942144] nix-daemon[848]: reaped child process 1386, status = succeeded5626client # [ 137.952962] nix[1252]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5627worker2 # [ 137.958617] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)5628worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25629worker2 # Checking switch inhibitors... done5630worker2 # activating the configuration...5631worker2 # [ 140.482780] nixos[1431]: switching to system configuration /nix/store/3ydbgxjx8wpxk0hzmy5qz34wrd2xncr6-nixos-system-worker2-test5632worker2 # [ 140.487421] systemd[1]: Stopped target Remote File Systems.5633worker2 # [ 140.489043] systemd[1]: Stopped target Local File Systems.5634worker2 # setting up /etc...5635lb # [ 141.031489] envoy[913]: conn peer=[2001:db8:1::1]:55468 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2944 tx=754 ms=35705636worker2 # [ 141.281202] systemd[1]: Reload requested from client PID 1431 ('.switch-to-conf') (unit backdoor.service)...5637worker2 # [ 141.283249] systemd[1]: Reloading...5638worker2 # [ 142.169623] systemd[1]: Reloading finished in 883 ms.5639worker2 # restarting sysinit-reactivation.target5640worker2 # [ 142.249879] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5641worker2 # [ 142.400466] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5642worker2 # [ 142.404489] systemd[1]: Reached target Reactivate sysinit units.5643worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5644worker2 # [ 142.409415] systemd[1]: Reloading D-Bus System Message Bus...5645worker2 # [ 142.421401] dbus-broker-launch[494]: Caught SIGHUP, trigger reload.5646worker2 # [ 142.423329] systemd[1]: Reloading Nix worker-protocol over gRPC...5647worker2 # [ 142.455797] dbus-broker-launch[494]: Looking up NSS user entry for 'systemd-timesync'...5648worker2 # [ 142.459185] dbus-broker-launch[494]: NSS returned no entry for 'systemd-timesync'5649worker2 # [ 142.461350] dbus-broker-launch[494]: Invalid user-name in /nix/store/c8a4kacp5lnfwrcxrjmwczbl8ba32m4a-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5650worker2 # [ 142.472731] systemd[1]: Reloaded Nix worker-protocol over gRPC.5651worker2 # [ 142.505582] systemd[1]: Reloaded D-Bus System Message Bus.5652worker2 # [ 142.517897] systemd[1]: Reached target Local File Systems.5653worker2 # [ 142.521576] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5654worker2 # [ 142.527818] systemd[1]: Starting Create SUID/SGID Wrappers...5655worker2 # [ 142.532357] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5656worker2 # [ 142.537038] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675657worker2 # [ 142.547915] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5658worker2 # [ 142.549682] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5659worker2 # [ 142.550886] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5660worker2 # [ 142.556192] systemd[1]: Reached target Remote File Systems.5661worker2 # [ 142.556996] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5662worker2 # [ 142.558211] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5663worker2 # [ 143.035529] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5664worker2 # [ 143.040846] systemd[1]: Finished Create SUID/SGID Wrappers.5665worker2 # [ 143.042850] systemd[1]: Starting linger-users.service...5666worker2 # [ 143.088596] systemd[1]: linger-users.service: Deactivated successfully.5667worker2 # [ 143.092067] systemd[1]: Finished linger-users.service.5668worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5669worker2 # [ 143.347344] nixos[1431]: finished switching to system configuration /nix/store/3ydbgxjx8wpxk0hzmy5qz34wrd2xncr6-nixos-system-worker2-test5670worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.70 seconds)5671worker2: must succeed: pgrep -f 'read -t [9]0 x'5672worker2 # [ 143.384743] nix-grpc-daemon[1178]: ts=2026-09-20T12:48:53Z level=info event=draining5673worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.07 seconds)5674client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25675client # don't know how to build these paths:5676client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5677worker1 # [ 143.647870] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:421005678worker1 # [ 143.656359] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005679worker1 # [ 143.681914] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05680worker1 # [ 143.689349] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15681worker1 # [ 143.733564] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005682worker1 # [ 143.754262] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005683client # copying 1 paths to 'lb:50051'...5684worker1 # [ 143.771613] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421005685client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5686worker1 # [ 143.795631] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5687worker1 # [ 144.010439] nix-daemon[835]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)5688worker1 # [ 144.030797] nix-daemon[835]: reaped child process 1706, status = succeeded5689worker1 # [ 144.209552] nix-daemon[835]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)5690worker1 # [ 144.228597] nix-daemon[835]: reaped child process 1711, status = succeeded5691lb # [ 144.232594] niks3-server[1051]: 2026/09/20 12:48:54 INFO Received uploads request method=POST path=/api/pending_closures5692worker1 # [ 144.256484] nix-grpc-daemon[1624]: time=2026-09-20T12:48:54.777Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5693worker1 # [ 144.258025] nix-grpc-daemon[1624]: time=2026-09-20T12:48:54.777Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5694lb # [ 144.263404] niks3-server[1051]: 2026/09/20 12:48:54 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5695worker1 # [ 144.295835] nix-grpc-daemon[1624]: time=2026-09-20T12:48:54.817Z level=INFO msg="Uploading 1 narinfos"5696lb # [ 144.277563] niks3-server[1051]: 2026/09/20 12:48:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5697lb # [ 144.286258] niks3-server[1051]: 2026/09/20 12:48:54 INFO Signed narinfos id=39 count=15698lb # [ 144.294580] niks3-server[1051]: 2026/09/20 12:48:54 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5699lb # [ 144.306112] niks3-server[1051]: 2026/09/20 12:48:54 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5700lb # [ 144.315672] niks3-server[1051]: 2026/09/20 12:48:54 INFO Completed upload id=395701worker1 # [ 144.328155] nix-grpc-daemon[1624]: time=2026-09-20T12:48:54.850Z level=INFO msg="Upload complete. (514ms)"5702lb # [ 144.318786] niks3-server[1051]: 2026/09/20 12:48:54 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5703worker1 # [ 144.338925] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=5285704worker1 # [ 144.348755] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005705worker1 # [ 144.362025] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5706worker1 # [ 144.365083] nix-daemon[835]: reaped child process 1701, status = succeeded5707client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5708client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5709worker1 # [ 144.379106] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5710client # farm-a> LOG-farm-a-during-drain5711worker1 # [ 145.612305] nix-daemon[835]: reaped child process 1715, status = succeeded5712worker1 # [ 145.788293] nix-daemon[835]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)5713worker1 # [ 145.805837] nix-daemon[835]: reaped child process 1733, status = succeeded5714lb # [ 145.807338] niks3-server[1051]: 2026/09/20 12:48:56 INFO Received uploads request method=POST path=/api/pending_closures5715worker1 # [ 145.832484] nix-grpc-daemon[1624]: time=2026-09-20T12:48:56.350Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5716worker1 # [ 145.834223] nix-grpc-daemon[1624]: time=2026-09-20T12:48:56.350Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5717lb # [ 145.836154] niks3-server[1051]: 2026/09/20 12:48:56 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5718lb # [ 145.855048] niks3-server[1051]: 2026/09/20 12:48:56 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5719lb # [ 145.870103] niks3-server[1051]: 2026/09/20 12:48:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5720worker1 # [ 145.891649] nix-grpc-daemon[1624]: time=2026-09-20T12:48:56.412Z level=INFO msg="Uploading 1 narinfos"5721lb # [ 145.879172] niks3-server[1051]: 2026/09/20 12:48:56 INFO Signed narinfos id=40 count=15722lb # [ 145.885924] niks3-server[1051]: 2026/09/20 12:48:56 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5723lb # [ 145.898885] niks3-server[1051]: 2026/09/20 12:48:56 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5724lb # [ 145.908516] niks3-server[1051]: 2026/09/20 12:48:56 INFO Completed upload id=405725lb # [ 145.911566] niks3-server[1051]: 2026/09/20 12:48:56 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5726worker1 # [ 145.920704] nix-grpc-daemon[1624]: time=2026-09-20T12:48:56.442Z level=INFO msg="Upload complete. (330ms)"5727worker1 # [ 145.939350] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=15728worker1 # [ 145.956310] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:56Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05729worker1 # [ 145.959960] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:56Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15730worker1 # [ 145.964708] nix-daemon[835]: reaped child process 1713, status = succeeded5731worker1 # [ 145.991024] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005732worker1 # [ 146.006078] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005733worker1 # [ 146.020335] nix-grpc-daemon[1624]: time=2026-09-20T12:48:56.542Z level=INFO msg="All 1 paths already cached"5734client # copying 1 paths to 'lb:50051'...5735worker1 # [ 146.030930] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421005736client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5737worker1 # [ 146.054873] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5738worker1 # [ 146.263325] nix-daemon[835]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)5739worker1 # [ 146.280994] nix-daemon[835]: reaped child process 1742, status = succeeded5740worker1 # [ 146.453317] nix-daemon[835]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)5741worker1 # [ 146.471284] nix-daemon[835]: reaped child process 1747, status = succeeded5742lb # [ 146.471025] niks3-server[1051]: 2026/09/20 12:48:56 INFO Received uploads request method=POST path=/api/pending_closures5743worker1 # [ 146.492791] nix-grpc-daemon[1624]: time=2026-09-20T12:48:57.014Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5744worker1 # [ 146.494268] nix-grpc-daemon[1624]: time=2026-09-20T12:48:57.014Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5745lb # [ 146.500186] niks3-server[1051]: 2026/09/20 12:48:57 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5746lb # [ 146.512265] niks3-server[1051]: 2026/09/20 12:48:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5747worker1 # [ 146.531336] nix-grpc-daemon[1624]: time=2026-09-20T12:48:57.053Z level=INFO msg="Uploading 1 narinfos"5748lb # [ 146.523098] niks3-server[1051]: 2026/09/20 12:48:57 INFO Signed narinfos id=41 count=15749lb # [ 146.530183] niks3-server[1051]: 2026/09/20 12:48:57 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5750lb # [ 146.539334] niks3-server[1051]: 2026/09/20 12:48:57 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5751lb # [ 146.546476] niks3-server[1051]: 2026/09/20 12:48:57 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5752lb # [ 146.552081] niks3-server[1051]: 2026/09/20 12:48:57 INFO Completed upload id=415753worker1 # [ 146.564526] nix-grpc-daemon[1624]: time=2026-09-20T12:48:57.086Z level=INFO msg="Upload complete. (490ms)"5754worker1 # [ 146.573168] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=6405755worker1 # [ 146.588496] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005756worker1 # [ 146.594292] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5757worker1 # [ 146.599057] nix-daemon[835]: reaped child process 1737, status = succeeded5758client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5759client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5760worker1 # [ 146.614421] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5761worker1 # [ 147.836556] nix-daemon[835]: reaped child process 1751, status = succeeded5762client # farm-b> LOG-farm-b-during-drain5763worker1 # [ 147.997041] nix-daemon[835]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)5764worker1 # [ 148.017807] nix-daemon[835]: reaped child process 1769, status = succeeded5765lb # [ 148.017354] niks3-server[1051]: 2026/09/20 12:48:58 INFO Received uploads request method=POST path=/api/pending_closures5766worker1 # [ 148.040408] nix-grpc-daemon[1624]: time=2026-09-20T12:48:58.561Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5767worker1 # [ 148.041918] nix-grpc-daemon[1624]: time=2026-09-20T12:48:58.561Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5768lb # [ 148.050977] niks3-server[1051]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5769lb # [ 148.064613] niks3-server[1051]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5770lb # [ 148.076991] niks3-server[1051]: 2026/09/20 12:48:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5771worker1 # [ 148.094602] nix-grpc-daemon[1624]: time=2026-09-20T12:48:58.616Z level=INFO msg="Uploading 1 narinfos"5772lb # [ 148.088771] niks3-server[1051]: 2026/09/20 12:48:58 INFO Signed narinfos id=42 count=15773lb # [ 148.098179] niks3-server[1051]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5774lb # [ 148.110415] niks3-server[1051]: 2026/09/20 12:48:58 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5775lb # [ 148.115809] niks3-server[1051]: 2026/09/20 12:48:58 INFO Completed upload id=425776worker1 # [ 148.128166] nix-grpc-daemon[1624]: time=2026-09-20T12:48:58.650Z level=INFO msg="Upload complete. (302ms)"5777lb # [ 148.118681] niks3-server[1051]: 2026/09/20 12:48:58 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5778worker1 # [ 148.145904] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=15779worker1 # [ 148.162493] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:58Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05780worker1 # [ 148.166496] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:58Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15781worker1 # [ 148.170731] nix-daemon[835]: reaped child process 1749, status = succeeded5782worker1 # [ 148.188808] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005783worker1 # [ 148.204906] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005784worker1 # [ 148.220332] nix-grpc-daemon[1624]: time=2026-09-20T12:48:58.742Z level=INFO msg="All 2 paths already cached"5785client # copying 1 paths to 'lb:50051'...5786client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5787worker1 # [ 148.230894] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421005788worker1 # [ 148.248824] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5789worker1 # [ 148.464230] nix-daemon[835]: accepted connection from pid 1775, user nix-grpc-daemon (trusted)5790worker1 # [ 148.482332] nix-daemon[835]: reaped child process 1778, status = succeeded5791worker1 # [ 148.660528] nix-daemon[835]: accepted connection from pid 1780, user nix-grpc-daemon (trusted)5792worker1 # [ 148.678540] nix-daemon[835]: reaped child process 1783, status = succeeded5793lb # [ 148.678714] niks3-server[1051]: 2026/09/20 12:48:59 INFO Received uploads request method=POST path=/api/pending_closures5794worker1 # [ 148.701355] nix-grpc-daemon[1624]: time=2026-09-20T12:48:59.223Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5795worker1 # [ 148.702873] nix-grpc-daemon[1624]: time=2026-09-20T12:48:59.223Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5796lb # [ 148.708810] niks3-server[1051]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5797lb # [ 148.721344] niks3-server[1051]: 2026/09/20 12:48:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5798worker1 # [ 148.741524] nix-grpc-daemon[1624]: time=2026-09-20T12:48:59.263Z level=INFO msg="Uploading 1 narinfos"5799lb # [ 148.732917] niks3-server[1051]: 2026/09/20 12:48:59 INFO Signed narinfos id=43 count=15800lb # [ 148.739873] niks3-server[1051]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5801lb # [ 148.747655] niks3-server[1051]: 2026/09/20 12:48:59 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5802lb # [ 148.755464] niks3-server[1051]: 2026/09/20 12:48:59 INFO Completed upload id=435803worker1 # [ 148.767595] nix-grpc-daemon[1624]: time=2026-09-20T12:48:59.289Z level=INFO msg="Upload complete. (501ms)"5804lb # [ 148.757650] niks3-server[1051]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5805worker1 # [ 148.777028] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=7685806worker1 # [ 148.790253] nix-grpc-daemon[1550]: ts=2026-09-20T12:48:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005807worker1 # [ 148.795945] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5808worker1 # [ 148.800847] nix-daemon[835]: reaped child process 1773, status = succeeded5809client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5810client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5811worker1 # [ 148.813928] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5812client # farm-top> LOG-farm-top-during-drain5813worker1 # [ 149.860421] nix-daemon[835]: reaped child process 1787, status = succeeded5814worker1 # [ 150.035141] nix-daemon[835]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)5815worker1 # [ 150.053404] nix-daemon[835]: reaped child process 1805, status = succeeded5816lb # [ 150.053383] niks3-server[1051]: 2026/09/20 12:49:00 INFO Received uploads request method=POST path=/api/pending_closures5817worker1 # [ 150.076419] nix-grpc-daemon[1624]: time=2026-09-20T12:49:00.597Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5818worker1 # [ 150.077899] nix-grpc-daemon[1624]: time=2026-09-20T12:49:00.597Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5819lb # [ 150.092629] niks3-server[1051]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5820lb # [ 150.104424] niks3-server[1051]: 2026/09/20 12:49:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5821worker1 # [ 150.122819] nix-grpc-daemon[1624]: time=2026-09-20T12:49:00.644Z level=INFO msg="Uploading 1 narinfos"5822lb # [ 150.116767] niks3-server[1051]: 2026/09/20 12:49:00 INFO Signed narinfos id=44 count=15823lb # [ 150.123794] niks3-server[1051]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5824lb # [ 150.139004] niks3-server[1051]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5825lb # [ 150.142900] niks3-server[1051]: 2026/09/20 12:49:00 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5826lb # [ 150.148159] niks3-server[1051]: 2026/09/20 12:49:00 INFO Completed upload id=445827worker1 # [ 150.159714] nix-grpc-daemon[1624]: time=2026-09-20T12:49:00.681Z level=INFO msg="Upload complete. (334ms)"5828lb # [ 150.150915] niks3-server[1051]: 2026/09/20 12:49:00 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5829worker1 # [ 150.173065] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=15830worker1 # [ 150.189906] nix-daemon[835]: reaped child process 1785, status = succeeded5831client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.83 seconds)5832client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5833client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.24 seconds)5834worker2: must succeed: pkill -f 'read -t [9]0 x'5835worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5836client: waiting for success: ! systemctl is-active sw5837worker2 # [ 150.593157] nix-daemon[848]: reaped child process 1400, status = succeeded5838worker2 # [ 150.754528] nix-daemon[848]: accepted connection from pid 1571, user nix-grpc-daemon (trusted)5839worker2 # [ 150.772736] nix-daemon[848]: reaped child process 1574, status = succeeded5840lb # [ 150.739040] niks3-server[1051]: 2026/09/20 12:49:01 INFO Received uploads request method=POST path=/api/pending_closures5841worker2 # [ 150.793757] nix-grpc-daemon[1288]: time=2026-09-20T12:49:01.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5842worker2 # [ 150.796091] nix-grpc-daemon[1288]: time=2026-09-20T12:49:01.309Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5843lb # [ 150.774981] niks3-server[1051]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5844lb # [ 150.791181] niks3-server[1051]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5845lb # [ 150.799207] niks3-server[1051]: 2026/09/20 12:49:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5846worker2 # [ 150.850947] nix-grpc-daemon[1288]: time=2026-09-20T12:49:01.364Z level=INFO msg="Uploading 1 narinfos"5847lb # [ 150.805457] niks3-server[1051]: 2026/09/20 12:49:01 INFO Signed narinfos id=45 count=15848lb # [ 150.812196] niks3-server[1051]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5849lb # [ 150.822693] niks3-server[1051]: 2026/09/20 12:49:01 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5850lb # [ 150.832321] niks3-server[1051]: 2026/09/20 12:49:01 INFO Completed upload id=455851worker2 # [ 150.877667] nix-grpc-daemon[1288]: time=2026-09-20T12:49:01.391Z level=INFO msg="Upload complete. (296ms)"5852lb # [ 150.835739] niks3-server[1051]: 2026/09/20 12:49:01 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5853worker2 # [ 150.889100] nix-grpc-daemon[1178]: ts=2026-09-20T12:49:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51570 duration_s=12 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=15854worker2 # [ 150.905863] nix-daemon[848]: reaped child process 1398, status = succeeded5855client # [ 150.931552] systemd[1]: sw.service: Deactivated successfully.5856client # [ 150.936485] systemd[1]: sw.service: Consumed 169ms CPU time over 13.945s wall clock time, 39M memory peak, 8.3K incoming IP traffic, 8.8K outgoing IP traffic.5857lb # [ 151.037378] envoy[913]: conn peer=[2001:db8:1::1]:33128 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9582 tx=3849 ms=65445858lb # [ 151.042385] envoy[913]: conn peer=[2001:db8:1::1]:55474 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2922 tx=1335 ms=136545859worker2 # [ 151.388587] nix-grpc-daemon[1178]: ts=2026-09-20T12:49:01Z level=info event=signal_exit5860client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)5861client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5862client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)5863worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5864worker2 # [ 151.829921] systemd[1]: Starting scheduler-token.service...5865worker2 # [ 151.910472] systemd[1]: scheduler-token.service: Deactivated successfully.5866worker2 # [ 151.912921] systemd[1]: Finished scheduler-token.service.5867worker2 # [ 151.913802] systemd[1]: scheduler-token.service: Consumed 30ms CPU time over 82ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5868worker2 # [ 152.622682] nix-grpc-daemon[1178]: ts=2026-09-20T12:49:03Z level=info event=scheduler_restarting5869worker1 # [ 152.600607] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:03Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:602365870worker1 # [ 152.600918] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:03Z level=info event=worker_gone addr=192.168.1.4:500515871worker2 # [ 152.650450] nix-grpc-daemon[1178]: ts=2026-09-20T12:49:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5872worker2 # [ 152.675654] nix-daemon[848]: reaped child process 1281, status = succeeded5873worker2 # [ 153.459709] nix-grpc-daemon[1178]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5874worker2 # [ 153.464406] nix-grpc-daemon[1178]: E0000 00:00:1789908543.973204 1221 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true5875worker2 # [ 153.468720] nix-grpc-daemon[1178]: E0000 00:00:1789908543.982331 1221 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5876worker2 # [ 153.863846] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5877worker2 # [ 153.871171] systemd[1]: nix-grpc-daemon.service: Consumed 1.155s CPU time over 46.696s wall clock time, 31.2M memory peak, 27.7K incoming IP traffic, 50.8K outgoing IP traffic.5878worker2 # [ 153.892452] systemd[1]: Starting Nix worker-protocol over gRPC...5879worker2 # [ 154.087239] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5880worker2 # [ 154.089852] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5881worker2 # [ 154.094009] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15882worker2 # [ 154.101979] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5883worker2 # [ 154.103729] systemd[1]: Started Nix worker-protocol over gRPC.5884worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.38 seconds)5885lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5886lb: must succeed: curl -sf localhost:9901/clusters5887worker2 # [ 154.124894] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515888worker1 # [ 154.097846] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:04Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:602365889worker1 # [ 154.104144] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05890lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5891lb: must succeed: curl -sf localhost:9901/clusters5892lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5893lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.18 seconds)5894lb: must succeed: curl -sf localhost:9901/clusters5895lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5896worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5897worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5898(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.51 seconds)5899subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5900client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15901worker2 # [ 155.677680] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428425902worker2 # [ 155.691890] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)5903worker1 # [ 155.681843] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005904worker1 # [ 155.730305] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05905worker1 # [ 155.737369] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15906worker1 # [ 155.744293] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05907worker1 # [ 155.749554] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05908worker1 # [ 155.776994] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005909worker1 # [ 155.793498] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005910worker1 # [ 155.808612] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421005911worker1 # [ 155.824492] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5912worker1 # [ 156.034720] nix-daemon[835]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)5913worker1 # [ 156.053819] nix-daemon[835]: reaped child process 1817, status = succeeded5914worker1 # [ 156.225521] nix-daemon[835]: accepted connection from pid 1819, user nix-grpc-daemon (trusted)5915worker1 # [ 156.245262] nix-daemon[835]: reaped child process 1822, status = succeeded5916lb # [ 156.247931] niks3-server[1051]: 2026/09/20 12:49:06 INFO Received uploads request method=POST path=/api/pending_closures5917worker1 # [ 156.269856] nix-grpc-daemon[1624]: time=2026-09-20T12:49:06.791Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5918worker1 # [ 156.271349] nix-grpc-daemon[1624]: time=2026-09-20T12:49:06.791Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5919lb # [ 156.275850] niks3-server[1051]: 2026/09/20 12:49:06 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5920lb # [ 156.295040] niks3-server[1051]: 2026/09/20 12:49:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5921worker1 # [ 156.313466] nix-grpc-daemon[1624]: time=2026-09-20T12:49:06.835Z level=INFO msg="Uploading 1 narinfos"5922lb # [ 156.308458] niks3-server[1051]: 2026/09/20 12:49:06 INFO Signed narinfos id=46 count=15923lb # [ 156.315572] niks3-server[1051]: 2026/09/20 12:49:06 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5924lb # [ 156.327289] niks3-server[1051]: 2026/09/20 12:49:06 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5925lb # [ 156.332217] niks3-server[1051]: 2026/09/20 12:49:06 INFO Completed upload id=465926worker1 # [ 156.344267] nix-grpc-daemon[1624]: time=2026-09-20T12:49:06.866Z level=INFO msg="Upload complete. (502ms)"5927lb # [ 156.335998] niks3-server[1051]: 2026/09/20 12:49:06 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5928worker1 # [ 156.357097] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=5125929worker1 # [ 156.368356] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005930worker1 # [ 156.380172] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5931worker1 # [ 156.383169] nix-daemon[835]: reaped child process 1812, status = succeeded5932worker1 # [ 156.398032] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5933worker1 # [ 157.428797] nix-daemon[835]: reaped child process 1826, status = succeeded5934worker1 # [ 157.601227] nix-daemon[835]: accepted connection from pid 1841, user nix-grpc-daemon (trusted)5935worker1 # [ 157.619372] nix-daemon[835]: reaped child process 1844, status = succeeded5936lb # [ 157.619693] niks3-server[1051]: 2026/09/20 12:49:08 INFO Received uploads request method=POST path=/api/pending_closures5937worker1 # [ 157.641364] nix-grpc-daemon[1624]: time=2026-09-20T12:49:08.163Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5938worker1 # [ 157.645890] nix-grpc-daemon[1624]: time=2026-09-20T12:49:08.165Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5939lb # [ 157.655366] niks3-server[1051]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5940lb # [ 157.668415] niks3-server[1051]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5941worker1 # [ 157.697243] nix-grpc-daemon[1624]: time=2026-09-20T12:49:08.218Z level=INFO msg="Uploading 1 narinfos"5942lb # [ 157.684499] niks3-server[1051]: 2026/09/20 12:49:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5943lb # [ 157.695414] niks3-server[1051]: 2026/09/20 12:49:08 INFO Signed narinfos id=47 count=15944lb # [ 157.701888] niks3-server[1051]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5945lb # [ 157.710354] niks3-server[1051]: 2026/09/20 12:49:08 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5946lb # [ 157.717003] niks3-server[1051]: 2026/09/20 12:49:08 INFO Completed upload id=475947worker1 # [ 157.729519] nix-grpc-daemon[1624]: time=2026-09-20T12:49:08.251Z level=INFO msg="Upload complete. (320ms)"5948lb # [ 157.720810] niks3-server[1051]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5949worker1 # [ 157.747927] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=15950worker1 # [ 157.764248] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:08Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15951worker1 # [ 157.775943] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005952worker1 # [ 157.781276] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:08Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05953worker1 # [ 157.789358] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005954worker1 # [ 157.799288] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421005955worker1 # [ 157.804124] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5956worker1 # [ 157.808357] nix-daemon[835]: reaped child process 1824, status = succeeded5957worker1 # [ 157.985473] nix-daemon[835]: accepted connection from pid 1850, user nix-grpc-daemon (trusted)5958worker1 # [ 158.003425] nix-daemon[835]: reaped child process 1853, status = succeeded5959worker1 # [ 158.176550] nix-daemon[835]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)5960worker1 # [ 158.193880] nix-daemon[835]: reaped child process 1858, status = succeeded5961lb # [ 158.194610] niks3-server[1051]: 2026/09/20 12:49:08 INFO Received uploads request method=POST path=/api/pending_closures5962worker1 # [ 158.217508] nix-grpc-daemon[1624]: time=2026-09-20T12:49:08.739Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5963worker1 # [ 158.219159] nix-grpc-daemon[1624]: time=2026-09-20T12:49:08.739Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5964lb # [ 158.224103] niks3-server[1051]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5965worker1 # [ 158.249965] nix-grpc-daemon[1624]: time=2026-09-20T12:49:08.771Z level=INFO msg="Uploading 1 narinfos"5966lb # [ 158.241014] niks3-server[1051]: 2026/09/20 12:49:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5967lb # [ 158.250892] niks3-server[1051]: 2026/09/20 12:49:08 INFO Signed narinfos id=48 count=15968lb # [ 158.256281] niks3-server[1051]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5969lb # [ 158.265103] niks3-server[1051]: 2026/09/20 12:49:08 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5970lb # [ 158.270062] niks3-server[1051]: 2026/09/20 12:49:08 INFO Completed upload id=485971worker1 # [ 158.282426] nix-grpc-daemon[1624]: time=2026-09-20T12:49:08.804Z level=INFO msg="Upload complete. (474ms)"5972lb # [ 158.272925] niks3-server[1051]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5973worker1 # [ 158.292530] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=5125974worker1 # [ 158.300653] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005975worker1 # [ 158.310911] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5976worker1 # [ 158.315960] nix-daemon[835]: reaped child process 1848, status = succeeded5977worker1 # [ 158.328268] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5978worker1 # [ 159.558627] nix-daemon[835]: reaped child process 1862, status = succeeded5979worker1 # [ 159.721118] nix-daemon[835]: accepted connection from pid 1877, user nix-grpc-daemon (trusted)5980worker1 # [ 159.741045] nix-daemon[835]: reaped child process 1880, status = succeeded5981lb # [ 159.741664] niks3-server[1051]: 2026/09/20 12:49:10 INFO Received uploads request method=POST path=/api/pending_closures5982worker1 # [ 159.764402] nix-grpc-daemon[1624]: time=2026-09-20T12:49:10.284Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5983worker1 # [ 159.765881] nix-grpc-daemon[1624]: time=2026-09-20T12:49:10.284Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5984lb # [ 159.778796] niks3-server[1051]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5985lb # [ 159.791079] niks3-server[1051]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5986worker1 # [ 159.823669] nix-grpc-daemon[1624]: time=2026-09-20T12:49:10.345Z level=INFO msg="Uploading 1 narinfos"5987lb # [ 159.805460] niks3-server[1051]: 2026/09/20 12:49:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5988lb # [ 159.815831] niks3-server[1051]: 2026/09/20 12:49:10 INFO Signed narinfos id=49 count=15989lb # [ 159.821772] niks3-server[1051]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5990lb # [ 159.829858] niks3-server[1051]: 2026/09/20 12:49:10 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5991lb # [ 159.840511] niks3-server[1051]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo5992lb # [ 159.845856] niks3-server[1051]: 2026/09/20 12:49:10 INFO Completed upload id=495993worker1 # [ 159.858257] nix-grpc-daemon[1624]: time=2026-09-20T12:49:10.380Z level=INFO msg="Upload complete. (318ms)"5994worker1 # [ 159.875793] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=15995worker1 # [ 159.895031] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:10Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=15996worker1 # [ 159.899539] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421005997worker1 # [ 159.904616] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:10Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=05998worker1 # [ 159.909965] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421005999worker1 # [ 159.914910] nix-daemon[835]: reaped child process 1860, status = succeeded6000worker1 # [ 159.918680] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421006001worker1 # [ 159.928163] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6002worker1 # [ 160.115612] nix-daemon[835]: accepted connection from pid 1886, user nix-grpc-daemon (trusted)6003worker1 # [ 160.133113] nix-daemon[835]: reaped child process 1889, status = succeeded6004worker1 # [ 160.310312] nix-daemon[835]: accepted connection from pid 1891, user nix-grpc-daemon (trusted)6005worker1 # [ 160.328552] nix-daemon[835]: reaped child process 1894, status = succeeded6006lb # [ 160.327501] niks3-server[1051]: 2026/09/20 12:49:10 INFO Received uploads request method=POST path=/api/pending_closures6007worker1 # [ 160.348665] nix-grpc-daemon[1624]: time=2026-09-20T12:49:10.870Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6008worker1 # [ 160.350143] nix-grpc-daemon[1624]: time=2026-09-20T12:49:10.870Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6009lb # [ 160.357587] niks3-server[1051]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6010worker1 # [ 160.378611] nix-grpc-daemon[1624]: time=2026-09-20T12:49:10.900Z level=INFO msg="Uploading 1 narinfos"6011lb # [ 160.371278] niks3-server[1051]: 2026/09/20 12:49:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6012lb # [ 160.381956] niks3-server[1051]: 2026/09/20 12:49:10 INFO Signed narinfos id=50 count=16013lb # [ 160.386160] niks3-server[1051]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6014worker1 # [ 160.406771] nix-grpc-daemon[1624]: time=2026-09-20T12:49:10.928Z level=INFO msg="Upload complete. (468ms)"6015lb # [ 160.395109] niks3-server[1051]: 2026/09/20 12:49:10 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6016lb # [ 160.399893] niks3-server[1051]: 2026/09/20 12:49:10 INFO Completed upload id=506017lb # [ 160.402846] niks3-server[1051]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6018worker1 # [ 160.415770] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=5126019worker1 # [ 160.426182] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006020worker1 # [ 160.431357] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6021worker1 # [ 160.435690] nix-daemon[835]: reaped child process 1884, status = succeeded6022worker1 # [ 160.447517] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6023lb # [ 161.035959] envoy[913]: conn peer=[2001:db8:1::4]:38566 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=13762 tx=958 ms=452486024worker1 # [ 161.685480] nix-daemon[835]: reaped child process 1898, status = succeeded6025worker1 # [ 161.855154] nix-daemon[835]: accepted connection from pid 1913, user nix-grpc-daemon (trusted)6026worker1 # [ 161.875204] nix-daemon[835]: reaped child process 1916, status = succeeded6027lb # [ 161.875208] niks3-server[1051]: 2026/09/20 12:49:12 INFO Received uploads request method=POST path=/api/pending_closures6028worker1 # [ 161.898964] nix-grpc-daemon[1624]: time=2026-09-20T12:49:12.419Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6029worker1 # [ 161.900597] nix-grpc-daemon[1624]: time=2026-09-20T12:49:12.419Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6030lb # [ 161.908402] niks3-server[1051]: 2026/09/20 12:49:12 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6031lb # [ 161.929579] niks3-server[1051]: 2026/09/20 12:49:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6032lb # [ 161.942977] niks3-server[1051]: 2026/09/20 12:49:12 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6033lb # [ 161.951315] niks3-server[1051]: 2026/09/20 12:49:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6034lb # [ 161.957370] niks3-server[1051]: 2026/09/20 12:49:12 INFO Signed narinfos id=51 count=16035worker1 # [ 161.970042] nix-grpc-daemon[1624]: time=2026-09-20T12:49:12.491Z level=INFO msg="Uploading 1 narinfos"6036lb # [ 161.964536] niks3-server[1051]: 2026/09/20 12:49:12 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6037lb # [ 161.971400] niks3-server[1051]: 2026/09/20 12:49:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6038lb # [ 161.982520] niks3-server[1051]: 2026/09/20 12:49:12 INFO Completed upload id=516039worker1 # [ 161.995022] nix-grpc-daemon[1624]: time=2026-09-20T12:49:12.516Z level=INFO msg="Upload complete. (337ms)"6040worker1 # [ 162.009255] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=16041worker1 # [ 162.026839] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:12Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16042worker1 # [ 162.032736] nix-daemon[835]: reaped child process 1896, status = succeeded6043worker1 # [ 162.049368] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006044worker1 # [ 162.066101] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421006045worker1 # [ 162.074134] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:12Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06046worker1 # [ 162.085657] nix-grpc-daemon[1624]: time=2026-09-20T12:49:12.607Z level=INFO msg="All 1 paths already cached"6047worker1 # [ 162.094999] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421006048worker1 # [ 162.108712] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6049worker1 # [ 162.303982] nix-daemon[835]: accepted connection from pid 1922, user nix-grpc-daemon (trusted)6050worker1 # [ 162.324620] nix-daemon[835]: reaped child process 1925, status = succeeded6051worker1 # [ 162.500597] nix-daemon[835]: accepted connection from pid 1927, user nix-grpc-daemon (trusted)6052worker1 # [ 162.520223] nix-daemon[835]: reaped child process 1930, status = succeeded6053lb # [ 162.519337] niks3-server[1051]: 2026/09/20 12:49:13 INFO Received uploads request method=POST path=/api/pending_closures6054worker1 # [ 162.545765] nix-grpc-daemon[1624]: time=2026-09-20T12:49:13.067Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6055worker1 # [ 162.547387] nix-grpc-daemon[1624]: time=2026-09-20T12:49:13.067Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6056lb # [ 162.555888] niks3-server[1051]: 2026/09/20 12:49:13 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6057worker1 # [ 162.583019] nix-grpc-daemon[1624]: time=2026-09-20T12:49:13.104Z level=INFO msg="Uploading 1 narinfos"6058lb # [ 162.570657] niks3-server[1051]: 2026/09/20 12:49:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6059lb # [ 162.581930] niks3-server[1051]: 2026/09/20 12:49:13 INFO Signed narinfos id=52 count=16060lb # [ 162.588823] niks3-server[1051]: 2026/09/20 12:49:13 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6061lb # [ 162.596636] niks3-server[1051]: 2026/09/20 12:49:13 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6062lb # [ 162.604254] niks3-server[1051]: 2026/09/20 12:49:13 INFO Completed upload id=526063lb # [ 162.607688] niks3-server[1051]: 2026/09/20 12:49:13 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6064worker1 # [ 162.614041] nix-grpc-daemon[1624]: time=2026-09-20T12:49:13.135Z level=INFO msg="Upload complete. (489ms)"6065worker1 # [ 162.624601] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=6246066worker1 # [ 162.635668] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006067worker1 # [ 162.642392] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6068worker1 # [ 162.647068] nix-daemon[835]: reaped child process 1920, status = succeeded6069worker1 # [ 162.659766] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6070worker1 # [ 163.486198] nix-daemon[835]: reaped child process 1934, status = succeeded6071worker1 # [ 163.655193] nix-daemon[835]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6072worker1 # [ 163.675058] nix-daemon[835]: reaped child process 1952, status = succeeded6073lb # [ 163.675343] niks3-server[1051]: 2026/09/20 12:49:14 INFO Received uploads request method=POST path=/api/pending_closures6074worker1 # [ 163.700422] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.219Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6075worker1 # [ 163.701911] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.219Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6076lb # [ 163.712659] niks3-server[1051]: 2026/09/20 12:49:14 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6077lb # [ 163.730128] niks3-server[1051]: 2026/09/20 12:49:14 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6078lb # [ 163.741420] niks3-server[1051]: 2026/09/20 12:49:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6079worker1 # [ 163.760767] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.282Z level=INFO msg="Uploading 1 narinfos"6080lb # [ 163.750174] niks3-server[1051]: 2026/09/20 12:49:14 INFO Signed narinfos id=53 count=16081lb # [ 163.756073] niks3-server[1051]: 2026/09/20 12:49:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6082lb # [ 163.767718] niks3-server[1051]: 2026/09/20 12:49:14 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6083lb # [ 163.774382] niks3-server[1051]: 2026/09/20 12:49:14 INFO Completed upload id=536084worker1 # [ 163.786790] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.308Z level=INFO msg="Upload complete. (315ms)"6085lb # [ 163.777570] niks3-server[1051]: 2026/09/20 12:49:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6086worker1 # [ 163.804583] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=16087worker1 # [ 163.821707] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:14Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16088worker1 # [ 163.832994] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006089worker1 # [ 163.839193] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:14Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06090worker1 # [ 163.852244] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421006091worker1 # [ 163.858562] nix-daemon[835]: reaped child process 1932, status = succeeded6092worker1 # [ 163.862886] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6093worker1 # [ 163.866194] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.354Z level=INFO msg="All 1 paths already cached"6094worker1 # [ 163.869584] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421006095worker1 # [ 164.045708] nix-daemon[835]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6096worker1 # [ 164.064750] nix-daemon[835]: reaped child process 1961, status = succeeded6097worker1 # [ 164.239122] nix-daemon[835]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6098worker1 # [ 164.256726] nix-daemon[835]: reaped child process 1966, status = succeeded6099lb # [ 164.258856] niks3-server[1051]: 2026/09/20 12:49:14 INFO Received uploads request method=POST path=/api/pending_closures6100worker1 # [ 164.280155] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.801Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6101worker1 # [ 164.281713] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.801Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6102lb # [ 164.289418] niks3-server[1051]: 2026/09/20 12:49:14 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6103lb # [ 164.303750] niks3-server[1051]: 2026/09/20 12:49:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6104worker1 # [ 164.322786] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.844Z level=INFO msg="Uploading 1 narinfos"6105lb # [ 164.315758] niks3-server[1051]: 2026/09/20 12:49:14 INFO Signed narinfos id=54 count=16106lb # [ 164.322673] niks3-server[1051]: 2026/09/20 12:49:14 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6107worker1 # [ 164.344863] nix-grpc-daemon[1624]: time=2026-09-20T12:49:14.866Z level=INFO msg="Upload complete. (482ms)"6108lb # [ 164.332749] niks3-server[1051]: 2026/09/20 12:49:14 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6109lb # [ 164.338009] niks3-server[1051]: 2026/09/20 12:49:14 INFO Completed upload id=546110worker1 # [ 164.352177] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=6246111lb # [ 164.341379] niks3-server[1051]: 2026/09/20 12:49:14 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6112worker1 # [ 164.366716] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006113worker1 # [ 164.372842] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6114worker1 # [ 164.378110] nix-daemon[835]: reaped child process 1956, status = succeeded6115worker1 # [ 164.389410] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6116worker1 # [ 165.825586] nix-daemon[835]: reaped child process 1970, status = succeeded6117worker1 # [ 166.001985] nix-daemon[835]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6118worker1 # [ 166.019605] nix-daemon[835]: reaped child process 1988, status = succeeded6119lb # [ 166.021556] niks3-server[1051]: 2026/09/20 12:49:16 INFO Received uploads request method=POST path=/api/pending_closures6120worker1 # [ 166.046264] nix-grpc-daemon[1624]: time=2026-09-20T12:49:16.566Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6121worker1 # [ 166.048274] nix-grpc-daemon[1624]: time=2026-09-20T12:49:16.566Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6122lb # [ 166.059674] niks3-server[1051]: 2026/09/20 12:49:16 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6123lb # [ 166.073659] niks3-server[1051]: 2026/09/20 12:49:16 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6124lb # [ 166.084212] niks3-server[1051]: 2026/09/20 12:49:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6125worker1 # [ 166.102255] nix-grpc-daemon[1624]: time=2026-09-20T12:49:16.623Z level=INFO msg="Uploading 1 narinfos"6126lb # [ 166.092917] niks3-server[1051]: 2026/09/20 12:49:16 INFO Signed narinfos id=55 count=16127lb # [ 166.098442] niks3-server[1051]: 2026/09/20 12:49:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6128lb # [ 166.107014] niks3-server[1051]: 2026/09/20 12:49:16 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6129lb # [ 166.114560] niks3-server[1051]: 2026/09/20 12:49:16 INFO Completed upload id=556130worker1 # [ 166.126352] nix-grpc-daemon[1624]: time=2026-09-20T12:49:16.648Z level=INFO msg="Upload complete. (324ms)"6131lb # [ 166.117695] niks3-server[1051]: 2026/09/20 12:49:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6132worker1 # [ 166.144851] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=16133worker1 # [ 166.166343] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:16Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16134worker1 # [ 166.170674] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006135worker1 # [ 166.176204] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:16Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06136worker1 # [ 166.182303] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421006137worker1 # [ 166.187341] nix-daemon[835]: reaped child process 1968, status = succeeded6138worker1 # [ 166.189584] nix-grpc-daemon[1624]: time=2026-09-20T12:49:16.708Z level=INFO msg="All 1 paths already cached"6139worker1 # [ 166.194657] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421006140worker1 # [ 166.210055] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6141worker1 # [ 166.397392] nix-daemon[835]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6142worker1 # [ 166.415538] nix-daemon[835]: reaped child process 1997, status = succeeded6143worker1 # [ 166.596357] nix-daemon[835]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)6144worker1 # [ 166.613433] nix-daemon[835]: reaped child process 2002, status = succeeded6145lb # [ 166.615129] niks3-server[1051]: 2026/09/20 12:49:17 INFO Received uploads request method=POST path=/api/pending_closures6146worker1 # [ 166.636615] nix-grpc-daemon[1624]: time=2026-09-20T12:49:17.158Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6147worker1 # [ 166.638137] nix-grpc-daemon[1624]: time=2026-09-20T12:49:17.158Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6148lb # [ 166.645187] niks3-server[1051]: 2026/09/20 12:49:17 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6149worker1 # [ 166.670579] nix-grpc-daemon[1624]: time=2026-09-20T12:49:17.192Z level=INFO msg="Uploading 1 narinfos"6150lb # [ 166.662669] niks3-server[1051]: 2026/09/20 12:49:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6151lb # [ 166.673104] niks3-server[1051]: 2026/09/20 12:49:17 INFO Signed narinfos id=56 count=16152lb # [ 166.678185] niks3-server[1051]: 2026/09/20 12:49:17 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6153lb # [ 166.687840] niks3-server[1051]: 2026/09/20 12:49:17 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6154worker1 # [ 166.706179] nix-grpc-daemon[1624]: time=2026-09-20T12:49:17.228Z level=INFO msg="Upload complete. (490ms)"6155lb # [ 166.694485] niks3-server[1051]: 2026/09/20 12:49:17 INFO Completed upload id=566156lb # [ 166.698923] niks3-server[1051]: 2026/09/20 12:49:17 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6157worker1 # [ 166.717002] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=6246158worker1 # [ 166.726720] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006159worker1 # [ 166.734000] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6160worker1 # [ 166.738594] nix-daemon[835]: reaped child process 1992, status = succeeded6161worker1 # [ 166.751953] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6162worker1 # [ 167.597943] nix-daemon[835]: reaped child process 2006, status = succeeded6163worker1 # [ 167.783519] nix-daemon[835]: accepted connection from pid 2021, user nix-grpc-daemon (trusted)6164worker1 # [ 167.801158] nix-daemon[835]: reaped child process 2024, status = succeeded6165lb # [ 167.802922] niks3-server[1051]: 2026/09/20 12:49:18 INFO Received uploads request method=POST path=/api/pending_closures6166worker1 # [ 167.827712] nix-grpc-daemon[1624]: time=2026-09-20T12:49:18.346Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6167worker1 # [ 167.829458] nix-grpc-daemon[1624]: time=2026-09-20T12:49:18.346Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6168lb # [ 167.833820] niks3-server[1051]: 2026/09/20 12:49:18 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6169lb # [ 167.847375] niks3-server[1051]: 2026/09/20 12:49:18 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6170lb # [ 167.866283] niks3-server[1051]: 2026/09/20 12:49:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6171worker1 # [ 167.884131] nix-grpc-daemon[1624]: time=2026-09-20T12:49:18.405Z level=INFO msg="Uploading 1 narinfos"6172lb # [ 167.878402] niks3-server[1051]: 2026/09/20 12:49:18 INFO Signed narinfos id=57 count=16173lb # [ 167.885223] niks3-server[1051]: 2026/09/20 12:49:18 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6174lb # [ 167.894558] niks3-server[1051]: 2026/09/20 12:49:18 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6175lb # [ 167.900127] niks3-server[1051]: 2026/09/20 12:49:18 INFO Completed upload id=576176worker1 # [ 167.911890] nix-grpc-daemon[1624]: time=2026-09-20T12:49:18.433Z level=INFO msg="Upload complete. (348ms)"6177lb # [ 167.903331] niks3-server[1051]: 2026/09/20 12:49:18 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6178worker1 # [ 167.925220] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=16179worker1 # [ 167.942903] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:18Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16180worker1 # [ 167.947624] nix-daemon[835]: reaped child process 2004, status = succeeded6181worker1 # [ 167.966463] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006182worker1 # [ 167.977372] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:18Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06183worker1 # [ 167.989759] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421006184worker1 # [ 168.005796] nix-grpc-daemon[1624]: time=2026-09-20T12:49:18.527Z level=INFO msg="All 2 paths already cached"6185worker1 # [ 168.015086] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421006186worker1 # [ 168.029518] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6187worker1 # [ 168.229691] nix-daemon[835]: accepted connection from pid 2030, user nix-grpc-daemon (trusted)6188worker1 # [ 168.250637] nix-daemon[835]: reaped child process 2033, status = succeeded6189worker1 # [ 168.426274] nix-daemon[835]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)6190worker1 # [ 168.445037] nix-daemon[835]: reaped child process 2038, status = succeeded6191lb # [ 168.443762] niks3-server[1051]: 2026/09/20 12:49:18 INFO Received uploads request method=POST path=/api/pending_closures6192worker1 # [ 168.465703] nix-grpc-daemon[1624]: time=2026-09-20T12:49:18.987Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6193worker1 # [ 168.467236] nix-grpc-daemon[1624]: time=2026-09-20T12:49:18.987Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6194lb # [ 168.472830] niks3-server[1051]: 2026/09/20 12:49:18 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6195worker1 # [ 168.497295] nix-grpc-daemon[1624]: time=2026-09-20T12:49:19.019Z level=INFO msg="Uploading 1 narinfos"6196lb # [ 168.489334] niks3-server[1051]: 2026/09/20 12:49:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6197lb # [ 168.500587] niks3-server[1051]: 2026/09/20 12:49:19 INFO Signed narinfos id=58 count=16198lb # [ 168.505748] niks3-server[1051]: 2026/09/20 12:49:19 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6199worker1 # [ 168.527513] nix-grpc-daemon[1624]: time=2026-09-20T12:49:19.049Z level=INFO msg="Upload complete. (484ms)"6200lb # [ 168.517019] niks3-server[1051]: 2026/09/20 12:49:19 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6201lb # [ 168.523446] niks3-server[1051]: 2026/09/20 12:49:19 INFO Completed upload id=586202worker1 # [ 168.535791] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=7526203lb # [ 168.526544] niks3-server[1051]: 2026/09/20 12:49:19 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6204worker1 # [ 168.545215] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006205worker1 # [ 168.555425] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6206worker1 # [ 168.560569] nix-daemon[835]: reaped child process 2028, status = succeeded6207worker1 # [ 168.575070] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6208worker1 # [ 169.621108] nix-daemon[835]: reaped child process 2042, status = succeeded6209worker1 # [ 169.799710] nix-daemon[835]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)6210worker1 # [ 169.817282] nix-daemon[835]: reaped child process 2060, status = succeeded6211lb # [ 169.822398] niks3-server[1051]: 2026/09/20 12:49:20 INFO Received uploads request method=POST path=/api/pending_closures6212worker1 # [ 169.843574] nix-grpc-daemon[1624]: time=2026-09-20T12:49:20.364Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6213worker1 # [ 169.845226] nix-grpc-daemon[1624]: time=2026-09-20T12:49:20.364Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6214lb # [ 169.864907] niks3-server[1051]: 2026/09/20 12:49:20 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6215worker1 # [ 169.887167] nix-grpc-daemon[1624]: time=2026-09-20T12:49:20.408Z level=INFO msg="Uploading 1 narinfos"6216lb # [ 169.877405] niks3-server[1051]: 2026/09/20 12:49:20 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6217lb # [ 169.887000] niks3-server[1051]: 2026/09/20 12:49:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6218lb # [ 169.891907] niks3-server[1051]: 2026/09/20 12:49:20 INFO Signed narinfos id=59 count=16219worker1 # [ 169.911206] nix-grpc-daemon[1624]: time=2026-09-20T12:49:20.433Z level=INFO msg="Upload complete. (325ms)"6220lb # [ 169.899501] niks3-server[1051]: 2026/09/20 12:49:20 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6221lb # [ 169.905081] niks3-server[1051]: 2026/09/20 12:49:20 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6222lb # [ 169.909517] niks3-server[1051]: 2026/09/20 12:49:20 INFO Completed upload id=596223worker1 # [ 169.920955] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=16224lb # [ 169.911833] niks3-server[1051]: 2026/09/20 12:49:20 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6225worker1 # [ 169.937071] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:20Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16226worker1 # [ 169.941202] nix-daemon[835]: reaped child process 2040, status = succeeded6227worker1 # [ 169.964610] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006228worker1 # [ 169.979117] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421006229worker1 # [ 169.992634] nix-grpc-daemon[1624]: time=2026-09-20T12:49:20.514Z level=INFO msg="All 2 paths already cached"6230worker1 # [ 170.001441] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421006231worker1 # [ 170.014328] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6232worker1 # [ 170.213345] nix-daemon[835]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)6233worker1 # [ 170.231360] nix-daemon[835]: reaped child process 2067, status = succeeded6234worker1 # [ 170.409554] nix-daemon[835]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6235worker1 # [ 170.428255] nix-daemon[835]: reaped child process 2072, status = succeeded6236lb # [ 170.427292] niks3-server[1051]: 2026/09/20 12:49:20 INFO Received uploads request method=POST path=/api/pending_closures6237worker1 # [ 170.448557] nix-grpc-daemon[1624]: time=2026-09-20T12:49:20.970Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6238worker1 # [ 170.450079] nix-grpc-daemon[1624]: time=2026-09-20T12:49:20.970Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6239lb # [ 170.456104] niks3-server[1051]: 2026/09/20 12:49:20 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6240worker1 # [ 170.480989] nix-grpc-daemon[1624]: time=2026-09-20T12:49:21.002Z level=INFO msg="Uploading 1 narinfos"6241lb # [ 170.471145] niks3-server[1051]: 2026/09/20 12:49:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6242lb # [ 170.482244] niks3-server[1051]: 2026/09/20 12:49:20 INFO Signed narinfos id=60 count=16243lb # [ 170.487499] niks3-server[1051]: 2026/09/20 12:49:21 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6244lb # [ 170.495840] niks3-server[1051]: 2026/09/20 12:49:21 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6245worker1 # [ 170.512673] nix-grpc-daemon[1624]: time=2026-09-20T12:49:21.034Z level=INFO msg="Upload complete. (483ms)"6246lb # [ 170.501084] niks3-server[1051]: 2026/09/20 12:49:21 INFO Completed upload id=606247lb # [ 170.504916] niks3-server[1051]: 2026/09/20 12:49:21 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6248worker1 # [ 170.520156] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=7526249worker1 # [ 170.528800] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006250worker1 # [ 170.540219] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6251worker1 # [ 170.544969] nix-daemon[835]: reaped child process 2062, status = succeeded6252worker1 # [ 170.558686] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6253worker1 # [ 171.379788] nix-daemon[835]: reaped child process 2076, status = succeeded6254worker1 # [ 171.538328] nix-daemon[835]: accepted connection from pid 2091, user nix-grpc-daemon (trusted)6255worker1 # [ 171.560170] nix-daemon[835]: reaped child process 2094, status = succeeded6256lb # [ 171.561692] niks3-server[1051]: 2026/09/20 12:49:22 INFO Received uploads request method=POST path=/api/pending_closures6257worker1 # [ 171.586872] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.106Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6258worker1 # [ 171.588533] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.106Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6259lb # [ 171.604240] niks3-server[1051]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6260lb # [ 171.616993] niks3-server[1051]: 2026/09/20 12:49:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6261worker1 # [ 171.634763] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.156Z level=INFO msg="Uploading 1 narinfos"6262lb # [ 171.625233] niks3-server[1051]: 2026/09/20 12:49:22 INFO Signed narinfos id=61 count=16263lb # [ 171.632521] niks3-server[1051]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6264lb # [ 171.642503] niks3-server[1051]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6265worker1 # [ 171.662050] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.183Z level=INFO msg="Upload complete. (294ms)"6266lb # [ 171.650402] niks3-server[1051]: 2026/09/20 12:49:22 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6267lb # [ 171.654345] niks3-server[1051]: 2026/09/20 12:49:22 INFO Completed upload id=616268lb # [ 171.658280] niks3-server[1051]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6269worker1 # [ 171.673052] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=16270worker1 # [ 171.692260] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:22Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16271worker1 # [ 171.696944] nix-daemon[835]: reaped child process 2074, status = succeeded6272worker1 # [ 171.717287] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006273worker1 # [ 171.733717] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:421006274worker1 # [ 171.749510] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.271Z level=INFO msg="All 2 paths already cached"6275worker1 # [ 171.760131] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:421006276worker1 # [ 171.774775] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6277worker1 # [ 171.979077] nix-daemon[835]: accepted connection from pid 2098, user nix-grpc-daemon (trusted)6278worker1 # [ 171.997992] nix-daemon[835]: reaped child process 2101, status = succeeded6279worker1 # [ 172.183818] nix-daemon[835]: accepted connection from pid 2103, user nix-grpc-daemon (trusted)6280worker1 # [ 172.202513] nix-daemon[835]: reaped child process 2106, status = succeeded6281lb # [ 172.204412] niks3-server[1051]: 2026/09/20 12:49:22 INFO Received uploads request method=POST path=/api/pending_closures6282worker1 # [ 172.227165] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.748Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6283worker1 # [ 172.228842] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.748Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6284lb # [ 172.235221] niks3-server[1051]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6285worker1 # [ 172.272661] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.794Z level=INFO msg="Uploading 1 narinfos"6286lb # [ 172.254036] niks3-server[1051]: 2026/09/20 12:49:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6287lb # [ 172.267177] niks3-server[1051]: 2026/09/20 12:49:22 INFO Signed narinfos id=62 count=16288lb # [ 172.274394] niks3-server[1051]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6289lb # [ 172.285665] niks3-server[1051]: 2026/09/20 12:49:22 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6290lb # [ 172.290614] niks3-server[1051]: 2026/09/20 12:49:22 INFO Completed upload id=626291worker1 # [ 172.302618] nix-grpc-daemon[1624]: time=2026-09-20T12:49:22.824Z level=INFO msg="Upload complete. (511ms)"6292lb # [ 172.293662] niks3-server[1051]: 2026/09/20 12:49:22 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6293worker1 # [ 172.312287] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=0 paths=1 nar_bytes_in=7526294worker1 # [ 172.320396] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:421006295worker1 # [ 172.334310] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6296worker1 # [ 172.339880] nix-daemon[835]: reaped child process 2096, status = succeeded6297worker1 # [ 172.353664] nix-daemon[835]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)6298worker1 # [ 173.201575] nix-daemon[835]: reaped child process 2110, status = succeeded6299worker1 # [ 173.383369] nix-daemon[835]: accepted connection from pid 2125, user nix-grpc-daemon (trusted)6300worker1 # [ 173.402529] nix-daemon[835]: reaped child process 2128, status = succeeded6301lb # [ 173.406825] niks3-server[1051]: 2026/09/20 12:49:23 INFO Received uploads request method=POST path=/api/pending_closures6302worker1 # [ 173.432588] nix-grpc-daemon[1624]: time=2026-09-20T12:49:23.953Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6303worker1 # [ 173.434656] nix-grpc-daemon[1624]: time=2026-09-20T12:49:23.953Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6304lb # [ 173.452665] niks3-server[1051]: 2026/09/20 12:49:23 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6305lb # [ 173.470914] niks3-server[1051]: 2026/09/20 12:49:23 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6306lb # [ 173.485597] niks3-server[1051]: 2026/09/20 12:49:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6307worker1 # [ 173.502789] nix-grpc-daemon[1624]: time=2026-09-20T12:49:24.024Z level=INFO msg="Uploading 1 narinfos"6308lb # [ 173.495130] niks3-server[1051]: 2026/09/20 12:49:24 INFO Signed narinfos id=63 count=16309lb # [ 173.500506] niks3-server[1051]: 2026/09/20 12:49:24 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6310lb # [ 173.508340] niks3-server[1051]: 2026/09/20 12:49:24 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6311lb # [ 173.516462] niks3-server[1051]: 2026/09/20 12:49:24 INFO Completed upload id=636312worker1 # [ 173.528327] nix-grpc-daemon[1624]: time=2026-09-20T12:49:24.050Z level=INFO msg="Upload complete. (360ms)"6313lb # [ 173.519757] niks3-server[1051]: 2026/09/20 12:49:24 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6314worker1 # [ 173.541542] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42100 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=16315worker1 # [ 173.563341] nix-daemon[835]: reaped child process 2108, status = succeeded6316client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.27 seconds)6317worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6318worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6319client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16320worker2 # [ 174.035753] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428426321worker2 # [ 174.047115] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428426322worker2 # [ 174.065981] nix-grpc-daemon[1591]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)6323worker2 # [ 174.090829] nix-grpc-daemon[1591]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5)6324worker2 # [ 174.131675] nix-grpc-daemon[1591]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 292 ms (attempt 3/5)6325worker2 # [ 174.432109] nix-grpc-daemon[1591]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 652 ms (attempt 4/5)6326worker2 # [ 175.086398] nix-grpc-daemon[1591]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6327worker1 # [ 175.092752] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:25Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06328client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 1.40 seconds)6329(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 19.75 seconds)6330subtest: low disk drains a worker and builds go to the other6331worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6332worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.15 seconds)6333worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6334worker1 # [ 176.573863] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:27Z level=info event=unhealthy reason=min_free6335worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.14 seconds)6336client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26337client # don't know how to build these paths:6338client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6339worker2 # [ 177.735080] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428426340worker2 # [ 177.743578] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428426341worker1 # [ 177.741107] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:28Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06342worker1 # [ 177.741558] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:28Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16343worker2 # [ 177.818602] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428426344worker2 # [ 177.837234] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428426345client # copying 1 paths to 'lb:50051'...6346client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6347worker2 # [ 177.855898] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428426348worker2 # [ 177.873361] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6349worker2 # [ 177.905723] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:28Z level=info event=niks3_push_started pid=16476350worker2 # [ 178.126982] nix-grpc-daemon[1647]: time=2026-09-20T12:49:28.639Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""6351worker2 # [ 178.317144] nix-daemon[848]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6352worker2 # [ 178.336747] nix-daemon[848]: reaped child process 1663, status = succeeded6353worker2 # [ 178.528771] nix-daemon[848]: accepted connection from pid 1665, user nix-grpc-daemon (trusted)6354worker2 # [ 178.546481] nix-daemon[848]: reaped child process 1668, status = succeeded6355lb # [ 178.518978] niks3-server[1051]: 2026/09/20 12:49:29 INFO Received uploads request method=POST path=/api/pending_closures6356worker2 # [ 178.575882] nix-grpc-daemon[1647]: time=2026-09-20T12:49:29.088Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6357worker2 # [ 178.577874] nix-grpc-daemon[1647]: time=2026-09-20T12:49:29.088Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6358lb # [ 178.564774] niks3-server[1051]: 2026/09/20 12:49:29 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6359lb # [ 178.573546] niks3-server[1051]: 2026/09/20 12:49:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6360worker2 # [ 178.625479] nix-grpc-daemon[1647]: time=2026-09-20T12:49:29.138Z level=INFO msg="Uploading 1 narinfos"6361lb # [ 178.587118] niks3-server[1051]: 2026/09/20 12:49:29 INFO Signed narinfos id=64 count=16362lb # [ 178.597362] niks3-server[1051]: 2026/09/20 12:49:29 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6363lb # [ 178.606838] niks3-server[1051]: 2026/09/20 12:49:29 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6364lb # [ 178.616512] niks3-server[1051]: 2026/09/20 12:49:29 INFO Completed upload id=646365worker2 # [ 178.658607] nix-grpc-daemon[1647]: time=2026-09-20T12:49:29.172Z level=INFO msg="Upload complete. (527ms)"6366lb # [ 178.620647] niks3-server[1051]: 2026/09/20 12:49:29 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6367worker2 # [ 178.668621] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42842 duration_s=0 paths=1 nar_bytes_in=5126368worker2 # [ 178.686809] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428426369worker2 # [ 178.694266] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6370worker2 # [ 178.699407] nix-daemon[848]: reaped child process 1645, status = succeeded6371client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6372worker2 # [ 178.711160] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6373client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6374client # farm-a> LOG-farm-a-drain6375worker2 # [ 179.963159] nix-daemon[848]: reaped child process 1672, status = succeeded6376worker2 # [ 180.139196] nix-daemon[848]: accepted connection from pid 1687, user nix-grpc-daemon (trusted)6377worker2 # [ 180.159645] nix-daemon[848]: reaped child process 1690, status = succeeded6378lb # [ 180.127946] niks3-server[1051]: 2026/09/20 12:49:30 INFO Received uploads request method=POST path=/api/pending_closures6379worker2 # [ 180.183398] nix-grpc-daemon[1647]: time=2026-09-20T12:49:30.696Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6380worker2 # [ 180.188765] nix-grpc-daemon[1647]: time=2026-09-20T12:49:30.698Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6381lb # [ 180.164716] niks3-server[1051]: 2026/09/20 12:49:30 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6382lb # [ 180.180795] niks3-server[1051]: 2026/09/20 12:49:30 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6383lb # [ 180.191442] niks3-server[1051]: 2026/09/20 12:49:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6384worker2 # [ 180.243508] nix-grpc-daemon[1647]: time=2026-09-20T12:49:30.755Z level=INFO msg="Uploading 1 narinfos"6385lb # [ 180.197717] niks3-server[1051]: 2026/09/20 12:49:30 INFO Signed narinfos id=65 count=16386lb # [ 180.202848] niks3-server[1051]: 2026/09/20 12:49:30 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6387lb # [ 180.215476] niks3-server[1051]: 2026/09/20 12:49:30 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6388lb # [ 180.221902] niks3-server[1051]: 2026/09/20 12:49:30 INFO Completed upload id=656389worker2 # [ 180.267072] nix-grpc-daemon[1647]: time=2026-09-20T12:49:30.780Z level=INFO msg="Upload complete. (337ms)"6390lb # [ 180.224887] niks3-server[1051]: 2026/09/20 12:49:30 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6391worker2 # [ 180.277039] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42842 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=16392worker1 # [ 180.252694] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:30Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06393worker1 # [ 180.253073] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:30Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16394worker2 # [ 180.295587] nix-daemon[848]: reaped child process 1670, status = succeeded6395worker2 # [ 180.331064] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428426396worker2 # [ 180.347667] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428426397worker2 # [ 180.362754] nix-grpc-daemon[1647]: time=2026-09-20T12:49:30.876Z level=INFO msg="All 1 paths already cached"6398client # copying 1 paths to 'lb:50051'...6399client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6400worker2 # [ 180.373323] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428426401worker2 # [ 180.387770] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6402worker2 # [ 180.592380] nix-daemon[848]: accepted connection from pid 1694, user nix-grpc-daemon (trusted)6403worker2 # [ 180.613041] nix-daemon[848]: reaped child process 1697, status = succeeded6404worker2 # [ 180.789006] nix-daemon[848]: accepted connection from pid 1699, user nix-grpc-daemon (trusted)6405worker2 # [ 180.807165] nix-daemon[848]: reaped child process 1702, status = succeeded6406lb # [ 180.772898] niks3-server[1051]: 2026/09/20 12:49:31 INFO Received uploads request method=POST path=/api/pending_closures6407worker2 # [ 180.827969] nix-grpc-daemon[1647]: time=2026-09-20T12:49:31.341Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6408worker2 # [ 180.829608] nix-grpc-daemon[1647]: time=2026-09-20T12:49:31.341Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6409lb # [ 180.799003] niks3-server[1051]: 2026/09/20 12:49:31 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6410worker2 # [ 180.858779] nix-grpc-daemon[1647]: time=2026-09-20T12:49:31.372Z level=INFO msg="Uploading 1 narinfos"6411lb # [ 180.814396] niks3-server[1051]: 2026/09/20 12:49:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6412lb # [ 180.828787] niks3-server[1051]: 2026/09/20 12:49:31 INFO Signed narinfos id=66 count=16413lb # [ 180.831614] niks3-server[1051]: 2026/09/20 12:49:31 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6414lb # [ 180.841464] niks3-server[1051]: 2026/09/20 12:49:31 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6415lb # [ 180.848674] niks3-server[1051]: 2026/09/20 12:49:31 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6416lb # [ 180.854655] niks3-server[1051]: 2026/09/20 12:49:31 INFO Completed upload id=666417worker2 # [ 180.900103] nix-grpc-daemon[1647]: time=2026-09-20T12:49:31.413Z level=INFO msg="Upload complete. (495ms)"6418worker2 # [ 180.908529] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42842 duration_s=0 paths=1 nar_bytes_in=6326419worker2 # [ 180.920539] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428426420worker2 # [ 180.930710] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6421worker2 # [ 180.940297] nix-daemon[848]: reaped child process 1692, status = succeeded6422client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6423client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6424worker2 # [ 180.952877] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6425lb # [ 181.033878] envoy[913]: conn peer=[2001:db8:1::1]:53372 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27101 tx=10611 ms=179656426lb # [ 181.039191] envoy[913]: conn peer=[2001:db8:1::1]:40210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=524 ms=11186427client # farm-b> LOG-farm-b-drain6428worker2 # [ 181.799248] nix-daemon[848]: reaped child process 1706, status = succeeded6429worker2 # [ 181.979727] nix-daemon[848]: accepted connection from pid 1721, user nix-grpc-daemon (trusted)6430worker2 # [ 181.997935] nix-daemon[848]: reaped child process 1724, status = succeeded6431lb # [ 181.966176] niks3-server[1051]: 2026/09/20 12:49:32 INFO Received uploads request method=POST path=/api/pending_closures6432worker2 # [ 182.021852] nix-grpc-daemon[1647]: time=2026-09-20T12:49:32.534Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6433worker2 # [ 182.023369] nix-grpc-daemon[1647]: time=2026-09-20T12:49:32.534Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6434lb # [ 181.998546] niks3-server[1051]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6435lb # [ 182.012523] niks3-server[1051]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6436lb # [ 182.024971] niks3-server[1051]: 2026/09/20 12:49:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6437worker2 # [ 182.075188] nix-grpc-daemon[1647]: time=2026-09-20T12:49:32.588Z level=INFO msg="Uploading 1 narinfos"6438lb # [ 182.032342] niks3-server[1051]: 2026/09/20 12:49:32 INFO Signed narinfos id=67 count=16439lb # [ 182.039495] niks3-server[1051]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6440lb # [ 182.048360] niks3-server[1051]: 2026/09/20 12:49:32 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6441lb # [ 182.056316] niks3-server[1051]: 2026/09/20 12:49:32 INFO Completed upload id=676442worker2 # [ 182.101753] nix-grpc-daemon[1647]: time=2026-09-20T12:49:32.615Z level=INFO msg="Upload complete. (337ms)"6443lb # [ 182.059095] niks3-server[1051]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6444worker2 # [ 182.113188] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42842 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=16445worker1 # [ 182.086425] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:32Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06446worker1 # [ 182.092640] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:32Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16447worker2 # [ 182.134087] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428426448worker2 # [ 182.143162] nix-daemon[848]: reaped child process 1704, status = succeeded6449worker2 # [ 182.151031] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428426450worker2 # [ 182.163872] nix-grpc-daemon[1647]: time=2026-09-20T12:49:32.677Z level=INFO msg="All 2 paths already cached"6451client # copying 1 paths to 'lb:50051'...6452client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6453worker2 # [ 182.173043] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428426454worker2 # [ 182.184795] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6455worker2 # [ 182.378023] nix-daemon[848]: accepted connection from pid 1728, user nix-grpc-daemon (trusted)6456worker2 # [ 182.398356] nix-daemon[848]: reaped child process 1731, status = succeeded6457worker2 # [ 182.571654] nix-daemon[848]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6458worker2 # [ 182.589799] nix-daemon[848]: reaped child process 1736, status = succeeded6459lb # [ 182.557482] niks3-server[1051]: 2026/09/20 12:49:33 INFO Received uploads request method=POST path=/api/pending_closures6460worker2 # [ 182.613202] nix-grpc-daemon[1647]: time=2026-09-20T12:49:33.126Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6461worker2 # [ 182.614750] nix-grpc-daemon[1647]: time=2026-09-20T12:49:33.126Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6462lb # [ 182.583569] niks3-server[1051]: 2026/09/20 12:49:33 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6463lb # [ 182.597439] niks3-server[1051]: 2026/09/20 12:49:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6464worker2 # [ 182.652525] nix-grpc-daemon[1647]: time=2026-09-20T12:49:33.165Z level=INFO msg="Uploading 1 narinfos"6465lb # [ 182.609917] niks3-server[1051]: 2026/09/20 12:49:33 INFO Signed narinfos id=68 count=16466lb # [ 182.615781] niks3-server[1051]: 2026/09/20 12:49:33 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6467lb # [ 182.625125] niks3-server[1051]: 2026/09/20 12:49:33 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6468lb # [ 182.631720] niks3-server[1051]: 2026/09/20 12:49:33 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6469lb # [ 182.636362] niks3-server[1051]: 2026/09/20 12:49:33 INFO Completed upload id=686470worker2 # [ 182.681450] nix-grpc-daemon[1647]: time=2026-09-20T12:49:33.195Z level=INFO msg="Upload complete. (484ms)"6471worker2 # [ 182.688211] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42842 duration_s=0 paths=1 nar_bytes_in=7606472worker2 # [ 182.698630] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428426473worker2 # [ 182.709620] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6474worker2 # [ 182.719955] nix-daemon[848]: reaped child process 1726, status = succeeded6475client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6476worker2 # [ 182.727596] nix-daemon[848]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)6477client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6478client # farm-top> LOG-farm-top-drain6479worker2 # [ 183.753061] nix-daemon[848]: reaped child process 1740, status = succeeded6480worker2 # [ 183.928242] nix-daemon[848]: accepted connection from pid 1755, user nix-grpc-daemon (trusted)6481worker2 # [ 183.948330] nix-daemon[848]: reaped child process 1758, status = succeeded6482lb # [ 183.918437] niks3-server[1051]: 2026/09/20 12:49:34 INFO Received uploads request method=POST path=/api/pending_closures6483worker2 # [ 183.974721] nix-grpc-daemon[1647]: time=2026-09-20T12:49:34.486Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6484worker2 # [ 183.976357] nix-grpc-daemon[1647]: time=2026-09-20T12:49:34.486Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6485lb # [ 183.955672] niks3-server[1051]: 2026/09/20 12:49:34 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6486lb # [ 183.969626] niks3-server[1051]: 2026/09/20 12:49:34 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6487lb # [ 183.982958] niks3-server[1051]: 2026/09/20 12:49:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6488worker2 # [ 184.032876] nix-grpc-daemon[1647]: time=2026-09-20T12:49:34.546Z level=INFO msg="Uploading 1 narinfos"6489lb # [ 183.992351] niks3-server[1051]: 2026/09/20 12:49:34 INFO Signed narinfos id=69 count=16490lb # [ 183.998298] niks3-server[1051]: 2026/09/20 12:49:34 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6491lb # [ 184.009291] niks3-server[1051]: 2026/09/20 12:49:34 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6492lb # [ 184.015071] niks3-server[1051]: 2026/09/20 12:49:34 INFO Completed upload id=696493worker2 # [ 184.059769] nix-grpc-daemon[1647]: time=2026-09-20T12:49:34.573Z level=INFO msg="Upload complete. (323ms)"6494lb # [ 184.018196] niks3-server[1051]: 2026/09/20 12:49:34 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6495worker2 # [ 184.073174] nix-grpc-daemon[1591]: ts=2026-09-20T12:49:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42842 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=16496worker2 # [ 184.089425] nix-daemon[848]: reaped child process 1738, status = succeeded6497client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.66 seconds)6498client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6499client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.24 seconds)6500worker1: must succeed: rm /nix/.rw-store/fill6501worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6502worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6503worker1 # [ 185.587650] nix-grpc-daemon[1550]: ts=2026-09-20T12:49:36Z level=info event=healthy reason=min_free6504worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6505(finished: subtest: low disk drains a worker and builds go to the other, in 10.57 seconds)6506(finished: run the VM test script, in 186.85 seconds)6507test script finished in 187.15s6508cleanup6509kill QemuMachine (pid 51)6510client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6511kill QemuMachine (pid 49)6512lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6513kill QemuMachine (pid 52)6514worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6515kill QemuMachine (pid 50)6516worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6517(finished: cleanup, in 1.23 seconds)6518additionally exposed symbols:6519 client, lb, worker1, worker2,6520 vlan1,6521 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh