vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #349
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1: QEMU running (pid 52)14client # Disk image does not exist, creating the virtualisation disk image...15client: QEMU running (pid 50)16client # Formatting '/build/vm-state-client/tmp.3iLKYm4MNe', fmt=raw size=107374182417worker1 # Formatting '/build/vm-state-worker1/tmp.enVQS1iAYK', fmt=raw size=107374182418lb # Disk image does not exist, creating the virtualisation disk image...19worker1 # mke2fs 1.47.4 (6-Mar-2025)20client # mke2fs 1.47.4 (6-Mar-2025)21worker1 # Discarding device blocks: 0/262144 done22lb: QEMU running (pid 49)23lb # Formatting '/build/vm-state-lb/tmp.GKs2Disy4x', fmt=raw size=107374182424client # Discarding device blocks: 0/262144 done25worker2 # Disk image does not exist, creating the virtualisation disk image...26worker2: QEMU running (pid 51)27client # Creating filesystem with 262144 4k blocks and 65536 inodes28worker2 # Formatting '/build/vm-state-worker2/tmp.sldrnh1yX4', fmt=raw size=107374182429lb # mke2fs 1.47.4 (6-Mar-2025)30worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes31(finished: start all VMs, in 1.10 seconds)32lb # Discarding device blocks: 0/262144 done33worker1 # Filesystem UUID: 38eadec4-c5df-4faa-81bb-fc92f892367c34client: waiting for unit network-addresses-eth1.service35lb # Creating filesystem with 262144 4k blocks and 65536 inodes36worker1 # Superblock backups stored on blocks:37client: waiting for the VM to finish booting38lb # Filesystem UUID: ab6c7878-7972-4949-87ca-f8448a43654639worker1 # 32768, 98304, 163840, 22937640worker2 # mke2fs 1.47.4 (6-Mar-2025)41client # Filesystem UUID: 5b80c133-8ae6-446a-bdf4-68cc378e341342lb # Superblock backups stored on blocks:43worker1 # 44worker2 # Discarding device blocks: 0/262144 done45client # Superblock backups stored on blocks:46lb # 32768, 98304, 163840, 22937647worker1 # Allocating group tables: 0/8 done48worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes49client # 32768, 98304, 163840, 22937650lb # 51worker1 # Writing inode tables: 0/8 done52worker2 # Filesystem UUID: d6fc7b16-315c-48c4-bf99-75702371229653client # 54lb # Allocating group tables: 0/8 done55worker1 # Creating journal (8192 blocks): done56worker2 # Superblock backups stored on blocks:57client # Allocating group tables: 0/8 done58lb # Writing inode tables: 0/8 done59worker1 # Writing superblocks and filesystem accounting information: 0/8 done60worker2 # 32768, 98304, 163840, 22937661client # Writing inode tables: 0/8 done62lb # Creating journal (8192 blocks): done63worker1 # 64worker2 # 65client # Creating journal (8192 blocks): done66lb # Writing superblocks and filesystem accounting information: 0/8 done67worker1 # Virtualisation disk image created.68worker2 # Allocating group tables: 0/8 done69client # Writing superblocks and filesystem accounting information: 0/8 done70worker2 # Writing inode tables: 0/8 done71client # 72worker2 # Creating journal (8192 blocks): done73client # Virtualisation disk image created.74worker2 # Writing superblocks and filesystem accounting information: 0/8 done75lb # 76worker2 # 77lb # Virtualisation disk image created.78worker2 # Virtualisation disk image created.79client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681client # [ 0.000000] KASLR enabled82client # [ 0.000000] random: crng init done83client # [ 0.000000] Machine model: linux,dummy-virt84client # [ 0.000000] efi: UEFI not found.85client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88client # [ 0.000000] Zone ranges:89client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90client # [ 0.000000] DMA32 empty91client # [ 0.000000] Normal empty92client # [ 0.000000] Device empty93client # [ 0.000000] Movable zone start for each node94client # [ 0.000000] Early memory node ranges95client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098client # [ 0.000000] psci: probing for conduit method from DT.99client # [ 0.000000] psci: PSCIv1.3 detected in firmware.100client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101client # [ 0.000000] psci: Trusted OS migration not required102client # [ 0.000000] psci: SMC Calling Convention v1.1103client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105client # [ 0.000000] Detected PIPT I-cache on CPU0106client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107client # [ 0.000000] CPU features: detected: GICv3 CPU interface108client # [ 0.000000] CPU features: detected: Spectre-v4109client # [ 0.000000] CPU features: detected: Spectre-BHB110client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112client # [ 0.000000] alternatives: applying boot alternatives113client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test/init regInfo=/nix/store/nq3jd61rq8ynjk18jfmxb6am9bzw19x4-closure-info/registration console=ttyAMA0,115200n8 console=tty0114client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/nq3jd61rq8ynjk18jfmxb6am9bzw19x4-closure-info/registration", will be passed to user space.115client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119client # [ 0.000000] software IO TLB: area num 1.120client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122client # [ 0.000000] Fallback order for Node 0: 0123client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216124client # [ 0.000000] Policy zone: DMA125client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off126worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]127client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1128client # [ 0.000000] allocated 3145728 bytes of page_ext129client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages130worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026131worker1 # [ 0.000000] KASLR enabled132client # [ 0.000000] ftrace: allocated 294 pages with 4 groups133worker1 # [ 0.000000] random: crng init done134client # [ 0.000000] rcu: Hierarchical RCU implementation.135worker1 # [ 0.000000] Machine model: linux,dummy-virt136worker1 # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] rcu: RCU event tracing is enabled.138client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.139worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT140client # [ 0.000000] Trampoline variant of Tasks RCU enabled.141worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] Rude variant of Tasks RCU enabled.143worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]144client # [ 0.000000] Tracing variant of Tasks RCU enabled.145worker1 # [ 0.000000] Zone ranges:146client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.147worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]148worker1 # [ 0.000000] DMA32 empty149worker1 # [ 0.000000] Normal empty150client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1151worker1 # [ 0.000000] Device empty152worker1 # [ 0.000000] Movable zone start for each node153client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.154worker1 # [ 0.000000] Early memory node ranges155worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]156client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.157worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]158client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.159worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000160client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0161worker1 # [ 0.000000] psci: probing for conduit method from DT.162client # [ 0.000000] GICv3: 256 SPIs implemented163worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.164client # [ 0.000000] GICv3: 0 Extended SPIs implemented165worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs166client # [ 0.000000] Root IRQ handler: gic_handle_irq167worker1 # [ 0.000000] psci: Trusted OS migration not required168client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI169worker1 # [ 0.000000] psci: SMC Calling Convention v1.1170client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0171client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000172worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)173client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]174worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296175worker1 # [ 0.000000] Detected PIPT I-cache on CPU0176client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)177worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)178client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)179worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface180client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000181worker1 # [ 0.000000] CPU features: detected: Spectre-v4182worker1 # [ 0.000000] CPU features: detected: Spectre-BHB183client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000184worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38185client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.186worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23187worker1 # [ 0.000000] alternatives: applying boot alternatives188client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns189client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).190client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns191client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns192client # [ 0.000039] arm-pv: using stolen time PV193worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/w518yd74l6j92j3q0axjs0fin71pjwgz-nixos-system-worker1-test/init regInfo=/nix/store/xd3py3agj8bb304p7akayq6ykixgr29n-closure-info/registration console=ttyAMA0,115200n8 console=tty0194client # [ 0.000749] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)195client # [ 0.000959] Console: colour dummy device 80x25196worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/xd3py3agj8bb304p7akayq6ykixgr29n-closure-info/registration", will be passed to user space.197client # [ 0.000967] printk: legacy console [tty0] enabled198worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes199client # [ 0.001168] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)200worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)201client # [ 0.001176] pid_max: default: 32768 minimum: 301202worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)203lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]204client # [ 0.001255] LSM: initializing lsm=capability,landlock,yama,bpf,ima205client # [ 0.001440] landlock: Up and running.206worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB207client # [ 0.001443] Yama: becoming mindful.208worker1 # [ 0.000000] software IO TLB: area num 1.209client # [ 0.002143] LSM support for eBPF active210lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026211lb # [ 0.000000] KASLR enabled212worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB213client # [ 0.002304] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)214worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]215lb # [ 0.000000] random: crng init done216worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)217lb # [ 0.000000] Machine model: linux,dummy-virt218client # [ 0.002336] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)219worker1 # [ 0.000000] Fallback order for Node 0: 0220lb # [ 0.000000] efi: UEFI not found.221worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026222client # [ 0.003524] cacheinfo: Unable to detect cache hierarchy for CPU 0223worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216224lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT225worker2 # [ 0.000000] KASLR enabled226worker1 # [ 0.000000] Policy zone: DMA227client # [ 0.004336] rcu: Hierarchical SRCU implementation.228worker2 # [ 0.000000] random: crng init done229lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]230client # [ 0.004341] rcu: Max phase no-delay instances is 1000.231worker2 # [ 0.000000] Machine model: linux,dummy-virt232worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off233worker2 # [ 0.000000] efi: UEFI not found.234lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]235client # [ 0.005589] fsl-mc MSI: its@8080000 domain created236lb # [ 0.000000] Zone ranges:237worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1238client # [ 0.005680] EFI services will not be available.239worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT240worker1 # [ 0.000000] allocated 3145728 bytes of page_ext241lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]242client # [ 0.005772] smp: Bringing up secondary CPUs ...243lb # [ 0.000000] DMA32 empty244worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages245worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]246client # [ 0.005781] smp: Brought up 1 node, 1 CPU247lb # [ 0.000000] Normal empty248worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups249lb # [ 0.000000] Device empty250client # [ 0.005784] SMP: Total of 1 processors activated.251worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]252worker2 # [ 0.000000] Zone ranges:253lb # [ 0.000000] Movable zone start for each node254worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.255client # [ 0.005790] CPU: All CPU(s) started at EL1256lb # [ 0.000000] Early memory node ranges257worker1 # [ 0.000000] rcu: RCU event tracing is enabled.258worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]259client # [ 0.005800] CPU features: detected: Branch Target Identification260worker2 # [ 0.000000] DMA32 empty261lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]262worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.263worker2 # [ 0.000000] Normal empty264client # [ 0.005805] CPU features: detected: ARMv8.4 Translation Table Level265worker2 # [ 0.000000] Device empty266lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]267worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.268worker2 # [ 0.000000] Movable zone start for each node269worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.270lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000271worker2 # [ 0.000000] Early memory node ranges272client # [ 0.005809] CPU features: detected: Instruction cache invalidation not required for I/D coherence273worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.274lb # [ 0.000000] psci: probing for conduit method from DT.275worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]276client # [ 0.005812] CPU features: detected: Data cache clean to the PoU not required for I/D coherence277lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.278worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.279worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]280client # [ 0.005816] CPU features: detected: Common not Private translations281lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs282worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1283worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000284lb # [ 0.000000] psci: Trusted OS migration not required285client # [ 0.005819] CPU features: detected: CRC32 instructions286lb # [ 0.000000] psci: SMC Calling Convention v1.1287worker2 # [ 0.000000] psci: probing for conduit method from DT.288client # [ 0.005822] CPU features: detected: Data cache clean to Point of Deep Persistence289worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.290worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.291lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)292client # [ 0.005826] CPU features: detected: Data cache clean to Point of Persistence293worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs294worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.295worker2 # [ 0.000000] psci: Trusted OS migration not required296lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296297client # [ 0.005829] CPU features: detected: Data independent timing control (DIT)298worker2 # [ 0.000000] psci: SMC Calling Convention v1.1299client # [ 0.005832] CPU features: detected: E0PD300lb # [ 0.000000] Detected PIPT I-cache on CPU0301worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.302client # [ 0.005834] CPU features: detected: Enhanced Counter Virtualization303worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0304lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)305worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)306worker1 # [ 0.000000] GICv3: 256 SPIs implemented307lb # [ 0.000000] CPU features: detected: GICv3 CPU interface308client # [ 0.005837] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)309worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296310worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented311lb # [ 0.000000] CPU features: detected: Spectre-v4312worker2 # [ 0.000000] Detected PIPT I-cache on CPU0313client # [ 0.005841] CPU features: detected: Enhanced Virtualization Traps314worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq315lb # [ 0.000000] CPU features: detected: Spectre-BHB316worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI317client # [ 0.005844] CPU features: detected: Fine Grained Traps318worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)319lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38320worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0321worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface322client # [ 0.005847] CPU features: detected: Generic authentication (architected QARMA5 algorithm)323lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23324worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000325worker2 # [ 0.000000] CPU features: detected: Spectre-v4326client # [ 0.005852] CPU features: detected: RCpc load-acquire (LDAPR)327worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]328lb # [ 0.000000] alternatives: applying boot alternatives329worker2 # [ 0.000000] CPU features: detected: Spectre-BHB330client # [ 0.005854] CPU features: detected: LSE atomic instructions331worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38332worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)333client # [ 0.005857] CPU features: detected: Privileged Access Never334client # [ 0.005860] CPU features: detected: PMUv3335worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23336client # [ 0.005863] CPU features: detected: RAS Extension Support337worker2 # [ 0.000000] alternatives: applying boot alternatives338worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)339client # [ 0.005865] CPU features: detected: RASv1p1 Extension Support340worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000341lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/lai4n7chb0cnvnygfis27sipihspygia-nixos-system-lb-test/init regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration console=ttyAMA0,115200n8 console=tty0342worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000343client # [ 0.005868] CPU features: detected: Random Number Generator344client # [ 0.005870] CPU features: detected: Speculation barrier (SB)345worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.346client # [ 0.005873] CPU features: detected: Stage-2 Force Write-Back347client # [ 0.005876] CPU features: detected: TLB range maintenance instructions348worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns349worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/f0cp7qw3cm27n34d47jydhc08lp1yy22-nixos-system-worker2-test/init regInfo=/nix/store/8r3nnn9b2hsc90akzwn4pnxvw20y65vr-closure-info/registration console=ttyAMA0,115200n8 console=tty0350lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/qdg5w64gnxykmz82pw25fgmx46q7lp99-closure-info/registration", will be passed to user space.351client # [ 0.005907] CPU features: detected: Speculative Store Bypassing Safe (SSBS)352worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).353lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes354client # [ 0.005950] alternatives: applying system-wide alternatives355lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)356client # [ 0.009018] CPU features: detected: BBM Level 2 without TLB conflict abort357worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns358worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/8r3nnn9b2hsc90akzwn4pnxvw20y65vr-closure-info/registration", will be passed to user space.359lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)360worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns361worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes362worker1 # [ 0.000038] arm-pv: using stolen time PV363lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB364client # [ 0.009202] Memory: 1406496K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)365client # [ 0.009636] devtmpfs: initialized366lb # [ 0.000000] software IO TLB: area num 1.367worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)368worker1 # [ 0.000614] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)369client # [ 0.011537] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)370lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB371worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)372worker1 # [ 0.000774] Console: colour dummy device 80x25373worker1 # [ 0.000782] printk: legacy console [tty0] enabled374lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)375worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB376client # [ 0.011565] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).377lb # [ 0.000000] Fallback order for Node 0: 0378worker2 # [ 0.000000] software IO TLB: area num 1.379client # [ 0.011756] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL380worker1 # [ 0.000983] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)381worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB382client # [ 0.011760] 0 pages in range for non-PLT usage383lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216384worker1 # [ 0.000989] pid_max: default: 32768 minimum: 301385lb # [ 0.000000] Policy zone: DMA386client # [ 0.011761] 508288 pages in range for PLT usage387worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)388worker1 # [ 0.001070] LSM: initializing lsm=capability,landlock,yama,bpf,ima389client # [ 0.011884] pinctrl core: initialized pinctrl subsystem390lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off391worker2 # [ 0.000000] Fallback order for Node 0: 0392client # [ 0.012708] DMI not present or invalid.393worker1 # [ 0.001237] landlock: Up and running.394lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1395worker1 # [ 0.001239] Yama: becoming mindful.396worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216397client # [ 0.016572] NET: Registered PF_NETLINK/PF_ROUTE protocol family398lb # [ 0.000000] allocated 3145728 bytes of page_ext399worker1 # [ 0.001859] LSM support for eBPF active400worker2 # [ 0.000000] Policy zone: DMA401lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages402client # [ 0.018836] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations403worker1 # [ 0.002009] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)404worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off405lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups406worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1407client # [ 0.019041] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations408worker1 # [ 0.002048] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)409lb # [ 0.000000] rcu: Hierarchical RCU implementation.410worker2 # [ 0.000000] allocated 3145728 bytes of page_ext411lb # [ 0.000000] rcu: RCU event tracing is enabled.412worker1 # [ 0.003173] cacheinfo: Unable to detect cache hierarchy for CPU 0413client # [ 0.019251] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations414worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages415worker1 # [ 0.003912] rcu: Hierarchical SRCU implementation.416client # [ 0.019278] audit: initializing netlink subsys (disabled)417lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.418worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups419worker1 # [ 0.003917] rcu: Max phase no-delay instances is 1000.420lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.421client # [ 0.019939] thermal_sys: Registered thermal governor 'fair_share'422worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.423worker1 # [ 0.005134] fsl-mc MSI: its@8080000 domain created424lb # [ 0.000000] Rude variant of Tasks RCU enabled.425worker2 # [ 0.000000] rcu: RCU event tracing is enabled.426client # [ 0.019941] thermal_sys: Registered thermal governor 'bang_bang'427worker1 # [ 0.005226] EFI services will not be available.428lb # [ 0.000000] Tracing variant of Tasks RCU enabled.429client # [ 0.019945] thermal_sys: Registered thermal governor 'step_wise'430worker1 # [ 0.005353] smp: Bringing up secondary CPUs ...431worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.432lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.433worker1 # [ 0.005361] smp: Brought up 1 node, 1 CPU434worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.435client # [ 0.019951] thermal_sys: Registered thermal governor 'user_space'436worker1 # [ 0.005364] SMP: Total of 1 processors activated.437worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.438lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1439client # [ 0.019953] thermal_sys: Registered thermal governor 'power_allocator'440worker1 # [ 0.005370] CPU: All CPU(s) started at EL1441worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.442worker1 # [ 0.005381] CPU features: detected: Branch Target Identification443lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.444client # [ 0.019981] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1445worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.446client # [ 0.019990] cpuidle: using governor ladder447worker1 # [ 0.005385] CPU features: detected: ARMv8.4 Translation Table Level448client # [ 0.019996] cpuidle: using governor menu449worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1450lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.451worker1 # [ 0.005389] CPU features: detected: Instruction cache invalidation not required for I/D coherence452client # [ 0.020197] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.453worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.454lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.455client # [ 0.020213] ASID allocator initialised with 65536 entries456lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0457worker1 # [ 0.005393] CPU features: detected: Data cache clean to the PoU not required for I/D coherence458client # [ 0.021468] Serial: AMBA PL011 UART driver459worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.460lb # [ 0.000000] GICv3: 256 SPIs implemented461worker1 # [ 0.005396] CPU features: detected: Common not Private translations462lb # [ 0.000000] GICv3: 0 Extended SPIs implemented463client # [ 0.026963] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1464worker1 # [ 0.005399] CPU features: detected: CRC32 instructions465worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.466lb # [ 0.000000] Root IRQ handler: gic_handle_irq467client # [ 0.027151] printk: console [ttyAMA0] enabled468lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI469worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0470worker1 # [ 0.005402] CPU features: detected: Data cache clean to Point of Deep Persistence471worker2 # [ 0.000000] GICv3: 256 SPIs implemented472lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0473worker1 # [ 0.005406] CPU features: detected: Data cache clean to Point of Persistence474worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented475lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000476worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq477worker1 # [ 0.005409] CPU features: detected: Data independent timing control (DIT)478lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]479worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI480worker1 # [ 0.005412] CPU features: detected: E0PD481worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0482worker1 # [ 0.005414] CPU features: detected: Enhanced Counter Virtualization483lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)484worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000485worker1 # [ 0.005417] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)486worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]487lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)488worker1 # [ 0.005421] CPU features: detected: Enhanced Virtualization Traps489lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000490worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)491worker1 # [ 0.005423] CPU features: detected: Fine Grained Traps492lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000493worker1 # [ 0.005427] CPU features: detected: Generic authentication (architected QARMA5 algorithm)494worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)495lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.496worker1 # [ 0.005432] CPU features: detected: RCpc load-acquire (LDAPR)497worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000498worker1 # [ 0.005435] CPU features: detected: LSE atomic instructions499lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns500worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000501worker1 # [ 0.005438] CPU features: detected: Privileged Access Never502lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).503worker1 # [ 0.005440] CPU features: detected: PMUv3504worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.505client # [ 0.150350] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages506worker1 # [ 0.005443] CPU features: detected: RAS Extension Support507client # [ 0.150372] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page508lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns509worker1 # [ 0.005446] CPU features: detected: RASv1p1 Extension Support510worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns511worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).512client # [ 0.150377] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages513worker1 # [ 0.005448] CPU features: detected: Random Number Generator514lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns515lb # [ 0.000031] arm-pv: using stolen time PV516client # [ 0.150381] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page517worker1 # [ 0.005451] CPU features: detected: Speculation barrier (SB)518worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns519worker1 # [ 0.005454] CPU features: detected: Stage-2 Force Write-Back520client # [ 0.150386] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages521lb # [ 0.000434] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)522worker1 # [ 0.005457] CPU features: detected: TLB range maintenance instructions523worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns524client # [ 0.150390] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page525lb # [ 0.000583] Console: colour dummy device 80x25526worker2 # [ 0.000031] arm-pv: using stolen time PV527lb # [ 0.000590] printk: legacy console [tty0] enabled528worker1 # [ 0.005461] CPU features: detected: Speculative Store Bypassing Safe (SSBS)529client # [ 0.150394] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages530worker1 # [ 0.005501] alternatives: applying system-wide alternatives531worker2 # [ 0.000422] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)532client # [ 0.150399] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page533lb # [ 0.000776] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)534worker2 # [ 0.000608] Console: colour dummy device 80x25535worker1 # [ 0.008531] CPU features: detected: BBM Level 2 without TLB conflict abort536lb # [ 0.000782] pid_max: default: 32768 minimum: 301537worker2 # [ 0.000616] printk: legacy console [tty0] enabled538lb # [ 0.000861] LSM: initializing lsm=capability,landlock,yama,bpf,ima539lb # [ 0.001026] landlock: Up and running.540worker1 # [ 0.008765] Memory: 1406536K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)541worker2 # [ 0.000826] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)542client # [ 0.157647] fbcon: Taking over console543lb # [ 0.001029] Yama: becoming mindful.544worker1 # [ 0.009207] devtmpfs: initialized545worker2 # [ 0.000834] pid_max: default: 32768 minimum: 301546lb # [ 0.001523] LSM support for eBPF active547client # [ 0.157663] ACPI: Interpreter disabled.548worker2 # [ 0.000903] LSM: initializing lsm=capability,landlock,yama,bpf,ima549worker1 # [ 0.011038] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)550lb # [ 0.001687] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)551worker2 # [ 0.001045] landlock: Up and running.552worker1 # [ 0.011065] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).553lb # [ 0.001718] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)554worker2 # [ 0.001048] Yama: becoming mindful.555worker2 # [ 0.001496] LSM support for eBPF active556worker1 # [ 0.011261] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL557lb # [ 0.002809] cacheinfo: Unable to detect cache hierarchy for CPU 0558worker2 # [ 0.001649] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)559worker1 # [ 0.011265] 0 pages in range for non-PLT usage560lb # [ 0.003540] rcu: Hierarchical SRCU implementation.561worker1 # [ 0.011266] 508288 pages in range for PLT usage562worker2 # [ 0.001681] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)563lb # [ 0.003544] rcu: Max phase no-delay instances is 1000.564worker1 # [ 0.011362] pinctrl core: initialized pinctrl subsystem565lb # [ 0.004732] fsl-mc MSI: its@8080000 domain created566worker2 # [ 0.002799] cacheinfo: Unable to detect cache hierarchy for CPU 0567worker1 # [ 0.012157] DMI not present or invalid.568lb # [ 0.004832] EFI services will not be available.569worker2 # [ 0.003555] rcu: Hierarchical SRCU implementation.570worker1 # [ 0.015650] NET: Registered PF_NETLINK/PF_ROUTE protocol family571lb # [ 0.004897] smp: Bringing up secondary CPUs ...572worker2 # [ 0.003559] rcu: Max phase no-delay instances is 1000.573lb # [ 0.004906] smp: Brought up 1 node, 1 CPU574worker2 # [ 0.004787] fsl-mc MSI: its@8080000 domain created575worker1 # [ 0.017953] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations576lb # [ 0.004909] SMP: Total of 1 processors activated.577worker2 # [ 0.004879] EFI services will not be available.578lb # [ 0.004914] CPU: All CPU(s) started at EL1579worker1 # [ 0.018133] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations580worker2 # [ 0.004949] smp: Bringing up secondary CPUs ...581lb # [ 0.004925] CPU features: detected: Branch Target Identification582client # [ 0.166412] iommu: Default domain type: Translated583worker1 # [ 0.018341] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations584worker2 # [ 0.004959] smp: Brought up 1 node, 1 CPU585worker2 # [ 0.004962] SMP: Total of 1 processors activated.586lb # [ 0.004930] CPU features: detected: ARMv8.4 Translation Table Level587worker1 # [ 0.018367] audit: initializing netlink subsys (disabled)588client # [ 0.166424] iommu: DMA domain TLB invalidation policy: strict mode589worker2 # [ 0.004968] CPU: All CPU(s) started at EL1590client # [ 0.166808] SCSI subsystem initialized591worker1 # [ 0.018978] thermal_sys: Registered thermal governor 'fair_share'592lb # [ 0.004933] CPU features: detected: Instruction cache invalidation not required for I/D coherence593worker2 # [ 0.004981] CPU features: detected: Branch Target Identification594worker1 # [ 0.018980] thermal_sys: Registered thermal governor 'bang_bang'595worker2 # [ 0.004986] CPU features: detected: ARMv8.4 Translation Table Level596lb # [ 0.004937] CPU features: detected: Data cache clean to the PoU not required for I/D coherence597client # [ 0.168754] usbcore: registered new interface driver usbfs598lb # [ 0.004940] CPU features: detected: Common not Private translations599client # [ 0.168787] usbcore: registered new interface driver hub600worker2 # [ 0.004989] CPU features: detected: Instruction cache invalidation not required for I/D coherence601worker1 # [ 0.018984] thermal_sys: Registered thermal governor 'step_wise'602lb # [ 0.004944] CPU features: detected: CRC32 instructions603client # [ 0.168806] usbcore: registered new device driver usb604worker2 # [ 0.004993] CPU features: detected: Data cache clean to the PoU not required for I/D coherence605client # [ 0.169101] pps_core: LinuxPPS API ver. 1 registered606worker1 # [ 0.018990] thermal_sys: Registered thermal governor 'user_space'607lb # [ 0.004947] CPU features: detected: Data cache clean to Point of Deep Persistence608worker2 # [ 0.004996] CPU features: detected: Common not Private translations609worker1 # [ 0.018992] thermal_sys: Registered thermal governor 'power_allocator'610client # [ 0.169107] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>611lb # [ 0.004951] CPU features: detected: Data cache clean to Point of Persistence612worker2 # [ 0.005000] CPU features: detected: CRC32 instructions613client # [ 0.169117] PTP clock support registered614worker1 # [ 0.019017] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1615lb # [ 0.004954] CPU features: detected: Data independent timing control (DIT)616client # [ 0.169169] EDAC MC: Ver: 3.0.0617worker2 # [ 0.005003] CPU features: detected: Data cache clean to Point of Deep Persistence618worker1 # [ 0.019026] cpuidle: using governor ladder619lb # [ 0.004957] CPU features: detected: E0PD620worker1 # [ 0.019032] cpuidle: using governor menu621client # [ 0.173703] scmi_core: SCMI protocol bus registered622worker2 # [ 0.005007] CPU features: detected: Data cache clean to Point of Persistence623lb # [ 0.004959] CPU features: detected: Enhanced Counter Virtualization624worker1 # [ 0.019227] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.625worker2 # [ 0.005010] CPU features: detected: Data independent timing control (DIT)626client # [ 0.174666] FPGA manager framework627lb # [ 0.004962] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)628worker2 # [ 0.005013] CPU features: detected: E0PD629worker1 # [ 0.019243] ASID allocator initialised with 65536 entries630client # [ 0.175587] vgaarb: loaded631worker1 # [ 0.020499] Serial: AMBA PL011 UART driver632lb # [ 0.004965] CPU features: detected: Enhanced Virtualization Traps633worker2 # [ 0.005016] CPU features: detected: Enhanced Counter Virtualization634lb # [ 0.004968] CPU features: detected: Fine Grained Traps635client # [ 0.176206] clocksource: Switched to clocksource arch_sys_counter636worker1 # [ 0.025766] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1637worker2 # [ 0.005019] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)638worker1 # [ 0.025925] printk: console [ttyAMA0] enabled639lb # [ 0.004972] CPU features: detected: Generic authentication (architected QARMA5 algorithm)640worker2 # [ 0.005022] CPU features: detected: Enhanced Virtualization Traps641lb # [ 0.004977] CPU features: detected: RCpc load-acquire (LDAPR)642worker2 # [ 0.005026] CPU features: detected: Fine Grained Traps643client # [ 0.178597] VFS: Disk quotas dquot_6.6.0644lb # [ 0.004980] CPU features: detected: LSE atomic instructions645worker2 # [ 0.005030] CPU features: detected: Generic authentication (architected QARMA5 algorithm)646lb # [ 0.004983] CPU features: detected: Privileged Access Never647worker2 # [ 0.005035] CPU features: detected: RCpc load-acquire (LDAPR)648lb # [ 0.004986] CPU features: detected: PMUv3649worker2 # [ 0.005038] CPU features: detected: LSE atomic instructions650lb # [ 0.004988] CPU features: detected: RAS Extension Support651worker2 # [ 0.005041] CPU features: detected: Privileged Access Never652lb # [ 0.004991] CPU features: detected: RASv1p1 Extension Support653worker2 # [ 0.005044] CPU features: detected: PMUv3654client # [ 0.181182] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)655lb # [ 0.004994] CPU features: detected: Random Number Generator656worker2 # [ 0.005047] CPU features: detected: RAS Extension Support657client # [ 0.182358] netfs: FS-Cache loaded658lb # [ 0.004996] CPU features: detected: Speculation barrier (SB)659client # [ 0.182493] pnp: PnP ACPI: disabled660worker2 # [ 0.005049] CPU features: detected: RASv1p1 Extension Support661lb # [ 0.004999] CPU features: detected: Stage-2 Force Write-Back662worker2 # [ 0.005052] CPU features: detected: Random Number Generator663lb # [ 0.005002] CPU features: detected: TLB range maintenance instructions664worker2 # [ 0.005054] CPU features: detected: Speculation barrier (SB)665lb # [ 0.005007] CPU features: detected: Speculative Store Bypassing Safe (SSBS)666worker2 # [ 0.005057] CPU features: detected: Stage-2 Force Write-Back667lb # [ 0.005046] alternatives: applying system-wide alternatives668worker2 # [ 0.005061] CPU features: detected: TLB range maintenance instructions669lb # [ 0.008230] CPU features: detected: BBM Level 2 without TLB conflict abort670worker2 # [ 0.005066] CPU features: detected: Speculative Store Bypassing Safe (SSBS)671worker1 # [ 0.148680] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages672worker2 # [ 0.005105] alternatives: applying system-wide alternatives673worker1 # [ 0.148703] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page674lb # [ 0.008422] Memory: 1406540K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)675worker2 # [ 0.008127] CPU features: detected: BBM Level 2 without TLB conflict abort676client # [ 0.186290] NET: Registered PF_INET protocol family677worker1 # [ 0.148708] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages678lb # [ 0.008860] devtmpfs: initialized679client # [ 0.186552] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)680worker1 # [ 0.148713] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page681worker2 # [ 0.008346] Memory: 1406552K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)682worker1 # [ 0.148717] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages683worker2 # [ 0.008772] devtmpfs: initialized684worker1 # [ 0.148721] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page685worker2 # [ 0.010589] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)686worker1 # [ 0.148725] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages687worker2 # [ 0.010612] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).688worker1 # [ 0.148730] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page689worker2 # [ 0.010812] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL690worker2 # [ 0.010816] 0 pages in range for non-PLT usage691worker2 # [ 0.010817] 508288 pages in range for PLT usage692worker1 # [ 0.156241] fbcon: Taking over console693worker2 # [ 0.010969] pinctrl core: initialized pinctrl subsystem694worker1 # [ 0.156259] ACPI: Interpreter disabled.695worker2 # [ 0.011699] DMI not present or invalid.696worker2 # [ 0.014964] NET: Registered PF_NETLINK/PF_ROUTE protocol family697worker1 # [ 0.158168] iommu: Default domain type: Translated698worker2 # [ 0.017148] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations699worker1 # [ 0.158178] iommu: DMA domain TLB invalidation policy: strict mode700lb # [ 0.010758] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)701worker1 # [ 0.159887] SCSI subsystem initialized702worker2 # [ 0.017332] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations703lb # [ 0.010781] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).704worker2 # [ 0.017537] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations705worker1 # [ 0.160571] usbcore: registered new interface driver usbfs706worker2 # [ 0.017563] audit: initializing netlink subsys (disabled)707lb # [ 0.010963] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL708worker1 # [ 0.160601] usbcore: registered new interface driver hub709lb # [ 0.010968] 0 pages in range for non-PLT usage710worker1 # [ 0.160619] usbcore: registered new device driver usb711worker2 # [ 0.018154] thermal_sys: Registered thermal governor 'fair_share'712lb # [ 0.010969] 508288 pages in range for PLT usage713worker2 # [ 0.018156] thermal_sys: Registered thermal governor 'bang_bang'714lb # [ 0.011068] pinctrl core: initialized pinctrl subsystem715lb # [ 0.011800] DMI not present or invalid.716worker2 # [ 0.018160] thermal_sys: Registered thermal governor 'step_wise'717lb # [ 0.014912] NET: Registered PF_NETLINK/PF_ROUTE protocol family718worker2 # [ 0.018165] thermal_sys: Registered thermal governor 'user_space'719lb # [ 0.017306] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations720worker2 # [ 0.018168] thermal_sys: Registered thermal governor 'power_allocator'721lb # [ 0.017488] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations722worker2 # [ 0.018192] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1723worker2 # [ 0.018201] cpuidle: using governor ladder724lb # [ 0.017721] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations725worker2 # [ 0.018207] cpuidle: using governor menu726lb # [ 0.017747] audit: initializing netlink subsys (disabled)727worker2 # [ 0.018414] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.728lb # [ 0.018409] thermal_sys: Registered thermal governor 'fair_share'729worker2 # [ 0.018430] ASID allocator initialised with 65536 entries730worker1 # [ 0.168924] pps_core: LinuxPPS API ver. 1 registered731worker2 # [ 0.019646] Serial: AMBA PL011 UART driver732lb # [ 0.018411] thermal_sys: Registered thermal governor 'bang_bang'733lb # [ 0.018415] thermal_sys: Registered thermal governor 'step_wise'734worker1 # [ 0.168934] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>735worker2 # [ 0.024826] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1736worker1 # [ 0.168951] PTP clock support registered737lb # [ 0.018421] thermal_sys: Registered thermal governor 'user_space'738worker2 # [ 0.024953] printk: console [ttyAMA0] enabled739worker1 # [ 0.169006] EDAC MC: Ver: 3.0.0740lb # [ 0.018423] thermal_sys: Registered thermal governor 'power_allocator'741worker1 # [ 0.171682] scmi_core: SCMI protocol bus registered742lb # [ 0.018449] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1743worker1 # [ 0.172671] FPGA manager framework744lb # [ 0.018458] cpuidle: using governor ladder745worker1 # [ 0.173649] vgaarb: loaded746lb # [ 0.018463] cpuidle: using governor menu747lb # [ 0.018655] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.748worker1 # [ 0.174299] clocksource: Switched to clocksource arch_sys_counter749lb # [ 0.018671] ASID allocator initialised with 65536 entries750lb # [ 0.019910] Serial: AMBA PL011 UART driver751lb # [ 0.025253] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1752worker1 # [ 0.176562] VFS: Disk quotas dquot_6.6.0753lb # [ 0.025382] printk: console [ttyAMA0] enabled754worker1 # [ 0.176597] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)755lb # [ 0.147471] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages756lb # [ 0.147492] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page757lb # [ 0.147498] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages758worker2 # [ 0.146344] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages759worker1 # [ 0.180345] netfs: FS-Cache loaded760worker1 # [ 0.180454] pnp: PnP ACPI: disabled761lb # [ 0.147502] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page762worker2 # [ 0.146362] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page763worker2 # [ 0.146367] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages764lb # [ 0.147506] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages765worker2 # [ 0.146372] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page766lb # [ 0.147511] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page767worker2 # [ 0.146376] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages768lb # [ 0.147515] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages769lb # [ 0.147519] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page770worker2 # [ 0.146380] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page771worker2 # [ 0.146385] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages772worker2 # [ 0.146389] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page773lb # [ 0.155133] fbcon: Taking over console774worker1 # [ 0.184259] NET: Registered PF_INET protocol family775lb # [ 0.155149] ACPI: Interpreter disabled.776client # [ 0.216544] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)777worker1 # [ 0.184522] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)778lb # [ 0.157048] iommu: Default domain type: Translated779worker2 # [ 0.153949] fbcon: Taking over console780client # [ 0.216601] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)781worker2 # [ 0.153966] ACPI: Interpreter disabled.782lb # [ 0.157058] iommu: DMA domain TLB invalidation policy: strict mode783client # [ 0.216635] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)784worker2 # [ 0.155834] iommu: Default domain type: Translated785lb # [ 0.158746] SCSI subsystem initialized786client # [ 0.216728] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)787worker2 # [ 0.155845] iommu: DMA domain TLB invalidation policy: strict mode788client # [ 0.216872] TCP: Hash tables configured (established 16384 bind 16384)789worker2 # [ 0.157574] SCSI subsystem initialized790client # [ 0.216962] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)791client # [ 0.217005] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)792client # [ 0.217042] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)793client # [ 0.217148] NET: Registered PF_UNIX/PF_LOCAL protocol family794client # [ 0.217187] NET: Registered PF_XDP protocol family795client # [ 0.217207] PCI: CLS 0 bytes, default 64796lb # [ 0.163939] usbcore: registered new interface driver usbfs797client # [ 0.217462] Trying to unpack rootfs image as initramfs...798lb # [ 0.163972] usbcore: registered new interface driver hub799lb # [ 0.163990] usbcore: registered new device driver usb800worker2 # [ 0.162808] usbcore: registered new interface driver usbfs801lb # [ 0.164293] pps_core: LinuxPPS API ver. 1 registered802worker2 # [ 0.162840] usbcore: registered new interface driver hub803worker2 # [ 0.162858] usbcore: registered new device driver usb804lb # [ 0.164299] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>805worker2 # [ 0.163158] pps_core: LinuxPPS API ver. 1 registered806lb # [ 0.164309] PTP clock support registered807lb # [ 0.164370] EDAC MC: Ver: 3.0.0808worker2 # [ 0.163164] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>809lb # [ 0.169186] scmi_core: SCMI protocol bus registered810worker2 # [ 0.163174] PTP clock support registered811worker2 # [ 0.163223] EDAC MC: Ver: 3.0.0812lb # [ 0.170180] FPGA manager framework813worker2 # [ 0.167926] scmi_core: SCMI protocol bus registered814lb # [ 0.171177] vgaarb: loaded815client # [ 0.234088] kvm [1]: HYP mode not available816worker2 # [ 0.168945] FPGA manager framework817lb # [ 0.171830] clocksource: Switched to clocksource arch_sys_counter818worker2 # [ 0.169913] vgaarb: loaded819worker2 # [ 0.170576] clocksource: Switched to clocksource arch_sys_counter820worker2 # [ 0.174478] VFS: Disk quotas dquot_6.6.0821worker2 # [ 0.174515] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)822lb # [ 0.178028] VFS: Disk quotas dquot_6.6.0823lb # [ 0.178077] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)824lb # [ 0.179806] netfs: FS-Cache loaded825lb # [ 0.180402] pnp: PnP ACPI: disabled826worker2 # [ 0.178374] netfs: FS-Cache loaded827worker2 # [ 0.178512] pnp: PnP ACPI: disabled828worker2 # [ 0.182553] NET: Registered PF_INET protocol family829lb # [ 0.183973] NET: Registered PF_INET protocol family830lb # [ 0.184228] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)831worker2 # [ 0.183478] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)832worker1 # [ 0.214901] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)833worker1 # [ 0.214961] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)834worker1 # [ 0.214993] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)835worker1 # [ 0.215074] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)836worker1 # [ 0.215218] TCP: Hash tables configured (established 16384 bind 16384)837worker1 # [ 0.215329] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)838worker1 # [ 0.215371] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)839worker1 # [ 0.215409] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)840worker1 # [ 0.215502] NET: Registered PF_UNIX/PF_LOCAL protocol family841worker1 # [ 0.215528] NET: Registered PF_XDP protocol family842worker1 # [ 0.215549] PCI: CLS 0 bytes, default 64843worker1 # [ 0.215813] Trying to unpack rootfs image as initramfs...844worker1 # [ 0.232242] kvm [1]: HYP mode not available845lb # [ 0.215124] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)846lb # [ 0.215184] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)847worker2 # [ 0.213395] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)848lb # [ 0.215217] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)849worker2 # [ 0.213449] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)850lb # [ 0.215304] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)851worker2 # [ 0.213481] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)852lb # [ 0.215449] TCP: Hash tables configured (established 16384 bind 16384)853worker2 # [ 0.213572] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)854lb # [ 0.215545] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)855worker2 # [ 0.213716] TCP: Hash tables configured (established 16384 bind 16384)856lb # [ 0.215587] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)857worker2 # [ 0.213807] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)858lb # [ 0.215626] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)859worker2 # [ 0.213844] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)860lb # [ 0.215729] NET: Registered PF_UNIX/PF_LOCAL protocol family861lb # [ 0.215759] NET: Registered PF_XDP protocol family862worker2 # [ 0.213881] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)863lb # [ 0.215780] PCI: CLS 0 bytes, default 64864worker2 # [ 0.213972] NET: Registered PF_UNIX/PF_LOCAL protocol family865lb # [ 0.215925] Trying to unpack rootfs image as initramfs...866worker2 # [ 0.213996] NET: Registered PF_XDP protocol family867worker2 # [ 0.214015] PCI: CLS 0 bytes, default 64868worker2 # [ 0.214273] Trying to unpack rootfs image as initramfs...869lb # [ 0.233990] kvm [1]: HYP mode not available870worker2 # [ 0.232520] kvm [1]: HYP mode not available871client # [ 0.364723] Initialise system trusted keyrings872client # [ 0.372277] workingset: timestamp_bits=42 max_order=19 bucket_order=0873client # [ 0.373737] squashfs: version 4.0 (2009/01/31) Phillip Lougher874client # [ 0.374576] 9p: Installing v9fs 9p2000 file system support875client # [ 0.395314] Key type asymmetric registered876client # [ 0.395344] Asymmetric key parser 'x509' registered877client # [ 0.395421] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878worker1 # [ 0.362344] Initialise system trusted keyrings879worker1 # [ 0.363144] workingset: timestamp_bits=42 max_order=19 bucket_order=0880worker1 # [ 0.364510] squashfs: version 4.0 (2009/01/31) Phillip Lougher881client # [ 0.404271] io scheduler mq-deadline registered882client # [ 0.404293] io scheduler kyber registered883client # [ 0.409458] pl061_gpio 9030000.pl061: PL061 GPIO chip registered884worker1 # [ 0.370355] 9p: Installing v9fs 9p2000 file system support885client # [ 0.416278] ledtrig-cpu: registered to indicate activity on CPUs886client # [ 0.416831] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:887client # [ 0.416853] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000888client # [ 0.416865] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000889worker1 # [ 0.390340] Key type asymmetric registered890client # [ 0.416873] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000891worker1 # [ 0.390368] Asymmetric key parser 'x509' registered892worker1 # [ 0.390446] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)893client # [ 0.416901] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits894worker1 # [ 0.392719] io scheduler mq-deadline registered895client # [ 0.416930] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]896worker1 # [ 0.392731] io scheduler kyber registered897client # [ 0.417017] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00898client # [ 0.417026] pci_bus 0000:00: root bus resource [bus 00-ff]899client # [ 0.417041] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]900worker2 # [ 0.362183] Initialise system trusted keyrings901client # [ 0.417047] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]902worker2 # [ 0.363013] workingset: timestamp_bits=42 max_order=19 bucket_order=0903client # [ 0.417053] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]904worker2 # [ 0.364335] squashfs: version 4.0 (2009/01/31) Phillip Lougher905client # [ 0.417141] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint906client # [ 0.417590] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint907client # [ 0.417778] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]908worker2 # [ 0.365113] 9p: Installing v9fs 9p2000 file system support909client # [ 0.417795] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]910client # [ 0.417824] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]911client # [ 0.417840] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]912lb # [ 0.372418] Initialise system trusted keyrings913client # [ 0.418340] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint914client # [ 0.418523] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]915lb # [ 0.373196] workingset: timestamp_bits=42 max_order=19 bucket_order=0916client # [ 0.418539] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]917lb # [ 0.374578] squashfs: version 4.0 (2009/01/31) Phillip Lougher918client # [ 0.418568] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]919client # [ 0.419030] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint920lb # [ 0.375396] 9p: Installing v9fs 9p2000 file system support921client # [ 0.419215] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]922client # [ 0.419230] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]923client # [ 0.419260] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]924client # [ 0.419706] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint925client # [ 0.419888] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]926client # [ 0.419904] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]927client # [ 0.419941] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]928client # [ 0.420471] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint929client # [ 0.420661] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]930client # [ 0.420677] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]931worker1 # [ 0.406451] pl061_gpio 9030000.pl061: PL061 GPIO chip registered932worker1 # [ 0.407084] ledtrig-cpu: registered to indicate activity on CPUs933client # [ 0.420707] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]934worker1 # [ 0.407483] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:935client # [ 0.421196] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint936client # [ 0.421391] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]937worker1 # [ 0.407499] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000938client # [ 0.421407] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]939worker1 # [ 0.407511] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000940client # [ 0.421437] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]941worker1 # [ 0.407520] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000942client # [ 0.421897] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint943client # [ 0.422082] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]944worker1 # [ 0.407541] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits945client # [ 0.422098] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]946worker1 # [ 0.407567] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]947client # [ 0.422127] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]948worker1 # [ 0.407651] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00949worker2 # [ 0.389078] Key type asymmetric registered950client # [ 0.422144] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]951worker2 # [ 0.389111] Asymmetric key parser 'x509' registered952worker1 # [ 0.407660] pci_bus 0000:00: root bus resource [bus 00-ff]953client # [ 0.422608] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint954worker1 # [ 0.407666] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]955worker2 # [ 0.389187] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)956client # [ 0.422798] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]957worker1 # [ 0.407671] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]958worker2 # [ 0.391374] io scheduler mq-deadline registered959client # [ 0.422828] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]960worker2 # [ 0.391384] io scheduler kyber registered961worker1 # [ 0.407676] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]962client # [ 0.423290] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint963worker1 # [ 0.407800] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint964client # [ 0.423479] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]965worker1 # [ 0.408269] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint966client # [ 0.423509] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]967worker1 # [ 0.408458] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]968client # [ 0.423916] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint969worker1 # [ 0.408474] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]970client # [ 0.424113] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]971worker1 # [ 0.408504] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]972worker1 # [ 0.408520] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]973worker1 # [ 0.408984] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint974worker1 # [ 0.409170] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]975worker1 # [ 0.409185] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]976worker1 # [ 0.409214] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]977lb # [ 0.404111] Key type asymmetric registered978worker1 # [ 0.409665] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint979lb # [ 0.404141] Asymmetric key parser 'x509' registered980worker1 # [ 0.409850] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]981lb # [ 0.404217] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)982worker1 # [ 0.409867] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]983lb # [ 0.406404] io scheduler mq-deadline registered984worker1 # [ 0.409895] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]985lb # [ 0.406415] io scheduler kyber registered986worker1 # [ 0.410366] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint987client # [ 0.472489] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint988worker1 # [ 0.410560] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]989client # [ 0.472818] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]990worker1 # [ 0.410576] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]991client # [ 0.472836] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]992worker1 # [ 0.410609] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]993client # [ 0.472867] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]994worker1 # [ 0.411086] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint995client # [ 0.473343] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint996worker1 # [ 0.411273] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]997client # [ 0.473532] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]998worker1 # [ 0.411288] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]999client # [ 0.473548] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1000worker1 # [ 0.411318] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1001client # [ 0.473578] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1002worker1 # [ 0.411784] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1003client # [ 0.474186] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1004worker1 # [ 0.411968] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1005client # [ 0.474198] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1006worker1 # [ 0.411983] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1007client # [ 0.474204] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1008worker1 # [ 0.412012] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1009lb # [ 0.416096] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1010client # [ 0.474250] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1011worker2 # [ 0.406734] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1012worker1 # [ 0.412475] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1013worker1 # [ 0.412659] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1014worker2 # [ 0.407453] ledtrig-cpu: registered to indicate activity on CPUs1015client # [ 0.474298] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1016worker1 # [ 0.412674] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1017worker2 # [ 0.407924] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1018client # [ 0.474346] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1019worker1 # [ 0.412704] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1020lb # [ 0.417627] ledtrig-cpu: registered to indicate activity on CPUs1021worker2 # [ 0.407942] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001022client # [ 0.474393] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1023worker1 # [ 0.412719] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1024lb # [ 0.418073] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1025worker2 # [ 0.407955] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001026client # [ 0.474441] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1027worker1 # [ 0.413205] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1028lb # [ 0.418091] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001029worker1 # [ 0.413400] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1030client # [ 0.474488] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1031worker2 # [ 0.407962] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001032worker1 # [ 0.413429] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033lb # [ 0.418112] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001034client # [ 0.474537] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1035worker2 # [ 0.407991] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1036worker1 # [ 0.413888] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1037lb # [ 0.418120] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001038client # [ 0.474585] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1039worker2 # [ 0.408020] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1040worker1 # [ 0.414080] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1041client # [ 0.474633] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1042lb # [ 0.418141] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1043worker2 # [ 0.408107] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001044worker1 # [ 0.414111] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1045worker2 # [ 0.408116] pci_bus 0000:00: root bus resource [bus 00-ff]1046client # [ 0.474706] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1047lb # [ 0.418168] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1048worker2 # [ 0.408122] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1049client # [ 0.474753] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1050lb # [ 0.418247] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001051worker2 # [ 0.408128] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1052client # [ 0.474774] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1053lb # [ 0.418257] pci_bus 0000:00: root bus resource [bus 00-ff]1054worker2 # [ 0.408133] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1055lb # [ 0.418263] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1056client # [ 0.474795] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1057worker2 # [ 0.408257] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1058client # [ 0.474817] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1059lb # [ 0.418268] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1060client # [ 0.474842] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1061lb # [ 0.418273] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1062worker2 # [ 0.408702] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1063client # [ 0.474863] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1064worker2 # [ 0.408891] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1065lb # [ 0.418333] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1066worker2 # [ 0.408907] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1067client # [ 0.474885] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1068lb # [ 0.418779] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1069worker2 # [ 0.408937] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070client # [ 0.474907] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1071lb # [ 0.418972] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1072worker2 # [ 0.408958] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1073client # [ 0.474929] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1074lb # [ 0.418988] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1075client # [ 0.474951] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1076worker2 # [ 0.409436] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1077lb # [ 0.419018] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1078worker2 # [ 0.409626] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1079client # [ 0.474975] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1080lb # [ 0.419034] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1081worker2 # [ 0.409642] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1082client # [ 0.474997] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1083lb # [ 0.419487] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1084worker2 # [ 0.409672] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085worker1 # [ 0.462739] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1086client # [ 0.475018] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1087lb # [ 0.419675] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1088client # [ 0.475039] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1089worker1 # [ 0.462975] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1090worker2 # [ 0.410143] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1091lb # [ 0.419691] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1092client # [ 0.475060] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1093worker2 # [ 0.410328] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1094worker1 # [ 0.463249] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1095lb # [ 0.419720] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1096client # [ 0.475082] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1097worker1 # [ 0.463539] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1098worker2 # [ 0.410343] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1099client # [ 0.475103] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1100worker1 # [ 0.463556] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1101worker2 # [ 0.410373] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1102client # [ 0.475124] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1103worker1 # [ 0.463586] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104worker2 # [ 0.410908] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1105client # [ 0.475145] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1106worker2 # [ 0.411094] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1107worker1 # [ 0.464058] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1108client # [ 0.475167] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1109worker2 # [ 0.411110] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1110worker1 # [ 0.464272] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1111client # [ 0.475188] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1112worker1 # [ 0.464288] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1113worker2 # [ 0.411143] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114client # [ 0.475214] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1115worker1 # [ 0.464318] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1116worker2 # [ 0.411620] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1117client # [ 0.475224] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1118worker1 # [ 0.464914] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1119worker2 # [ 0.411810] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1120client # [ 0.475229] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1121worker2 # [ 0.411827] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1122worker1 # [ 0.464925] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1123worker2 # [ 0.411857] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124worker1 # [ 0.464930] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1125worker2 # [ 0.412318] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1126worker1 # [ 0.464975] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1127worker2 # [ 0.412504] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1128worker1 # [ 0.465023] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1129worker2 # [ 0.412519] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1130worker2 # [ 0.412549] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1131worker1 # [ 0.465070] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1132worker2 # [ 0.413008] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1133worker1 # [ 0.465117] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1134worker2 # [ 0.413194] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1135worker1 # [ 0.465163] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1136worker2 # [ 0.413209] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1137worker1 # [ 0.465211] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1138worker2 # [ 0.413239] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1139lb # [ 0.444369] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1140worker2 # [ 0.413255] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1141worker1 # [ 0.465259] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1142lb # [ 0.444598] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1143lb # [ 0.444615] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1144worker2 # [ 0.413788] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1145worker1 # [ 0.465307] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1146client # [ 0.520540] pci 0000:00:0a.0: enabling device (0000 -> 0002)1147worker2 # [ 0.413976] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1148lb # [ 0.444646] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker1 # [ 0.465354] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1150worker2 # [ 0.414006] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1151lb # [ 0.445132] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1152worker1 # [ 0.465468] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1153lb # [ 0.445324] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1154worker2 # [ 0.414467] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1155worker1 # [ 0.465515] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1156lb # [ 0.445340] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1157worker1 # [ 0.465536] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1158lb # [ 0.445373] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1159worker1 # [ 0.465558] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1160lb # [ 0.445868] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1161worker1 # [ 0.465579] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1162lb # [ 0.446062] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1163lb # [ 0.446078] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1164worker1 # [ 0.465604] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1165lb # [ 0.446109] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1166worker1 # [ 0.465625] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1167worker1 # [ 0.465646] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1168lb # [ 0.446623] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1169lb # [ 0.446820] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1170worker1 # [ 0.465667] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1171lb # [ 0.446836] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1172worker1 # [ 0.465689] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1173lb # [ 0.446866] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1174worker1 # [ 0.465711] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1175worker1 # [ 0.465733] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1176lb # [ 0.447335] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1177worker2 # [ 0.458725] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1178lb # [ 0.447527] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1179worker1 # [ 0.465755] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1180worker2 # [ 0.458781] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1181lb # [ 0.447542] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker1 # [ 0.465776] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1183worker2 # [ 0.459307] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1184worker1 # [ 0.465797] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1185lb # [ 0.447572] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1186worker2 # [ 0.459498] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1187worker1 # [ 0.465818] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1188lb # [ 0.447588] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1189worker1 # [ 0.465839] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1190worker2 # [ 0.459749] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1191lb # [ 0.448080] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1192worker1 # [ 0.465859] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1193worker2 # [ 0.460034] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1194lb # [ 0.448276] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1195worker1 # [ 0.465880] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1196worker2 # [ 0.460052] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1197lb # [ 0.448306] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1198worker1 # [ 0.465902] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1199worker2 # [ 0.460082] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200lb # [ 0.448769] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1201worker1 # [ 0.465922] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1202worker2 # [ 0.460556] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1203lb # [ 0.448977] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1204worker1 # [ 0.465944] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1205worker2 # [ 0.460743] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1206worker1 # [ 0.465971] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1207lb # [ 0.449006] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1208worker2 # [ 0.460759] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1209worker1 # [ 0.465981] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1210lb # [ 0.449407] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1211worker2 # [ 0.460789] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1212worker1 # [ 0.465985] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1213lb # [ 0.449604] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1214worker2 # [ 0.461381] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1215lb # [ 0.449855] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1216worker2 # [ 0.461393] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1217lb # [ 0.450175] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1218client # [ 0.541608] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1219worker2 # [ 0.461399] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1220lb # [ 0.450193] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1221client # [ 0.544147] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1222worker2 # [ 0.461445] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1223lb # [ 0.450225] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1224worker2 # [ 0.461493] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1225lb # [ 0.450712] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1226lb # [ 0.450905] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1227worker2 # [ 0.461540] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1228lb # [ 0.450921] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1229worker2 # [ 0.461587] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1230lb # [ 0.450951] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1231worker2 # [ 0.461634] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1232lb # [ 0.451572] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1233worker1 # [ 0.511284] pci 0000:00:0a.0: enabling device (0000 -> 0002)1234worker2 # [ 0.461682] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1235lb # [ 0.451584] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1236worker2 # [ 0.461731] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1237lb # [ 0.451590] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1238worker2 # [ 0.461780] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1239lb # [ 0.451642] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1240worker2 # [ 0.461827] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1241lb # [ 0.451696] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1242worker2 # [ 0.461891] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1243lb # [ 0.451747] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1244worker2 # [ 0.461937] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1245lb # [ 0.451796] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1246client # [ 0.555036] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1247worker2 # [ 0.461959] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1248worker2 # [ 0.461988] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1249worker2 # [ 0.462010] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1250worker2 # [ 0.462036] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1251worker2 # [ 0.462058] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1252worker2 # [ 0.462080] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1253worker2 # [ 0.462102] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1254worker2 # [ 0.462124] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1255worker2 # [ 0.462147] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1256worker2 # [ 0.462170] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1257lb # [ 0.491892] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1258worker2 # [ 0.462193] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1259lb # [ 0.491959] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1260client # [ 0.561415] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1261worker2 # [ 0.462215] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1262client # [ 0.563559] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1263worker2 # [ 0.462237] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1264lb # [ 0.492010] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1265worker2 # [ 0.462258] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1266lb # [ 0.492060] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1267worker2 # [ 0.462280] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1268lb # [ 0.492111] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1269worker2 # [ 0.462303] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1270worker2 # [ 0.462324] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1271lb # [ 0.492244] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1272worker2 # [ 0.462348] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1273lb # [ 0.492296] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1274worker2 # [ 0.462370] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1275lb # [ 0.492319] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1276worker2 # [ 0.462392] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1277lb # [ 0.492343] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1278worker1 # [ 0.532394] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1279worker2 # [ 0.462420] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1280lb # [ 0.492366] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1281worker2 # [ 0.462429] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1282lb # [ 0.492393] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1283worker2 # [ 0.462434] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1284lb # [ 0.492415] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1285lb # [ 0.492437] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1286lb # [ 0.492461] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1287lb # [ 0.492483] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1288client # [ 0.573806] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1289lb # [ 0.492506] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1290client # [ 0.576057] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1291lb # [ 0.492530] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1292lb # [ 0.492553] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1293lb # [ 0.492575] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1294lb # [ 0.492597] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1295lb # [ 0.492619] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1296lb # [ 0.492640] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1297lb # [ 0.492662] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1298lb # [ 0.492684] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1299worker2 # [ 0.511788] pci 0000:00:0a.0: enabling device (0000 -> 0002)1300worker1 # [ 0.542621] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1301lb # [ 0.492707] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1302worker1 # [ 0.546151] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1303lb # [ 0.492729] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1304lb # [ 0.492751] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1305lb # [ 0.492785] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1306lb # [ 0.492795] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1307client # [ 0.587650] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1308lb # [ 0.492800] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1309lb # [ 0.494027] pci 0000:00:0a.0: enabling device (0000 -> 0002)1310client # [ 0.594159] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1311worker1 # [ 0.555772] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1312client # [ 0.600449] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1313client # [ 0.602939] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1314worker2 # [ 0.533005] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1315worker1 # [ 0.567019] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1316worker1 # [ 0.569158] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1317worker1 # [ 0.571226] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1318worker1 # [ 0.573495] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1319lb # [ 0.547000] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1320worker2 # [ 0.543399] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1321worker2 # [ 0.548202] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1322client # [ 0.621540] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1323worker2 # [ 0.550277] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1324worker1 # [ 0.584191] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1325client # [ 0.624594] msm_serial: driver initialized1326lb # [ 0.557451] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1327client # [ 0.624738] SuperH (H)SCI(F) driver initialized1328client # [ 0.624793] STM32 USART driver initialized1329worker1 # [ 0.590555] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1330worker1 # [ 0.593140] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1331lb # [ 0.561502] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1332lb # [ 0.563634] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333lb # [ 0.565828] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1334worker2 # [ 0.564143] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1335worker2 # [ 0.566382] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1336worker2 # [ 0.570348] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1337worker2 # [ 0.573601] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1338worker1 # [ 0.611421] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1339worker1 # [ 0.614264] msm_serial: driver initialized1340worker1 # [ 0.615023] SuperH (H)SCI(F) driver initialized1341worker1 # [ 0.615082] STM32 USART driver initialized1342worker2 # [ 0.584410] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1343lb # [ 0.584151] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1344lb # [ 0.586443] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1345lb # [ 0.588757] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1346client # [ 0.662053] loop: module loaded1347lb # [ 0.591128] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1348client # [ 0.662289] virtio_blk virtio5: 1/0/0 default/read/poll queues1349client # [ 0.663141] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1350lb # [ 0.601446] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1351worker2 # [ 0.594849] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1352worker2 # [ 0.597450] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1353worker2 # [ 0.607890] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1354client # [ 0.673098] megasas: 07.734.00.00-rc11355client # [ 0.673981] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1356worker2 # [ 0.610560] msm_serial: driver initialized1357client # [ 0.676166] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001358client # [ 0.676246] Intel/Sharp Extended Query Table at 0x00311359client # [ 0.678104] Using buffer write method1360client # [ 0.678227] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1361lb # [ 0.612247] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1362lb # [ 0.619126] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1363worker2 # [ 0.614940] SuperH (H)SCI(F) driver initialized1364worker1 # [ 0.650596] loop: module loaded1365worker2 # [ 0.615011] STM32 USART driver initialized1366worker1 # [ 0.650813] virtio_blk virtio5: 1/0/0 default/read/poll queues1367worker1 # [ 0.651628] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1368lb # [ 0.625258] msm_serial: driver initialized1369client # [ 0.692279] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001370lb # [ 0.625435] SuperH (H)SCI(F) driver initialized1371client # [ 0.692321] Intel/Sharp Extended Query Table at 0x00311372lb # [ 0.625492] STM32 USART driver initialized1373client # [ 0.694125] Using buffer write method1374client # [ 0.694167] Concatenating MTD devices:1375client # [ 0.694171] (0): "0.flash"1376client # [ 0.694175] (1): "0.flash"1377client # [ 0.694179] into device "0.flash"1378worker1 # [ 0.659046] megasas: 07.734.00.00-rc11379worker1 # [ 0.659791] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1380worker1 # [ 0.662095] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001381worker1 # [ 0.662128] Intel/Sharp Extended Query Table at 0x00311382worker1 # [ 0.664028] Using buffer write method1383worker1 # [ 0.664114] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1384worker1 # [ 0.666026] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001385worker1 # [ 0.666062] Intel/Sharp Extended Query Table at 0x00311386worker2 # [ 0.648930] loop: module loaded1387worker2 # [ 0.649157] virtio_blk virtio5: 1/0/0 default/read/poll queues1388worker1 # [ 0.685001] Using buffer write method1389worker1 # [ 0.685052] Concatenating MTD devices:1390worker1 # [ 0.685057] (0): "0.flash"1391worker2 # [ 0.649998] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1392worker1 # [ 0.685061] (1): "0.flash"1393worker1 # [ 0.685065] into device "0.flash"1394lb # [ 0.663035] loop: module loaded1395lb # [ 0.663261] virtio_blk virtio5: 1/0/0 default/read/poll queues1396lb # [ 0.665349] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1397worker2 # [ 0.658744] megasas: 07.734.00.00-rc11398worker2 # [ 0.659723] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399worker2 # [ 0.661836] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001400worker2 # [ 0.661871] Intel/Sharp Extended Query Table at 0x00311401worker2 # [ 0.663666] Using buffer write method1402worker2 # [ 0.663747] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1403worker2 # [ 0.665691] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001404worker2 # [ 0.665723] Intel/Sharp Extended Query Table at 0x00311405lb # [ 0.676659] megasas: 07.734.00.00-rc11406lb # [ 0.677443] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1407lb # [ 0.679522] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408lb # [ 0.679557] Intel/Sharp Extended Query Table at 0x00311409worker2 # [ 0.685373] Using buffer write method1410lb # [ 0.681269] Using buffer write method1411worker2 # [ 0.685428] Concatenating MTD devices:1412worker2 # [ 0.685432] (0): "0.flash"1413worker2 # [ 0.685437] (1): "0.flash"1414lb # [ 0.681364] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1415worker2 # [ 0.685440] into device "0.flash"1416lb # [ 0.683232] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417lb # [ 0.683270] Intel/Sharp Extended Query Table at 0x00311418lb # [ 0.703715] Using buffer write method1419lb # [ 0.703771] Concatenating MTD devices:1420lb # [ 0.703775] (0): "0.flash"1421lb # [ 0.703780] (1): "0.flash"1422lb # [ 0.703783] into device "0.flash"1423client # [ 0.989263] Freeing initrd memory: 26160K1424client # [ 0.996292] tun: Universal TUN/TAP device driver, 1.61425client # [ 1.000678] thunder_xcv, ver 1.01426client # [ 1.000732] thunder_bgx, ver 1.01427client # [ 1.000756] nicpf, ver 1.01428client # [ 1.001355] e1000: Intel(R) PRO/1000 Network Driver1429client # [ 1.001364] e1000: Copyright (c) 1999-2006 Intel Corporation.1430client # [ 1.001390] e1000e: Intel(R) PRO/1000 Network Driver1431client # [ 1.001400] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1432client # [ 1.001435] igb: Intel(R) Gigabit Ethernet Network Driver1433worker1 # [ 0.970006] Freeing initrd memory: 26148K1434client # [ 1.001441] igb: Copyright (c) 2007-2014 Intel Corporation.1435client # [ 1.001471] igbvf: Intel(R) Gigabit Virtual Function Network Driver1436client # [ 1.001477] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1437client # [ 1.001625] sky2: driver version 1.301438client # [ 1.003384] usbcore: registered new interface driver usb-storage1439client # [ 1.003440] usbcore: registered new interface driver usbserial_generic1440client # [ 1.003456] usbserial: USB Serial support registered for generic1441worker1 # [ 0.976643] tun: Universal TUN/TAP device driver, 1.61442client # [ 1.004087] hv_vmbus: registering driver hyperv_keyboard1443client # [ 1.005013] ehci-pci 0000:00:0a.0: EHCI Host Controller1444client # [ 1.005055] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11445client # [ 1.005241] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001446client # [ 1.016500] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001447client # [ 1.016958] hub 1-0:1.0: USB hub found1448worker1 # [ 0.980780] thunder_xcv, ver 1.01449client # [ 1.016976] hub 1-0:1.0: 6 ports detected1450worker1 # [ 0.980829] thunder_bgx, ver 1.01451worker1 # [ 0.980856] nicpf, ver 1.01452worker1 # [ 0.981434] e1000: Intel(R) PRO/1000 Network Driver1453worker1 # [ 0.981442] e1000: Copyright (c) 1999-2006 Intel Corporation.1454worker1 # [ 0.981467] e1000e: Intel(R) PRO/1000 Network Driver1455client # [ 1.021842] rtc-pl031 9010000.pl031: registered as rtc01456worker1 # [ 0.981476] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457worker1 # [ 0.981509] igb: Intel(R) Gigabit Ethernet Network Driver1458client # [ 1.021901] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:47:29 UTC (1789908449)1459worker1 # [ 0.981515] igb: Copyright (c) 2007-2014 Intel Corporation.1460client # [ 1.022243] i2c_dev: i2c /dev entries driver1461worker1 # [ 0.981539] igbvf: Intel(R) Gigabit Virtual Function Network Driver1462worker1 # [ 0.981545] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1463worker1 # [ 0.981685] sky2: driver version 1.301464client # [ 1.027478] sdhci: Secure Digital Host Controller Interface driver1465client # [ 1.027498] sdhci: Copyright(c) Pierre Ossman1466client # [ 1.027777] Synopsys Designware Multimedia Card Interface Driver1467client # [ 1.028169] sdhci-pltfm: SDHCI platform and OF driver helper1468worker1 # [ 0.990878] usbcore: registered new interface driver usb-storage1469worker1 # [ 0.991007] usbcore: registered new interface driver usbserial_generic1470worker1 # [ 0.991022] usbserial: USB Serial support registered for generic1471worker1 # [ 0.991648] hv_vmbus: registering driver hyperv_keyboard1472worker2 # [ 0.961315] Freeing initrd memory: 26148K1473worker1 # [ 0.993068] ehci-pci 0000:00:0a.0: EHCI Host Controller1474client # [ 1.032768] hid: raw HID events driver (C) Jiri Kosina1475client # [ 1.033037] usbcore: registered new interface driver usbhid1476worker1 # [ 0.993099] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11477client # [ 1.033044] usbhid: USB HID core driver1478worker1 # [ 0.993317] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001479client # [ 1.036039] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1480worker1 # [ 0.999067] rtc-pl031 9010000.pl031: registered as rtc01481worker1 # [ 0.999100] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:47:29 UTC (1789908449)1482worker2 # [ 0.968128] tun: Universal TUN/TAP device driver, 1.61483client # [ 1.038812] drop_monitor: Initializing network drop monitor service1484worker1 # [ 0.999548] i2c_dev: i2c /dev entries driver1485client # [ 1.039038] NET: Registered PF_INET6 protocol family1486worker1 # [ 1.002745] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001487client # [ 1.041288] Segment Routing with IPv61488worker1 # [ 1.003081] hub 1-0:1.0: USB hub found1489client # [ 1.041321] In-situ OAM (IOAM) with IPv61490worker1 # [ 1.003103] hub 1-0:1.0: 6 ports detected1491client # [ 1.041358] NET: Registered PF_PACKET protocol family1492client # [ 1.043069] 9pnet: Installing 9P2000 support1493lb # [ 0.978180] Freeing initrd memory: 26140K1494worker2 # [ 0.972303] thunder_xcv, ver 1.01495worker2 # [ 0.972358] thunder_bgx, ver 1.01496worker2 # [ 0.972380] nicpf, ver 1.01497worker2 # [ 0.972985] e1000: Intel(R) PRO/1000 Network Driver1498worker1 # [ 1.007309] sdhci: Secure Digital Host Controller Interface driver1499client # [ 1.046153] Key type dns_resolver registered1500worker2 # [ 0.972993] e1000: Copyright (c) 1999-2006 Intel Corporation.1501worker1 # [ 1.007321] sdhci: Copyright(c) Pierre Ossman1502worker2 # [ 0.973019] e1000e: Intel(R) PRO/1000 Network Driver1503worker1 # [ 1.007594] Synopsys Designware Multimedia Card Interface Driver1504worker2 # [ 0.973028] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1505worker1 # [ 1.007984] sdhci-pltfm: SDHCI platform and OF driver helper1506worker2 # [ 0.973058] igb: Intel(R) Gigabit Ethernet Network Driver1507worker1 # [ 1.009779] hid: raw HID events driver (C) Jiri Kosina1508worker2 # [ 0.973063] igb: Copyright (c) 2007-2014 Intel Corporation.1509worker1 # [ 1.010018] usbcore: registered new interface driver usbhid1510worker1 # [ 1.010025] usbhid: USB HID core driver1511worker2 # [ 0.973089] igbvf: Intel(R) Gigabit Virtual Function Network Driver1512worker2 # [ 0.973094] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1513worker2 # [ 0.973237] sky2: driver version 1.301514lb # [ 0.984901] tun: Universal TUN/TAP device driver, 1.61515worker2 # [ 0.975406] ehci-pci 0000:00:0a.0: EHCI Host Controller1516worker2 # [ 0.975437] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11517worker2 # [ 0.975645] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001518worker1 # [ 1.015975] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1519client # [ 1.053098] registered taskstats version 11520worker1 # [ 1.017579] drop_monitor: Initializing network drop monitor service1521client # [ 1.053269] Loading compiled-in X.509 certificates1522worker1 # [ 1.017770] NET: Registered PF_INET6 protocol family1523worker2 # [ 0.985709] usbcore: registered new interface driver usb-storage1524worker1 # [ 1.021016] Segment Routing with IPv61525worker1 # [ 1.021037] In-situ OAM (IOAM) with IPv61526worker2 # [ 0.985821] usbcore: registered new interface driver usbserial_generic1527worker1 # [ 1.021068] NET: Registered PF_PACKET protocol family1528worker2 # [ 0.985843] usbserial: USB Serial support registered for generic1529worker2 # [ 0.986482] hv_vmbus: registering driver hyperv_keyboard1530lb # [ 0.989254] thunder_xcv, ver 1.01531lb # [ 0.989306] thunder_bgx, ver 1.01532worker2 # [ 0.986635] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001533lb # [ 0.989330] nicpf, ver 1.01534worker2 # [ 0.987057] hub 1-0:1.0: USB hub found1535lb # [ 0.989969] e1000: Intel(R) PRO/1000 Network Driver1536worker2 # [ 0.987078] hub 1-0:1.0: 6 ports detected1537client # [ 1.062340] Demotion targets for Node 0: null1538lb # [ 0.989976] e1000: Copyright (c) 1999-2006 Intel Corporation.1539client # [ 1.062477] Key type .fscrypt registered1540lb # [ 0.990002] e1000e: Intel(R) PRO/1000 Network Driver1541client # [ 1.062483] Key type fscrypt-provisioning registered1542worker1 # [ 1.022806] 9pnet: Installing 9P2000 support1543lb # [ 0.990010] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1544worker1 # [ 1.025234] Key type dns_resolver registered1545client # [ 1.062596] ima: No TPM chip found, activating TPM-bypass!1546client # [ 1.062618] ima: Allocated hash algorithm: sha11547lb # [ 0.990040] igb: Intel(R) Gigabit Ethernet Network Driver1548worker2 # [ 0.993188] rtc-pl031 9010000.pl031: registered as rtc01549client # [ 1.062642] ima: No architecture policies found1550lb # [ 0.990046] igb: Copyright (c) 2007-2014 Intel Corporation.1551worker2 # [ 0.993228] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:47:29 UTC (1789908449)1552lb # [ 0.990069] igbvf: Intel(R) Gigabit Virtual Function Network Driver1553worker2 # [ 0.993559] i2c_dev: i2c /dev entries driver1554client # [ 1.066929] input: gpio-keys as /devices/platform/gpio-keys/input/input01555lb # [ 0.990075] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1556lb # [ 0.990223] sky2: driver version 1.301557lb # [ 0.992600] ehci-pci 0000:00:0a.0: EHCI Host Controller1558lb # [ 0.992651] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11559lb # [ 0.992827] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001560worker1 # [ 1.032399] registered taskstats version 11561worker1 # [ 1.032577] Loading compiled-in X.509 certificates1562lb # [ 1.002779] usbcore: registered new interface driver usb-storage1563worker2 # [ 0.998838] sdhci: Secure Digital Host Controller Interface driver1564lb # [ 1.002898] usbcore: registered new interface driver usbserial_generic1565worker2 # [ 0.998850] sdhci: Copyright(c) Pierre Ossman1566lb # [ 1.002921] usbserial: USB Serial support registered for generic1567worker2 # [ 0.999153] Synopsys Designware Multimedia Card Interface Driver1568lb # [ 1.003589] hv_vmbus: registering driver hyperv_keyboard1569worker2 # [ 0.999551] sdhci-pltfm: SDHCI platform and OF driver helper1570lb # [ 1.003908] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001571worker2 # [ 1.001389] hid: raw HID events driver (C) Jiri Kosina1572lb # [ 1.004293] hub 1-0:1.0: USB hub found1573lb # [ 1.004315] hub 1-0:1.0: 6 ports detected1574worker2 # [ 1.001676] usbcore: registered new interface driver usbhid1575worker2 # [ 1.001682] usbhid: USB HID core driver1576worker1 # [ 1.041508] Demotion targets for Node 0: null1577lb # [ 1.010365] rtc-pl031 9010000.pl031: registered as rtc01578worker1 # [ 1.041649] Key type .fscrypt registered1579worker1 # [ 1.041657] Key type fscrypt-provisioning registered1580lb # [ 1.010406] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T12:47:29 UTC (1789908449)1581worker2 # [ 1.007549] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1582worker1 # [ 1.041770] ima: No TPM chip found, activating TPM-bypass!1583lb # [ 1.010755] i2c_dev: i2c /dev entries driver1584worker2 # [ 1.009268] drop_monitor: Initializing network drop monitor service1585worker1 # [ 1.041792] ima: Allocated hash algorithm: sha11586worker2 # [ 1.009457] NET: Registered PF_INET6 protocol family1587worker1 # [ 1.041815] ima: No architecture policies found1588worker2 # [ 1.012741] Segment Routing with IPv61589worker1 # [ 1.046073] input: gpio-keys as /devices/platform/gpio-keys/input/input01590worker2 # [ 1.012764] In-situ OAM (IOAM) with IPv61591worker2 # [ 1.012795] NET: Registered PF_PACKET protocol family1592worker2 # [ 1.014479] 9pnet: Installing 9P2000 support1593client # [ 1.087370] clk: Disabling unused clocks1594lb # [ 1.016160] sdhci: Secure Digital Host Controller Interface driver1595client # [ 1.087414] PM: genpd: Disabling unused power domains1596lb # [ 1.016171] sdhci: Copyright(c) Pierre Ossman1597worker2 # [ 1.017371] Key type dns_resolver registered1598lb # [ 1.016468] Synopsys Designware Multimedia Card Interface Driver1599lb # [ 1.016875] sdhci-pltfm: SDHCI platform and OF driver helper1600lb # [ 1.018766] hid: raw HID events driver (C) Jiri Kosina1601lb # [ 1.019029] usbcore: registered new interface driver usbhid1602lb # [ 1.019036] usbhid: USB HID core driver1603client # [ 1.091816] Freeing unused kernel memory: 4736K1604client # [ 1.092061] Run /init as init process1605lb # [ 1.025002] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1606lb # [ 1.026745] drop_monitor: Initializing network drop monitor service1607lb # [ 1.026946] NET: Registered PF_INET6 protocol family1608worker2 # [ 1.024287] registered taskstats version 11609worker2 # [ 1.024449] Loading compiled-in X.509 certificates1610lb # [ 1.030488] Segment Routing with IPv61611lb # [ 1.030511] In-situ OAM (IOAM) with IPv61612lb # [ 1.030544] NET: Registered PF_PACKET protocol family1613worker1 # [ 1.065821] clk: Disabling unused clocks1614lb # [ 1.032319] 9pnet: Installing 9P2000 support1615worker1 # [ 1.065864] PM: genpd: Disabling unused power domains1616lb # [ 1.034751] Key type dns_resolver registered1617worker2 # [ 1.033431] Demotion targets for Node 0: null1618worker2 # [ 1.033565] Key type .fscrypt registered1619worker2 # [ 1.033572] Key type fscrypt-provisioning registered1620worker2 # [ 1.033670] ima: No TPM chip found, activating TPM-bypass!1621worker2 # [ 1.033689] ima: Allocated hash algorithm: sha11622worker2 # [ 1.033712] ima: No architecture policies found1623worker1 # [ 1.070224] Freeing unused kernel memory: 4736K1624worker1 # [ 1.071086] Run /init as init process1625worker2 # [ 1.038089] input: gpio-keys as /devices/platform/gpio-keys/input/input01626client # [ 1.111006] systemd[1]: Successfully made /usr/ read-only.1627lb # [ 1.042746] registered taskstats version 11628lb # [ 1.042915] Loading compiled-in X.509 certificates1629lb # [ 1.051806] Demotion targets for Node 0: null1630lb # [ 1.052579] Key type .fscrypt registered1631lb # [ 1.052592] Key type fscrypt-provisioning registered1632lb # [ 1.052706] ima: No TPM chip found, activating TPM-bypass!1633lb # [ 1.052726] ima: Allocated hash algorithm: sha11634lb # [ 1.052752] ima: No architecture policies found1635lb # [ 1.056594] input: gpio-keys as /devices/platform/gpio-keys/input/input01636worker1 # [ 1.088882] systemd[1]: Successfully made /usr/ read-only.1637worker2 # [ 1.057427] clk: Disabling unused clocks1638worker2 # [ 1.057466] PM: genpd: Disabling unused power domains1639worker2 # [ 1.061859] Freeing unused kernel memory: 4736K1640worker2 # [ 1.062088] Run /init as init process1641lb # [ 1.077249] clk: Disabling unused clocks1642worker2 # [ 1.077171] systemd[1]: Successfully made /usr/ read-only.1643lb # [ 1.077293] PM: genpd: Disabling unused power domains1644lb # [ 1.081747] Freeing unused kernel memory: 4736K1645lb # [ 1.081990] Run /init as init process1646lb # [ 1.098525] systemd[1]: Successfully made /usr/ read-only.1647client # [ 1.264272] usb 1-1: new high-speed USB device number 2 using ehci-pci1648worker1 # [ 1.250404] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker2 # [ 1.234699] usb 1-1: new high-speed USB device number 2 using ehci-pci1650lb # [ 1.251930] usb 1-1: new high-speed USB device number 2 using ehci-pci1651client # [ 1.420712] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652worker1 # [ 1.404856] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653client # [ 1.445939] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654client # [ 1.450058] systemd[1]: Detected virtualization qemu.1655client # [ 1.450769] systemd[1]: Detected architecture arm64.1656client # [ 1.451419] systemd[1]: Running in initrd.1657client # [ 1.452428] systemd[1]: Initializing machine ID from random generator.1658client # [ 1.453410] systemd[1]: Hostname set to <client>.1659worker2 # [ 1.384402] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11660worker1 # [ 1.423980] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1661worker1 # [ 1.436491] systemd[1]: Detected virtualization qemu.1662worker1 # [ 1.438834] systemd[1]: Detected architecture arm64.1663worker1 # [ 1.440740] systemd[1]: Running in initrd.1664lb # [ 1.406330] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11665worker1 # [ 1.443279] systemd[1]: Initializing machine ID from random generator.1666worker1 # [ 1.446089] systemd[1]: Hostname set to <worker1>.1667worker2 # [ 1.412199] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668worker2 # [ 1.424656] systemd[1]: Detected virtualization qemu.1669worker2 # [ 1.424768] systemd[1]: Detected architecture arm64.1670worker2 # [ 1.424796] systemd[1]: Running in initrd.1671worker2 # [ 1.425792] systemd[1]: Initializing machine ID from random generator.1672client # [ 1.504521] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01673worker2 # [ 1.426093] systemd[1]: Hostname set to <worker2>.1674lb # [ 1.433518] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1675lb # [ 1.445789] systemd[1]: Detected virtualization qemu.1676lb # [ 1.447801] systemd[1]: Detected architecture arm64.1677lb # [ 1.449744] systemd[1]: Running in initrd.1678lb # [ 1.452365] systemd[1]: Initializing machine ID from random generator.1679lb # [ 1.455281] systemd[1]: Hostname set to <lb>.1680worker1 # [ 1.490637] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681worker2 # [ 1.471146] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682lb # [ 1.500236] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683client # [ 1.624279] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker1 # [ 1.614440] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker2 # [ 1.598713] usb 1-2: new high-speed USB device number 3 using ehci-pci1686lb # [ 1.623919] usb 1-2: new high-speed USB device number 3 using ehci-pci1687client # [ 1.698125] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688client # [ 1.759117] systemd[1]: Queued start job for default target Initrd Default Target.1689client # [ 1.770407] systemd[1]: Created slice Slice /system/modprobe.1690client # [ 1.771444] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1691client # [ 1.772600] systemd[1]: Expecting device /dev/disk/by-label/nixos...1692client # [ 1.773690] systemd[1]: Reached target Path Units.1693client # [ 1.774356] systemd[1]: Reached target Slice Units.1694client # [ 1.775119] systemd[1]: Reached target Swaps.1695client # [ 1.775917] systemd[1]: Reached target Timer Units.1696client # [ 1.776100] systemd[1]: Listening on D-Bus System Message Bus Socket.1697client # [ 1.777850] systemd[1]: Listening on Journal Socket (/dev/log).1698client # [ 1.778959] systemd[1]: Listening on Journal Sockets.1699client # [ 1.779096] systemd[1]: Listening on udev Control Socket.1700client # [ 1.779199] systemd[1]: Listening on udev Kernel Socket.1701client # [ 1.779223] systemd[1]: Reached target Socket Units.1702client # [ 1.783782] systemd[1]: Starting Create List of Static Device Nodes...1703client # [ 1.788369] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1704client # [ 1.791008] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1705client # [ 1.791560] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21706client # [ 1.791742] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01707client # [ 1.804510] systemd[1]: Mounting Kernel Configuration File System...1708client # [ 1.820685] systemd[1]: Starting Journal Service...1709worker1 # [ 1.782978] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21710worker2 # [ 1.752692] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21711worker1 # [ 1.788297] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01712worker1 # [ 1.792746] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713worker2 # [ 1.758384] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01714lb # [ 1.764733] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715client # [ 1.841017] systemd[1]: Starting Load Kernel Modules...1716client # [ 1.841938] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1717client # [ 1.852530] systemd[1]: Starting Coldplug All udev Devices...1718lb # [ 1.784340] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719lb # [ 1.787493] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01720client # [ 1.868379] systemd[1]: Finished Create List of Static Device Nodes.1721client # [ 1.869356] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1722client # [ 1.869674] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1723client # [ 1.869997] systemd[1]: Mounted Kernel Configuration File System.1724client # [ 1.888866] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1725worker1 # [ 1.882085] systemd[1]: Queued start job for default target Initrd Default Target.1726lb # [ 1.851542] systemd[1]: Queued start job for default target Initrd Default Target.1727worker2 # [ 1.854289] systemd[1]: bpf-restrict-fs: LSM BPF program attached1728client # [ 1.925389] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1729worker1 # [ 1.892704] systemd[1]: Created slice Slice /system/modprobe.1730lb # [ 1.862247] systemd[1]: Created slice Slice /system/modprobe.1731lb # [ 1.863375] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1732worker1 # [ 1.893875] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1733worker1 # [ 1.895277] systemd[1]: Expecting device /dev/disk/by-label/nixos...1734lb # [ 1.864635] systemd[1]: Expecting device /dev/disk/by-label/nixos...1735worker1 # [ 1.896285] systemd[1]: Reached target Path Units.1736lb # [ 1.865713] systemd[1]: Reached target Path Units.1737client # [ 1.933848] systemd-journald[73]: Collecting audit messages is disabled.1738worker1 # [ 1.897030] systemd[1]: Reached target Slice Units.1739lb # [ 1.866433] systemd[1]: Reached target Slice Units.1740lb # [ 1.867165] systemd[1]: Reached target Swaps.1741worker1 # [ 1.897789] systemd[1]: Reached target Swaps.1742lb # [ 1.867880] systemd[1]: Reached target Timer Units.1743worker1 # [ 1.898532] systemd[1]: Reached target Timer Units.1744lb # [ 1.868819] systemd[1]: Listening on D-Bus System Message Bus Socket.1745worker1 # [ 1.899517] systemd[1]: Listening on D-Bus System Message Bus Socket.1746lb # [ 1.869962] systemd[1]: Listening on Journal Socket (/dev/log).1747worker1 # [ 1.900669] systemd[1]: Listening on Journal Socket (/dev/log).1748lb # [ 1.870985] systemd[1]: Listening on Journal Sockets.1749worker1 # [ 1.901712] systemd[1]: Listening on Journal Sockets.1750lb # [ 1.871906] systemd[1]: Listening on udev Control Socket.1751client # [ 1.940291] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1752lb # [ 1.872836] systemd[1]: Listening on udev Kernel Socket.1753worker1 # [ 1.902664] systemd[1]: Listening on udev Control Socket.1754lb # [ 1.873667] systemd[1]: Reached target Socket Units.1755worker1 # [ 1.902781] systemd[1]: Listening on udev Kernel Socket.1756worker1 # [ 1.902804] systemd[1]: Reached target Socket Units.1757lb # [ 1.875971] systemd[1]: Starting Create List of Static Device Nodes...1758worker1 # [ 1.907011] systemd[1]: Starting Create List of Static Device Nodes...1759client # [ 1.951912] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01760lb # [ 1.884110] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1761client # [ 1.952164] [drm] features: -virgl +edid -resource_blob -host_visible1762client # [ 1.952175] [drm] features: -context_init1763lb # [ 1.885191] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1764worker1 # [ 1.917712] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1765client # [ 1.955076] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1766worker1 # [ 1.919055] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1767lb # [ 1.891723] systemd[1]: Mounting Kernel Configuration File System...1768client # [ 1.960963] systemd[1]: Starting Create Static Device Nodes in /dev...1769client # [ 1.969101] [drm] number of scanouts: 11770client # [ 1.969140] [drm] number of cap sets: 01771worker1 # [ 1.932132] systemd[1]: Mounting Kernel Configuration File System...1772lb # [ 1.904290] systemd[1]: Starting Journal Service...1773client # [ 1.976568] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1774client # [ 1.976591] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01775worker1 # [ 1.951214] systemd[1]: Starting Journal Service...1776lb # [ 1.923589] systemd[1]: Starting Load Kernel Modules...1777lb # [ 1.924748] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1778worker1 # [ 1.962548] systemd[1]: Starting Load Kernel Modules...1779worker1 # [ 1.963491] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1780worker2 # [ 1.932166] systemd[1]: Queued start job for default target Initrd Default Target.1781worker1 # [ 1.967696] systemd[1]: Starting Coldplug All udev Devices...1782worker2 # [ 1.941151] systemd[1]: Created slice Slice /system/modprobe.1783client # [ 2.000549] Console: switching to colour frame buffer device 160x501784worker2 # [ 1.942287] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1785worker2 # [ 1.943619] systemd[1]: Expecting device /dev/disk/by-label/nixos...1786worker2 # [ 1.944862] systemd[1]: Reached target Path Units.1787worker1 # [ 1.978664] systemd[1]: Finished Create List of Static Device Nodes.1788worker2 # [ 1.945582] systemd[1]: Reached target Slice Units.1789worker2 # [ 1.946329] systemd[1]: Reached target Swaps.1790worker2 # [ 1.947073] systemd[1]: Reached target Timer Units.1791worker1 # [ 1.980571] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1792lb # [ 1.949494] systemd[1]: Starting Coldplug All udev Devices...1793worker2 # [ 1.948622] systemd[1]: Listening on D-Bus System Message Bus Socket.1794worker2 # [ 1.949768] systemd[1]: Listening on Journal Socket (/dev/log).1795worker1 # [ 1.983244] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1796lb # [ 1.952174] systemd[1]: Finished Create List of Static Device Nodes.1797lb # [ 1.953068] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1798worker2 # [ 1.950850] systemd[1]: Listening on Journal Sockets.1799worker2 # [ 1.951015] systemd[1]: Listening on udev Control Socket.1800worker2 # [ 1.951126] systemd[1]: Listening on udev Kernel Socket.1801worker2 # [ 1.951147] systemd[1]: Reached target Socket Units.1802worker2 # [ 1.955985] systemd[1]: Starting Create List of Static Device Nodes...1803lb # [ 1.959789] systemd-journald[73]: Collecting audit messages is disabled.1804client # [ 2.028988] systemd[1]: Finished Create Static Device Nodes in /dev.1805client # [ 2.029213] systemd[1]: Reached target Preparation for Local File Systems.1806client # [ 2.029238] systemd[1]: Reached target Local File Systems.1807client # [ 2.033369] systemd[1]: Starting Rule-based Manager for Device Events and Files...1808worker2 # [ 1.966808] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1809lb # [ 1.968502] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1810worker2 # [ 1.967964] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1811client # [ 2.037957] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1812lb # [ 1.976364] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1813lb # [ 1.977788] systemd[1]: Mounted Kernel Configuration File System.1814worker1 # [ 2.006532] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1815worker2 # [ 1.983259] systemd[1]: Mounting Kernel Configuration File System...1816worker1 # [ 2.024194] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1817client # [ 2.060503] systemd[1]: Finished Load Kernel Modules.1818client # [ 2.064375] systemd[1]: Starting Apply Kernel Variables...1819worker2 # [ 2.003006] systemd[1]: Starting Journal Service...1820lb # [ 2.010408] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1821worker1 # [ 2.046829] systemd[1]: Mounted Kernel Configuration File System.1822worker2 # [ 2.014353] systemd[1]: Starting Load Kernel Modules...1823worker2 # [ 2.015524] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1824worker1 # [ 2.049272] systemd-journald[73]: Collecting audit messages is disabled.1825worker1 # [ 2.050287] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1826worker1 # [ 2.052707] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1827worker1 # [ 2.062622] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01828worker1 # [ 2.062871] [drm] features: -virgl +edid -resource_blob -host_visible1829worker1 # [ 2.062881] [drm] features: -context_init1830worker1 # [ 2.063616] [drm] number of scanouts: 11831worker1 # [ 2.063634] [drm] number of cap sets: 01832worker2 # [ 2.031269] systemd[1]: Starting Coldplug All udev Devices...1833worker1 # [ 2.067456] systemd[1]: Starting Create Static Device Nodes in /dev...1834lb # [ 2.035990] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1835lb # [ 2.040874] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01836lb # [ 2.041126] [drm] features: -virgl +edid -resource_blob -host_visible1837lb # [ 2.041137] [drm] features: -context_init1838lb # [ 2.041906] [drm] number of scanouts: 11839lb # [ 2.041925] [drm] number of cap sets: 01840client # [ 2.119722] systemd[1]: Started Journal Service.1841worker1 # [ 2.082766] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1842worker1 # [ 2.082795] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01843lb # [ 2.056559] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1844lb # [ 2.060342] systemd[1]: Starting Create Static Device Nodes in /dev...1845client # [ 2.113297] systemd-modules-load[74]: Inserted module 'dm_mod'1846client # [ 2.114502] systemd-modules-load[74]: Module 'virtio_balloon' is built in1847worker2 # [ 2.058811] systemd[1]: Finished Create List of Static Device Nodes.1848worker2 # [ 2.060163] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1849worker2 # [ 2.060477] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1850worker2 # [ 2.060707] systemd[1]: Mounted Kernel Configuration File System.1851lb # [ 2.064337] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1852lb # [ 2.064365] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01853worker2 # [ 2.068328] systemd-journald[73]: Collecting audit messages is disabled.1854client # [ 2.124412] systemd-modules-load[74]: Module 'virtio_console' is built in1855client # [ 2.125718] systemd-modules-load[74]: Inserted module 'virtio_gpu'1856worker2 # [ 2.071249] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1857client # [ 2.128471] systemd-modules-load[74]: Module 'virtio_rng' is built in1858client # [ 2.136393] systemd[1]: Starting Create System Files and Directories...1859worker2 # [ 2.082691] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1860client # [ 2.139282] systemd[1]: Finished Apply Kernel Variables.1861worker1 # [ 2.110626] Console: switching to colour frame buffer device 160x501862worker2 # [ 2.093756] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1863lb # [ 2.096200] Console: switching to colour frame buffer device 160x501864lb # [ 2.102809] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1865worker2 # [ 2.102930] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01866worker1 # [ 2.138997] systemd[1]: Finished Create Static Device Nodes in /dev.1867worker2 # [ 2.103178] [drm] features: -virgl +edid -resource_blob -host_visible1868worker2 # [ 2.103190] [drm] features: -context_init1869worker1 # [ 2.139244] systemd[1]: Reached target Preparation for Local File Systems.1870worker2 # [ 2.103942] [drm] number of scanouts: 11871worker1 # [ 2.139271] systemd[1]: Reached target Local File Systems.1872worker2 # [ 2.103961] [drm] number of cap sets: 01873client # [ 2.164683] systemd-udevd[79]: Using default interface naming scheme 'v261'.1874worker1 # [ 2.143720] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1875worker1 # [ 2.147238] systemd[1]: Starting Rule-based Manager for Device Events and Files...1876worker2 # [ 2.123019] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1877worker2 # [ 2.123051] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01878lb # [ 2.129476] systemd[1]: Finished Create Static Device Nodes in /dev.1879lb # [ 2.129770] systemd[1]: Reached target Preparation for Local File Systems.1880lb # [ 2.129807] systemd[1]: Reached target Local File Systems.1881client # [ 2.187068] systemd[1]: Finished Create System Files and Directories.1882worker1 # [ 2.166601] systemd[1]: Finished Load Kernel Modules.1883lb # [ 2.136338] systemd[1]: Starting Rule-based Manager for Device Events and Files...1884worker1 # [ 2.174694] systemd[1]: Starting Apply Kernel Variables...1885client # [ 2.205376] systemd[1]: Started Rule-based Manager for Device Events and Files.1886lb # [ 2.140611] systemd-modules-load[74]: Inserted module 'dm_mod'1887lb # [ 2.157717] systemd[1]: Started Journal Service.1888worker2 # [ 2.155361] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1889lb # [ 2.144756] systemd-modules-load[74]: Module 'virtio_balloon' is built in1890lb # [ 2.145957] systemd-modules-load[74]: Module 'virtio_console' is built in1891lb # [ 2.147051] systemd-modules-load[74]: Inserted module 'virtio_gpu'1892worker2 # [ 2.157964] Console: switching to colour frame buffer device 160x501893lb # [ 2.152723] systemd-modules-load[74]: Module 'virtio_rng' is built in1894lb # [ 2.157524] systemd[1]: Finished Load Kernel Modules.1895worker2 # [ 2.166558] systemd[1]: Starting Create Static Device Nodes in /dev...1896lb # [ 2.164503] systemd[1]: Starting Apply Kernel Variables...1897worker1 # [ 2.213986] systemd[1]: Started Journal Service.1898lb # [ 2.172256] systemd[1]: Starting Create System Files and Directories...1899worker2 # [ 2.187241] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1900worker1 # [ 2.220315] systemd-modules-load[74]: Inserted module 'dm_mod'1901worker1 # [ 2.221527] systemd-modules-load[74]: Module 'virtio_balloon' is built in1902worker1 # [ 2.222671] systemd-modules-load[74]: Module 'virtio_console' is built in1903worker1 # [ 2.223746] systemd-modules-load[74]: Inserted module 'virtio_gpu'1904worker2 # [ 2.203387] systemd[1]: Finished Load Kernel Modules.1905worker2 # [ 2.211136] systemd[1]: Starting Apply Kernel Variables...1906worker1 # [ 2.237358] systemd-modules-load[74]: Module 'virtio_rng' is built in1907worker1 # [ 2.238498] systemd[1]: Starting Create System Files and Directories...1908client # [ 2.282884] systemd[1]: Starting Virtual Console Setup...1909worker1 # [ 2.252129] systemd[1]: Finished Apply Kernel Variables.1910lb # [ 2.221481] systemd[1]: Finished Create System Files and Directories.1911worker2 # [ 2.239056] systemd[1]: Finished Create Static Device Nodes in /dev.1912worker2 # [ 2.239344] systemd[1]: Reached target Preparation for Local File Systems.1913worker2 # [ 2.239381] systemd[1]: Reached target Local File Systems.1914worker1 # [ 2.270105] systemd[1]: Finished Create System Files and Directories.1915worker2 # [ 2.247080] systemd[1]: Starting Rule-based Manager for Device Events and Files...1916lb # [ 2.241732] systemd[1]: Finished Apply Kernel Variables.1917worker2 # [ 2.246298] systemd-modules-load[74]: Inserted module 'dm_mod'1918worker1 # [ 2.280641] systemd-udevd[80]: Using default interface naming scheme 'v261'.1919lb # [ 2.251739] systemd-udevd[80]: Using default interface naming scheme 'v261'.1920worker2 # [ 2.266088] systemd[1]: Started Journal Service.1921worker2 # [ 2.254265] systemd-modules-load[74]: Module 'virtio_balloon' is built in1922worker2 # [ 2.255436] systemd-modules-load[74]: Module 'virtio_console' is built in1923worker2 # [ 2.260631] systemd-modules-load[74]: Inserted module 'virtio_gpu'1924worker2 # [ 2.268453] systemd-modules-load[74]: Module 'virtio_rng' is built in1925worker1 # [ 2.305835] systemd[1]: Started Rule-based Manager for Device Events and Files.1926worker2 # [ 2.272384] systemd[1]: Starting Create System Files and Directories...1927worker2 # [ 2.277147] systemd[1]: Finished Apply Kernel Variables.1928lb # [ 2.282535] systemd[1]: Started Rule-based Manager for Device Events and Files.1929client # [ 2.363257] systemd-vconsole-setup[105]: Configuration of first virtual console was skipped, ignoring remaining ones.1930client # [ 2.367249] systemd[1]: Finished Virtual Console Setup.1931worker2 # [ 2.319727] systemd[1]: Finished Create System Files and Directories.1932worker1 # [ 2.370668] systemd[1]: Starting Virtual Console Setup...1933worker2 # [ 2.337350] systemd-udevd[81]: Using default interface naming scheme 'v261'.1934lb # [ 2.341300] systemd[1]: Starting Virtual Console Setup...1935worker2 # [ 2.369278] systemd[1]: Started Rule-based Manager for Device Events and Files.1936worker1 # [ 2.428586] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1937lb # [ 2.396580] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938worker1 # [ 2.432344] systemd[1]: Finished Virtual Console Setup.1939lb # [ 2.404197] systemd[1]: Finished Virtual Console Setup.1940worker2 # [ 2.427940] systemd[1]: Starting Virtual Console Setup...1941worker2 # [ 2.480520] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942worker2 # [ 2.484373] systemd[1]: Finished Virtual Console Setup.1943client # [ 2.975539] systemd[1]: Finished Coldplug All udev Devices.1944client # [ 2.977247] systemd[1]: Reached target System Initialization.1945client # [ 2.980121] systemd[1]: Reached target Basic System.1946worker1 # [ 3.011107] systemd[1]: Finished Coldplug All udev Devices.1947worker1 # [ 3.012582] systemd[1]: Reached target System Initialization.1948worker1 # [ 3.016459] systemd[1]: Reached target Basic System.1949lb # [ 3.021067] systemd[1]: Finished Coldplug All udev Devices.1950lb # [ 3.022733] systemd[1]: Reached target System Initialization.1951lb # [ 3.024115] systemd[1]: Reached target Basic System.1952client # [ 3.096816] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1953client # [ 3.162351] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1954client # [ 3.168338] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1955worker2 # [ 3.140513] systemd[1]: Finished Coldplug All udev Devices.1956worker2 # [ 3.141992] systemd[1]: Reached target System Initialization.1957worker2 # [ 3.144189] systemd[1]: Reached target Basic System.1958worker1 # [ 3.179956] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1959client # [ 3.222000] systemd[1]: Found device /dev/disk/by-label/nixos.1960client # [ 3.225124] systemd[1]: Reached target Initrd Root Device.1961client # [ 3.229603] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1962lb # [ 3.183225] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1963worker1 # [ 3.215880] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1964worker1 # [ 3.220847] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1965lb # [ 3.205781] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1966lb # [ 3.212249] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1967client # [ 3.289354] systemd-fsck[114]: nixos: clean, 12/65536 files, 13019/262144 blocks1968worker1 # [ 3.266892] systemd[1]: Found device /dev/disk/by-label/nixos.1969worker1 # [ 3.268820] systemd[1]: Reached target Initrd Root Device.1970worker1 # [ 3.273188] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1971client # [ 3.312101] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1972client # [ 3.317036] systemd[1]: Mounting /sysroot...1973worker2 # [ 3.288382] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1974lb # [ 3.297257] systemd[1]: Found device /dev/disk/by-label/nixos.1975worker1 # [ 3.330552] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1976client # [ 3.382628] EXT4-fs (vda): mounted filesystem 5b80c133-8ae6-446a-bdf4-68cc378e3413 r/w with ordered data mode. Quota mode: none.1977lb # [ 3.303913] systemd[1]: Reached target Initrd Root Device.1978client # [ 3.372849] systemd[1]: Mounted /sysroot.1979worker2 # [ 3.303273] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1980client # [ 3.375199] systemd[1]: Reached target Initrd Root File System.1981lb # [ 3.309837] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982client # [ 3.381258] systemd[1]: Starting Mountpoints Configured in the Real Root...1983worker1 # [ 3.346156] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1984worker1 # [ 3.352135] systemd[1]: Mounting /sysroot...1985worker2 # [ 3.335576] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1986client # [ 3.413395] systemd-sysroot-fstab-check[122]: /sysroot should be mounted in the initrd, will request daemon-reload.1987client # [ 3.420113] systemd[1]: Reload requested from client PID 122 ('systemd-sysroot') (unit initrd-parse-etc.service)...1988client # [ 3.423699] systemd[1]: Reloading...1989worker1 # [ 3.406584] EXT4-fs (vda): mounted filesystem 38eadec4-c5df-4faa-81bb-fc92f892367c r/w with ordered data mode. Quota mode: none.1990worker1 # [ 3.398791] systemd[1]: Mounted /sysroot.1991lb # [ 3.368519] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1992worker1 # [ 3.401398] systemd[1]: Reached target Initrd Root File System.1993worker1 # [ 3.406536] systemd[1]: Starting Mountpoints Configured in the Real Root...1994worker1 # [ 3.438371] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1995worker1 # [ 3.446340] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1996worker1 # [ 3.451657] systemd[1]: Reloading...1997worker2 # [ 3.428612] systemd[1]: Found device /dev/disk/by-label/nixos.1998worker2 # [ 3.431529] systemd[1]: Reached target Initrd Root Device.1999worker2 # [ 3.436987] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...2000worker2 # [ 3.478500] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks
loading 1,525 hidden lines…
3526lb # [ 21.475108] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.filters.upstream_network: envoy.filters.upstream_network.reverse_tunnel_lifecycle3527lb # [ 21.478884] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.filters.listener: envoy.filters.listener.dynamic_modules, envoy.filters.listener.http_inspector, envoy.filters.listener.local_ratelimit, envoy.filters.listener.original_dst, envoy.filters.listener.original_src, envoy.filters.listener.proxy_protocol, envoy.filters.listener.set_filter_state, envoy.filters.listener.tls_inspector, envoy.listener.http_inspector, envoy.listener.original_dst, envoy.listener.original_src, envoy.listener.proxy_protocol, envoy.listener.tls_inspector3528lb # [ 21.486438] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.protocols: dubbo3529lb # [ 21.488815] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.matching.network.input: envoy.matching.inputs.application_protocol, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.filter_state, envoy.matching.inputs.network_namespace, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.subject, envoy.matching.inputs.transport_protocol, envoy.matching.inputs.uri_san3530lb # [ 21.497765] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.transport_sockets.downstream: envoy.transport_sockets.alts, envoy.transport_sockets.dynamic_modules, envoy.transport_sockets.quic, envoy.transport_sockets.raw_buffer, envoy.transport_sockets.starttls, envoy.transport_sockets.tap, envoy.transport_sockets.tcp_stats, envoy.transport_sockets.tls, raw_buffer, starttls, tls3531lb # [ 21.502741] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.matching.http.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3532lb # [ 21.505863] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.matching.stat_matching_data.input: stat_full_name_match_input3533lb # [ 21.507845] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.stateful_session: envoy.http.stateful_session.cookie, envoy.http.stateful_session.envelope, envoy.http.stateful_session.header3534lb # [ 21.510958] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.filters.http.upstream: envoy.buffer, envoy.ext_proc, envoy.extensions.filters.http.dynamic_modules, envoy.filters.http.admission_control, envoy.filters.http.aws_lambda, envoy.filters.http.aws_request_signing, envoy.filters.http.buffer, envoy.filters.http.composite, envoy.filters.http.credential_injector, envoy.filters.http.ext_proc, envoy.filters.http.header_mutation, envoy.filters.http.lua, envoy.filters.http.match_delegate, envoy.filters.http.upstream_codec, envoy.filters.http.upstream_rbac, envoy.filters.http.wasm3535lb # [ 21.518797] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.ext_proc.processing_request_modifiers: envoy.extensions.http.ext_proc.mapped_attribute_builder3536lb # [ 21.521884] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.config_subscription: envoy.config_subscription.ads, envoy.config_subscription.ads_collection, envoy.config_subscription.aggregated_grpc_collection, envoy.config_subscription.delta_grpc, envoy.config_subscription.delta_grpc_collection, envoy.config_subscription.filesystem, envoy.config_subscription.filesystem_collection, envoy.config_subscription.grpc, envoy.config_subscription.rest3537lb # [ 21.527720] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.listener_manager_impl: envoy.listener_manager_impl.default, envoy.listener_manager_impl.validation3538lb # [ 21.530631] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.tracers.opentelemetry.samplers: envoy.tracers.opentelemetry.samplers.always_on, envoy.tracers.opentelemetry.samplers.cel, envoy.tracers.opentelemetry.samplers.dynatrace, envoy.tracers.opentelemetry.samplers.parent_based, envoy.tracers.opentelemetry.samplers.trace_id_ratio_based3539lb # [ 21.535502] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.custom_response: envoy.extensions.http.custom_response.local_response_policy, envoy.extensions.http.custom_response.redirect_policy3540lb # [ 21.538597] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.rds_factory: envoy.rds_factory.default3541lb # [ 21.540782] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.matching.http.input: envoy.matching.inputs.cel_data_input, envoy.matching.inputs.destination_ip, envoy.matching.inputs.destination_port, envoy.matching.inputs.direct_source_ip, envoy.matching.inputs.dns_san, envoy.matching.inputs.dynamic_metadata, envoy.matching.inputs.dynamic_module_data_input, envoy.matching.inputs.filter_state, envoy.matching.inputs.local_reply, envoy.matching.inputs.network_namespace, envoy.matching.inputs.request_headers, envoy.matching.inputs.request_trailers, envoy.matching.inputs.response_headers, envoy.matching.inputs.response_trailers, envoy.matching.inputs.server_name, envoy.matching.inputs.source_ip, envoy.matching.inputs.source_port, envoy.matching.inputs.source_type, envoy.matching.inputs.status_code_class_input, envoy.matching.inputs.status_code_input, envoy.matching.inputs.subject, envoy.matching.inputs.uri_san, query_params3542lb # [ 21.552110] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.health_check.event_sinks: envoy.health_check.event_sink.file3543lb # [ 21.554563] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.string_matcher: envoy.string_matcher.lua3544lb # [ 21.556527] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] quic.http_server_connection: quic.http_server_connection.default3545lb # [ 21.558531] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.retry_host_predicates: envoy.retry_host_predicates.omit_canary_hosts, envoy.retry_host_predicates.omit_host_metadata, envoy.retry_host_predicates.previous_hosts3546lb # [ 21.561946] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.dubbo_proxy.filters: envoy.filters.dubbo.router3547lb # [ 21.563881] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.matching.common_inputs: envoy.matching.common_inputs.environment_variable3548lb # [ 21.566300] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_mappers: envoy.tls.certificate_mappers.static_name, envoy.tls.upstream_certificate_mappers.filter_state_override3549lb # [ 21.569865] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.cache_v2: envoy.extensions.http.cache_v2.file_system_http_cache, envoy.extensions.http.cache_v2.simple3550lb # [ 21.572785] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.quic.server.crypto_stream: envoy.quic.crypto_stream.server.quiche3551lb # [ 21.574833] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.route.early_data_policy: envoy.route.early_data_policy.default3552lb # [ 21.577017] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.injected_credentials: envoy.http.injected_credentials.generic, envoy.http.injected_credentials.oauth23553lb # [ 21.579518] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.thrift_proxy.transports: auto, framed, header, unframed3554lb # [ 21.581818] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.quic.server_preferred_address: quic.server_preferred_address.datasource, quic.server_preferred_address.fixed3555lb # [ 21.584459] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.tls.upstream_certificate_selectors: envoy.tls.certificate_selectors.on_demand_secret3556lb # [ 21.586696] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.tracers: envoy.tracers.datadog, envoy.tracers.dynamic_modules, envoy.tracers.fluentd, envoy.tracers.opentelemetry, envoy.tracers.skywalking, envoy.tracers.xray, envoy.tracers.zipkin, envoy.zipkin3557lb # [ 21.590633] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.generic_proxy.filters: envoy.filters.generic.router3558lb # [ 21.592845] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.connection_handler: envoy.connection_handler.default3559lb # [ 21.594799] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.router.cluster_specifier_plugin: envoy.router.cluster_specifier_plugin.lua, envoy.router.cluster_specifier_plugin.matcher3560lb # [ 21.597661] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.compression.compressor: envoy.compression.brotli.compressor, envoy.compression.gzip.compressor, envoy.compression.zstd.compressor3561lb # [ 21.600689] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.stateful_header_formatters: envoy.http.stateful_header_formatters.preserve_case, preserve_case3562lb # [ 21.603105] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.resolvers: envoy.ip, envoy.resolvers.reverse_connection3563lb # [ 21.605233] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.quic.connection_debug_visitor: envoy.quic.connection_debug_visitor.basic, envoy.quic.connection_debug_visitor.quic_stats3564lb # [ 21.607848] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.original_ip_detection: envoy.http.original_ip_detection.custom_header, envoy.http.original_ip_detection.xff3565lb # [ 21.610684] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.wasm.runtime: envoy.wasm.runtime.null, envoy.wasm.runtime.v83566lb # [ 21.612857] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.early_header_mutation: envoy.http.early_header_mutation.header_mutation3567lb # [ 21.615271] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.bootstrap: envoy.bootstrap.dynamic_modules, envoy.bootstrap.internal_listener, envoy.bootstrap.reverse_tunnel.downstream_socket_interface, envoy.bootstrap.reverse_tunnel.upstream_socket_interface, envoy.bootstrap.wasm, envoy.extensions.network.socket_interface.default_socket_interface, envoy.extensions.network.socket_interface.sockmap3568lb # [ 21.620670] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.tls.cert_validator: envoy.tls.cert_validator.default, envoy.tls.cert_validator.dynamic_modules, envoy.tls.cert_validator.spiffe3569lb # [ 21.623408] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.http.header_validators: envoy.http.header_validators.envoy_default3570lb # [ 21.625928] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.request_id: envoy.request_id.uuid3571lb # [ 21.627693] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.matching.network.custom_matchers: envoy.matching.custom_matchers.domain_matcher, envoy.matching.custom_matchers.ip_range_matcher3572lb # [ 21.630643] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] filter_state.object: envoy.filters.listener.original_dst.local_ip, envoy.filters.listener.original_dst.remote_ip, envoy.hashable_string, envoy.network.application_protocols, envoy.network.ip, envoy.network.network_namespace, envoy.network.transport_socket.http_11_proxy.info, envoy.network.transport_socket.original_dst_address, envoy.network.upstream_server_name, envoy.network.upstream_subject_alt_names, envoy.ratelimit.hits_addend, envoy.router.debug_config, envoy.string, envoy.tcp_proxy.cluster, envoy.tcp_proxy.disable_tunneling, envoy.tcp_proxy.per_connection_idle_timeout_ms, envoy.upstream.dynamic_host, envoy.upstream.dynamic_port3573lb # [ 21.639319] envoy[922]: [2026-09-20 12:47:49.724][922][info][main] [source/server/server.cc:471] envoy.rbac.principals: envoy.rbac.principals.mtls_authenticated3574lb # [ 21.641528] envoy[922]: [2026-09-20 12:47:49.793][922][info][main] [source/server/server.cc:531] HTTP header map info:3575lb # [ 21.643018] envoy[922]: [2026-09-20 12:47:49.795][922][info][main] [source/server/server.cc:534] request header map: 664 bytes: :authority,:method,:path,:protocol,:scheme,accept,accept-encoding,access-control-request-headers,access-control-request-method,access-control-request-private-network,authentication,authorization,cache-control,cdn-loop,connection,content-encoding,content-length,content-type,expect,grpc-accept-encoding,grpc-timeout,if-match,if-modified-since,if-none-match,if-range,if-unmodified-since,keep-alive,origin,pragma,proxy-connection,proxy-status,referer,te,transfer-encoding,upgrade,user-agent,via,x-client-trace-id,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-downstream-service-cluster,x-envoy-downstream-service-node,x-envoy-expected-rq-timeout-ms,x-envoy-external-address,x-envoy-force-trace,x-envoy-hedge-on-per-try-timeout,x-envoy-internal,x-envoy-ip-tags,x-envoy-is-timeout-retry,x-envoy-max-retries,x-envoy-original-host,x-envoy-original-path,x-envoy-original-url,x-envoy-retriable-header-names,x-envoy-retriable-status-codes,x-envoy-retry-grpc-on,x-envoy-retry-on,x-envoy-upstream-alt-stat-name,x-envoy-upstream-rq-per-try-timeout-ms,x-envoy-upstream-rq-timeout-alt-response,x-envoy-upstream-rq-timeout-ms,x-envoy-upstream-stream-duration-ms,x-forwarded-client-cert,x-forwarded-for,x-forwarded-host,x-forwarded-port,x-forwarded-proto,x-request-id3576lb # [ 21.659464] envoy[922]: [2026-09-20 12:47:49.796][922][info][main] [source/server/server.cc:534] request trailer map: 120 bytes:3577lb # [ 21.661271] envoy[922]: [2026-09-20 12:47:49.796][922][info][main] [source/server/server.cc:534] response header map: 432 bytes: :status,access-control-allow-credentials,access-control-allow-headers,access-control-allow-methods,access-control-allow-origin,access-control-allow-private-network,access-control-expose-headers,access-control-max-age,age,cache-control,connection,content-encoding,content-length,content-type,date,etag,expires,grpc-message,grpc-status,keep-alive,last-modified,location,proxy-connection,proxy-status,server,transfer-encoding,upgrade,vary,via,x-envoy-attempt-count,x-envoy-decorator-operation,x-envoy-degraded,x-envoy-immediate-health-check-fail,x-envoy-ratelimited,x-envoy-upstream-canary,x-envoy-upstream-healthchecked-cluster,x-envoy-upstream-service-time,x-request-id3578lb # [ 21.670903] envoy[922]: [2026-09-20 12:47:49.796][922][info][main] [source/server/server.cc:534] response trailer map: 144 bytes: grpc-message,grpc-status3579lb # [ 21.673275] envoy[922]: [2026-09-20 12:47:49.977][922][info][main] [source/server/server.cc:968] runtime: {}3580lb # [ 21.674725] envoy[922]: [2026-09-20 12:47:50.002][922][info][main] [source/server/server.cc:420] Raised file descriptor limits from 1024 to 524288.3581lb # [ 21.676752] envoy[922]: [2026-09-20 12:47:50.002][922][info][main] [source/server/server.cc:796] Starting admin HTTP server at 127.0.0.1:99013582lb # [ 21.678677] envoy[922]: [2026-09-20 12:47:50.002][922][info][admin] [source/server/admin/admin.cc:66] admin address: 127.0.0.1:99013583lb # [ 21.681758] envoy[922]: [2026-09-20 12:47:50.011][922][info][config] [source/server/configuration_impl.cc:181] loading tracing configuration3584lb # [ 21.683503] envoy[922]: [2026-09-20 12:47:50.011][922][info][config] [source/server/configuration_impl.cc:132] loading 0 static secret(s)3585lb # [ 21.685485] envoy[922]: [2026-09-20 12:47:50.011][922][info][config] [source/server/configuration_impl.cc:138] loading 2 cluster(s)3586lb # [ 21.704708] envoy[922]: [2026-09-20 12:47:50.210][922][info][config] [source/server/configuration_impl.cc:148] loading 1 listener(s)3587worker2 # [ 21.707491] systemd[1]: Reached target Network is Online.3588lb # [ 21.716206] envoy[922]: [2026-09-20 12:47:50.221][922][info][config] [source/server/configuration_impl.cc:164] loading stats configuration3589lb # [ 21.719062] envoy[922]: [2026-09-20 12:47:50.224][922][warning][main] [source/server/server.cc:1038] There is no configured limit to the number of allowed active downstream connections. Configure a limit in `envoy.resource_monitors.global_downstream_max_connections` resource monitor.3590worker1 # [ 21.752413] systemd[1]: Starting Nix worker-protocol over gRPC...3591worker2 # [ 21.719270] systemd[1]: Starting scheduler-token.service...3592lb # [ 21.722775] envoy[922]: [2026-09-20 12:47:50.224][922][info][main] [source/server/server.cc:1077] starting main dispatch loop3593worker2 # [ 21.845486] systemd[1]: scheduler-token.service: Deactivated successfully.3594worker2 # [ 21.849075] systemd[1]: Finished scheduler-token.service.3595worker2 # [ 21.850510] systemd[1]: scheduler-token.service: Consumed 53ms CPU time over 129ms wall clock time, 2.9M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3596worker2 # [ 21.860789] systemd[1]: Starting Nix worker-protocol over gRPC...3597worker1 # [ 22.007244] nix-grpc-daemon[777]: ts=2026-09-20T12:47:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3598worker1 # [ 22.019450] nix-grpc-daemon[777]: ts=2026-09-20T12:47:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3599worker1 # [ 22.021888] nix-grpc-daemon[777]: ts=2026-09-20T12:47:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=13600worker1 # [ 22.067799] nix-grpc-daemon[777]: ts=2026-09-20T12:47:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3601worker1 # [ 22.072749] systemd[1]: Started Nix worker-protocol over gRPC.3602worker1 # [ 22.103240] nix-grpc-daemon[777]: ts=2026-09-20T12:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3603lb # [ 22.075431] envoy[922]: [2026-09-20 12:47:50.580][922][info][runtime] [source/common/runtime/runtime_impl.cc:567] RTDS has finished initialization3604lb # [ 22.082647] envoy[922]: [2026-09-20 12:47:50.580][922][info][upstream] [source/common/upstream/cluster_manager_impl.cc:250] cm init: all clusters initialized3605worker2 # [ 22.087985] nix-grpc-daemon[835]: ts=2026-09-20T12:47:50Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"3606lb # [ 22.088590] envoy[922]: [2026-09-20 12:47:50.580][922][info][main] [source/server/server.cc:1057] all clusters initialized. initializing init manager3607lb # [ 22.094880] envoy[922]: [2026-09-20 12:47:50.580][922][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3608worker2 # [ 22.100077] nix-grpc-daemon[835]: ts=2026-09-20T12:47:50Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc3609worker2 # [ 22.102520] nix-grpc-daemon[835]: ts=2026-09-20T12:47:50Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=13610worker2 # [ 22.151838] nix-grpc-daemon[835]: ts=2026-09-20T12:47:50Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket3611worker2 # [ 22.158285] systemd[1]: Started Nix worker-protocol over gRPC.3612worker2 # [ 22.195394] nix-grpc-daemon[835]: ts=2026-09-20T12:47:50Z level=info event=scheduler_connected addr=lb:500513613worker2 # [ 22.199867] nix-grpc-daemon[835]: ts=2026-09-20T12:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3614worker1 # [ 22.304211] nix-grpc-daemon[777]: ts=2026-09-20T12:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3615worker2 # [ 22.401987] nix-grpc-daemon[835]: ts=2026-09-20T12:47:50Z level=info event=scheduler_connected addr=lb:500513616worker2 # [ 22.407451] nix-grpc-daemon[835]: ts=2026-09-20T12:47:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3617worker2 # [ 22.609143] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_connected addr=lb:500513618worker2 # [ 22.614547] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3619worker1 # [ 22.704716] nix-grpc-daemon[777]: ts=2026-09-20T12:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.1.2:50051: Failed to connect to remote host: Connection refused"3620worker2 # [ 22.816105] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_connected addr=lb:500513621worker2 # [ 22.821448] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3622worker2 # [ 23.022822] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_connected addr=lb:500513623worker2 # [ 23.028278] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3624worker2 # [ 23.231508] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_connected addr=lb:500513625worker2 # [ 23.235778] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3626worker2 # [ 23.432413] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_connected addr=lb:500513627worker2 # [ 23.437746] nix-grpc-daemon[835]: ts=2026-09-20T12:47:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3628worker1 # [ 23.507889] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513629worker1 # [ 23.512063] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3630worker2 # [ 23.639344] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513631worker2 # [ 23.644929] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3632worker1 # [ 23.708876] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513633worker1 # [ 23.714182] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3634worker2 # [ 23.846160] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513635worker2 # [ 23.851501] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3636worker1 # [ 23.915426] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513637worker1 # [ 23.921036] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3638worker2 # [ 24.052975] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513639worker2 # [ 24.058374] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3640worker1 # [ 24.121865] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513641worker1 # [ 24.127159] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3642worker2 # [ 24.260831] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513643worker2 # [ 24.265525] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3644worker1 # [ 24.328691] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513645worker1 # [ 24.333932] nix-grpc-daemon[777]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3646worker2 # [ 24.466896] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_connected addr=lb:500513647worker2 # [ 24.472815] nix-grpc-daemon[835]: ts=2026-09-20T12:47:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3648worker1 # [ 24.537029] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513649worker1 # [ 24.541258] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3650worker2 # [ 24.674315] nix-grpc-daemon[835]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513651worker2 # [ 24.679895] nix-grpc-daemon[835]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3652worker1 # [ 24.737905] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513653worker1 # [ 24.743315] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3654worker2 # [ 24.881574] nix-grpc-daemon[835]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513655worker2 # [ 24.887224] nix-grpc-daemon[835]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3656worker1 # [ 24.944569] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513657worker1 # [ 24.950286] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3658worker2 # [ 25.088662] nix-grpc-daemon[835]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513659worker2 # [ 25.094337] nix-grpc-daemon[835]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3660worker1 # [ 25.151592] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513661worker1 # [ 25.158351] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3662worker2 # [ 25.298315] nix-grpc-daemon[835]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513663worker2 # [ 25.302931] nix-grpc-daemon[835]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3664worker1 # [ 25.359468] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_connected addr=lb:500513665worker1 # [ 25.366084] nix-grpc-daemon[777]: ts=2026-09-20T12:47:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3666worker2 # [ 25.499876] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513667worker2 # [ 25.505677] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3668worker1 # [ 25.568821] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513669worker1 # [ 25.573472] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3670worker2 # [ 25.707405] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513671worker2 # [ 25.713552] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3672worker1 # [ 25.769328] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513673worker1 # [ 25.775174] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3674worker2 # [ 25.915066] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513675worker2 # [ 25.921302] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3676worker1 # [ 25.976682] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513677worker1 # [ 25.982380] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3678worker2 # [ 26.123030] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513679worker2 # [ 26.129027] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3680worker1 # [ 26.183101] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513681worker1 # [ 26.186764] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3682worker2 # [ 26.332444] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513683worker2 # [ 26.337619] nix-grpc-daemon[835]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3684worker1 # [ 26.388212] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_connected addr=lb:500513685worker1 # [ 26.394345] nix-grpc-daemon[777]: ts=2026-09-20T12:47:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3686worker2 # [ 26.539842] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513687worker2 # [ 26.545118] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3688worker1 # [ 26.597363] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513689worker1 # [ 26.602636] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3690worker2 # [ 26.742793] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513691worker2 # [ 26.748947] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3692worker1 # [ 26.800413] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513693worker1 # [ 26.806409] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3694worker2 # [ 26.950541] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513695worker2 # [ 26.956467] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3696worker1 # [ 27.007230] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513697worker1 # [ 27.013395] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3698worker2 # [ 27.157709] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513699worker2 # [ 27.163695] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3700worker1 # [ 27.214731] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513701worker1 # [ 27.220454] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3702worker2 # [ 27.368656] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513703worker2 # [ 27.375014] nix-grpc-daemon[835]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3704worker1 # [ 27.421370] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_connected addr=lb:500513705worker1 # [ 27.427051] nix-grpc-daemon[777]: ts=2026-09-20T12:47:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3706worker2 # [ 27.569419] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513707worker2 # [ 27.575106] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3708worker1 # [ 27.630640] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513709worker1 # [ 27.635023] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3710worker2 # [ 27.776714] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513711worker2 # [ 27.782350] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3712worker1 # [ 27.831508] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513713worker1 # [ 27.837374] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3714worker2 # [ 27.983616] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513715worker2 # [ 27.989337] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3716worker1 # [ 28.038773] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513717worker1 # [ 28.044313] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3718worker2 # [ 28.190466] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513719worker2 # [ 28.196158] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3720worker1 # [ 28.245120] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513721worker1 # [ 28.250842] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3722worker2 # [ 28.399656] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513723worker2 # [ 28.404270] nix-grpc-daemon[835]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3724worker1 # [ 28.451875] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_connected addr=lb:500513725worker1 # [ 28.458248] nix-grpc-daemon[777]: ts=2026-09-20T12:47:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3726worker2 # [ 28.601515] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513727worker2 # [ 28.607049] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3728worker1 # [ 28.661573] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513729worker1 # [ 28.665890] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3730worker2 # [ 28.808531] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513731worker2 # [ 28.814121] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3732worker1 # [ 28.862592] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513733worker1 # [ 28.868197] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3734worker2 # [ 29.015487] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513735worker2 # [ 29.021283] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3736worker1 # [ 29.069170] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513737worker1 # [ 29.075260] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3738worker2 # [ 29.222213] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513739worker2 # [ 29.227799] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3740worker1 # [ 29.276447] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513741worker1 # [ 29.281784] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3742worker2 # [ 29.430636] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513743worker2 # [ 29.436208] nix-grpc-daemon[835]: ts=2026-09-20T12:47:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3744worker1 # [ 29.483375] nix-grpc-daemon[777]: ts=2026-09-20T12:47:57Z level=info event=scheduler_connected addr=lb:500513745worker1 # [ 29.494352] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3746worker2 # [ 29.637732] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513747worker2 # [ 29.643481] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3748worker1 # [ 29.694218] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513749worker1 # [ 29.698622] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3750worker2 # [ 29.845028] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513751worker2 # [ 29.850760] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3752worker1 # [ 29.894589] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513753worker1 # [ 29.900183] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3754worker2 # [ 30.052260] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513755worker2 # [ 30.057895] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3756worker1 # [ 30.101479] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513757worker1 # [ 30.106976] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3758worker2 # [ 30.259465] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513759worker2 # [ 30.265822] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3760worker1 # [ 30.308200] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513761worker1 # [ 30.314086] nix-grpc-daemon[777]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3762worker2 # [ 30.470091] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_connected addr=lb:500513763worker2 # [ 30.474526] nix-grpc-daemon[835]: ts=2026-09-20T12:47:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3764worker1 # [ 30.515261] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513765worker1 # [ 30.521352] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3766worker2 # [ 30.671625] nix-grpc-daemon[835]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513767worker2 # [ 30.677692] nix-grpc-daemon[835]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3768worker1 # [ 30.724699] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513769worker1 # [ 30.728949] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3770worker2 # [ 30.879352] nix-grpc-daemon[835]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513771worker2 # [ 30.885394] nix-grpc-daemon[835]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3772worker1 # [ 30.925615] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513773worker1 # [ 30.931371] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3774worker2 # [ 31.086824] nix-grpc-daemon[835]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513775worker2 # [ 31.092721] nix-grpc-daemon[835]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3776worker1 # [ 31.133890] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513777worker1 # [ 31.139760] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3778worker2 # [ 31.293886] nix-grpc-daemon[835]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513779worker2 # [ 31.299497] nix-grpc-daemon[835]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3780worker1 # [ 31.340866] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_connected addr=lb:500513781worker1 # [ 31.346068] nix-grpc-daemon[777]: ts=2026-09-20T12:47:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3782worker2 # [ 31.503316] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513783worker2 # [ 31.507781] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3784worker1 # [ 31.547227] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513785worker1 # [ 31.553062] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3786worker2 # [ 31.704925] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513787worker2 # [ 31.710701] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3788worker1 # [ 31.757079] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513789worker1 # [ 31.761482] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3790worker2 # [ 31.912229] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513791worker2 # [ 31.917807] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3792worker1 # [ 31.957062] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513793worker1 # [ 31.962890] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3794worker2 # [ 32.119331] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513795worker2 # [ 32.125265] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3796worker1 # [ 32.163742] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513797worker1 # [ 32.169431] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3798worker2 # [ 32.326746] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513799worker2 # [ 32.332773] nix-grpc-daemon[835]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3800worker1 # [ 32.370975] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_connected addr=lb:500513801worker1 # [ 32.376681] nix-grpc-daemon[777]: ts=2026-09-20T12:48:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3802worker2 # [ 32.536730] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513803worker2 # [ 32.541302] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3804worker1 # [ 32.578098] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513805worker1 # [ 32.583813] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3806worker2 # [ 32.738165] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513807worker2 # [ 32.744353] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3808worker1 # [ 32.791076] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513809worker1 # [ 32.798193] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3810worker2 # [ 32.945441] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513811worker2 # [ 32.951788] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3812worker1 # [ 32.991905] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513813worker1 # [ 32.997478] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3814worker2 # [ 33.153155] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513815worker2 # [ 33.159875] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3816worker1 # [ 33.198065] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513817worker1 # [ 33.203611] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3818worker2 # [ 33.360629] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513819worker2 # [ 33.366728] nix-grpc-daemon[835]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3820worker1 # [ 33.404594] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_connected addr=lb:500513821worker1 # [ 33.410119] nix-grpc-daemon[777]: ts=2026-09-20T12:48:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3822lb # [ 33.527354] rustfs-bucket-start[1049]: mb s3://farm3823lb # [ 33.543279] systemd[1]: Finished rustfs-bucket.service.3824lb # [ 33.559259] systemd[1]: Starting niks3 server...3825worker2 # [ 33.570272] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513826worker2 # [ 33.574759] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3827worker1 # [ 33.611305] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513828worker1 # [ 33.617079] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3829worker2 # [ 33.771149] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513830worker2 # [ 33.777040] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3831worker1 # [ 33.821266] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513832worker1 # [ 33.825558] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3833lb # [ 33.821175] postgres[1067]: [1067] ERROR: relation "goose_db_version" does not exist at character 363834lb # [ 33.822539] postgres[1067]: [1067] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC3835lb # [ 33.858317] niks3-server[1060]: 2026/09/20 12:48:02 OK 20241026095416_initial_model.sql (23.34ms)3836lb # [ 33.862006] niks3-server[1060]: 2026/09/20 12:48:02 OK 20251210153512_drop_unused_gin_index.sql (4ms)3837lb # [ 33.868205] niks3-server[1060]: 2026/09/20 12:48:02 OK 20251218171726_add_pins.sql (6.01ms)3838lb # [ 33.873765] niks3-server[1060]: 2026/09/20 12:48:02 OK 20260628120000_add_object_size_and_stats.sql (5.5ms)3839lb # [ 33.879320] niks3-server[1060]: 2026/09/20 12:48:02 OK 20260905000000_add_claims.sql (5.57ms)3840lb # [ 33.884275] niks3-server[1060]: 2026/09/20 12:48:02 OK 20260920000000_drop_claims.sql (4.8ms)3841lb # [ 33.885533] niks3-server[1060]: 2026/09/20 12:48:02 goose: successfully migrated database to version: 202609200000003842lb # [ 33.890877] niks3-server[1060]: 2026/09/20 12:48:02 OK 1_commit_pending_closure.sql (6.64ms)3843lb # [ 33.893723] niks3-server[1060]: 2026/09/20 12:48:02 OK 2_object_stats_trigger.sql (1.44ms)3844lb # [ 33.895124] niks3-server[1060]: 2026/09/20 12:48:02 goose: up to current file version: 23845lb # [ 33.899152] niks3-server[1060]: 2026/09/20 12:48:02 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key3846lb # [ 33.946187] niks3-server[1060]: 2026/09/20 12:48:02 INFO Read proxy enabled — serving cache objects from S33847lb # [ 33.947636] niks3-server[1060]: 2026/09/20 12:48:02 INFO Using socket-activated listener address=0.0.0.0:57513848lb # [ 33.951241] systemd[1]: Started niks3 server.3849lb # [ 33.955344] niks3-server[1060]: 2026/09/20 12:48:02 INFO systemd watchdog enabled interval=15s3850lb # [ 33.958291] niks3-server[1060]: 2026/09/20 12:48:02 INFO Starting HTTP server address=0.0.0.0:57513851lb # [ 33.959920] systemd[1]: Reached target Multi-User System.3852lb # [ 33.961520] systemd[1]: Startup finished in 1.071s (kernel) + 5.577s (initrd) + 27.306s (userspace) = 33.955s.3853lb # [ 33.970222] niks3-server[1060]: 2026/09/20 12:48:02 INFO lead: acquired remote=192.168.1.4:430663854worker2 # [ 33.976514] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_take_over3855worker2 # [ 33.982304] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513856worker1 # [ 34.021098] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513857worker2 # [ 33.988102] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3858worker1 # [ 34.026908] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3859worker2 # [ 34.189143] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513860worker2 # [ 34.194014] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3861worker1 # [ 34.227669] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513862worker1 # [ 34.233494] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"3863worker2 # [ 34.395452] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513864worker1 # [ 34.434826] nix-grpc-daemon[777]: ts=2026-09-20T12:48:02Z level=info event=scheduler_connected addr=lb:500513865worker2 # [ 34.408347] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:505943866worker2 # [ 34.415936] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=03867worker2 # [ 34.430765] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=oidc_jwks provider=mock keys=13868worker2 # [ 34.435881] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:505943869worker2 # [ 34.441318] nix-grpc-daemon[835]: ts=2026-09-20T12:48:02Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=03870lb: (finished: waiting for unit niks3.service, in 19.45 seconds)3871lb: waiting for TCP port 5751 on localhost3872lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!3873lb: (finished: waiting for TCP port 5751 on localhost, in 0.10 seconds)3874lb: waiting for unit envoy.service3875lb: (finished: waiting for unit envoy.service, in 0.07 seconds)3876lb: waiting for TCP port 50051 on localhost3877lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!3878lb # [ 34.677666] envoy[922]: conn peer=[::1]:56494 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=33879lb: (finished: waiting for TCP port 50051 on localhost, in 0.05 seconds)3880worker2 # [ 34.877024] systemd[1]: Starting scheduler-token.service...3881worker2 # [ 34.976673] systemd[1]: scheduler-token.service: Deactivated successfully.3882worker2 # [ 34.982022] systemd[1]: Finished scheduler-token.service.3883worker2 # [ 34.983892] systemd[1]: scheduler-token.service: Consumed 43ms CPU time over 101ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.3884lb: waiting until exactly no endpoints of aarch64-linux are unhealthy3885??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.3886 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1283887lb: must succeed: curl -sf localhost:9901/clusters3888lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)3889lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)3890subtest: one node schedules and both builders hold a WorkerSession on it3891lb: must succeed: curl -sf localhost:9901/clusters3892lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3893worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics3894worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)3895lb: must succeed: curl -sf localhost:9901/clusters3896lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)3897lb: must succeed: curl -sf localhost:9901/clusters3898lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)3899worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'3900worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds)3901(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.35 seconds)3902subtest: balancer auth: client cert, bearer token, nothing, unknown CN3903worker2 # [ 35.799157] nix-grpc-daemon[835]: ts=2026-09-20T12:48:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506043904worker2 # [ 35.818340] systemd[1]: Started Nix Daemon.3905worker2 # [ 36.082314] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)3906worker2 # [ 36.120342] nix-grpc-daemon[835]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5)3907worker2 # [ 36.139410] nix-grpc-daemon[835]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 2/5)3908worker2 # [ 36.278961] nix-grpc-daemon[835]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 360 ms (attempt 3/5)3909worker2 # [ 36.639453] nix-grpc-daemon[835]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 457 ms (attempt 4/5)3910worker2 # [ 37.109665] nix-grpc-daemon[835]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3911worker1 # [ 37.183798] nix-grpc-daemon[777]: ts=2026-09-20T12:48:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330063912worker1 # [ 37.205139] systemd[1]: Started Nix Daemon.3913worker1 # [ 37.520806] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3914worker1 # [ 37.549874] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5)3915worker1 # [ 37.590145] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 2/5)3916worker1 # [ 37.731140] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 3/5)3917worker1 # [ 37.896862] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 235 ms (attempt 4/5)3918worker1 # [ 38.142688] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org3919worker1 # [ 38.162547] nix-grpc-daemon[777]: ts=2026-09-20T12:48:06Z level=debug event=path_info_miss path=ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test error="error: path '/nix/store/ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test' is required, but there is no substituter that can build it"3920worker1 # [ 38.176725] nix-grpc-daemon[777]: ts=2026-09-20T12:48:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330063921worker1 # [ 38.192408] nix-grpc-daemon[777]: ts=2026-09-20T12:48:06Z level=debug event=path_info_miss path=ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test error="error: path '/nix/store/ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test' is required, but there is no substituter that can build it"3922client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt3923client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds)3924client: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key3925client: (finished: must succeed: install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)3926worker2 # [ 38.615053] nix-grpc-daemon[835]: ts=2026-09-20T12:48:07Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:506043927worker2 # [ 38.634348] nix-grpc-daemon[835]: ts=2026-09-20T12:48:07Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:506043928worker2 # [ 38.651511] nix-grpc-daemon[835]: ts=2026-09-20T12:48:07Z level=debug event=path_info_miss path=ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test error="error: path '/nix/store/ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test' is required, but there is no substituter that can build it"3929worker1 # [ 38.703641] nix-grpc-daemon[777]: ts=2026-09-20T12:48:07Z level=info event=oidc_jwks provider=mock keys=13930worker1 # [ 38.710096] nix-grpc-daemon[777]: ts=2026-09-20T12:48:07Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:330063931worker1 # [ 38.719674] nix-grpc-daemon[777]: ts=2026-09-20T12:48:07Z level=debug event=path_info_miss path=ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test error="error: path '/nix/store/ml1jx6qcw1prpa5z0v7856q1y99ffw44-nixos-system-client-test' is required, but there is no substituter that can build it"3932client: must succeed: rm -r /var/lib/nix-grpc-store3933client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)3934worker2 # [ 39.000279] nix-grpc-daemon[835]: ts=2026-09-20T12:48:07Z level=info event=denied method=QueryMissing cn=- role=none3935worker2 # [ 39.268874] nix-grpc-daemon[835]: ts=2026-09-20T12:48:07Z level=info event=denied method=QueryMissing cn=stranger role=none3936(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.85 seconds)3937subtest: DAG build is scheduled across workers and lands in the cache3938client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&23939worker2 # [ 39.530535] nix-grpc-daemon[835]: ts=2026-09-20T12:48:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506043940client # don't know how to build these paths:3941client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv3942worker2 # [ 39.548588] nix-grpc-daemon[835]: ts=2026-09-20T12:48:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506043943worker2 # [ 39.594351] nix-grpc-daemon[835]: ts=2026-09-20T12:48:08Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=03944worker2 # [ 39.601449] nix-grpc-daemon[835]: ts=2026-09-20T12:48:08Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=13945worker1 # [ 39.691146] nix-grpc-daemon[777]: ts=2026-09-20T12:48:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330063946worker1 # [ 39.713426] nix-grpc-daemon[777]: ts=2026-09-20T12:48:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330063947client # copying 1 paths to 'lb:50051'...3948client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...3949worker1 # [ 39.730900] nix-grpc-daemon[777]: ts=2026-09-20T12:48:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330063950worker1 # [ 39.753909] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3951worker1 # [ 39.791240] nix-grpc-daemon[777]: ts=2026-09-20T12:48:08Z level=info event=niks3_push_started pid=8463952worker1 # [ 40.107092] nix-grpc-daemon[846]: time=2026-09-20T12:48:08.620Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""3953worker1 # [ 40.322491] nix-daemon[836]: accepted connection from pid 859, user nix-grpc-daemon (trusted)3954worker1 # [ 40.344702] nix-daemon[836]: reaped child process 862, status = succeeded3955worker1 # [ 40.543960] nix-daemon[836]: accepted connection from pid 864, user nix-grpc-daemon (trusted)3956worker1 # [ 40.563579] nix-daemon[836]: reaped child process 867, status = succeeded3957lb # [ 40.547337] niks3-server[1060]: 2026/09/20 12:48:09 INFO Received uploads request method=POST path=/api/pending_closures3958worker1 # [ 40.610193] nix-grpc-daemon[846]: time=2026-09-20T12:48:09.125Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3959worker1 # [ 40.614151] nix-grpc-daemon[846]: time=2026-09-20T12:48:09.127Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"3960lb # [ 40.647648] niks3-server[1060]: 2026/09/20 12:48:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign3961lb # [ 40.661869] niks3-server[1060]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst3962lb # [ 40.707307] niks3-server[1060]: 2026/09/20 12:48:09 INFO Signed narinfos id=1 count=13963worker1 # [ 40.744154] nix-grpc-daemon[846]: time=2026-09-20T12:48:09.258Z level=INFO msg="Uploading 1 narinfos"3964lb # [ 40.722681] niks3-server[1060]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls3965lb # [ 40.727811] niks3-server[1060]: 2026/09/20 12:48:09 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete3966lb # [ 40.742140] niks3-server[1060]: 2026/09/20 12:48:09 INFO Completed upload id=13967worker1 # [ 40.774927] nix-grpc-daemon[846]: time=2026-09-20T12:48:09.289Z level=INFO msg="Upload complete. (658ms)"3968lb # [ 40.750758] niks3-server[1060]: 2026/09/20 12:48:09 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo3969worker1 # [ 40.789105] nix-grpc-daemon[777]: ts=2026-09-20T12:48:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 paths=1 nar_bytes_in=5043970worker1 # [ 40.803441] nix-grpc-daemon[777]: ts=2026-09-20T12:48:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330063971worker1 # [ 40.822410] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3972worker1 # [ 40.830995] nix-daemon[836]: reaped child process 844, status = succeeded3973client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...3974client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3975worker1 # [ 40.849152] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)3976lb # [ 41.709967] envoy[922]: conn peer=[2001:db8:1::1]:43384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=24083977lb # [ 41.715152] envoy[922]: conn peer=[2001:db8:1::1]:43388 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=361 ms=883978lb # [ 41.719805] envoy[922]: conn peer=[2001:db8:1::1]:43396 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=173979lb # [ 41.724684] envoy[922]: conn peer=[2001:db8:1::1]:43410 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=143980client # farm-a> LOG-farm-a-t13981worker1 # [ 41.876971] nix-daemon[836]: reaped child process 871, status = succeeded3982worker1 # [ 42.044926] nix-daemon[836]: accepted connection from pid 886, user nix-grpc-daemon (trusted)3983worker1 # [ 42.063618] nix-daemon[836]: reaped child process 889, status = succeeded3984lb # [ 42.046045] niks3-server[1060]: 2026/09/20 12:48:10 INFO Received uploads request method=POST path=/api/pending_closures3985worker1 # [ 42.095492] nix-grpc-daemon[846]: time=2026-09-20T12:48:10.609Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"3986worker1 # [ 42.097942] nix-grpc-daemon[846]: time=2026-09-20T12:48:10.609Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"3987lb # [ 42.099170] niks3-server[1060]: 2026/09/20 12:48:10 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv3988lb # [ 42.112440] niks3-server[1060]: 2026/09/20 12:48:10 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst3989lb # [ 42.121743] niks3-server[1060]: 2026/09/20 12:48:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign3990worker1 # [ 42.160229] nix-grpc-daemon[846]: time=2026-09-20T12:48:10.674Z level=INFO msg="Uploading 1 narinfos"3991lb # [ 42.129386] niks3-server[1060]: 2026/09/20 12:48:10 INFO Signed narinfos id=2 count=13992lb # [ 42.133508] niks3-server[1060]: 2026/09/20 12:48:10 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls3993lb # [ 42.148338] niks3-server[1060]: 2026/09/20 12:48:10 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete3994lb # [ 42.157902] niks3-server[1060]: 2026/09/20 12:48:10 INFO Completed upload id=23995worker1 # [ 42.190471] nix-grpc-daemon[846]: time=2026-09-20T12:48:10.705Z level=INFO msg="Upload complete. (329ms)"3996lb # [ 42.160834] niks3-server[1060]: 2026/09/20 12:48:10 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo3997worker1 # [ 42.205339] nix-grpc-daemon[777]: ts=2026-09-20T12:48:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=13998worker2 # [ 42.183767] nix-grpc-daemon[835]: ts=2026-09-20T12:48:10Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=03999worker1 # [ 42.223088] nix-daemon[836]: reaped child process 869, status = succeeded4000worker2 # [ 42.190380] nix-grpc-daemon[835]: ts=2026-09-20T12:48:10Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14001worker1 # [ 42.265575] nix-grpc-daemon[777]: ts=2026-09-20T12:48:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064002worker1 # [ 42.282897] nix-grpc-daemon[777]: ts=2026-09-20T12:48:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064003worker1 # [ 42.298947] nix-grpc-daemon[846]: time=2026-09-20T12:48:10.813Z level=INFO msg="All 1 paths already cached"4004client # copying 1 paths to 'lb:50051'...4005worker1 # [ 42.310643] nix-grpc-daemon[777]: ts=2026-09-20T12:48:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330064006client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4007worker1 # [ 42.332524] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4008worker1 # [ 42.542209] nix-daemon[836]: accepted connection from pid 893, user nix-grpc-daemon (trusted)4009worker1 # [ 42.564300] nix-daemon[836]: reaped child process 896, status = succeeded4010worker1 # [ 42.741692] nix-daemon[836]: accepted connection from pid 898, user nix-grpc-daemon (trusted)4011worker1 # [ 42.770206] nix-daemon[836]: reaped child process 901, status = succeeded4012lb # [ 42.747923] niks3-server[1060]: 2026/09/20 12:48:11 INFO Received uploads request method=POST path=/api/pending_closures4013worker1 # [ 42.793491] nix-grpc-daemon[846]: time=2026-09-20T12:48:11.307Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4014worker1 # [ 42.795041] nix-grpc-daemon[846]: time=2026-09-20T12:48:11.307Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4015lb # [ 42.782582] niks3-server[1060]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4016worker1 # [ 42.827604] nix-grpc-daemon[846]: time=2026-09-20T12:48:11.342Z level=INFO msg="Uploading 1 narinfos"4017lb # [ 42.799933] niks3-server[1060]: 2026/09/20 12:48:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4018lb # [ 42.809893] niks3-server[1060]: 2026/09/20 12:48:11 INFO Signed narinfos id=3 count=14019lb # [ 42.814001] niks3-server[1060]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4020lb # [ 42.821892] niks3-server[1060]: 2026/09/20 12:48:11 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4021lb # [ 42.828120] niks3-server[1060]: 2026/09/20 12:48:11 INFO Completed upload id=34022worker1 # [ 42.859073] nix-grpc-daemon[846]: time=2026-09-20T12:48:11.374Z level=INFO msg="Upload complete. (508ms)"4023worker1 # [ 42.864252] nix-grpc-daemon[777]: ts=2026-09-20T12:48:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_in=6244024lb # [ 42.831948] niks3-server[1060]: 2026/09/20 12:48:11 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4025worker1 # [ 42.871821] nix-grpc-daemon[777]: ts=2026-09-20T12:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064026worker1 # [ 42.873955] nix-daemon[836]: reaped child process 891, status = succeeded4027worker1 # [ 42.877708] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4028worker1 # [ 42.886490] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4029client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4030client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4031worker1 # [ 44.307408] nix-daemon[836]: reaped child process 906, status = succeeded4032client # farm-b> LOG-farm-b-t14033worker1 # [ 44.465066] nix-daemon[836]: accepted connection from pid 921, user nix-grpc-daemon (trusted)4034worker1 # [ 44.485015] nix-daemon[836]: reaped child process 924, status = succeeded4035lb # [ 44.470416] niks3-server[1060]: 2026/09/20 12:48:12 INFO Received uploads request method=POST path=/api/pending_closures4036worker1 # [ 44.511846] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.025Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4037worker1 # [ 44.513853] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.026Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4038lb # [ 44.513570] niks3-server[1060]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4039lb # [ 44.526039] niks3-server[1060]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4040worker1 # [ 44.565115] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.079Z level=INFO msg="Uploading 1 narinfos"4041lb # [ 44.536156] niks3-server[1060]: 2026/09/20 12:48:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4042lb # [ 44.543051] niks3-server[1060]: 2026/09/20 12:48:13 INFO Signed narinfos id=4 count=14043lb # [ 44.546441] niks3-server[1060]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4044worker1 # [ 44.587439] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.102Z level=INFO msg="Upload complete. (292ms)"4045lb # [ 44.555825] niks3-server[1060]: 2026/09/20 12:48:13 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4046lb # [ 44.558939] niks3-server[1060]: 2026/09/20 12:48:13 INFO Completed upload id=44047lb # [ 44.562711] niks3-server[1060]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4048worker1 # [ 44.597022] nix-grpc-daemon[777]: ts=2026-09-20T12:48:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14049worker2 # [ 44.574974] nix-grpc-daemon[835]: ts=2026-09-20T12:48:13Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04050worker1 # [ 44.614411] nix-daemon[836]: reaped child process 904, status = succeeded4051worker2 # [ 44.581331] nix-grpc-daemon[835]: ts=2026-09-20T12:48:13Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14052worker1 # [ 44.653169] nix-grpc-daemon[777]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064053worker1 # [ 44.671227] nix-grpc-daemon[777]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064054worker1 # [ 44.687837] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.202Z level=INFO msg="All 2 paths already cached"4055client # copying 1 paths to 'lb:50051'...4056client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4057worker1 # [ 44.699487] nix-grpc-daemon[777]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330064058worker1 # [ 44.718866] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4059worker1 # [ 44.930586] nix-daemon[836]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4060worker1 # [ 44.949941] nix-daemon[836]: reaped child process 931, status = succeeded4061worker1 # [ 45.132420] nix-daemon[836]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4062worker1 # [ 45.153371] nix-daemon[836]: reaped child process 936, status = succeeded4063lb # [ 45.132662] niks3-server[1060]: 2026/09/20 12:48:13 INFO Received uploads request method=POST path=/api/pending_closures4064worker1 # [ 45.177318] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.692Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4065worker1 # [ 45.179354] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.694Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4066lb # [ 45.167254] niks3-server[1060]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4067worker1 # [ 45.220288] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.735Z level=INFO msg="Uploading 1 narinfos"4068lb # [ 45.180549] niks3-server[1060]: 2026/09/20 12:48:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4069lb # [ 45.192848] niks3-server[1060]: 2026/09/20 12:48:13 INFO Signed narinfos id=5 count=14070lb # [ 45.200109] niks3-server[1060]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4071lb # [ 45.210171] niks3-server[1060]: 2026/09/20 12:48:13 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4072lb # [ 45.218470] niks3-server[1060]: 2026/09/20 12:48:13 INFO Completed upload id=54073worker1 # [ 45.250516] nix-grpc-daemon[846]: time=2026-09-20T12:48:13.765Z level=INFO msg="Upload complete. (512ms)"4074lb # [ 45.221405] niks3-server[1060]: 2026/09/20 12:48:13 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4075worker1 # [ 45.260362] nix-grpc-daemon[777]: ts=2026-09-20T12:48:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_in=7524076worker1 # [ 45.271162] nix-grpc-daemon[777]: ts=2026-09-20T12:48:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064077worker1 # [ 45.281996] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4078worker1 # [ 45.286703] nix-daemon[836]: reaped child process 926, status = succeeded4079client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4080worker1 # [ 45.298560] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4081client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4082client # farm-top> LOG-farm-top-t14083worker1 # [ 46.548880] nix-daemon[836]: reaped child process 940, status = succeeded4084worker1 # [ 46.726718] nix-daemon[836]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4085worker1 # [ 46.747705] nix-daemon[836]: reaped child process 959, status = succeeded4086lb # [ 46.729028] niks3-server[1060]: 2026/09/20 12:48:15 INFO Received uploads request method=POST path=/api/pending_closures4087worker1 # [ 46.774827] nix-grpc-daemon[846]: time=2026-09-20T12:48:15.287Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4088worker1 # [ 46.777130] nix-grpc-daemon[846]: time=2026-09-20T12:48:15.287Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4089lb # [ 46.771445] niks3-server[1060]: 2026/09/20 12:48:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4090worker1 # [ 46.813913] nix-grpc-daemon[846]: time=2026-09-20T12:48:15.328Z level=INFO msg="Uploading 1 narinfos"4091lb # [ 46.788368] niks3-server[1060]: 2026/09/20 12:48:15 INFO Signed narinfos id=6 count=14092lb # [ 46.791562] niks3-server[1060]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4093lb # [ 46.809276] niks3-server[1060]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4094lb # [ 46.823621] niks3-server[1060]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4095worker1 # [ 46.855269] nix-grpc-daemon[846]: time=2026-09-20T12:48:15.370Z level=INFO msg="Upload complete. (343ms)"4096lb # [ 46.827295] niks3-server[1060]: 2026/09/20 12:48:15 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4097lb # [ 46.833618] niks3-server[1060]: 2026/09/20 12:48:15 INFO Completed upload id=64098lb # [ 46.839247] niks3-server[1060]: 2026/09/20 12:48:15 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4099worker1 # [ 46.869253] nix-grpc-daemon[777]: ts=2026-09-20T12:48:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14100worker1 # [ 46.887034] nix-daemon[836]: reaped child process 938, status = succeeded4101client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.59 seconds)4102client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4103client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.24 seconds)4104client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4105client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4106client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4107client # copying 3 paths...4108client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4109client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4110client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4111client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4112(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.28 seconds)4113subtest: repeat is answered Cached by the scheduler without building4114worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4115worker1 # finding garbage collector roots...4116worker1 # deleting specified paths...4117worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4118worker1 # deleting unused links...4119worker1 # note: hard linking is currently saving 0.0 KiB4120worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.27 seconds)4121worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4122worker2 # finding garbage collector roots...4123worker2 # deleting specified paths...4124worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4125worker2 # deleting unused links...4126worker2 # note: hard linking is currently saving 0.0 KiB4127worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4128client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24129client # don't know how to build these paths:4130client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4131worker2 # [ 48.407521] nix-grpc-daemon[835]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506044132worker1 # [ 48.446338] nix-grpc-daemon[777]: ts=2026-09-20T12:48:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064133worker1 # [ 48.459744] nix-grpc-daemon[846]: time=2026-09-20T12:48:16.974Z level=INFO msg="All 2 paths already cached"4134worker2 # [ 48.445934] nix-grpc-daemon[835]: ts=2026-09-20T12:48:16Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14135client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.34 seconds)4136client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4137client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.22 seconds)4138lb: must succeed: curl -sf localhost:9901/clusters4139lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4140lb: must succeed: curl -sf localhost:9901/clusters4141lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4142worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4143worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4144(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.46 seconds)4145subtest: inputs already in the cache are not sent to the scheduler4146lb: must succeed: curl -sf localhost:9901/clusters4147lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4148lb: must succeed: curl -sf localhost:9901/clusters4149lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4150worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4151worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4152client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24153worker1 # [ 49.534488] nix-grpc-daemon[777]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330064154client # don't know how to build these paths:4155client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4156worker1 # [ 49.544669] nix-grpc-daemon[777]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064157worker1 # [ 49.558402] nix-grpc-daemon[846]: time=2026-09-20T12:48:18.072Z level=INFO msg="All 2 paths already cached"4158worker2 # [ 49.543646] nix-grpc-daemon[835]: ts=2026-09-20T12:48:18Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04159worker2 # [ 49.550015] nix-grpc-daemon[835]: ts=2026-09-20T12:48:18Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14160worker1 # [ 49.621465] nix-grpc-daemon[777]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064161worker1 # [ 49.638294] nix-grpc-daemon[777]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064162worker1 # [ 49.654522] nix-grpc-daemon[846]: time=2026-09-20T12:48:18.169Z level=INFO msg="All 2 paths already cached"4163client # copying 1 paths to 'lb:50051'...4164client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4165worker1 # [ 49.667112] nix-grpc-daemon[777]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330064166worker1 # [ 49.682936] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4167worker1 # [ 49.893543] nix-daemon[836]: accepted connection from pid 976, user nix-grpc-daemon (trusted)4168worker1 # [ 49.914201] nix-daemon[836]: reaped child process 979, status = succeeded4169worker1 # [ 50.091711] nix-daemon[836]: accepted connection from pid 981, user nix-grpc-daemon (trusted)4170worker1 # [ 50.111881] nix-daemon[836]: reaped child process 984, status = succeeded4171lb # [ 50.093378] niks3-server[1060]: 2026/09/20 12:48:18 INFO Received uploads request method=POST path=/api/pending_closures4172worker1 # [ 50.136411] nix-grpc-daemon[846]: time=2026-09-20T12:48:18.651Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4173worker1 # [ 50.137986] nix-grpc-daemon[846]: time=2026-09-20T12:48:18.651Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4174lb # [ 50.128511] niks3-server[1060]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4175worker1 # [ 50.172645] nix-grpc-daemon[846]: time=2026-09-20T12:48:18.687Z level=INFO msg="Uploading 1 narinfos"4176lb # [ 50.143630] niks3-server[1060]: 2026/09/20 12:48:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4177lb # [ 50.155118] niks3-server[1060]: 2026/09/20 12:48:18 INFO Signed narinfos id=7 count=14178lb # [ 50.159618] niks3-server[1060]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4179lb # [ 50.169372] niks3-server[1060]: 2026/09/20 12:48:18 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4180lb # [ 50.175010] niks3-server[1060]: 2026/09/20 12:48:18 INFO Completed upload id=74181worker1 # [ 50.207532] nix-grpc-daemon[846]: time=2026-09-20T12:48:18.722Z level=INFO msg="Upload complete. (506ms)"4182lb # [ 50.177720] niks3-server[1060]: 2026/09/20 12:48:18 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4183worker1 # [ 50.216329] nix-grpc-daemon[777]: ts=2026-09-20T12:48:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_in=7524184worker1 # [ 50.231162] nix-grpc-daemon[777]: ts=2026-09-20T12:48:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064185worker1 # [ 50.238152] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4186worker1 # [ 50.243214] nix-daemon[836]: reaped child process 974, status = succeeded4187client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4188worker1 # [ 50.256071] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4189client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4190client # farm-top> LOG-farm-top-t1b4191worker1 # [ 51.707323] nix-daemon[836]: reaped child process 988, status = succeeded4192lb # [ 51.710896] envoy[922]: conn peer=[2001:db8:1::1]:43414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9579 tx=3879 ms=73324193lb # [ 51.716181] envoy[922]: conn peer=[2001:db8:1::1]:44658 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=614194worker1 # [ 51.881595] nix-daemon[836]: accepted connection from pid 1003, user nix-grpc-daemon (trusted)4195worker1 # [ 51.902758] nix-daemon[836]: reaped child process 1006, status = succeeded4196lb # [ 51.885405] niks3-server[1060]: 2026/09/20 12:48:20 INFO Received uploads request method=POST path=/api/pending_closures4197worker1 # [ 51.955089] nix-grpc-daemon[846]: time=2026-09-20T12:48:20.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4198worker1 # [ 51.964810] nix-grpc-daemon[846]: time=2026-09-20T12:48:20.474Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4199lb # [ 51.961227] niks3-server[1060]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4200lb # [ 51.974263] niks3-server[1060]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4201lb # [ 51.982218] niks3-server[1060]: 2026/09/20 12:48:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4202worker1 # [ 52.019754] nix-grpc-daemon[846]: time=2026-09-20T12:48:20.534Z level=INFO msg="Uploading 1 narinfos"4203lb # [ 51.992410] niks3-server[1060]: 2026/09/20 12:48:20 INFO Signed narinfos id=8 count=14204lb # [ 51.998673] niks3-server[1060]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4205lb # [ 52.008764] niks3-server[1060]: 2026/09/20 12:48:20 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4206lb # [ 52.018155] niks3-server[1060]: 2026/09/20 12:48:20 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4207worker1 # [ 52.059521] nix-grpc-daemon[846]: time=2026-09-20T12:48:20.574Z level=INFO msg="Upload complete. (385ms)"4208lb # [ 52.027805] niks3-server[1060]: 2026/09/20 12:48:20 INFO Completed upload id=84209worker1 # [ 52.073181] nix-grpc-daemon[777]: ts=2026-09-20T12:48:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14210worker1 # [ 52.091638] nix-daemon[836]: reaped child process 986, status = succeeded4211client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.83 seconds)4212client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4213client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4214lb: must succeed: curl -sf localhost:9901/clusters4215lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4216lb: must succeed: curl -sf localhost:9901/clusters4217lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4218worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4219worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4220(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.46 seconds)4221subtest: two clients wanting the same drv build it once4222client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4223client # [ 52.648790] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4224client # Running as unit: dedup1.service; invocation ID: 4847131988ee4ca3b8defa7f6c4a0ac44225client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.09 seconds)4226client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4227client # [ 52.898167] nix[880]: don't know how to build these paths:4228client # [ 52.899225] nix[880]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4229worker2 # [ 52.828526] nix-grpc-daemon[835]: ts=2026-09-20T12:48:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506044230worker2 # [ 52.843002] nix-grpc-daemon[835]: ts=2026-09-20T12:48:21Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04231worker2 # [ 52.850624] nix-grpc-daemon[835]: ts=2026-09-20T12:48:21Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14232worker2 # [ 52.890948] nix-grpc-daemon[835]: ts=2026-09-20T12:48:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506044233worker2 # [ 52.910721] nix-grpc-daemon[835]: ts=2026-09-20T12:48:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506044234client # [ 52.992971] nix[880]: copying 1 paths to 'lb:50051'...4235client # [ 52.997930] nix[880]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4236worker2 # [ 52.928914] nix-grpc-daemon[835]: ts=2026-09-20T12:48:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506044237worker2 # [ 52.945702] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4238worker2 # [ 52.987901] nix-grpc-daemon[835]: ts=2026-09-20T12:48:21Z level=info event=niks3_push_started pid=9574239worker2 # [ 53.264895] nix-grpc-daemon[957]: time=2026-09-20T12:48:21.782Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4240worker2 # [ 53.460816] nix-daemon[904]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4241worker2 # [ 53.480817] nix-daemon[904]: reaped child process 973, status = succeeded4242worker2 # [ 53.655289] nix-daemon[904]: accepted connection from pid 975, user nix-grpc-daemon (trusted)4243worker2 # [ 53.675388] nix-daemon[904]: reaped child process 978, status = succeeded4244lb # [ 53.690205] niks3-server[1060]: 2026/09/20 12:48:22 INFO Received uploads request method=POST path=/api/pending_closures4245worker2 # [ 53.698893] nix-grpc-daemon[957]: time=2026-09-20T12:48:22.220Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4246worker2 # [ 53.702508] nix-grpc-daemon[957]: time=2026-09-20T12:48:22.221Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4247client # [ 53.763776] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4248client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.13 seconds)4249lb # [ 53.732634] niks3-server[1060]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4250worker2 # [ 53.751505] nix-grpc-daemon[957]: time=2026-09-20T12:48:22.272Z level=INFO msg="Uploading 1 narinfos"4251lb # [ 53.745673] niks3-server[1060]: 2026/09/20 12:48:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4252lb # [ 53.758064] niks3-server[1060]: 2026/09/20 12:48:22 INFO Signed narinfos id=9 count=14253lb # [ 53.765611] niks3-server[1060]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4254lb # [ 53.778020] niks3-server[1060]: 2026/09/20 12:48:22 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4255lb # [ 53.785913] niks3-server[1060]: 2026/09/20 12:48:22 INFO Completed upload id=94256worker2 # [ 53.784069] nix-grpc-daemon[957]: time=2026-09-20T12:48:22.305Z level=INFO msg="Upload complete. (512ms)"4257lb # [ 53.788867] niks3-server[1060]: 2026/09/20 12:48:22 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4258worker2 # [ 53.792241] nix-grpc-daemon[835]: ts=2026-09-20T12:48:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=0 paths=1 nar_bytes_in=4324259worker2 # [ 53.808656] nix-grpc-daemon[835]: ts=2026-09-20T12:48:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506044260worker2 # [ 53.815212] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4261worker2 # [ 53.825699] nix-daemon[904]: reaped child process 955, status = succeeded4262client # Running as unit: dedup2.service; invocation ID: 07d51f03c64d487fb5a4a64cc53fe2694263client # [ 53.906551] nix[880]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4264worker2 # [ 53.841391] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4265client: must succeed: sleep 24266worker1 # [ 55.138245] nix-grpc-daemon[777]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330064267client # [ 55.184290] nix[896]: don't know how to build these paths:4268client # [ 55.184464] nix[896]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4269worker2 # [ 55.122159] nix-grpc-daemon[835]: ts=2026-09-20T12:48:23Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04270worker2 # [ 55.126939] nix-grpc-daemon[835]: ts=2026-09-20T12:48:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506044271client # [ 55.199797] nix[896]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4272client: (finished: must succeed: sleep 2, in 2.06 seconds)4273??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4274 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394275client: waiting for success: ! systemctl is-active dedup1 dedup24276??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4277 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394278worker2 # [ 57.493474] nix-daemon[904]: reaped child process 985, status = succeeded4279worker2 # [ 57.677201] nix-daemon[904]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)4280worker2 # [ 57.696655] nix-daemon[904]: reaped child process 1016, status = succeeded4281lb # [ 57.713298] niks3-server[1060]: 2026/09/20 12:48:26 INFO Received uploads request method=POST path=/api/pending_closures4282worker2 # [ 57.787307] nix-grpc-daemon[957]: time=2026-09-20T12:48:26.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4283worker2 # [ 57.793703] nix-grpc-daemon[957]: time=2026-09-20T12:48:26.312Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4284lb # [ 57.828845] niks3-server[1060]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4285lb # [ 57.845841] niks3-server[1060]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4286lb # [ 57.859185] niks3-server[1060]: 2026/09/20 12:48:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4287lb # [ 57.867640] niks3-server[1060]: 2026/09/20 12:48:26 INFO Signed narinfos id=10 count=14288worker2 # [ 57.868138] nix-grpc-daemon[957]: time=2026-09-20T12:48:26.387Z level=INFO msg="Uploading 1 narinfos"4289lb # [ 57.871860] niks3-server[1060]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4290lb # [ 57.889943] niks3-server[1060]: 2026/09/20 12:48:26 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4291lb # [ 57.899168] niks3-server[1060]: 2026/09/20 12:48:26 INFO Completed upload id=104292worker2 # [ 57.898111] nix-grpc-daemon[957]: time=2026-09-20T12:48:26.419Z level=INFO msg="Upload complete. (436ms)"4293lb # [ 57.902500] niks3-server[1060]: 2026/09/20 12:48:26 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4294worker2 # [ 57.916655] nix-grpc-daemon[835]: ts=2026-09-20T12:48:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=2 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14295worker2 # [ 57.932420] nix-grpc-daemon[835]: ts=2026-09-20T12:48:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14296worker2 # [ 57.939831] nix-daemon[904]: reaped child process 982, status = succeeded4297client # [ 58.040840] systemd[1]: dedup1.service: Deactivated successfully.4298client # [ 58.046188] systemd[1]: dedup1.service: Consumed 156ms CPU time over 5.395s wall clock time, 37.2M memory peak, 8.7K incoming IP traffic, 9.1K outgoing IP traffic.4299client # [ 58.057091] systemd[1]: dedup2.service: Deactivated successfully.4300client # [ 58.059059] systemd[1]: dedup2.service: Consumed 222ms CPU time over 4.290s wall clock time, 31.2M memory peak, 7.4K incoming IP traffic, 6.4K outgoing IP traffic.4301client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.15 seconds)4302client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4303client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4304worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4305worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4306worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4307worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4308worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4309worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)4310worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4311worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4312(finished: subtest: two clients wanting the same drv build it once, in 6.32 seconds)4313subtest: build hook: nix-daemon with builders = grpc://lb4314client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4315client # [ 59.121750] systemd[1]: Started Nix Daemon.4316client # [ 59.314485] nix-daemon[938]: accepted connection from pid 935, user root (trusted)4317worker1 # [ 59.483869] nix-grpc-daemon[777]: ts=2026-09-20T12:48:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:330064318worker1 # [ 59.490834] nix-grpc-daemon[777]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064319worker2 # [ 59.472666] nix-grpc-daemon[835]: ts=2026-09-20T12:48:27Z level=debug event=want client=6 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04320worker2 # [ 59.473059] nix-grpc-daemon[835]: ts=2026-09-20T12:48:27Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14321worker1 # [ 59.553416] nix-grpc-daemon[777]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064322worker1 # [ 59.570658] nix-grpc-daemon[777]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064323worker1 # [ 59.586974] nix-grpc-daemon[777]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330064324worker1 # [ 59.603117] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4325worker1 # [ 59.820739] nix-daemon[836]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4326worker1 # [ 59.842510] nix-daemon[836]: reaped child process 1034, status = succeeded4327worker1 # [ 60.021293] nix-daemon[836]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)4328worker1 # [ 60.041536] nix-daemon[836]: reaped child process 1039, status = succeeded4329lb # [ 60.025689] niks3-server[1060]: 2026/09/20 12:48:28 INFO Received uploads request method=POST path=/api/pending_closures4330worker1 # [ 60.067137] nix-grpc-daemon[846]: time=2026-09-20T12:48:28.581Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4331worker1 # [ 60.068789] nix-grpc-daemon[846]: time=2026-09-20T12:48:28.581Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4332lb # [ 60.057293] niks3-server[1060]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4333worker1 # [ 60.108158] nix-grpc-daemon[846]: time=2026-09-20T12:48:28.622Z level=INFO msg="Uploading 1 narinfos"4334lb # [ 60.075089] niks3-server[1060]: 2026/09/20 12:48:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4335lb # [ 60.086168] niks3-server[1060]: 2026/09/20 12:48:28 INFO Signed narinfos id=11 count=14336lb # [ 60.093571] niks3-server[1060]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4337lb # [ 60.108554] niks3-server[1060]: 2026/09/20 12:48:28 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4338worker1 # [ 60.145941] nix-grpc-daemon[846]: time=2026-09-20T12:48:28.660Z level=INFO msg="Upload complete. (525ms)"4339lb # [ 60.114749] niks3-server[1060]: 2026/09/20 12:48:28 INFO Completed upload id=114340lb # [ 60.122638] niks3-server[1060]: 2026/09/20 12:48:28 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4341worker1 # [ 60.156203] nix-grpc-daemon[777]: ts=2026-09-20T12:48:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_in=5124342worker1 # [ 60.165042] nix-grpc-daemon[777]: ts=2026-09-20T12:48:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064343worker1 # [ 60.178424] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4344worker1 # [ 60.183540] nix-daemon[836]: reaped child process 1029, status = succeeded4345worker1 # [ 60.196271] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4346worker1 # [ 61.647366] nix-daemon[836]: reaped child process 1043, status = succeeded4347lb # [ 61.710573] envoy[922]: conn peer=[2001:db8:1::1]:44668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4159 tx=1728 ms=25514348lb # [ 61.715852] envoy[922]: conn peer=[2001:db8:1::1]:35846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1086 ms=28314349lb # [ 61.720990] envoy[922]: conn peer=[2001:db8:1::1]:35840 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2992 tx=1374 ms=51024350worker1 # [ 61.828725] nix-daemon[836]: accepted connection from pid 1058, user nix-grpc-daemon (trusted)4351worker1 # [ 61.851029] nix-daemon[836]: reaped child process 1061, status = succeeded4352lb # [ 61.830911] niks3-server[1060]: 2026/09/20 12:48:30 INFO Received uploads request method=POST path=/api/pending_closures4353worker1 # [ 61.876503] nix-grpc-daemon[846]: time=2026-09-20T12:48:30.387Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4354worker1 # [ 61.878088] nix-grpc-daemon[846]: time=2026-09-20T12:48:30.387Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4355lb # [ 61.884995] niks3-server[1060]: 2026/09/20 12:48:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4356worker1 # [ 61.924769] nix-grpc-daemon[846]: time=2026-09-20T12:48:30.439Z level=INFO msg="Uploading 1 narinfos"4357lb # [ 61.894111] niks3-server[1060]: 2026/09/20 12:48:30 INFO Signed narinfos id=12 count=14358lb # [ 61.905697] niks3-server[1060]: 2026/09/20 12:48:30 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4359lb # [ 61.942291] niks3-server[1060]: 2026/09/20 12:48:30 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4360lb # [ 61.950935] niks3-server[1060]: 2026/09/20 12:48:30 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4361lb # [ 61.963136] niks3-server[1060]: 2026/09/20 12:48:30 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4362lb # [ 61.969741] niks3-server[1060]: 2026/09/20 12:48:30 INFO Completed upload id=124363worker1 # [ 62.001212] nix-grpc-daemon[846]: time=2026-09-20T12:48:30.516Z level=INFO msg="Upload complete. (389ms)"4364lb # [ 61.974663] niks3-server[1060]: 2026/09/20 12:48:30 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4365worker1 # [ 62.013519] nix-grpc-daemon[777]: ts=2026-09-20T12:48:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=6 outputs=14366worker1 # [ 62.036002] nix-grpc-daemon[777]: ts=2026-09-20T12:48:30Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330064367worker1 # [ 62.041921] nix-daemon[836]: reaped child process 1041, status = succeeded4368worker2 # [ 62.025436] nix-grpc-daemon[835]: ts=2026-09-20T12:48:30Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:506044369worker2 # [ 62.042886] nix-grpc-daemon[835]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4370worker2 # [ 62.066222] nix-grpc-daemon[835]: ts=2026-09-20T12:48:30Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=0 paths=1 nar_bytes_out=1284371worker1 # [ 62.409549] nix-grpc-daemon[777]: ts=2026-09-20T12:48:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:330064372worker1 # [ 62.417531] nix-grpc-daemon[777]: ts=2026-09-20T12:48:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064373worker1 # [ 62.424529] nix-grpc-daemon[846]: time=2026-09-20T12:48:30.938Z level=INFO msg="All 1 paths already cached"4374worker2 # [ 62.405601] nix-grpc-daemon[835]: ts=2026-09-20T12:48:30Z level=debug event=want client=7 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04375worker2 # [ 62.412735] nix-grpc-daemon[835]: ts=2026-09-20T12:48:30Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14376worker1 # [ 62.450242] nix-grpc-daemon[777]: ts=2026-09-20T12:48:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064377worker1 # [ 62.466758] nix-grpc-daemon[777]: ts=2026-09-20T12:48:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064378worker1 # [ 62.482943] nix-grpc-daemon[846]: time=2026-09-20T12:48:30.997Z level=INFO msg="All 1 paths already cached"4379worker1 # [ 62.494202] nix-grpc-daemon[777]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330064380worker1 # [ 62.509169] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4381worker1 # [ 62.720849] nix-daemon[836]: accepted connection from pid 1065, user nix-grpc-daemon (trusted)4382worker1 # [ 62.743863] nix-daemon[836]: reaped child process 1068, status = succeeded4383worker1 # [ 62.922673] nix-daemon[836]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4384worker1 # [ 62.942305] nix-daemon[836]: reaped child process 1073, status = succeeded4385lb # [ 62.921435] niks3-server[1060]: 2026/09/20 12:48:31 INFO Received uploads request method=POST path=/api/pending_closures4386worker1 # [ 62.994337] nix-grpc-daemon[846]: time=2026-09-20T12:48:31.508Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4387worker1 # [ 62.995989] nix-grpc-daemon[846]: time=2026-09-20T12:48:31.508Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4388lb # [ 62.988617] niks3-server[1060]: 2026/09/20 12:48:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4389worker1 # [ 63.030504] nix-grpc-daemon[846]: time=2026-09-20T12:48:31.545Z level=INFO msg="Uploading 1 narinfos"4390lb # [ 63.001330] niks3-server[1060]: 2026/09/20 12:48:31 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4391lb # [ 63.012849] niks3-server[1060]: 2026/09/20 12:48:31 INFO Signed narinfos id=13 count=14392lb # [ 63.022158] niks3-server[1060]: 2026/09/20 12:48:31 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4393lb # [ 63.030070] niks3-server[1060]: 2026/09/20 12:48:31 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4394lb # [ 63.374368] niks3-server[1060]: 2026/09/20 12:48:31 INFO Completed upload id=134395worker1 # [ 63.410564] nix-grpc-daemon[846]: time=2026-09-20T12:48:31.925Z level=INFO msg="Upload complete. (881ms)"4396worker1 # [ 63.422664] nix-grpc-daemon[777]: ts=2026-09-20T12:48:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_in=6244397lb # [ 63.392647] niks3-server[1060]: 2026/09/20 12:48:31 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4398worker1 # [ 63.443193] nix-grpc-daemon[777]: ts=2026-09-20T12:48:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064399worker1 # [ 63.455911] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4400worker1 # [ 63.461646] nix-daemon[836]: reaped child process 1063, status = succeeded4401worker1 # [ 63.479970] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4402worker1 # [ 64.934981] nix-daemon[836]: reaped child process 1077, status = succeeded4403worker1 # [ 65.118048] nix-daemon[836]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)4404worker1 # [ 65.137583] nix-daemon[836]: reaped child process 1095, status = succeeded4405lb # [ 65.119303] niks3-server[1060]: 2026/09/20 12:48:33 INFO Received uploads request method=POST path=/api/pending_closures4406worker1 # [ 65.162109] nix-grpc-daemon[846]: time=2026-09-20T12:48:33.675Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4407worker1 # [ 65.163898] nix-grpc-daemon[846]: time=2026-09-20T12:48:33.675Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4408lb # [ 65.156393] niks3-server[1060]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4409lb # [ 65.169332] niks3-server[1060]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4410worker1 # [ 65.212625] nix-grpc-daemon[846]: time=2026-09-20T12:48:33.727Z level=INFO msg="Uploading 1 narinfos"4411lb # [ 65.180403] niks3-server[1060]: 2026/09/20 12:48:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4412lb # [ 65.183987] niks3-server[1060]: 2026/09/20 12:48:33 INFO Signed narinfos id=14 count=14413lb # [ 65.194652] niks3-server[1060]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4414lb # [ 65.204110] niks3-server[1060]: 2026/09/20 12:48:33 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4415lb # [ 65.209087] niks3-server[1060]: 2026/09/20 12:48:33 INFO Completed upload id=144416worker1 # [ 65.241664] nix-grpc-daemon[846]: time=2026-09-20T12:48:33.756Z level=INFO msg="Upload complete. (346ms)"4417lb # [ 65.211364] niks3-server[1060]: 2026/09/20 12:48:33 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4418worker1 # [ 65.253438] nix-grpc-daemon[777]: ts=2026-09-20T12:48:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=7 outputs=14419worker1 # [ 65.272767] nix-daemon[836]: reaped child process 1075, status = succeeded4420worker1 # [ 65.297212] nix-grpc-daemon[777]: ts=2026-09-20T12:48:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330064421worker1 # [ 65.326971] nix-grpc-daemon[777]: ts=2026-09-20T12:48:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:330064422worker1 # [ 65.333946] nix-grpc-daemon[777]: ts=2026-09-20T12:48:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_out=1764423worker2 # [ 65.619138] nix-grpc-daemon[835]: ts=2026-09-20T12:48:34Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:506044424worker1 # [ 65.656115] nix-grpc-daemon[777]: ts=2026-09-20T12:48:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064425worker1 # [ 65.667677] nix-grpc-daemon[846]: time=2026-09-20T12:48:34.182Z level=INFO msg="All 2 paths already cached"4426worker2 # [ 65.654878] nix-grpc-daemon[835]: ts=2026-09-20T12:48:34Z level=debug event=want client=8 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04427worker2 # [ 65.661652] nix-grpc-daemon[835]: ts=2026-09-20T12:48:34Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14428worker1 # [ 65.699455] nix-grpc-daemon[777]: ts=2026-09-20T12:48:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064429worker1 # [ 65.720160] nix-grpc-daemon[777]: ts=2026-09-20T12:48:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064430worker1 # [ 65.734241] nix-grpc-daemon[846]: time=2026-09-20T12:48:34.249Z level=INFO msg="All 2 paths already cached"4431worker1 # [ 65.744534] nix-grpc-daemon[777]: ts=2026-09-20T12:48:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330064432worker1 # [ 65.758712] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4433worker1 # [ 65.968087] nix-daemon[836]: accepted connection from pid 1099, user nix-grpc-daemon (trusted)4434worker1 # [ 65.987873] nix-daemon[836]: reaped child process 1102, status = succeeded4435worker1 # [ 66.164081] nix-daemon[836]: accepted connection from pid 1104, user nix-grpc-daemon (trusted)4436worker1 # [ 66.183665] nix-daemon[836]: reaped child process 1107, status = succeeded4437lb # [ 66.165282] niks3-server[1060]: 2026/09/20 12:48:34 INFO Received uploads request method=POST path=/api/pending_closures4438worker1 # [ 66.214774] nix-grpc-daemon[846]: time=2026-09-20T12:48:34.729Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4439worker1 # [ 66.216379] nix-grpc-daemon[846]: time=2026-09-20T12:48:34.729Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4440lb # [ 66.208476] niks3-server[1060]: 2026/09/20 12:48:34 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4441worker1 # [ 66.253922] nix-grpc-daemon[846]: time=2026-09-20T12:48:34.768Z level=INFO msg="Uploading 1 narinfos"4442lb # [ 66.222411] niks3-server[1060]: 2026/09/20 12:48:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4443lb # [ 66.232429] niks3-server[1060]: 2026/09/20 12:48:34 INFO Signed narinfos id=15 count=14444lb # [ 66.238758] niks3-server[1060]: 2026/09/20 12:48:34 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4445lb # [ 66.250552] niks3-server[1060]: 2026/09/20 12:48:34 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4446lb # [ 66.309535] niks3-server[1060]: 2026/09/20 12:48:34 INFO Completed upload id=154447worker1 # [ 66.342044] nix-grpc-daemon[846]: time=2026-09-20T12:48:34.857Z level=INFO msg="Upload complete. (564ms)"4448lb # [ 66.313751] niks3-server[1060]: 2026/09/20 12:48:34 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4449worker1 # [ 66.353212] nix-grpc-daemon[777]: ts=2026-09-20T12:48:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_in=7524450worker1 # [ 66.372443] nix-grpc-daemon[777]: ts=2026-09-20T12:48:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064451worker1 # [ 66.383677] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4452worker1 # [ 66.391596] nix-daemon[836]: reaped child process 1097, status = succeeded4453worker1 # [ 66.410946] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4454worker1 # [ 67.657647] nix-daemon[836]: reaped child process 1111, status = succeeded4455worker1 # [ 67.859805] nix-daemon[836]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)4456worker1 # [ 67.879485] nix-daemon[836]: reaped child process 1129, status = succeeded4457lb # [ 67.861031] niks3-server[1060]: 2026/09/20 12:48:36 INFO Received uploads request method=POST path=/api/pending_closures4458worker1 # [ 67.990600] nix-grpc-daemon[846]: time=2026-09-20T12:48:36.493Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4459worker1 # [ 67.995476] nix-grpc-daemon[846]: time=2026-09-20T12:48:36.493Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4460lb # [ 67.988718] niks3-server[1060]: 2026/09/20 12:48:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4461worker1 # [ 68.032544] nix-grpc-daemon[846]: time=2026-09-20T12:48:36.547Z level=INFO msg="Uploading 1 narinfos"4462lb # [ 68.004775] niks3-server[1060]: 2026/09/20 12:48:36 INFO Signed narinfos id=16 count=14463lb # [ 68.015226] niks3-server[1060]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4464lb # [ 68.024726] niks3-server[1060]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4465worker1 # [ 68.068567] nix-grpc-daemon[846]: time=2026-09-20T12:48:36.583Z level=INFO msg="Upload complete. (442ms)"4466lb # [ 68.037126] niks3-server[1060]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4467lb # [ 68.040623] niks3-server[1060]: 2026/09/20 12:48:36 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4468lb # [ 68.047011] niks3-server[1060]: 2026/09/20 12:48:36 INFO Completed upload id=164469lb # [ 68.049904] niks3-server[1060]: 2026/09/20 12:48:36 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4470worker1 # [ 68.079524] nix-grpc-daemon[777]: ts=2026-09-20T12:48:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=8 outputs=14471worker2 # [ 68.059036] nix-grpc-daemon[835]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:506044472worker1 # [ 68.101907] nix-daemon[836]: reaped child process 1109, status = succeeded4473worker2 # [ 68.080592] nix-grpc-daemon[835]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4474worker2 # [ 68.097839] nix-grpc-daemon[835]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4475worker2 # [ 68.140164] nix-grpc-daemon[835]: ts=2026-09-20T12:48:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:506044476worker2 # [ 68.146992] nix-grpc-daemon[835]: ts=2026-09-20T12:48:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=0 paths=1 nar_bytes_out=2324477client # [ 68.348731] nix-daemon[938]: reaped child process 940, status = succeeded4478client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 9.48 seconds)4479client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4480client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4481client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4482client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.06 seconds)4483client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4484client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4485(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 9.62 seconds)4486subtest: an input only one worker has still reaches the builder4487client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4488client # this derivation will be built:4489client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4490client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4491client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.58 seconds)4492client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4493client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.21 seconds)4494worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4495worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.22 seconds)4496worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4497worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4498client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24499client # [ 69.786795] nix-daemon[938]: accepted connection from pid 1021, user root (trusted)4500client # this derivation will be built:4501client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4502worker1 # [ 69.990903] nix-grpc-daemon[777]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:330064503client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4504worker2 # [ 69.961870] nix-grpc-daemon[835]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506044505client # copying 1 paths...4506client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4507worker2 # [ 69.978951] nix-grpc-daemon[835]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506044508worker2 # [ 69.994856] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4509worker2 # [ 70.215371] nix-daemon[904]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4510worker2 # [ 70.237774] nix-daemon[904]: reaped child process 1044, status = succeeded4511lb # [ 70.251536] niks3-server[1060]: 2026/09/20 12:48:38 INFO Received uploads request method=POST path=/api/pending_closures4512worker2 # [ 70.261369] nix-grpc-daemon[957]: time=2026-09-20T12:48:38.781Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4513worker2 # [ 70.263466] nix-grpc-daemon[957]: time=2026-09-20T12:48:38.781Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4514lb # [ 70.284833] niks3-server[1060]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4515lb # [ 70.299637] niks3-server[1060]: 2026/09/20 12:48:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4516worker2 # [ 70.305833] nix-grpc-daemon[957]: time=2026-09-20T12:48:38.826Z level=INFO msg="Uploading 1 narinfos"4517lb # [ 70.311091] niks3-server[1060]: 2026/09/20 12:48:38 INFO Signed narinfos id=17 count=14518lb # [ 70.318024] niks3-server[1060]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4519lb # [ 70.328846] niks3-server[1060]: 2026/09/20 12:48:38 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4520lb # [ 70.337144] niks3-server[1060]: 2026/09/20 12:48:38 INFO Completed upload id=174521worker2 # [ 70.336056] nix-grpc-daemon[957]: time=2026-09-20T12:48:38.857Z level=INFO msg="Upload complete. (324ms)"4522lb # [ 70.340090] niks3-server[1060]: 2026/09/20 12:48:38 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4523worker2 # [ 70.348382] nix-grpc-daemon[835]: ts=2026-09-20T12:48:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=0 paths=1 nar_bytes_in=1204524worker2 # [ 70.370504] nix-grpc-daemon[835]: ts=2026-09-20T12:48:38Z level=debug event=want client=9 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04525worker2 # [ 70.376706] nix-grpc-daemon[835]: ts=2026-09-20T12:48:38Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14526worker2 # [ 70.381916] nix-daemon[904]: reaped child process 1039, status = succeeded4527worker2 # [ 70.416108] nix-grpc-daemon[835]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506044528worker2 # [ 70.433068] nix-grpc-daemon[835]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506044529client # copying 2 paths to 'lb:50051'...4530client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4531worker2 # [ 70.455267] nix-grpc-daemon[835]: ts=2026-09-20T12:48:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506044532client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4533worker2 # [ 70.470661] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4534worker2 # [ 70.686566] nix-daemon[904]: accepted connection from pid 1050, user nix-grpc-daemon (trusted)4535worker2 # [ 70.709677] nix-daemon[904]: reaped child process 1053, status = succeeded4536worker2 # [ 70.885149] nix-daemon[904]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4537worker2 # [ 70.903483] nix-daemon[904]: reaped child process 1058, status = succeeded4538lb # [ 70.916758] niks3-server[1060]: 2026/09/20 12:48:39 INFO Received uploads request method=POST path=/api/pending_closures4539lb # [ 70.928199] niks3-server[1060]: 2026/09/20 12:48:39 INFO Received uploads request method=POST path=/api/pending_closures4540worker2 # [ 70.936531] nix-grpc-daemon[957]: time=2026-09-20T12:48:39.457Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4541worker2 # [ 70.938078] nix-grpc-daemon[957]: time=2026-09-20T12:48:39.457Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4542worker2 # [ 70.939802] nix-grpc-daemon[957]: time=2026-09-20T12:48:39.457Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4543lb # [ 70.969695] niks3-server[1060]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4544lb # [ 70.983218] niks3-server[1060]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4545lb # [ 70.993555] niks3-server[1060]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4546lb # [ 71.000172] niks3-server[1060]: 2026/09/20 12:48:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4547lb # [ 71.006022] niks3-server[1060]: 2026/09/20 12:48:39 INFO Signed narinfos id=18 count=14548worker2 # [ 71.005797] nix-grpc-daemon[957]: time=2026-09-20T12:48:39.526Z level=INFO msg="Uploading 3 narinfos"4549lb # [ 71.010884] niks3-server[1060]: 2026/09/20 12:48:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4550lb # [ 71.016905] niks3-server[1060]: 2026/09/20 12:48:39 INFO Signed narinfos id=19 count=24551lb # [ 71.018866] niks3-server[1060]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4552lb # [ 71.038463] niks3-server[1060]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4553lb # [ 71.044687] niks3-server[1060]: 2026/09/20 12:48:39 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4554lb # [ 71.049364] niks3-server[1060]: 2026/09/20 12:48:39 INFO Completed upload id=184555lb # [ 71.055174] niks3-server[1060]: 2026/09/20 12:48:39 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4556lb # [ 71.060488] niks3-server[1060]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4557lb # [ 71.063607] niks3-server[1060]: 2026/09/20 12:48:39 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4558lb # [ 71.066293] niks3-server[1060]: 2026/09/20 12:48:39 INFO Completed upload id=194559worker2 # [ 71.062437] nix-grpc-daemon[957]: time=2026-09-20T12:48:39.583Z level=INFO msg="Upload complete. (571ms)"4560worker2 # [ 71.074256] nix-grpc-daemon[835]: ts=2026-09-20T12:48:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=0 paths=2 nar_bytes_in=9844561worker2 # [ 71.088431] nix-grpc-daemon[835]: ts=2026-09-20T12:48:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506044562worker2 # [ 71.105646] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4563worker2 # [ 71.110655] nix-daemon[904]: reaped child process 1048, status = succeeded4564client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4565worker2 # [ 71.123473] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4566client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4567lb # [ 71.712915] envoy[922]: conn peer=[2001:db8:1::1]:35852 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3404 tx=1662 ms=26754568lb # [ 71.718099] envoy[922]: conn peer=[2001:db8:1::1]:41810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1114569lb # [ 71.722919] envoy[922]: conn peer=[2001:db8:1::1]:41812 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=1876 ms=29844570lb # [ 71.727822] envoy[922]: conn peer=[2001:db8:1::1]:41828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=764571lb # [ 71.733383] envoy[922]: conn peer=[2001:db8:1::1]:41842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4313 tx=2223 ms=25914572lb # [ 71.738237] envoy[922]: conn peer=[2001:db8:1::1]:41844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=684573worker2 # [ 72.573700] nix-daemon[904]: reaped child process 1062, status = succeeded4574worker2 # [ 72.759717] nix-daemon[904]: accepted connection from pid 1077, user nix-grpc-daemon (trusted)4575worker2 # [ 72.778584] nix-daemon[904]: reaped child process 1080, status = succeeded4576lb # [ 72.795892] niks3-server[1060]: 2026/09/20 12:48:41 INFO Received uploads request method=POST path=/api/pending_closures4577worker2 # [ 72.821051] nix-grpc-daemon[957]: time=2026-09-20T12:48:41.341Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4578worker2 # [ 72.852652] nix-grpc-daemon[957]: time=2026-09-20T12:48:41.373Z level=INFO msg="Uploading 1 narinfos"4579lb # [ 72.846079] niks3-server[1060]: 2026/09/20 12:48:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4580lb # [ 72.859480] niks3-server[1060]: 2026/09/20 12:48:41 INFO Signed narinfos id=20 count=14581lb # [ 72.868361] niks3-server[1060]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4582lb # [ 72.882901] niks3-server[1060]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4583lb # [ 72.890114] niks3-server[1060]: 2026/09/20 12:48:41 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4584worker2 # [ 72.895160] nix-grpc-daemon[957]: time=2026-09-20T12:48:41.416Z level=INFO msg="Upload complete. (355ms)"4585lb # [ 72.899599] niks3-server[1060]: 2026/09/20 12:48:41 INFO Completed upload id=204586lb # [ 72.907338] niks3-server[1060]: 2026/09/20 12:48:41 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4587worker2 # [ 72.909277] nix-grpc-daemon[835]: ts=2026-09-20T12:48:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=9 outputs=14588worker2 # [ 72.927757] nix-daemon[904]: reaped child process 1060, status = succeeded4589client # copying 1 paths...4590worker2 # [ 72.953978] nix-grpc-daemon[835]: ts=2026-09-20T12:48:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:506044591client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4592worker1 # [ 73.016845] nix-grpc-daemon[777]: ts=2026-09-20T12:48:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:330064593worker1 # [ 73.034228] nix-grpc-daemon[777]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4594worker1 # [ 73.057642] nix-grpc-daemon[777]: ts=2026-09-20T12:48:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_out=1204595client # [ 73.226742] nix-daemon[938]: reaped child process 1024, status = succeeded4596client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.68 seconds)4597client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24598client # [ 73.452475] nix-daemon[938]: accepted connection from pid 1043, user root (trusted)4599client # this derivation will be built:4600client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4601worker1 # [ 73.626700] nix-grpc-daemon[777]: ts=2026-09-20T12:48:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:330064602client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4603worker2 # [ 73.598195] nix-grpc-daemon[835]: ts=2026-09-20T12:48:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506044604worker2 # [ 73.610495] nix-grpc-daemon[957]: time=2026-09-20T12:48:42.131Z level=INFO msg="All 1 paths already cached"4605worker2 # [ 73.629452] nix-grpc-daemon[835]: ts=2026-09-20T12:48:42Z level=debug event=want client=10 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04606worker2 # [ 73.636129] nix-grpc-daemon[835]: ts=2026-09-20T12:48:42Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14607worker1 # [ 73.673156] nix-grpc-daemon[777]: ts=2026-09-20T12:48:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064608worker1 # [ 73.689142] nix-grpc-daemon[777]: ts=2026-09-20T12:48:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064609client # copying 1 paths to 'lb:50051'...4610worker1 # [ 73.712436] nix-grpc-daemon[777]: ts=2026-09-20T12:48:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:330064611client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4612worker1 # [ 73.732494] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4613worker1 # [ 73.758133] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 77 ms (attempt 1/5)4614worker1 # [ 73.842190] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 145 ms (attempt 2/5)4615worker1 # [ 73.995472] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 3/5)4616worker1 # [ 74.126657] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 548 ms (attempt 4/5)4617worker1 # [ 74.683320] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4618worker1 # [ 74.694530] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4619worker1 # [ 74.954929] nix-daemon[836]: accepted connection from pid 1143, user nix-grpc-daemon (trusted)4620worker1 # [ 74.976244] nix-daemon[836]: reaped child process 1146, status = succeeded4621worker1 # [ 75.164835] nix-daemon[836]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)4622worker1 # [ 75.185087] nix-daemon[836]: reaped child process 1151, status = succeeded4623lb # [ 75.166572] niks3-server[1060]: 2026/09/20 12:48:43 INFO Received uploads request method=POST path=/api/pending_closures4624worker1 # [ 75.211534] nix-grpc-daemon[846]: time=2026-09-20T12:48:43.725Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4625worker1 # [ 75.213244] nix-grpc-daemon[846]: time=2026-09-20T12:48:43.725Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4626lb # [ 75.207211] niks3-server[1060]: 2026/09/20 12:48:43 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4627lb # [ 75.226099] niks3-server[1060]: 2026/09/20 12:48:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4628worker1 # [ 75.265391] nix-grpc-daemon[846]: time=2026-09-20T12:48:43.780Z level=INFO msg="Uploading 1 narinfos"4629lb # [ 75.238154] niks3-server[1060]: 2026/09/20 12:48:43 INFO Signed narinfos id=21 count=14630lb # [ 75.244486] niks3-server[1060]: 2026/09/20 12:48:43 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4631lb # [ 75.254955] niks3-server[1060]: 2026/09/20 12:48:43 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4632lb # [ 75.263986] niks3-server[1060]: 2026/09/20 12:48:43 INFO Completed upload id=214633worker1 # [ 75.296502] nix-grpc-daemon[846]: time=2026-09-20T12:48:43.811Z level=INFO msg="Upload complete. (565ms)"4634lb # [ 75.267070] niks3-server[1060]: 2026/09/20 12:48:43 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4635worker1 # [ 75.310086] nix-grpc-daemon[777]: ts=2026-09-20T12:48:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 paths=1 nar_bytes_in=5684636worker1 # [ 75.323425] nix-grpc-daemon[777]: ts=2026-09-20T12:48:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:330064637worker1 # [ 75.334898] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4638worker1 # [ 75.342505] nix-daemon[836]: reaped child process 1137, status = succeeded4639client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4640worker1 # [ 75.354152] nix-daemon[836]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4641client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4642worker1 # [ 76.615063] nix-daemon[836]: reaped child process 1155, status = succeeded4643worker1 # [ 76.806642] nix-daemon[836]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)4644worker1 # [ 76.826704] nix-daemon[836]: reaped child process 1173, status = succeeded4645lb # [ 76.807371] niks3-server[1060]: 2026/09/20 12:48:45 INFO Received uploads request method=POST path=/api/pending_closures4646worker1 # [ 76.853661] nix-grpc-daemon[846]: time=2026-09-20T12:48:45.367Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4647lb # [ 76.845278] niks3-server[1060]: 2026/09/20 12:48:45 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4648worker1 # [ 76.892007] nix-grpc-daemon[846]: time=2026-09-20T12:48:45.406Z level=INFO msg="Uploading 1 narinfos"4649lb # [ 76.859322] niks3-server[1060]: 2026/09/20 12:48:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4650lb # [ 76.869469] niks3-server[1060]: 2026/09/20 12:48:45 INFO Signed narinfos id=22 count=14651lb # [ 76.881905] niks3-server[1060]: 2026/09/20 12:48:45 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4652lb # [ 76.892607] niks3-server[1060]: 2026/09/20 12:48:45 INFO Completed upload id=224653worker1 # [ 76.923953] nix-grpc-daemon[846]: time=2026-09-20T12:48:45.439Z level=INFO msg="Upload complete. (352ms)"4654lb # [ 76.895731] niks3-server[1060]: 2026/09/20 12:48:45 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4655lb # [ 76.901552] niks3-server[1060]: 2026/09/20 12:48:45 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4656worker1 # [ 76.932903] nix-grpc-daemon[777]: ts=2026-09-20T12:48:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=10 outputs=14657worker1 # [ 76.952477] nix-daemon[836]: reaped child process 1153, status = succeeded4658client # copying 1 paths...4659worker1 # [ 76.982759] nix-grpc-daemon[777]: ts=2026-09-20T12:48:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330064660client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4661worker1 # [ 77.023476] nix-grpc-daemon[777]: ts=2026-09-20T12:48:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:330064662worker1 # [ 77.029522] nix-grpc-daemon[777]: ts=2026-09-20T12:48:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_out=1204663client # [ 77.191793] nix-daemon[938]: reaped child process 1046, status = succeeded4664client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.96 seconds)4665client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24666client # [ 77.425582] nix-daemon[938]: accepted connection from pid 1065, user root (trusted)4667client # this derivation will be built:4668client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4669worker2 # [ 77.570482] nix-grpc-daemon[835]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:506044670client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux'...4671worker1 # [ 77.608785] nix-grpc-daemon[777]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:330064672worker1 # [ 77.621393] nix-grpc-daemon[846]: time=2026-09-20T12:48:46.136Z level=INFO msg="All 1 paths already cached"4673worker2 # [ 77.608224] nix-grpc-daemon[835]: ts=2026-09-20T12:48:46Z level=debug event=want client=11 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04674worker2 # [ 77.616913] nix-grpc-daemon[835]: ts=2026-09-20T12:48:46Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14675worker2 # [ 77.628286] nix-grpc-daemon[835]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506044676worker2 # [ 77.641908] nix-grpc-daemon[835]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506044677client # copying 1 paths to 'lb:50051'...4678worker2 # [ 77.660992] nix-grpc-daemon[957]: time=2026-09-20T12:48:46.180Z level=INFO msg="All 1 paths already cached"4679client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4680worker2 # [ 77.667902] nix-grpc-daemon[835]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506044681worker2 # [ 77.684350] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4682worker2 # [ 77.900131] nix-daemon[904]: accepted connection from pid 1088, user nix-grpc-daemon (trusted)4683worker2 # [ 77.919774] nix-daemon[904]: reaped child process 1091, status = succeeded4684worker2 # [ 78.100064] nix-daemon[904]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)4685worker2 # [ 78.120470] nix-daemon[904]: reaped child process 1096, status = succeeded4686lb # [ 78.134766] niks3-server[1060]: 2026/09/20 12:48:46 INFO Received uploads request method=POST path=/api/pending_closures4687worker2 # [ 78.148076] nix-grpc-daemon[957]: time=2026-09-20T12:48:46.669Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4688worker2 # [ 78.149610] nix-grpc-daemon[957]: time=2026-09-20T12:48:46.669Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4689lb # [ 78.172497] niks3-server[1060]: 2026/09/20 12:48:46 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4690lb # [ 78.190882] niks3-server[1060]: 2026/09/20 12:48:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4691worker2 # [ 78.195882] nix-grpc-daemon[957]: time=2026-09-20T12:48:46.716Z level=INFO msg="Uploading 1 narinfos"4692lb # [ 78.205062] niks3-server[1060]: 2026/09/20 12:48:46 INFO Signed narinfos id=23 count=14693lb # [ 78.213212] niks3-server[1060]: 2026/09/20 12:48:46 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4694lb # [ 78.221291] niks3-server[1060]: 2026/09/20 12:48:46 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4695lb # [ 78.230183] niks3-server[1060]: 2026/09/20 12:48:46 INFO Completed upload id=234696worker2 # [ 78.228498] nix-grpc-daemon[957]: time=2026-09-20T12:48:46.749Z level=INFO msg="Upload complete. (526ms)"4697lb # [ 78.232668] niks3-server[1060]: 2026/09/20 12:48:46 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4698worker2 # [ 78.240209] nix-grpc-daemon[835]: ts=2026-09-20T12:48:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=0 paths=1 nar_bytes_in=5684699worker2 # [ 78.249509] nix-grpc-daemon[835]: ts=2026-09-20T12:48:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506044700worker2 # [ 78.262691] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4701worker2 # [ 78.268002] nix-daemon[904]: reaped child process 1086, status = succeeded4702client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4703worker2 # [ 78.284330] nix-daemon[904]: accepted connection from pid 835, user nix-grpc-daemon (trusted)4704client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4705worker2 # [ 79.341895] nix-daemon[904]: reaped child process 1100, status = succeeded4706worker2 # [ 79.540254] nix-daemon[904]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4707worker2 # [ 79.558951] nix-daemon[904]: reaped child process 1118, status = succeeded4708lb # [ 79.574394] niks3-server[1060]: 2026/09/20 12:48:48 INFO Received uploads request method=POST path=/api/pending_closures4709worker2 # [ 79.584398] nix-grpc-daemon[957]: time=2026-09-20T12:48:48.105Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4710lb # [ 79.610184] niks3-server[1060]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4711lb # [ 79.623872] niks3-server[1060]: 2026/09/20 12:48:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4712worker2 # [ 79.629752] nix-grpc-daemon[957]: time=2026-09-20T12:48:48.150Z level=INFO msg="Uploading 1 narinfos"4713lb # [ 79.638209] niks3-server[1060]: 2026/09/20 12:48:48 INFO Signed narinfos id=24 count=14714lb # [ 79.646111] niks3-server[1060]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4715lb # [ 79.657300] niks3-server[1060]: 2026/09/20 12:48:48 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4716lb # [ 79.665637] niks3-server[1060]: 2026/09/20 12:48:48 INFO Completed upload id=244717worker2 # [ 79.664685] nix-grpc-daemon[957]: time=2026-09-20T12:48:48.185Z level=INFO msg="Upload complete. (367ms)"4718lb # [ 79.669091] niks3-server[1060]: 2026/09/20 12:48:48 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4719worker2 # [ 79.677457] nix-grpc-daemon[835]: ts=2026-09-20T12:48:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50604 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=11 outputs=14720worker2 # [ 79.696983] nix-daemon[904]: reaped child process 1098, status = succeeded4721client # copying 1 paths...4722worker1 # [ 79.759624] nix-grpc-daemon[777]: ts=2026-09-20T12:48:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330064723worker1 # [ 79.776232] nix-grpc-daemon[777]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4724client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4725worker1 # [ 79.831005] nix-grpc-daemon[777]: ts=2026-09-20T12:48:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:330064726worker1 # [ 79.836957] nix-grpc-daemon[777]: ts=2026-09-20T12:48:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:33006 duration_s=0 paths=1 nar_bytes_out=1204727client # [ 80.007235] nix-daemon[938]: reaped child process 1068, status = succeeded4728client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.81 seconds)4729(finished: subtest: an input only one worker has still reaches the builder, in 11.51 seconds)4730subtest: upload whose reference the worker lacks is completed from the cache4731client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4732client # this derivation will be built:4733client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4734client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4735client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.48 seconds)4736client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4737client # this derivation will be built:4738client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4739client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4740client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.49 seconds)4741client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4742client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4743worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4744worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.25 seconds)4745worker2 # [ 81.452452] systemd[1]: Stopping Nix worker-protocol over gRPC...4746lb # [ 81.712410] envoy[922]: conn peer=[2001:db8:1::1]:41860 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4236 tx=1773 ms=31284747lb # [ 81.712740] envoy[922]: conn peer=[2001:db8:1::1]:59004 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1064748lb # [ 81.712869] envoy[922]: conn peer=[2001:db8:1::1]:59014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3724 tx=1804 ms=34564749lb # [ 81.712989] envoy[922]: conn peer=[2001:db8:1::1]:59022 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=684750lb # [ 81.713105] envoy[922]: conn peer=[2001:db8:1::1]:59030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3711 tx=1915 ms=22894751lb # [ 81.713217] envoy[922]: conn peer=[2001:db8:1::1]:59042 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=684752worker2 # [ 82.188897] nix-grpc-daemon[835]: ts=2026-09-20T12:48:50Z level=info event=signal_exit4753worker2 # [ 82.456826] nix-grpc-daemon[835]: ts=2026-09-20T12:48:50Z level=info event=scheduler_yield4754lb # [ 82.462506] niks3-server[1060]: 2026/09/20 12:48:50 INFO lead: released remote=192.168.1.4:430664755worker2 # [ 82.463236] nix-grpc-daemon[835]: ts=2026-09-20T12:48:50Z level=info event=scheduler_restarting addr=lb:500514756worker2 # [ 82.467650] nix-grpc-daemon[835]: ts=2026-09-20T12:48:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:505944757worker1 # [ 82.502641] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_restarting addr=lb:500514758worker2 # [ 82.472339] nix-grpc-daemon[835]: ts=2026-09-20T12:48:50Z level=info event=worker_gone addr=192.168.1.4:500514759worker2 # [ 82.478223] nix-grpc-daemon[835]: ts=2026-09-20T12:48:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:505944760worker2 # [ 82.482845] nix-grpc-daemon[835]: ts=2026-09-20T12:48:50Z level=info event=worker_gone addr=192.168.1.3:500514761worker2 # [ 82.663698] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_connected addr=lb:500514762worker1 # [ 82.702530] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_connected addr=lb:500514763worker2 # [ 82.669736] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4764worker1 # [ 82.707982] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4765worker2 # [ 82.871179] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_connected addr=lb:500514766worker1 # [ 82.908761] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_connected addr=lb:500514767worker2 # [ 82.876626] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4768worker1 # [ 82.914164] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4769worker2 # [ 83.077625] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_connected addr=lb:500514770worker1 # [ 83.115219] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_connected addr=lb:500514771worker2 # [ 83.082952] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4772worker1 # [ 83.121077] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4773worker2 # [ 83.286230] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_connected addr=lb:500514774worker1 # [ 83.322120] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_connected addr=lb:500514775worker2 # [ 83.290733] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4776worker1 # [ 83.327700] nix-grpc-daemon[777]: ts=2026-09-20T12:48:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4777worker2 # [ 83.462778] nix-grpc-daemon[835]: ts=2026-09-20T12:48:51Z level=info event=scheduler_restarting4778worker1 # [ 83.530968] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_connected addr=lb:500514779worker1 # [ 83.535340] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4780worker2 # [ 83.507032] nix-grpc-daemon[835]: ts=2026-09-20T12:48:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4781worker2 # [ 83.524245] nix-daemon[904]: reaped child process 906, status = succeeded4782worker1 # [ 83.731900] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_connected addr=lb:500514783worker1 # [ 83.737965] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4784worker1 # [ 83.938236] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_connected addr=lb:500514785worker1 # [ 83.944264] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4786worker1 # [ 83.995798] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_take_over4787lb # [ 83.964702] niks3-server[1060]: 2026/09/20 12:48:52 INFO lead: acquired remote=192.168.1.3:424304788worker1 # [ 84.145237] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_connected addr=lb:500514789worker1 # [ 84.151313] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4790worker1 # [ 84.351962] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=scheduler_connected addr=lb:500514791worker2 # [ 84.331230] nix-grpc-daemon[835]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4792worker2 # [ 84.335808] nix-grpc-daemon[835]: E0000 00:00:1789908532.850849 878 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true4793worker1 # [ 84.371720] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:372824794worker2 # [ 84.342494] nix-grpc-daemon[835]: E0000 00:00:1789908532.851116 878 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4795worker1 # [ 84.377552] nix-grpc-daemon[777]: ts=2026-09-20T12:48:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04796worker2 # [ 84.887417] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4797worker2 # [ 84.895760] systemd[1]: Stopped Nix worker-protocol over gRPC.4798worker2 # [ 84.903404] systemd[1]: nix-grpc-daemon.service: Consumed 2.130s CPU time over 1min 3.025s wall clock time, 52M memory peak, 59.6K incoming IP traffic, 147.9K outgoing IP traffic.4799worker2 # [ 84.918148] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4800worker2 # [ 84.920960] systemd[1]: Closed Nix worker-protocol over gRPC.4801worker2 # [ 84.924279] systemd[1]: nix-grpc-daemon.socket: Consumed 656us CPU time over 1min 11.960s wall clock time, 256K memory peak, 76.5K incoming IP traffic, 61.6K outgoing IP traffic.4802lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4803lb: must succeed: curl -sf localhost:9901/clusters4804lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4805lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4806client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24807worker1 # [ 85.290185] nix-grpc-daemon[777]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:330064808worker1 # [ 85.299738] nix-grpc-daemon[777]: ts=2026-09-20T12:48:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:330064809client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4810client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.29 seconds)4811worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4812worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)4813worker2 # [ 85.407220] systemd[1]: Listening on Nix worker-protocol over gRPC.4814worker1 # [ 85.518515] systemd[1]: Stopping Nix worker-protocol over gRPC...4815worker2 # [ 85.879351] systemd[1]: Starting scheduler-token.service...4816worker2 # [ 85.986922] systemd[1]: scheduler-token.service: Deactivated successfully.4817worker2 # [ 85.992569] systemd[1]: Finished scheduler-token.service.4818worker2 # [ 85.994064] systemd[1]: scheduler-token.service: Consumed 52ms CPU time over 110ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.4819worker2 # [ 86.003971] systemd[1]: Starting Nix worker-protocol over gRPC...4820worker1 # [ 86.110498] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=signal_exit4821worker2 # [ 86.188561] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:54Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4822worker2 # [ 86.191510] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:54Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4823worker2 # [ 86.195652] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:54Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14824worker2 # [ 86.203702] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:54Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4825worker2 # [ 86.212517] systemd[1]: Started Nix worker-protocol over gRPC.4826worker2 # [ 86.232444] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:54Z level=info event=scheduler_connected addr=lb:500514827worker1 # [ 86.269035] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:372824828worker1 # [ 86.275933] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04829worker1 # [ 86.371450] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=scheduler_yield4830worker2 # [ 86.340623] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:54Z level=info event=scheduler_restarting addr=lb:500514831lb # [ 86.342585] niks3-server[1060]: 2026/09/20 12:48:54 INFO lead: released remote=192.168.1.3:424304832worker1 # [ 86.377082] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:372824833worker1 # [ 86.381688] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=worker_gone addr=192.168.1.4:500514834worker1 # [ 86.388758] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=scheduler_restarting addr=lb:500514835worker1 # [ 86.393565] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:372824836worker1 # [ 86.398005] nix-grpc-daemon[777]: ts=2026-09-20T12:48:54Z level=info event=worker_gone addr=192.168.1.3:500514837worker2 # [ 86.542242] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514838worker2 # [ 86.547585] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4839worker1 # [ 86.588096] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514840worker1 # [ 86.593862] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4841worker2 # [ 86.748744] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514842worker2 # [ 86.753877] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4843worker1 # [ 86.794129] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514844worker1 # [ 86.800639] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4845worker2 # [ 86.955205] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514846worker2 # [ 86.960644] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4847worker1 # [ 87.000828] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514848worker1 # [ 87.007242] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4849worker2 # [ 87.161997] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514850worker2 # [ 87.167402] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4851worker1 # [ 87.208087] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514852worker1 # [ 87.213531] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4853worker1 # [ 87.384255] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=scheduler_restarting4854worker2 # [ 87.370506] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_connected addr=lb:500514855worker2 # [ 87.374864] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4856worker1 # [ 87.436561] nix-daemon[836]: reaped child process 838, status = succeeded4857worker1 # [ 87.438867] nix-grpc-daemon[777]: ts=2026-09-20T12:48:55Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4858worker2 # [ 87.571602] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_connected addr=lb:500514859worker2 # [ 87.577272] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4860worker2 # [ 87.778547] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_connected addr=lb:500514861worker2 # [ 87.784146] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4862worker2 # [ 87.985411] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_connected addr=lb:500514863worker2 # [ 87.990912] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4864worker2 # [ 88.192419] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_connected addr=lb:500514865worker2 # [ 88.197874] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4866worker1 # [ 88.263386] nix-grpc-daemon[777]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4867worker1 # [ 88.263803] nix-grpc-daemon[777]: E0000 00:00:1789908536.778139 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true4868worker1 # [ 88.275652] nix-grpc-daemon[777]: E0000 00:00:1789908536.790624 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4869worker2 # [ 88.400464] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_connected addr=lb:500514870worker2 # [ 88.405172] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4871worker2 # [ 88.606294] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_connected addr=lb:500514872worker2 # [ 88.612174] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4873worker2 # [ 88.813344] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_connected addr=lb:500514874worker1 # [ 88.850273] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4875worker2 # [ 88.818650] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4876worker1 # [ 88.859632] systemd[1]: Stopped Nix worker-protocol over gRPC.4877worker1 # [ 88.865282] systemd[1]: nix-grpc-daemon.service: Consumed 3.809s CPU time over 1min 7.102s wall clock time, 59.7M memory peak, 86.4K incoming IP traffic, 155.9K outgoing IP traffic.4878worker1 # [ 88.883619] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4879worker1 # [ 88.886598] systemd[1]: Closed Nix worker-protocol over gRPC.4880worker1 # [ 88.890460] systemd[1]: nix-grpc-daemon.socket: Consumed 637us CPU time over 1min 17.628s wall clock time, 256K memory peak, 85.8K incoming IP traffic, 71.2K outgoing IP traffic.4881lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4882lb: must succeed: curl -sf localhost:9901/clusters4883lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4884lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4885client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24886worker2 # [ 89.019967] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_connected addr=lb:500514887worker2 # [ 89.025260] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4888worker2 # [ 89.210966] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:337584889worker2 # [ 89.227789] nix-daemon[904]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4890worker2 # [ 89.244267] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_connected addr=lb:500514891worker2 # [ 89.252282] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4892worker2 # [ 89.267307] nix-grpc-daemon[1142]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 74 ms (attempt 1/5)4893worker2 # [ 89.347136] nix-grpc-daemon[1142]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 2/5)4894worker2 # [ 89.398672] nix-grpc-daemon[1142]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 3/5)4895worker2 # [ 89.430732] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_connected addr=lb:500514896worker2 # [ 89.435056] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4897worker2 # [ 89.446180] nix-grpc-daemon[1142]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 722 ms (attempt 4/5)4898worker2 # [ 89.631994] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_connected addr=lb:500514899worker2 # [ 89.637099] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4900worker2 # [ 89.838076] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_connected addr=lb:500514901worker2 # [ 89.843261] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4902worker2 # [ 90.044566] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_connected addr=lb:500514903worker2 # [ 90.049742] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4904worker2 # [ 90.176536] nix-grpc-daemon[1142]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4905client # copying 2 paths...4906client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...4907worker2 # [ 90.203577] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:337584908client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...4909worker2 # [ 90.222213] nix-daemon[904]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4910worker2 # [ 90.253758] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=niks3_push_started pid=11984911worker2 # [ 90.266500] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_connected addr=lb:500514912worker2 # [ 90.270461] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4913worker2 # [ 90.470501] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_connected addr=lb:500514914worker2 # [ 90.472759] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4915worker2 # [ 90.488474] nix-grpc-daemon[1198]: time=2026-09-20T12:48:59.008Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""4916worker2 # [ 90.675441] nix-daemon[904]: accepted connection from pid 1211, user nix-grpc-daemon (trusted)4917worker2 # [ 90.677379] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=scheduler_connected addr=lb:500514918worker2 # [ 90.678726] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4919worker2 # [ 90.698648] nix-daemon[904]: reaped child process 1214, status = succeeded4920lb # [ 90.714312] niks3-server[1060]: 2026/09/20 12:48:59 INFO Received uploads request method=POST path=/api/pending_closures4921lb # [ 90.724390] niks3-server[1060]: 2026/09/20 12:48:59 INFO Received uploads request method=POST path=/api/pending_closures4922worker2 # [ 90.732829] nix-grpc-daemon[1198]: time=2026-09-20T12:48:59.254Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4923worker2 # [ 90.738113] nix-grpc-daemon[1198]: time=2026-09-20T12:48:59.256Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"4924worker2 # [ 90.740138] nix-grpc-daemon[1198]: time=2026-09-20T12:48:59.256Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"4925lb # [ 90.775889] niks3-server[1060]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst4926lb # [ 90.798383] niks3-server[1060]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst4927lb # [ 90.809854] niks3-server[1060]: 2026/09/20 12:48:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4928lb # [ 90.818198] niks3-server[1060]: 2026/09/20 12:48:59 INFO Signed narinfos id=25 count=14929worker2 # [ 90.816199] nix-grpc-daemon[1198]: time=2026-09-20T12:48:59.335Z level=INFO msg="Uploading 3 narinfos"4930lb # [ 90.822580] niks3-server[1060]: 2026/09/20 12:48:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4931lb # [ 90.829481] niks3-server[1060]: 2026/09/20 12:48:59 INFO Signed narinfos id=26 count=24932lb # [ 90.831528] niks3-server[1060]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls4933lb # [ 90.840975] niks3-server[1060]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls4934lb # [ 90.849232] niks3-server[1060]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4935lb # [ 90.856097] niks3-server[1060]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo4936lb # [ 90.862339] niks3-server[1060]: 2026/09/20 12:48:59 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4937lb # [ 90.867476] niks3-server[1060]: 2026/09/20 12:48:59 INFO Completed upload id=254938lb # [ 90.870759] niks3-server[1060]: 2026/09/20 12:48:59 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4939lb # [ 90.873982] niks3-server[1060]: 2026/09/20 12:48:59 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo4940lb # [ 90.876595] niks3-server[1060]: 2026/09/20 12:48:59 INFO Completed upload id=264941worker2 # [ 90.872952] nix-grpc-daemon[1198]: time=2026-09-20T12:48:59.394Z level=INFO msg="Upload complete. (378ms)"4942worker2 # [ 90.884484] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33758 duration_s=0 paths=2 nar_bytes_in=3124943worker2 # [ 90.900344] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=scheduler_connected addr=lb:500514944worker2 # [ 90.903892] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4945worker2 # [ 90.910842] nix-daemon[904]: reaped child process 1196, status = succeeded4946client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.97 seconds)4947worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache4948worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)4949worker1 # [ 91.081282] systemd[1]: Listening on Nix worker-protocol over gRPC.4950worker1 # [ 91.089884] systemd[1]: Starting Nix worker-protocol over gRPC...4951worker2 # [ 91.081392] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=scheduler_connected addr=lb:500514952worker2 # [ 91.086674] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4953lb # [ 91.221309] niks3-server[1060]: 2026/09/20 12:48:59 INFO lead: acquired remote=192.168.1.4:385904954worker2 # [ 91.224325] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=scheduler_take_over4955worker1 # [ 91.292738] nix-grpc-daemon[1197]: ts=2026-09-20T12:48:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"4956worker1 # [ 91.294811] nix-grpc-daemon[1197]: ts=2026-09-20T12:48:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4957worker1 # [ 91.299559] nix-grpc-daemon[1197]: ts=2026-09-20T12:48:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=14958worker1 # [ 91.307888] nix-grpc-daemon[1197]: ts=2026-09-20T12:48:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4959worker1 # [ 91.316678] systemd[1]: Started Nix worker-protocol over gRPC.4960lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4961lb: must succeed: curl -sf localhost:9901/clusters4962worker2 # [ 91.287738] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=scheduler_connected addr=lb:500514963worker1 # [ 91.339918] nix-grpc-daemon[1197]: ts=2026-09-20T12:48:59Z level=info event=scheduler_connected addr=lb:500514964worker2 # [ 91.310208] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:484244965worker2 # [ 91.320330] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04966worker2 # [ 91.326428] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=oidc_jwks provider=mock keys=14967worker2 # [ 91.330423] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:484244968worker2 # [ 91.335689] nix-grpc-daemon[1142]: ts=2026-09-20T12:48:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04969lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4970lb # [ 91.712383] envoy[922]: conn peer=[2001:db8:1::4]:35210 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=49685 tx=2597 ms=613094971lb # [ 91.717457] envoy[922]: conn peer=[2001:db8:1::1]:35014 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=314972lb # [ 91.722151] envoy[922]: conn peer=192.168.1.3:54518 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=11018 tx=3148 ms=642904973lb # [ 91.727131] envoy[922]: conn peer=[2001:db8:1::1]:35030 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1007 tx=305 ms=16884974lb: must succeed: curl -sf localhost:9901/clusters4975lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4976lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)4977(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.51 seconds)4978subtest: scheduler down: the other node takes the lock and keeps it4979lb: must succeed: curl -sf localhost:9901/clusters4980lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4981lb: must succeed: curl -sf localhost:9901/clusters4982lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4983lb: must succeed: curl -sf localhost:9901/clusters4984lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4985lb: must succeed: curl -sf localhost:9901/clusters4986lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4987worker2 # [ 92.736221] systemd[1]: Stopping Nix worker-protocol over gRPC...4988worker2 # [ 93.213496] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:01Z level=info event=signal_exit4989worker2 # [ 93.427919] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:01Z level=info event=scheduler_yield4990lb # [ 93.433416] niks3-server[1060]: 2026/09/20 12:49:01 INFO lead: released remote=192.168.1.4:385904991worker2 # [ 93.434002] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:01Z level=info event=scheduler_restarting addr=lb:500514992worker2 # [ 93.439357] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:484244993worker1 # [ 93.475659] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:01Z level=info event=scheduler_restarting addr=lb:500514994worker2 # [ 93.445786] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:01Z level=info event=worker_gone addr=192.168.1.4:500514995worker2 # [ 93.449886] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:484244996worker2 # [ 93.454375] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:01Z level=info event=worker_gone addr=192.168.1.3:500514997worker2 # [ 93.635453] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_connected addr=lb:500514998worker2 # [ 93.640771] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4999worker1 # [ 93.675836] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:02Z level=info event=scheduler_connected addr=lb:500515000worker1 # [ 93.681345] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5001worker2 # [ 93.841916] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_connected addr=lb:500515002worker2 # [ 93.847166] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5003worker1 # [ 93.881825] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:02Z level=info event=scheduler_connected addr=lb:500515004worker1 # [ 93.887428] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5005worker2 # [ 94.048450] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_connected addr=lb:500515006worker2 # [ 94.053646] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5007worker1 # [ 94.088286] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:02Z level=info event=scheduler_connected addr=lb:500515008worker1 # [ 94.093477] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5009worker2 # [ 94.256547] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_connected addr=lb:500515010worker1 # [ 94.294701] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:02Z level=info event=scheduler_connected addr=lb:500515011worker2 # [ 94.260817] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5012worker1 # [ 94.299945] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5013worker2 # [ 94.430156] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=scheduler_restarting5014worker1 # [ 94.502771] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515015worker1 # [ 94.507664] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5016worker2 # [ 94.479241] nix-grpc-daemon[1142]: ts=2026-09-20T12:49:02Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5017worker2 # [ 94.485230] nix-daemon[904]: reaped child process 1191, status = succeeded5018worker1 # [ 94.703363] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515019worker1 # [ 94.708756] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5020worker1 # [ 94.909510] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515021worker1 # [ 94.914720] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5022worker1 # [ 95.115350] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515023worker1 # [ 95.120689] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5024worker2 # [ 95.242038] nix-grpc-daemon[1142]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5025worker2 # [ 95.242420] nix-grpc-daemon[1142]: E0000 00:00:1789908543.763034 1185 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5026worker2 # [ 95.254441] nix-grpc-daemon[1142]: E0000 00:00:1789908543.775555 1185 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5027worker1 # [ 95.321736] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_connected addr=lb:500515028worker1 # [ 95.327321] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5029worker1 # [ 95.530445] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515030worker1 # [ 95.534786] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5031worker1 # [ 95.730993] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515032worker1 # [ 95.736585] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5033worker1 # [ 95.937508] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515034worker1 # [ 95.943451] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5035worker1 # [ 96.144418] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515036worker1 # [ 96.149889] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5037worker2 # [ 96.244675] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5038worker2 # [ 96.248799] systemd[1]: Stopped Nix worker-protocol over gRPC.5039worker2 # [ 96.251231] systemd[1]: nix-grpc-daemon.service: Consumed 512ms CPU time over 10.236s wall clock time, 30.6M memory peak, 24.2K incoming IP traffic, 50K outgoing IP traffic.5040worker2 # [ 96.267580] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5041worker2 # [ 96.273340] systemd[1]: Closed Nix worker-protocol over gRPC.5042worker2 # [ 96.276170] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 10.870s wall clock time, 256K memory peak, 20.6K incoming IP traffic, 15.7K outgoing IP traffic.5043worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5044lb # [ 96.288683] niks3-server[1060]: 2026/09/20 12:49:04 INFO lead: acquired remote=192.168.1.3:444305045worker1 # [ 96.320637] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_take_over5046worker1 # [ 96.350829] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_connected addr=lb:500515047worker1 # [ 96.357944] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5048worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.10 seconds)5049worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5050worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5051worker1 # [ 96.559438] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515052worker1 # [ 96.563805] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5053worker1 # [ 96.760568] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515054worker1 # [ 96.765772] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5055worker1 # [ 96.966516] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:05Z level=info event=scheduler_connected addr=lb:500515056worker1 # [ 96.975913] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:400325057worker1 # [ 96.981271] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05058worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5059worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5060client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25061worker1 # [ 97.803718] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400405062worker1 # [ 97.820199] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5063client # don't know how to build these paths:5064client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5065worker1 # [ 97.844094] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400405066worker1 # [ 97.863369] nix-grpc-daemon[1197]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)5067worker1 # [ 97.889210] nix-grpc-daemon[1197]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 171 ms (attempt 2/5)5068worker1 # [ 98.068171] nix-grpc-daemon[1197]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 101 ms (attempt 3/5)5069worker1 # [ 98.176724] nix-grpc-daemon[1197]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 641 ms (attempt 4/5)5070worker1 # [ 98.830719] nix-grpc-daemon[1197]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5071worker1 # [ 98.862768] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:07Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05072worker1 # [ 98.870327] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:07Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15073worker1 # [ 98.909995] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400405074worker1 # [ 98.927598] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400405075client # copying 1 paths to 'lb:50051'...5076worker1 # [ 98.944111] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400405077client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5078worker1 # [ 98.962353] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5079worker1 # [ 98.989899] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:07Z level=info event=niks3_push_started pid=12675080worker1 # [ 99.219607] nix-grpc-daemon[1267]: time=2026-09-20T12:49:07.733Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5081worker1 # [ 99.397412] nix-daemon[836]: accepted connection from pid 1280, user nix-grpc-daemon (trusted)5082worker1 # [ 99.415735] nix-daemon[836]: reaped child process 1283, status = succeeded5083worker1 # [ 99.592476] nix-daemon[836]: accepted connection from pid 1285, user nix-grpc-daemon (trusted)5084worker1 # [ 99.609779] nix-daemon[836]: reaped child process 1288, status = succeeded5085lb # [ 99.594488] niks3-server[1060]: 2026/09/20 12:49:08 INFO Received uploads request method=POST path=/api/pending_closures5086worker1 # [ 99.640073] nix-grpc-daemon[1267]: time=2026-09-20T12:49:08.153Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5087worker1 # [ 99.641606] nix-grpc-daemon[1267]: time=2026-09-20T12:49:08.153Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5088lb # [ 99.636654] niks3-server[1060]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5089worker1 # [ 99.691037] nix-grpc-daemon[1267]: time=2026-09-20T12:49:08.205Z level=INFO msg="Uploading 1 narinfos"5090lb # [ 99.653017] niks3-server[1060]: 2026/09/20 12:49:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5091lb # [ 99.663825] niks3-server[1060]: 2026/09/20 12:49:08 INFO Signed narinfos id=27 count=15092lb # [ 99.669993] niks3-server[1060]: 2026/09/20 12:49:08 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5093lb # [ 99.680077] niks3-server[1060]: 2026/09/20 12:49:08 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5094lb # [ 99.688750] niks3-server[1060]: 2026/09/20 12:49:08 INFO Completed upload id=275095worker1 # [ 99.720423] nix-grpc-daemon[1267]: time=2026-09-20T12:49:08.235Z level=INFO msg="Upload complete. (497ms)"5096worker1 # [ 99.728169] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40040 duration_s=0 paths=1 nar_bytes_in=5205097worker1 # [ 99.747834] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400405098worker1 # [ 99.752446] nix-grpc-daemon[1267]: time=2026-09-20T12:49:08.252Z level=WARN msg="Failed to register uploaded object" key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo error="server returned 404: object is not pending upload\n"5099worker1 # [ 99.764306] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5100worker1 # [ 99.767641] nix-daemon[836]: reaped child process 1265, status = succeeded5101client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5102worker1 # [ 99.774975] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5103client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5104client # farm-a> LOG-farm-a-on-standby5105worker1 # [ 100.999745] nix-daemon[836]: reaped child process 1292, status = succeeded5106worker1 # [ 101.171803] nix-daemon[836]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5107worker1 # [ 101.189793] nix-daemon[836]: reaped child process 1310, status = succeeded5108lb # [ 101.171976] niks3-server[1060]: 2026/09/20 12:49:09 INFO Received uploads request method=POST path=/api/pending_closures5109worker1 # [ 101.220486] nix-grpc-daemon[1267]: time=2026-09-20T12:49:09.732Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5110worker1 # [ 101.222038] nix-grpc-daemon[1267]: time=2026-09-20T12:49:09.732Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5111lb # [ 101.237525] niks3-server[1060]: 2026/09/20 12:49:09 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5112worker1 # [ 101.280212] nix-grpc-daemon[1267]: time=2026-09-20T12:49:09.793Z level=INFO msg="Uploading 1 narinfos"5113lb # [ 101.250248] niks3-server[1060]: 2026/09/20 12:49:09 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5114lb # [ 101.261511] niks3-server[1060]: 2026/09/20 12:49:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5115lb # [ 101.271635] niks3-server[1060]: 2026/09/20 12:49:09 INFO Signed narinfos id=28 count=15116lb # [ 101.276464] niks3-server[1060]: 2026/09/20 12:49:09 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5117lb # [ 101.280990] niks3-server[1060]: 2026/09/20 12:49:09 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5118lb # [ 101.285795] niks3-server[1060]: 2026/09/20 12:49:09 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5119worker1 # [ 101.317543] nix-grpc-daemon[1267]: time=2026-09-20T12:49:09.832Z level=INFO msg="Upload complete. (333ms)"5120lb # [ 101.290213] niks3-server[1060]: 2026/09/20 12:49:09 INFO Completed upload id=285121worker1 # [ 101.342200] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40040 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15122worker1 # [ 101.359948] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:09Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05123worker1 # [ 101.364699] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:09Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15124worker1 # [ 101.369076] nix-daemon[836]: reaped child process 1290, status = succeeded5125worker1 # [ 101.391075] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400405126worker1 # [ 101.406687] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400405127worker1 # [ 101.421847] nix-grpc-daemon[1267]: time=2026-09-20T12:49:09.936Z level=INFO msg="All 1 paths already cached"5128client # copying 1 paths to 'lb:50051'...5129client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5130worker1 # [ 101.432110] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400405131worker1 # [ 101.447867] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5132worker1 # [ 101.652201] nix-daemon[836]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5133worker1 # [ 101.672778] nix-daemon[836]: reaped child process 1319, status = succeeded5134lb # [ 101.711882] envoy[922]: conn peer=[2001:db8:1::4]:60476 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=22478 tx=1180 ms=82355135worker1 # [ 101.845185] nix-daemon[836]: accepted connection from pid 1321, user nix-grpc-daemon (trusted)5136worker1 # [ 101.866473] nix-daemon[836]: reaped child process 1324, status = succeeded5137lb # [ 101.844291] niks3-server[1060]: 2026/09/20 12:49:10 INFO Received uploads request method=POST path=/api/pending_closures5138worker1 # [ 101.887123] nix-grpc-daemon[1267]: time=2026-09-20T12:49:10.401Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5139worker1 # [ 101.888792] nix-grpc-daemon[1267]: time=2026-09-20T12:49:10.401Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5140lb # [ 101.875338] niks3-server[1060]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5141worker1 # [ 101.922728] nix-grpc-daemon[1267]: time=2026-09-20T12:49:10.437Z level=INFO msg="Uploading 1 narinfos"5142lb # [ 101.893271] niks3-server[1060]: 2026/09/20 12:49:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5143lb # [ 101.904575] niks3-server[1060]: 2026/09/20 12:49:10 INFO Signed narinfos id=29 count=15144lb # [ 101.909650] niks3-server[1060]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5145worker1 # [ 101.951201] nix-grpc-daemon[1267]: time=2026-09-20T12:49:10.466Z level=INFO msg="Upload complete. (486ms)"5146lb # [ 101.920803] niks3-server[1060]: 2026/09/20 12:49:10 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5147lb # [ 101.926673] niks3-server[1060]: 2026/09/20 12:49:10 INFO Completed upload id=295148worker1 # [ 101.959148] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40040 duration_s=0 paths=1 nar_bytes_in=6405149lb # [ 101.929621] niks3-server[1060]: 2026/09/20 12:49:10 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5150worker1 # [ 101.969155] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400405151worker1 # [ 101.978629] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5152worker1 # [ 101.983109] nix-daemon[836]: reaped child process 1314, status = succeeded5153client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5154client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5155worker1 # [ 101.996458] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5156client # farm-b> LOG-farm-b-on-standby5157worker1 # [ 103.839504] nix-daemon[836]: reaped child process 1328, status = succeeded5158worker1 # [ 104.026048] nix-daemon[836]: accepted connection from pid 1343, user nix-grpc-daemon (trusted)5159worker1 # [ 104.043791] nix-daemon[836]: reaped child process 1346, status = succeeded5160lb # [ 104.027287] niks3-server[1060]: 2026/09/20 12:49:12 INFO Received uploads request method=POST path=/api/pending_closures5161worker1 # [ 104.071857] nix-grpc-daemon[1267]: time=2026-09-20T12:49:12.585Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5162worker1 # [ 104.073706] nix-grpc-daemon[1267]: time=2026-09-20T12:49:12.586Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5163lb # [ 104.065666] niks3-server[1060]: 2026/09/20 12:49:12 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5164lb # [ 104.079692] niks3-server[1060]: 2026/09/20 12:49:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5165lb # [ 104.092622] niks3-server[1060]: 2026/09/20 12:49:12 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5166lb # [ 104.102869] niks3-server[1060]: 2026/09/20 12:49:12 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5167lb # [ 104.110222] niks3-server[1060]: 2026/09/20 12:49:12 INFO Signed narinfos id=30 count=15168worker1 # [ 104.143145] nix-grpc-daemon[1267]: time=2026-09-20T12:49:12.657Z level=INFO msg="Uploading 1 narinfos"5169lb # [ 104.119757] niks3-server[1060]: 2026/09/20 12:49:12 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5170lb # [ 104.127887] niks3-server[1060]: 2026/09/20 12:49:12 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5171lb # [ 104.141786] niks3-server[1060]: 2026/09/20 12:49:12 INFO Completed upload id=305172worker1 # [ 104.174333] nix-grpc-daemon[1267]: time=2026-09-20T12:49:12.689Z level=INFO msg="Upload complete. (366ms)"5173worker1 # [ 104.189213] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40040 duration_s=2 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15174worker1 # [ 104.212457] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:12Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05175worker1 # [ 104.218489] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:12Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15176worker1 # [ 104.224132] nix-daemon[836]: reaped child process 1326, status = succeeded5177worker1 # [ 104.249039] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400405178worker1 # [ 104.264794] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400405179worker1 # [ 104.281373] nix-grpc-daemon[1267]: time=2026-09-20T12:49:12.796Z level=INFO msg="All 2 paths already cached"5180client # copying 1 paths to 'lb:50051'...5181client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5182worker1 # [ 104.291888] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400405183worker1 # [ 104.306106] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5184worker1 # [ 104.509479] nix-daemon[836]: accepted connection from pid 1352, user nix-grpc-daemon (trusted)5185worker1 # [ 104.529668] nix-daemon[836]: reaped child process 1355, status = succeeded5186worker1 # [ 104.711309] nix-daemon[836]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5187worker1 # [ 104.730256] nix-daemon[836]: reaped child process 1360, status = succeeded5188lb # [ 104.710787] niks3-server[1060]: 2026/09/20 12:49:13 INFO Received uploads request method=POST path=/api/pending_closures5189worker1 # [ 104.755202] nix-grpc-daemon[1267]: time=2026-09-20T12:49:13.269Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5190worker1 # [ 104.756912] nix-grpc-daemon[1267]: time=2026-09-20T12:49:13.269Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5191lb # [ 104.745501] niks3-server[1060]: 2026/09/20 12:49:13 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5192lb # [ 104.758011] niks3-server[1060]: 2026/09/20 12:49:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5193worker1 # [ 104.797715] nix-grpc-daemon[1267]: time=2026-09-20T12:49:13.312Z level=INFO msg="Uploading 1 narinfos"5194lb # [ 104.767775] niks3-server[1060]: 2026/09/20 12:49:13 INFO Signed narinfos id=31 count=15195lb # [ 104.773547] niks3-server[1060]: 2026/09/20 12:49:13 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5196lb # [ 104.781010] niks3-server[1060]: 2026/09/20 12:49:13 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5197lb # [ 104.786998] niks3-server[1060]: 2026/09/20 12:49:13 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5198lb # [ 104.792002] niks3-server[1060]: 2026/09/20 12:49:13 INFO Completed upload id=315199worker1 # [ 104.824521] nix-grpc-daemon[1267]: time=2026-09-20T12:49:13.339Z level=INFO msg="Upload complete. (500ms)"5200worker1 # [ 104.832643] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40040 duration_s=0 paths=1 nar_bytes_in=7685201worker1 # [ 104.842639] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400405202worker1 # [ 104.855007] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5203worker1 # [ 104.860672] nix-daemon[836]: reaped child process 1350, status = succeeded5204client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5205worker1 # [ 104.870468] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5206client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5207client # farm-top> LOG-farm-top-on-standby5208worker1 # [ 106.327226] nix-daemon[836]: reaped child process 1364, status = succeeded5209worker1 # [ 106.504868] nix-daemon[836]: accepted connection from pid 1379, user nix-grpc-daemon (trusted)5210worker1 # [ 106.522562] nix-daemon[836]: reaped child process 1382, status = succeeded5211lb # [ 106.507114] niks3-server[1060]: 2026/09/20 12:49:15 INFO Received uploads request method=POST path=/api/pending_closures5212worker1 # [ 106.551703] nix-grpc-daemon[1267]: time=2026-09-20T12:49:15.064Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5213worker1 # [ 106.553950] nix-grpc-daemon[1267]: time=2026-09-20T12:49:15.064Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5214lb # [ 106.539527] niks3-server[1060]: 2026/09/20 12:49:15 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5215lb # [ 106.556301] niks3-server[1060]: 2026/09/20 12:49:15 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5216lb # [ 106.571689] niks3-server[1060]: 2026/09/20 12:49:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5217worker1 # [ 106.610062] nix-grpc-daemon[1267]: time=2026-09-20T12:49:15.124Z level=INFO msg="Uploading 1 narinfos"5218lb # [ 106.581380] niks3-server[1060]: 2026/09/20 12:49:15 INFO Signed narinfos id=32 count=15219lb # [ 106.583904] niks3-server[1060]: 2026/09/20 12:49:15 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5220lb # [ 106.594545] niks3-server[1060]: 2026/09/20 12:49:15 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5221worker1 # [ 106.632918] nix-grpc-daemon[1267]: time=2026-09-20T12:49:15.147Z level=INFO msg="Upload complete. (345ms)"5222lb # [ 106.602123] niks3-server[1060]: 2026/09/20 12:49:15 INFO Completed upload id=325223lb # [ 106.604972] niks3-server[1060]: 2026/09/20 12:49:15 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5224worker1 # [ 106.649278] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40040 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15225worker1 # [ 106.665517] nix-daemon[836]: reaped child process 1362, status = succeeded5226client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 9.16 seconds)5227client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5228client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5229worker2 # [ 106.993132] systemd[1]: Listening on Nix worker-protocol over gRPC.5230worker2 # [ 107.000358] systemd[1]: Starting scheduler-token.service...5231worker2 # [ 107.096152] systemd[1]: scheduler-token.service: Deactivated successfully.5232worker2 # [ 107.102835] systemd[1]: Finished scheduler-token.service.5233worker2 # [ 107.104805] systemd[1]: scheduler-token.service: Consumed 39ms CPU time over 99ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5234worker2 # [ 107.111021] systemd[1]: Starting Nix worker-protocol over gRPC...5235worker2 # [ 107.291779] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:15Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5236worker2 # [ 107.294327] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:15Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5237worker2 # [ 107.298328] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:15Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15238worker2 # [ 107.306716] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:15Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5239worker2 # [ 107.320574] systemd[1]: Started Nix worker-protocol over gRPC.5240lb: must succeed: curl -sf localhost:9901/clusters5241worker2 # [ 107.335847] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:15Z level=info event=scheduler_connected addr=lb:500515242worker1 # [ 107.380191] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:15Z level=info event=oidc_jwks provider=mock keys=15243worker1 # [ 107.386155] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:15Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:400325244worker1 # [ 107.391697] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:15Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05245lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5246worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5247worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5248lb: must succeed: curl -sf localhost:9901/clusters5249lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5250lb: must succeed: curl -sf localhost:9901/clusters5251lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5252worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5253worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5254(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 15.23 seconds)5255subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5256client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5257client # [ 107.825571] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5258client # Running as unit: rs.service; invocation ID: c2aa438006e749998b6431f7622c228f5259client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.09 seconds)5260worker2 # [ 108.009562] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:529365261worker2 # [ 108.024890] nix-daemon[904]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5262client # [ 108.112658] nix[1169]: don't know how to build these paths:5263client # [ 108.113876] nix[1169]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5264worker1 # [ 108.086413] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:16Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05265worker1 # [ 108.093525] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:16Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15266worker1 # [ 108.141065] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400405267worker1 # [ 108.157253] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400405268client # [ 108.208177] nix[1169]: copying 1 paths to 'lb:50051'...5269worker1 # [ 108.173156] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400405270client # [ 108.210517] nix[1169]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5271worker1 # [ 108.190305] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5272worker1 # [ 108.404066] nix-daemon[836]: accepted connection from pid 1395, user nix-grpc-daemon (trusted)5273worker1 # [ 108.424702] nix-daemon[836]: reaped child process 1398, status = succeeded5274worker1 # [ 108.604732] nix-daemon[836]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5275worker1 # [ 108.622911] nix-daemon[836]: reaped child process 1403, status = succeeded5276lb # [ 108.605472] niks3-server[1060]: 2026/09/20 12:49:17 INFO Received uploads request method=POST path=/api/pending_closures5277worker1 # [ 108.647883] nix-grpc-daemon[1267]: time=2026-09-20T12:49:17.162Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5278worker1 # [ 108.649587] nix-grpc-daemon[1267]: time=2026-09-20T12:49:17.162Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5279lb # [ 108.632428] niks3-server[1060]: 2026/09/20 12:49:17 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5280lb # [ 108.647531] niks3-server[1060]: 2026/09/20 12:49:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5281worker1 # [ 108.686304] nix-grpc-daemon[1267]: time=2026-09-20T12:49:17.201Z level=INFO msg="Uploading 1 narinfos"5282lb # [ 108.659830] niks3-server[1060]: 2026/09/20 12:49:17 INFO Signed narinfos id=33 count=15283lb # [ 108.666893] niks3-server[1060]: 2026/09/20 12:49:17 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5284worker1 # [ 108.709784] nix-grpc-daemon[1267]: time=2026-09-20T12:49:17.224Z level=INFO msg="Upload complete. (501ms)"5285lb # [ 108.678130] niks3-server[1060]: 2026/09/20 12:49:17 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5286lb # [ 108.683196] niks3-server[1060]: 2026/09/20 12:49:17 INFO Completed upload id=335287lb # [ 108.686619] niks3-server[1060]: 2026/09/20 12:49:17 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5288worker1 # [ 108.719710] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40040 duration_s=0 paths=1 nar_bytes_in=4165289worker1 # [ 108.731508] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400405290worker1 # [ 108.738463] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5291worker1 # [ 108.743118] nix-daemon[836]: reaped child process 1393, status = succeeded5292worker1 # [ 108.754126] nix-daemon[836]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5293client # [ 108.788746] nix[1169]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5294lb: must succeed: curl -sf localhost:9901/clusters5295lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5296lb: must succeed: curl -sf localhost:9901/clusters5297lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5298lb: must succeed: curl -sf localhost:9901/clusters5299lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5300lb: must succeed: curl -sf localhost:9901/clusters5301lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5302worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5303worker2 # [ 110.723137] systemd-journald[273]: Received client request to rotate journal, rotating.5304worker2 # [ 110.743231] systemd-journald[273]: Vacuuming done, freed 0B of archived journals from /var/log/journal/d3dfbb8d8f9d4b50a24624b35913ef31.5305worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5306worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5307worker1 # [ 110.830389] systemd[1]: Stopping Nix worker-protocol over gRPC...5308worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5309??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5310 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735311client: must succeed: sleep 5.05312worker1 # [ 111.321642] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:19Z level=info event=draining5313lb # [ 111.711849] envoy[922]: conn peer=[2001:db8:1::1]:46884 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9640 tx=3905 ms=88645314client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5315worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5316worker1 # [ 116.424408] nix-daemon[836]: reaped child process 1407, status = succeeded5317worker1 # [ 116.625759] nix-daemon[836]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5318worker1 # [ 116.647604] nix-daemon[836]: reaped child process 1450, status = succeeded5319lb # [ 116.629692] niks3-server[1060]: 2026/09/20 12:49:25 INFO Received uploads request method=POST path=/api/pending_closures5320worker1 # [ 116.674077] nix-grpc-daemon[1267]: time=2026-09-20T12:49:25.188Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5321worker1 # [ 116.679267] nix-grpc-daemon[1267]: time=2026-09-20T12:49:25.190Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5322lb # [ 116.688038] niks3-server[1060]: 2026/09/20 12:49:25 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5323lb # [ 116.697513] niks3-server[1060]: 2026/09/20 12:49:25 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5324worker1 # [ 116.737139] nix-grpc-daemon[1267]: time=2026-09-20T12:49:25.251Z level=INFO msg="Uploading 1 narinfos"5325lb # [ 116.707716] niks3-server[1060]: 2026/09/20 12:49:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5326lb # [ 116.715879] niks3-server[1060]: 2026/09/20 12:49:25 INFO Signed narinfos id=34 count=15327lb # [ 116.721456] niks3-server[1060]: 2026/09/20 12:49:25 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5328lb # [ 116.732165] niks3-server[1060]: 2026/09/20 12:49:25 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5329worker1 # [ 116.763849] nix-grpc-daemon[1267]: time=2026-09-20T12:49:25.278Z level=INFO msg="Upload complete. (389ms)"5330lb # [ 116.735787] niks3-server[1060]: 2026/09/20 12:49:25 INFO Completed upload id=345331lb # [ 116.740531] niks3-server[1060]: 2026/09/20 12:49:25 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5332worker1 # [ 116.781588] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40040 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15333worker1 # [ 116.801806] nix-daemon[836]: reaped child process 1405, status = succeeded5334client # [ 116.868811] systemd[1]: rs.service: Deactivated successfully.5335client # [ 116.873647] systemd[1]: rs.service: Consumed 174ms CPU time over 9.047s wall clock time, 39.2M memory peak, 8.4K incoming IP traffic, 9K outgoing IP traffic.5336worker1 # [ 117.323780] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:25Z level=info event=signal_exit5337worker1 # [ 117.543014] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_yield5338lb # [ 117.517573] niks3-server[1060]: 2026/09/20 12:49:26 INFO lead: released remote=192.168.1.3:444305339worker1 # [ 117.551325] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_restarting addr=lb:500515340worker2 # [ 117.519641] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_restarting addr=lb:500515341worker1 # [ 117.557670] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400325342worker1 # [ 117.562573] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=worker_gone addr=192.168.1.4:500515343worker1 # [ 117.566857] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400325344worker1 # [ 117.571564] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=worker_gone addr=192.168.1.3:500515345worker1 # [ 117.752522] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_connected addr=lb:500515346worker2 # [ 117.720083] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_connected addr=lb:500515347worker2 # [ 117.727374] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5348worker1 # [ 117.761085] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5349worker2 # [ 117.929032] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_connected addr=lb:500515350worker1 # [ 117.964595] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_connected addr=lb:500515351worker2 # [ 117.934722] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5352worker1 # [ 117.969118] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker1 # [ 118.165661] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_connected addr=lb:500515354worker2 # [ 118.136558] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_connected addr=lb:500515355worker1 # [ 118.171797] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5356worker2 # [ 118.142240] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357worker1 # [ 118.372903] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_connected addr=lb:500515358worker2 # [ 118.344080] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_connected addr=lb:500515359worker1 # [ 118.378485] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5360worker2 # [ 118.350041] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5361worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.24 seconds)5362client: waiting for success: ! systemctl is-active rs5363worker1 # [ 118.550628] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:27Z level=info event=scheduler_restarting5364worker2 # [ 118.553388] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_connected addr=lb:500515365worker2 # [ 118.557779] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366worker1 # [ 118.602426] nix-grpc-daemon[1197]: ts=2026-09-20T12:49:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5367client: (finished: waiting for success: ! systemctl is-active rs, in 0.09 seconds)5368client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5369worker1 # [ 118.618837] nix-daemon[836]: reaped child process 1258, status = succeeded5370client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.07 seconds)5371client: must fail: journalctl -u rs | grep -q 'reconnecting'5372client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5373lb: must succeed: curl -sf localhost:9901/clusters5374worker2 # [ 118.754901] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_connected addr=lb:500515375worker2 # [ 118.760766] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5376lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5377worker1 # [ 118.897886] nix-grpc-daemon[1197]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5378worker1 # [ 118.902859] nix-grpc-daemon[1197]: E0000 00:00:1789908567.412616 1240 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5379worker1 # [ 118.911157] nix-grpc-daemon[1197]: E0000 00:00:1789908567.425994 1240 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5380worker2 # [ 118.962620] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_connected addr=lb:500515381worker2 # [ 118.968398] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5382worker2 # [ 119.169719] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_connected addr=lb:500515383worker2 # [ 119.175431] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker2 # [ 119.376465] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_connected addr=lb:500515385worker2 # [ 119.382484] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5386worker1 # [ 119.571126] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5387worker1 # [ 119.577094] systemd[1]: Stopped Nix worker-protocol over gRPC.5388worker1 # [ 119.579585] systemd[1]: nix-grpc-daemon.service: Consumed 1.928s CPU time over 28.481s wall clock time, 34.3M memory peak, 49K incoming IP traffic, 88.7K outgoing IP traffic.5389worker1 # [ 119.601358] systemd[1]: Starting Nix worker-protocol over gRPC...5390worker2 # [ 119.586324] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_connected addr=lb:500515391worker2 # [ 119.590658] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5392lb: must succeed: curl -sf localhost:9901/clusters5393worker1 # [ 119.819171] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5394worker1 # [ 119.821962] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5395worker2 # [ 119.787434] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_connected addr=lb:500515396worker1 # [ 119.826958] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15397worker2 # [ 119.793188] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5398worker1 # [ 119.835910] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5399worker1 # [ 119.840409] systemd[1]: Started Nix worker-protocol over gRPC.5400worker1 # [ 119.859895] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=scheduler_take_over5401lb # [ 119.830261] niks3-server[1060]: 2026/09/20 12:49:28 INFO lead: acquired remote=192.168.1.3:554605402worker1 # [ 119.869904] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=scheduler_connected addr=lb:500515403worker1 # [ 119.873041] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5404lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.12 seconds)5405worker2 # [ 119.994382] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_connected addr=lb:500515406worker2 # [ 119.999725] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker1 # [ 120.074417] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=scheduler_connected addr=lb:500515408worker1 # [ 120.080197] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5409worker2 # [ 120.201397] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_connected addr=lb:500515410worker2 # [ 120.207045] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5411worker1 # [ 120.281158] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=scheduler_connected addr=lb:500515412worker1 # [ 120.286939] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5413worker2 # [ 120.407923] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_connected addr=lb:500515414worker2 # [ 120.413638] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5415worker1 # [ 120.487960] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:29Z level=info event=scheduler_connected addr=lb:500515416worker1 # [ 120.493447] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5417worker2 # [ 120.616435] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:29Z level=info event=scheduler_connected addr=lb:500515418worker1 # [ 120.664434] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:29Z level=info event=oidc_jwks provider=mock keys=15419worker1 # [ 120.670928] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:29Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:459665420worker1 # [ 120.676468] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:29Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05421worker1 # [ 120.694090] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:29Z level=info event=scheduler_connected addr=lb:500515422worker1 # [ 120.699886] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:29Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:459665423worker1 # [ 120.705033] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:29Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05424lb: must succeed: curl -sf localhost:9901/clusters5425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5426worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5427worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5428(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.40 seconds)5429subtest: scheduler restart mid-build: build finishes, no double build5430client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5431client # [ 121.249361] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5432client # Running as unit: mid.service; invocation ID: ac841a73f4354da1ad08e879e5b4af445433client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.12 seconds)5434worker2 # [ 121.433922] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:529365435client # [ 121.513444] nix[1205]: don't know how to build these paths:5436client # [ 121.514412] nix[1205]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5437worker1 # [ 121.493523] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:30Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05438worker1 # [ 121.499947] nix-grpc-daemon[1457]: ts=2026-09-20T12:49:30Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15439worker2 # [ 121.504892] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529365440worker2 # [ 121.528380] nix-grpc-daemon[1238]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 1/5)5441worker2 # [ 121.551460] nix-grpc-daemon[1238]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 103 ms (attempt 2/5)5442worker2 # [ 121.656266] nix-grpc-daemon[1238]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 271 ms (attempt 3/5)5443lb # [ 121.711710] envoy[922]: conn peer=[2001:db8:1::1]:40830 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=1326 ms=87585444lb # [ 121.717180] envoy[922]: conn peer=[2001:db8:1::3]:52982 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4160 tx=1321 ms=272575445worker2 # [ 121.927718] nix-grpc-daemon[1238]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 793 ms (attempt 4/5)5446worker2 # [ 122.730569] nix-grpc-daemon[1238]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5447worker2 # [ 122.754481] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529365448client # [ 122.842153] nix[1205]: copying 1 paths to 'lb:50051'...5449worker2 # [ 122.771306] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529365450client # [ 122.848100] nix[1205]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5451worker2 # [ 122.793711] nix-daemon[904]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5452worker2 # [ 122.829280] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:31Z level=info event=niks3_push_started pid=13525453worker2 # [ 123.052260] nix-grpc-daemon[1352]: time=2026-09-20T12:49:31.572Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""5454worker2 # [ 123.234968] nix-daemon[904]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5455worker2 # [ 123.254757] nix-daemon[904]: reaped child process 1368, status = succeeded5456worker2 # [ 123.435984] nix-daemon[904]: accepted connection from pid 1370, user nix-grpc-daemon (trusted)5457worker2 # [ 123.455968] nix-daemon[904]: reaped child process 1373, status = succeeded5458lb # [ 123.469064] niks3-server[1060]: 2026/09/20 12:49:31 INFO Received uploads request method=POST path=/api/pending_closures5459worker2 # [ 123.474981] nix-grpc-daemon[1352]: time=2026-09-20T12:49:31.994Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5460worker2 # [ 123.476877] nix-grpc-daemon[1352]: time=2026-09-20T12:49:31.995Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5461lb # [ 123.505358] niks3-server[1060]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5462lb # [ 123.522220] niks3-server[1060]: 2026/09/20 12:49:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5463worker2 # [ 123.527151] nix-grpc-daemon[1352]: time=2026-09-20T12:49:32.047Z level=INFO msg="Uploading 1 narinfos"5464lb # [ 123.533603] niks3-server[1060]: 2026/09/20 12:49:32 INFO Signed narinfos id=35 count=15465lb # [ 123.539722] niks3-server[1060]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5466lb # [ 123.552681] niks3-server[1060]: 2026/09/20 12:49:32 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5467worker2 # [ 123.556971] nix-grpc-daemon[1352]: time=2026-09-20T12:49:32.078Z level=INFO msg="Upload complete. (501ms)"5468lb # [ 123.559110] niks3-server[1060]: 2026/09/20 12:49:32 INFO Completed upload id=355469lb # [ 123.561937] niks3-server[1060]: 2026/09/20 12:49:32 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5470worker2 # [ 123.568654] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52936 duration_s=0 paths=1 nar_bytes_in=4245471worker2 # [ 123.581830] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529365472worker2 # [ 123.588693] nix-daemon[904]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5473worker2 # [ 123.594211] nix-daemon[904]: reaped child process 1350, status = succeeded5474client # [ 123.673628] nix[1205]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5475worker2 # [ 123.608378] nix-daemon[904]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5476lb: must succeed: curl -sf localhost:9901/clusters5477lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5478lb: must succeed: curl -sf localhost:9901/clusters5479lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5480worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5481worker1 # [ 126.429385] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1457 (nix-grpc-daemon) on client request.5482worker1 # [ 126.445590] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5483lb # [ 126.414785] niks3-server[1060]: 2026/09/20 12:49:34 INFO lead: released remote=192.168.1.3:554605484worker2 # [ 126.428737] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:34Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"5485client # [ 126.499340] nix[1205]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5486worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.11 seconds)5487client: must succeed: sleep 5.05488worker1 # [ 126.484601] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5489worker1 # [ 126.487259] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5490worker1 # [ 126.494874] systemd[1]: nix-grpc-daemon.service: Consumed 216ms CPU time over 6.882s wall clock time, 11.1M memory peak, 9.9K incoming IP traffic, 10K outgoing IP traffic.5491worker1 # [ 126.588176] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5492worker1 # [ 126.601583] systemd[1]: Starting Nix worker-protocol over gRPC...5493worker2 # [ 126.629037] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:35Z level=info event=scheduler_connected addr=lb:500515494worker1 # [ 126.808559] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5495worker1 # [ 126.810566] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5496worker1 # [ 126.814881] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15497worker1 # [ 126.823440] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5498worker1 # [ 126.834958] systemd[1]: Started Nix worker-protocol over gRPC.5499worker1 # [ 126.836278] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=scheduler_take_over5500lb # [ 126.802368] niks3-server[1060]: 2026/09/20 12:49:35 INFO lead: acquired remote=192.168.1.3:447705501worker1 # [ 126.860238] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=scheduler_connected addr=lb:500515502worker1 # [ 126.864274] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=oidc_jwks provider=mock keys=15503worker1 # [ 126.865767] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:465145504worker1 # [ 126.867671] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:465145505worker1 # [ 126.869565] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05506worker1 # [ 126.871522] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:35Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=15507client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5508lb # [ 131.709915] envoy[922]: conn peer=[2001:db8:1::3]:58480 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=972 tx=385 ms=66135509client: waiting for success: ! systemctl is-active mid5510worker2 # [ 131.837293] nix-daemon[904]: reaped child process 1377, status = succeeded5511worker2 # [ 132.000137] nix-daemon[904]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)5512worker2 # [ 132.017841] nix-daemon[904]: reaped child process 1420, status = succeeded5513lb # [ 132.031726] niks3-server[1060]: 2026/09/20 12:49:40 INFO Received uploads request method=POST path=/api/pending_closures5514worker2 # [ 132.044423] nix-grpc-daemon[1352]: time=2026-09-20T12:49:40.563Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5515worker2 # [ 132.045928] nix-grpc-daemon[1352]: time=2026-09-20T12:49:40.563Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5516lb # [ 132.074131] niks3-server[1060]: 2026/09/20 12:49:40 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5517lb # [ 132.087233] niks3-server[1060]: 2026/09/20 12:49:40 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5518lb # [ 132.100380] niks3-server[1060]: 2026/09/20 12:49:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5519worker2 # [ 132.105716] nix-grpc-daemon[1352]: time=2026-09-20T12:49:40.625Z level=INFO msg="Uploading 1 narinfos"5520lb # [ 132.109424] niks3-server[1060]: 2026/09/20 12:49:40 INFO Signed narinfos id=36 count=15521lb # [ 132.123249] niks3-server[1060]: 2026/09/20 12:49:40 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5522lb # [ 132.135745] niks3-server[1060]: 2026/09/20 12:49:40 INFO Completed upload id=365523worker2 # [ 132.134727] nix-grpc-daemon[1352]: time=2026-09-20T12:49:40.655Z level=INFO msg="Upload complete. (309ms)"5524worker2 # [ 132.145204] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52936 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15525lb # [ 132.161438] niks3-server[1060]: 2026/09/20 12:49:40 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5526worker2 # [ 132.160752] nix-grpc-daemon[1352]: time=2026-09-20T12:49:40.681Z level=WARN msg="Failed to register uploaded object" key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo error="server returned 404: object is not pending upload\n"5527worker2 # [ 132.169526] nix-daemon[904]: reaped child process 1375, status = succeeded5528client # [ 132.260660] systemd[1]: mid.service: Deactivated successfully.5529client # [ 132.265546] systemd[1]: mid.service: Consumed 184ms CPU time over 11.017s wall clock time, 37M memory peak, 8.8K incoming IP traffic, 9.2K outgoing IP traffic.5530client: (finished: waiting for success: ! systemctl is-active mid, in 1.14 seconds)5531client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5532client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)5533(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 11.93 seconds)5534subtest: interrupting the client stops the build on the worker5535client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5536client # [ 133.124872] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5537client # Running as unit: intr.service; invocation ID: 40340415e3f94094b1fad9be39976bf15538client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)5539worker1 # [ 133.319193] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:385765540worker1 # [ 133.334771] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5541client # [ 133.389955] nix[1241]: don't know how to build these paths:5542client # [ 133.392071] nix[1241]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5543worker1 # [ 133.371409] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:41Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05544worker1 # [ 133.378514] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:41Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15545worker1 # [ 133.425194] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765546worker1 # [ 133.443730] nix-grpc-daemon[1542]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)5547worker1 # [ 133.480421] nix-grpc-daemon[1542]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)5548worker1 # [ 133.643877] nix-grpc-daemon[1542]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 215 ms (attempt 3/5)5549worker1 # [ 133.859268] nix-grpc-daemon[1542]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 387 ms (attempt 4/5)5550worker1 # [ 134.254539] nix-grpc-daemon[1542]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5551worker1 # [ 134.276290] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385765552client # [ 134.328191] nix[1241]: copying 1 paths to 'lb:50051'...5553client # [ 134.328644] nix[1241]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5554worker1 # [ 134.294132] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385765555worker1 # [ 134.309113] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5556worker1 # [ 134.345567] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:42Z level=info event=niks3_push_started pid=16165557worker1 # [ 134.558081] nix-grpc-daemon[1616]: time=2026-09-20T12:49:43.072Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker1.key ca=""5558worker1 # [ 134.732716] nix-daemon[836]: accepted connection from pid 1629, user nix-grpc-daemon (trusted)5559worker1 # [ 134.750283] nix-daemon[836]: reaped child process 1632, status = succeeded5560worker1 # [ 134.935323] nix-daemon[836]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)5561worker1 # [ 134.952421] nix-daemon[836]: reaped child process 1637, status = succeeded5562lb # [ 134.933849] niks3-server[1060]: 2026/09/20 12:49:43 INFO Received uploads request method=POST path=/api/pending_closures5563worker1 # [ 134.983681] nix-grpc-daemon[1616]: time=2026-09-20T12:49:43.497Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5564worker1 # [ 134.985439] nix-grpc-daemon[1616]: time=2026-09-20T12:49:43.497Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5565lb # [ 134.978530] niks3-server[1060]: 2026/09/20 12:49:43 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5566worker1 # [ 135.024663] nix-grpc-daemon[1616]: time=2026-09-20T12:49:43.539Z level=INFO msg="Uploading 1 narinfos"5567lb # [ 134.992511] niks3-server[1060]: 2026/09/20 12:49:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5568lb # [ 134.999745] niks3-server[1060]: 2026/09/20 12:49:43 INFO Signed narinfos id=37 count=15569lb # [ 135.010570] niks3-server[1060]: 2026/09/20 12:49:43 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5570lb # [ 135.019856] niks3-server[1060]: 2026/09/20 12:49:43 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5571lb # [ 135.033865] niks3-server[1060]: 2026/09/20 12:49:43 INFO Completed upload id=375572worker1 # [ 135.066422] nix-grpc-daemon[1616]: time=2026-09-20T12:49:43.581Z level=INFO msg="Upload complete. (503ms)"5573lb # [ 135.044871] niks3-server[1060]: 2026/09/20 12:49:43 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5574worker1 # [ 135.075795] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=4245575worker1 # [ 135.092362] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765576worker1 # [ 135.103149] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5577worker1 # [ 135.105839] nix-daemon[836]: reaped child process 1613, status = succeeded5578worker1 # [ 135.118201] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5579client # [ 135.153880] nix[1241]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5580client: must succeed: systemctl kill -s INT intr5581client # [ 136.914183] systemd[1]: intr.service: Sent signal SIGINT to main process 1241 (nix) on client request.5582client # [ 136.920428] nix[1241]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5583client # [ 136.927992] nix[1241]: error: interrupted by the user5584client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5585client # [ 136.960432] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5586client # [ 136.965323] systemd[1]: intr.service: Failed with result 'exit-code'.5587client # [ 136.967474] systemd[1]: intr.service: Consumed 145ms CPU time over 3.838s wall clock time, 39.2M memory peak, 7.6K incoming IP traffic, 8.8K outgoing IP traffic.5588worker1 # [ 136.960915] nix-daemon[836]: reaped child process 1639, status = succeeded5589worker1 # [ 136.964464] nix-daemon[1641]: unexpected Nix daemon error: error: interrupted by the user5590worker1 # [ 136.973651] nix-daemon[836]: reaped child process 1641, status = failed with exit code 15591(finished: subtest: interrupting the client stops the build on the worker, in 4.01 seconds)5592subtest: a deploy mid-build drains: the build finishes, the new generation takes over5593client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5594client # [ 137.159880] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5595client # Running as unit: sw.service; invocation ID: 1bc40242215a4e128e18cc9b292486a05596client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)5597client # [ 137.401498] nix[1261]: don't know how to build these paths:5598client # [ 137.402438] nix[1261]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5599worker1 # [ 137.364733] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:385765600worker1 # [ 137.381009] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:45Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05601worker1 # [ 137.386743] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:45Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15602worker2 # [ 137.396704] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529365603worker2 # [ 137.415920] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:529365604client # [ 137.500165] nix[1261]: copying 1 paths to 'lb:50051'...5605worker2 # [ 137.432520] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:529365606client # [ 137.504483] nix[1261]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5607worker2 # [ 137.448487] nix-daemon[904]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5608worker2 # [ 137.660615] nix-daemon[904]: accepted connection from pid 1448, user nix-grpc-daemon (trusted)5609worker2 # [ 137.679814] nix-daemon[904]: reaped child process 1451, status = succeeded5610worker2 # [ 137.857702] nix-daemon[904]: accepted connection from pid 1453, user nix-grpc-daemon (trusted)5611worker2 # [ 137.875633] nix-daemon[904]: reaped child process 1456, status = succeeded5612lb # [ 137.889310] niks3-server[1060]: 2026/09/20 12:49:46 INFO Received uploads request method=POST path=/api/pending_closures5613worker2 # [ 137.906761] nix-grpc-daemon[1352]: time=2026-09-20T12:49:46.427Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5614worker2 # [ 137.908416] nix-grpc-daemon[1352]: time=2026-09-20T12:49:46.427Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5615worker2 # [ 137.942845] nix-grpc-daemon[1352]: time=2026-09-20T12:49:46.463Z level=INFO msg="Uploading 1 narinfos"5616lb # [ 137.934168] niks3-server[1060]: 2026/09/20 12:49:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5617lb # [ 137.948546] niks3-server[1060]: 2026/09/20 12:49:46 INFO Signed narinfos id=38 count=15618lb # [ 137.951044] niks3-server[1060]: 2026/09/20 12:49:46 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5619lb # [ 137.961604] niks3-server[1060]: 2026/09/20 12:49:46 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5620lb # [ 137.971658] niks3-server[1060]: 2026/09/20 12:49:46 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5621lb # [ 137.977721] niks3-server[1060]: 2026/09/20 12:49:46 INFO Completed upload id=385622worker2 # [ 137.976544] nix-grpc-daemon[1352]: time=2026-09-20T12:49:46.497Z level=INFO msg="Upload complete. (510ms)"5623lb # [ 137.981315] niks3-server[1060]: 2026/09/20 12:49:46 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5624worker2 # [ 137.987679] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52936 duration_s=0 paths=1 nar_bytes_in=4165625worker2 # [ 138.001774] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:529365626worker2 # [ 138.008754] nix-daemon[904]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5627worker2 # [ 138.013997] nix-daemon[904]: reaped child process 1446, status = succeeded5628client # [ 138.093399] nix[1261]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5629worker2 # [ 138.028075] nix-daemon[904]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5630worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25631worker2 # Checking switch inhibitors... done5632worker2 # activating the configuration...5633worker2 # [ 140.585765] nixos[1491]: switching to system configuration /nix/store/8kwgng1ccw6xc0rnwacmxvmmsdhcl1if-nixos-system-worker2-test5634worker2 # [ 140.587591] systemd[1]: Stopped target Local File Systems.5635worker2 # [ 140.592238] systemd[1]: Stopped target Remote File Systems.5636worker2 # setting up /etc...5637worker2 # [ 141.386097] systemd[1]: Reload requested from client PID 1491 ('.switch-to-conf') (unit backdoor.service)...5638worker2 # [ 141.387938] systemd[1]: Reloading...5639lb # [ 141.712259] envoy[922]: conn peer=[2001:db8:1::1]:32928 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3035 tx=1424 ms=107305640lb # [ 141.712653] envoy[922]: conn peer=[2001:db8:1::1]:34850 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2931 tx=755 ms=35955641worker2 # [ 142.292400] systemd[1]: Reloading finished in 899 ms.5642worker2 # restarting sysinit-reactivation.target5643worker2 # [ 142.369834] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5644worker2 # [ 142.521466] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5645worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5646worker2 # [ 142.529883] systemd[1]: Reached target Reactivate sysinit units.5647worker2 # [ 142.531118] dbus-broker-launch[563]: Caught SIGHUP, trigger reload.5648worker2 # [ 142.534471] systemd[1]: Reloading D-Bus System Message Bus...5649worker2 # [ 142.545131] systemd[1]: Reloading Nix worker-protocol over gRPC...5650worker2 # [ 142.562038] dbus-broker-launch[563]: Looking up NSS user entry for 'systemd-timesync'...5651worker2 # [ 142.564929] dbus-broker-launch[563]: NSS returned no entry for 'systemd-timesync'5652worker2 # [ 142.568149] dbus-broker-launch[563]: Invalid user-name in /nix/store/c8a4kacp5lnfwrcxrjmwczbl8ba32m4a-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5653worker2 # [ 142.594431] systemd[1]: Reloaded Nix worker-protocol over gRPC.5654worker2 # [ 142.616735] systemd[1]: Reloaded D-Bus System Message Bus.5655worker2 # [ 142.624077] systemd[1]: Reached target Local File Systems.5656worker2 # [ 142.624907] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5657worker2 # [ 142.630222] systemd[1]: Starting Create SUID/SGID Wrappers...5658worker2 # [ 142.636577] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5659worker2 # [ 142.644151] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675660worker2 # [ 142.653958] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5661worker2 # [ 142.657746] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5662worker2 # [ 142.659017] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5663worker2 # [ 142.664360] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5664worker2 # [ 142.665749] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5665worker2 # [ 142.669500] systemd[1]: Reached target Remote File Systems.5666worker2 # [ 143.145304] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5667worker2 # [ 143.153039] systemd[1]: Finished Create SUID/SGID Wrappers.5668worker2 # [ 143.155714] systemd[1]: Starting linger-users.service...5669worker2 # [ 143.211224] systemd[1]: linger-users.service: Deactivated successfully.5670worker2 # [ 143.216203] systemd[1]: Finished linger-users.service.5671worker2 # [ 143.325199] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:51Z level=info event=draining5672worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5673worker2 # [ 143.468502] nixos[1491]: finished switching to system configuration /nix/store/8kwgng1ccw6xc0rnwacmxvmmsdhcl1if-nixos-system-worker2-test5674worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.73 seconds)5675worker2: must succeed: pgrep -f 'read -t [9]0 x'5676worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.07 seconds)5677client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&25678client # don't know how to build these paths:5679client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5680worker1 # [ 143.835800] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:385765681worker1 # [ 143.844288] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385765682worker1 # [ 143.866461] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:52Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05683worker1 # [ 143.878165] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:52Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15684worker1 # [ 143.916613] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765685worker1 # [ 143.933599] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385765686client # copying 1 paths to 'lb:50051'...5687worker1 # [ 143.949932] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385765688client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5689worker1 # [ 143.974759] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5690worker1 # [ 144.182373] nix-daemon[836]: accepted connection from pid 1691, user nix-grpc-daemon (trusted)5691worker1 # [ 144.202791] nix-daemon[836]: reaped child process 1694, status = succeeded5692worker1 # [ 144.379841] nix-daemon[836]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)5693worker1 # [ 144.399491] nix-daemon[836]: reaped child process 1699, status = succeeded5694lb # [ 144.379773] niks3-server[1060]: 2026/09/20 12:49:52 INFO Received uploads request method=POST path=/api/pending_closures5695worker1 # [ 144.422818] nix-grpc-daemon[1616]: time=2026-09-20T12:49:52.937Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5696worker1 # [ 144.425091] nix-grpc-daemon[1616]: time=2026-09-20T12:49:52.939Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5697lb # [ 144.412250] niks3-server[1060]: 2026/09/20 12:49:52 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5698lb # [ 144.426866] niks3-server[1060]: 2026/09/20 12:49:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5699worker1 # [ 144.465707] nix-grpc-daemon[1616]: time=2026-09-20T12:49:52.980Z level=INFO msg="Uploading 1 narinfos"5700lb # [ 144.440165] niks3-server[1060]: 2026/09/20 12:49:52 INFO Signed narinfos id=39 count=15701lb # [ 144.446866] niks3-server[1060]: 2026/09/20 12:49:52 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5702lb # [ 144.456973] niks3-server[1060]: 2026/09/20 12:49:52 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5703lb # [ 144.462592] niks3-server[1060]: 2026/09/20 12:49:52 INFO Completed upload id=395704lb # [ 144.465853] niks3-server[1060]: 2026/09/20 12:49:52 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5705worker1 # [ 144.494894] nix-grpc-daemon[1616]: time=2026-09-20T12:49:53.010Z level=INFO msg="Upload complete. (502ms)"5706worker1 # [ 144.504441] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=5285707worker1 # [ 144.515366] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765708worker1 # [ 144.522721] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5709worker1 # [ 144.527751] nix-daemon[836]: reaped child process 1689, status = succeeded5710client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5711client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5712worker1 # [ 144.542015] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5713client # farm-a> LOG-farm-a-during-drain5714worker1 # [ 145.788736] nix-daemon[836]: reaped child process 1703, status = succeeded5715worker1 # [ 145.975208] nix-daemon[836]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)5716worker1 # [ 145.993379] nix-daemon[836]: reaped child process 1721, status = succeeded5717lb # [ 145.972890] niks3-server[1060]: 2026/09/20 12:49:54 INFO Received uploads request method=POST path=/api/pending_closures5718worker1 # [ 146.016428] nix-grpc-daemon[1616]: time=2026-09-20T12:49:54.530Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5719worker1 # [ 146.017930] nix-grpc-daemon[1616]: time=2026-09-20T12:49:54.530Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5720lb # [ 146.014667] niks3-server[1060]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5721lb # [ 146.026980] niks3-server[1060]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5722lb # [ 146.037990] niks3-server[1060]: 2026/09/20 12:49:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5723worker1 # [ 146.076182] nix-grpc-daemon[1616]: time=2026-09-20T12:49:54.590Z level=INFO msg="Uploading 1 narinfos"5724lb # [ 146.044312] niks3-server[1060]: 2026/09/20 12:49:54 INFO Signed narinfos id=40 count=15725lb # [ 146.049328] niks3-server[1060]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5726lb # [ 146.058280] niks3-server[1060]: 2026/09/20 12:49:54 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5727lb # [ 146.064664] niks3-server[1060]: 2026/09/20 12:49:54 INFO Completed upload id=405728worker1 # [ 146.096475] nix-grpc-daemon[1616]: time=2026-09-20T12:49:54.611Z level=INFO msg="Upload complete. (343ms)"5729lb # [ 146.067274] niks3-server[1060]: 2026/09/20 12:49:54 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5730worker1 # [ 146.110423] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=15731worker1 # [ 146.128679] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:54Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05732worker1 # [ 146.133137] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:54Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15733worker1 # [ 146.137070] nix-daemon[836]: reaped child process 1701, status = succeeded5734worker1 # [ 146.170953] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765735worker1 # [ 146.186824] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385765736worker1 # [ 146.201398] nix-grpc-daemon[1616]: time=2026-09-20T12:49:54.716Z level=INFO msg="All 1 paths already cached"5737client # copying 1 paths to 'lb:50051'...5738worker1 # [ 146.211813] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385765739client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5740worker1 # [ 146.242124] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5741worker1 # [ 146.445251] nix-daemon[836]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)5742worker1 # [ 146.464777] nix-daemon[836]: reaped child process 1730, status = succeeded5743worker1 # [ 146.638455] nix-daemon[836]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)5744worker1 # [ 146.657555] nix-daemon[836]: reaped child process 1735, status = succeeded5745lb # [ 146.636246] niks3-server[1060]: 2026/09/20 12:49:55 INFO Received uploads request method=POST path=/api/pending_closures5746worker1 # [ 146.678382] nix-grpc-daemon[1616]: time=2026-09-20T12:49:55.193Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5747worker1 # [ 146.679884] nix-grpc-daemon[1616]: time=2026-09-20T12:49:55.193Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5748lb # [ 146.665364] niks3-server[1060]: 2026/09/20 12:49:55 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5749worker1 # [ 146.711250] nix-grpc-daemon[1616]: time=2026-09-20T12:49:55.226Z level=INFO msg="Uploading 1 narinfos"5750lb # [ 146.681296] niks3-server[1060]: 2026/09/20 12:49:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5751lb # [ 146.693288] niks3-server[1060]: 2026/09/20 12:49:55 INFO Signed narinfos id=41 count=15752lb # [ 146.703201] niks3-server[1060]: 2026/09/20 12:49:55 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5753lb # [ 146.706833] niks3-server[1060]: 2026/09/20 12:49:55 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5754worker1 # [ 146.743239] nix-grpc-daemon[1616]: time=2026-09-20T12:49:55.258Z level=INFO msg="Upload complete. (483ms)"5755lb # [ 146.711885] niks3-server[1060]: 2026/09/20 12:49:55 INFO Completed upload id=415756lb # [ 146.715578] niks3-server[1060]: 2026/09/20 12:49:55 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5757worker1 # [ 146.753063] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=6405758worker1 # [ 146.763498] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765759worker1 # [ 146.770155] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5760worker1 # [ 146.774541] nix-daemon[836]: reaped child process 1725, status = succeeded5761client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5762client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5763worker1 # [ 146.788348] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5764client # farm-b> LOG-farm-b-during-drain5765worker1 # [ 148.229778] nix-daemon[836]: reaped child process 1739, status = succeeded5766worker1 # [ 148.411748] nix-daemon[836]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)5767worker1 # [ 148.429255] nix-daemon[836]: reaped child process 1757, status = succeeded5768lb # [ 148.413562] niks3-server[1060]: 2026/09/20 12:49:56 INFO Received uploads request method=POST path=/api/pending_closures5769worker1 # [ 148.456077] nix-grpc-daemon[1616]: time=2026-09-20T12:49:56.970Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5770worker1 # [ 148.457915] nix-grpc-daemon[1616]: time=2026-09-20T12:49:56.970Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5771lb # [ 148.456336] niks3-server[1060]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5772worker1 # [ 148.498040] nix-grpc-daemon[1616]: time=2026-09-20T12:49:57.012Z level=INFO msg="Uploading 1 narinfos"5773lb # [ 148.470199] niks3-server[1060]: 2026/09/20 12:49:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5774lb # [ 148.481304] niks3-server[1060]: 2026/09/20 12:49:56 INFO Signed narinfos id=42 count=15775lb # [ 148.487151] niks3-server[1060]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5776worker1 # [ 148.530973] nix-grpc-daemon[1616]: time=2026-09-20T12:49:57.046Z level=INFO msg="Upload complete. (332ms)"5777lb # [ 148.499511] niks3-server[1060]: 2026/09/20 12:49:56 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5778lb # [ 148.505460] niks3-server[1060]: 2026/09/20 12:49:56 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5779lb # [ 148.510265] niks3-server[1060]: 2026/09/20 12:49:57 INFO Completed upload id=425780lb # [ 148.512870] niks3-server[1060]: 2026/09/20 12:49:57 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5781worker1 # [ 148.546290] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=15782worker1 # [ 148.567979] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:57Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05783worker1 # [ 148.572776] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:57Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15784worker1 # [ 148.577067] nix-daemon[836]: reaped child process 1737, status = succeeded5785worker1 # [ 148.615188] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765786worker1 # [ 148.631928] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385765787worker1 # [ 148.647074] nix-grpc-daemon[1616]: time=2026-09-20T12:49:57.162Z level=INFO msg="All 2 paths already cached"5788client # copying 1 paths to 'lb:50051'...5789worker1 # [ 148.657709] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385765790client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5791worker1 # [ 148.681683] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5792worker1 # [ 148.888628] nix-daemon[836]: accepted connection from pid 1763, user nix-grpc-daemon (trusted)5793worker1 # [ 148.906997] nix-daemon[836]: reaped child process 1766, status = succeeded5794worker1 # [ 149.081967] nix-daemon[836]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)5795worker1 # [ 149.099574] nix-daemon[836]: reaped child process 1771, status = succeeded5796lb # [ 149.078549] niks3-server[1060]: 2026/09/20 12:49:57 INFO Received uploads request method=POST path=/api/pending_closures5797worker1 # [ 149.122195] nix-grpc-daemon[1616]: time=2026-09-20T12:49:57.636Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5798worker1 # [ 149.123690] nix-grpc-daemon[1616]: time=2026-09-20T12:49:57.637Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5799lb # [ 149.110699] niks3-server[1060]: 2026/09/20 12:49:57 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5800worker1 # [ 149.156472] nix-grpc-daemon[1616]: time=2026-09-20T12:49:57.671Z level=INFO msg="Uploading 1 narinfos"5801lb # [ 149.126630] niks3-server[1060]: 2026/09/20 12:49:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5802lb # [ 149.135041] niks3-server[1060]: 2026/09/20 12:49:57 INFO Signed narinfos id=43 count=15803lb # [ 149.138930] niks3-server[1060]: 2026/09/20 12:49:57 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5804lb # [ 149.145009] niks3-server[1060]: 2026/09/20 12:49:57 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5805worker1 # [ 149.183032] nix-grpc-daemon[1616]: time=2026-09-20T12:49:57.698Z level=INFO msg="Upload complete. (483ms)"5806lb # [ 149.151182] niks3-server[1060]: 2026/09/20 12:49:57 INFO Completed upload id=435807lb # [ 149.154872] niks3-server[1060]: 2026/09/20 12:49:57 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5808worker1 # [ 149.192789] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=7685809worker1 # [ 149.202232] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765810worker1 # [ 149.208717] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5811worker1 # [ 149.213482] nix-daemon[836]: reaped child process 1761, status = succeeded5812client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5813client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5814worker1 # [ 149.227173] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5815client # farm-top> LOG-farm-top-during-drain5816worker1 # [ 150.274525] nix-daemon[836]: reaped child process 1775, status = succeeded5817worker1 # [ 150.456374] nix-daemon[836]: accepted connection from pid 1790, user nix-grpc-daemon (trusted)5818worker1 # [ 150.473880] nix-daemon[836]: reaped child process 1793, status = succeeded5819lb # [ 150.455683] niks3-server[1060]: 2026/09/20 12:49:58 INFO Received uploads request method=POST path=/api/pending_closures5820worker1 # [ 150.499438] nix-grpc-daemon[1616]: time=2026-09-20T12:49:59.012Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5821worker1 # [ 150.501200] nix-grpc-daemon[1616]: time=2026-09-20T12:49:59.012Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5822lb # [ 150.491705] niks3-server[1060]: 2026/09/20 12:49:58 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5823lb # [ 150.504155] niks3-server[1060]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5824worker1 # [ 150.551127] nix-grpc-daemon[1616]: time=2026-09-20T12:49:59.065Z level=INFO msg="Uploading 1 narinfos"5825lb # [ 150.519128] niks3-server[1060]: 2026/09/20 12:49:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5826lb # [ 150.528693] niks3-server[1060]: 2026/09/20 12:49:59 INFO Signed narinfos id=44 count=15827lb # [ 150.531225] niks3-server[1060]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5828lb # [ 150.539226] niks3-server[1060]: 2026/09/20 12:49:59 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5829lb # [ 150.546986] niks3-server[1060]: 2026/09/20 12:49:59 INFO Completed upload id=445830worker1 # [ 150.578678] nix-grpc-daemon[1616]: time=2026-09-20T12:49:59.093Z level=INFO msg="Upload complete. (340ms)"5831lb # [ 150.549573] niks3-server[1060]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5832worker1 # [ 150.593173] nix-grpc-daemon[1542]: ts=2026-09-20T12:49:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=15833worker1 # [ 150.609372] nix-daemon[836]: reaped child process 1773, status = succeeded5834client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 7.06 seconds)5835client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath5836client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5837worker2: must succeed: pkill -f 'read -t [9]0 x'5838worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5839client: waiting for success: ! systemctl is-active sw5840worker2 # [ 151.093207] nix-daemon[904]: reaped child process 1460, status = succeeded5841worker2 # [ 151.271493] nix-daemon[904]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)5842worker2 # [ 151.289152] nix-daemon[904]: reaped child process 1634, status = succeeded5843lb # [ 151.305956] niks3-server[1060]: 2026/09/20 12:49:59 INFO Received uploads request method=POST path=/api/pending_closures5844worker2 # [ 151.315088] nix-grpc-daemon[1352]: time=2026-09-20T12:49:59.835Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5845worker2 # [ 151.316693] nix-grpc-daemon[1352]: time=2026-09-20T12:49:59.835Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5846lb # [ 151.342023] niks3-server[1060]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5847lb # [ 151.354349] niks3-server[1060]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5848lb # [ 151.363545] niks3-server[1060]: 2026/09/20 12:49:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5849worker2 # [ 151.367338] nix-grpc-daemon[1352]: time=2026-09-20T12:49:59.888Z level=INFO msg="Uploading 1 narinfos"5850lb # [ 151.372684] niks3-server[1060]: 2026/09/20 12:49:59 INFO Signed narinfos id=45 count=15851lb # [ 151.378670] niks3-server[1060]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5852lb # [ 151.383444] niks3-server[1060]: 2026/09/20 12:49:59 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5853lb # [ 151.389942] niks3-server[1060]: 2026/09/20 12:49:59 INFO Completed upload id=455854worker2 # [ 151.388122] nix-grpc-daemon[1352]: time=2026-09-20T12:49:59.909Z level=INFO msg="Upload complete. (333ms)"5855lb # [ 151.392241] niks3-server[1060]: 2026/09/20 12:49:59 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5856worker2 # [ 151.400959] nix-grpc-daemon[1238]: ts=2026-09-20T12:49:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52936 duration_s=13 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=15857worker2 # [ 151.418554] nix-daemon[904]: reaped child process 1458, status = succeeded5858client # [ 151.510383] systemd[1]: sw.service: Deactivated successfully.5859client # [ 151.515745] systemd[1]: sw.service: Consumed 160ms CPU time over 14.353s wall clock time, 37M memory peak, 8.4K incoming IP traffic, 9K outgoing IP traffic.5860lb # [ 151.715244] envoy[922]: conn peer=[2001:db8:1::1]:49550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9616 tx=3874 ms=67765861lb # [ 151.720498] envoy[922]: conn peer=[2001:db8:1::1]:34866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2939 tx=1352 ms=140855862client: (finished: waiting for success: ! systemctl is-active sw, in 1.13 seconds)5863client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5864client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)5865worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5866worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.07 seconds)5867lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5868lb: must succeed: curl -sf localhost:9901/clusters5869lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5870worker2 # [ 152.331562] nix-grpc-daemon[1238]: ts=2026-09-20T12:50:00Z level=info event=signal_exit5871lb: must succeed: curl -sf localhost:9901/clusters5872lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5873worker2 # [ 153.569317] nix-grpc-daemon[1238]: ts=2026-09-20T12:50:02Z level=info event=scheduler_restarting5874worker1 # [ 153.614518] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:465145875worker1 # [ 153.619351] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:02Z level=info event=worker_gone addr=192.168.1.4:500515876worker2 # [ 153.602735] nix-grpc-daemon[1238]: ts=2026-09-20T12:50:02Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5877worker2 # [ 153.630012] nix-daemon[904]: reaped child process 1294, status = succeeded5878worker2 # [ 153.771138] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5879worker2 # [ 153.776461] systemd[1]: nix-grpc-daemon.service: Consumed 1.177s CPU time over 46.658s wall clock time, 31.6M memory peak, 27.8K incoming IP traffic, 51.2K outgoing IP traffic.5880lb: must succeed: curl -sf localhost:9901/clusters5881lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5882worker2 # [ 154.424260] systemd[1]: Starting scheduler-token.service...5883worker2 # [ 154.527478] systemd[1]: scheduler-token.service: Deactivated successfully.5884worker2 # [ 154.531641] systemd[1]: Finished scheduler-token.service.5885worker2 # [ 154.536622] systemd[1]: scheduler-token.service: Consumed 52ms CPU time over 106ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5886worker2 # [ 154.543826] systemd[1]: Starting Nix worker-protocol over gRPC...5887worker2 # [ 154.729993] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:03Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5888worker2 # [ 154.732727] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:03Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5889worker2 # [ 154.736876] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:03Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15890worker2 # [ 154.744851] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:03Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5891worker2 # [ 154.746605] systemd[1]: Started Nix worker-protocol over gRPC.5892worker2 # [ 154.772826] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:03Z level=info event=scheduler_connected addr=lb:500515893worker1 # [ 154.810723] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:03Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:465145894worker1 # [ 154.816653] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:03Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05895lb: must succeed: curl -sf localhost:9901/clusters5896lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5897lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 3.33 seconds)5898lb: must succeed: curl -sf localhost:9901/clusters5899lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5900worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5901worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5902(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 18.61 seconds)5903subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5904client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15905worker2 # [ 155.891294] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:551445906worker2 # [ 155.906570] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)5907worker2 # [ 155.927239] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551445908worker2 # [ 155.944195] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 1/5)5909worker2 # [ 156.007834] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 194 ms (attempt 2/5)5910worker2 # [ 156.202414] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 3/5)5911worker2 # [ 156.227149] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 211 ms (attempt 4/5)5912worker2 # [ 156.446794] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5913worker1 # [ 156.534280] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05914worker1 # [ 156.534653] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15915worker1 # [ 156.534786] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05916worker1 # [ 156.534961] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05917worker1 # [ 156.577506] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765918worker1 # [ 156.593757] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385765919worker1 # [ 156.608697] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385765920worker1 # [ 156.623444] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5921worker1 # [ 156.825589] nix-daemon[836]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)5922worker1 # [ 156.845954] nix-daemon[836]: reaped child process 1805, status = succeeded5923worker1 # [ 157.023469] nix-daemon[836]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)5924worker1 # [ 157.040475] nix-daemon[836]: reaped child process 1810, status = succeeded5925lb # [ 157.026360] niks3-server[1060]: 2026/09/20 12:50:05 INFO Received uploads request method=POST path=/api/pending_closures5926worker1 # [ 157.067068] nix-grpc-daemon[1616]: time=2026-09-20T12:50:05.581Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5927worker1 # [ 157.068708] nix-grpc-daemon[1616]: time=2026-09-20T12:50:05.581Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5928lb # [ 157.057194] niks3-server[1060]: 2026/09/20 12:50:05 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5929worker1 # [ 157.102464] nix-grpc-daemon[1616]: time=2026-09-20T12:50:05.617Z level=INFO msg="Uploading 1 narinfos"5930lb # [ 157.069940] niks3-server[1060]: 2026/09/20 12:50:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5931lb # [ 157.076343] niks3-server[1060]: 2026/09/20 12:50:05 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5932lb # [ 157.085525] niks3-server[1060]: 2026/09/20 12:50:05 INFO Signed narinfos id=46 count=15933lb # [ 157.091068] niks3-server[1060]: 2026/09/20 12:50:05 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5934worker1 # [ 157.133592] nix-grpc-daemon[1616]: time=2026-09-20T12:50:05.648Z level=INFO msg="Upload complete. (493ms)"5935lb # [ 157.101926] niks3-server[1060]: 2026/09/20 12:50:05 INFO Completed upload id=465936lb # [ 157.106300] niks3-server[1060]: 2026/09/20 12:50:05 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5937worker1 # [ 157.140286] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=5125938worker1 # [ 157.155483] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765939worker1 # [ 157.163339] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5940worker1 # [ 157.169549] nix-daemon[836]: reaped child process 1800, status = succeeded5941worker1 # [ 157.183778] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5942worker1 # [ 158.836501] nix-daemon[836]: reaped child process 1814, status = succeeded5943worker1 # [ 159.070862] nix-daemon[836]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)5944worker1 # [ 159.090941] nix-daemon[836]: reaped child process 1832, status = succeeded5945lb # [ 159.074068] niks3-server[1060]: 2026/09/20 12:50:07 INFO Received uploads request method=POST path=/api/pending_closures5946worker1 # [ 159.120152] nix-grpc-daemon[1616]: time=2026-09-20T12:50:07.632Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5947worker1 # [ 159.121703] nix-grpc-daemon[1616]: time=2026-09-20T12:50:07.632Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"5948worker1 # [ 159.165295] nix-grpc-daemon[1616]: time=2026-09-20T12:50:07.680Z level=INFO msg="Uploading 1 narinfos"5949lb # [ 159.123739] niks3-server[1060]: 2026/09/20 12:50:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5950lb # [ 159.140266] niks3-server[1060]: 2026/09/20 12:50:07 INFO Signed narinfos id=47 count=15951lb # [ 159.148708] niks3-server[1060]: 2026/09/20 12:50:07 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst5952lb # [ 159.161409] niks3-server[1060]: 2026/09/20 12:50:07 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv5953lb # [ 159.170885] niks3-server[1060]: 2026/09/20 12:50:07 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls5954lb # [ 159.178372] niks3-server[1060]: 2026/09/20 12:50:07 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5955lb # [ 159.183175] niks3-server[1060]: 2026/09/20 12:50:07 INFO Completed upload id=475956lb # [ 159.186398] niks3-server[1060]: 2026/09/20 12:50:07 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo5957worker1 # [ 159.214854] nix-grpc-daemon[1616]: time=2026-09-20T12:50:07.729Z level=INFO msg="Upload complete. (418ms)"5958worker1 # [ 159.238180] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=15959worker1 # [ 159.261501] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:07Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=15960worker1 # [ 159.276958] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765961worker1 # [ 159.282911] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:07Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=05962worker1 # [ 159.292182] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385765963worker1 # [ 159.295353] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385765964worker1 # [ 159.300515] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5965worker1 # [ 159.302414] nix-daemon[836]: reaped child process 1812, status = succeeded5966worker1 # [ 159.506295] nix-daemon[836]: accepted connection from pid 1838, user nix-grpc-daemon (trusted)5967worker1 # [ 159.527168] nix-daemon[836]: reaped child process 1841, status = succeeded5968worker1 # [ 159.723863] nix-daemon[836]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)5969worker1 # [ 159.743630] nix-daemon[836]: reaped child process 1846, status = succeeded5970lb # [ 159.723139] niks3-server[1060]: 2026/09/20 12:50:08 INFO Received uploads request method=POST path=/api/pending_closures5971worker1 # [ 159.766821] nix-grpc-daemon[1616]: time=2026-09-20T12:50:08.281Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5972worker1 # [ 159.768518] nix-grpc-daemon[1616]: time=2026-09-20T12:50:08.281Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"5973lb # [ 159.765624] niks3-server[1060]: 2026/09/20 12:50:08 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst5974worker1 # [ 159.820914] nix-grpc-daemon[1616]: time=2026-09-20T12:50:08.335Z level=INFO msg="Uploading 1 narinfos"5975lb # [ 159.780572] niks3-server[1060]: 2026/09/20 12:50:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5976lb # [ 159.796947] niks3-server[1060]: 2026/09/20 12:50:08 INFO Signed narinfos id=48 count=15977lb # [ 159.804380] niks3-server[1060]: 2026/09/20 12:50:08 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls5978lb # [ 159.813789] niks3-server[1060]: 2026/09/20 12:50:08 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5979lb # [ 159.820853] niks3-server[1060]: 2026/09/20 12:50:08 INFO Completed upload id=485980worker1 # [ 159.851265] nix-grpc-daemon[1616]: time=2026-09-20T12:50:08.366Z level=INFO msg="Upload complete. (540ms)"5981lb # [ 159.823598] niks3-server[1060]: 2026/09/20 12:50:08 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo5982worker1 # [ 159.864858] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=5125983worker1 # [ 159.880891] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385765984worker1 # [ 159.896310] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5985worker1 # [ 159.898936] nix-daemon[836]: reaped child process 1836, status = succeeded5986worker1 # [ 159.913092] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)5987worker1 # [ 160.967551] nix-daemon[836]: reaped child process 1850, status = succeeded5988worker1 # [ 161.174475] nix-daemon[836]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)5989worker1 # [ 161.195577] nix-daemon[836]: reaped child process 1868, status = succeeded5990lb # [ 161.178658] niks3-server[1060]: 2026/09/20 12:50:09 INFO Received uploads request method=POST path=/api/pending_closures5991worker1 # [ 161.220511] nix-grpc-daemon[1616]: time=2026-09-20T12:50:09.734Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5992worker1 # [ 161.222056] nix-grpc-daemon[1616]: time=2026-09-20T12:50:09.734Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"5993lb # [ 161.227761] niks3-server[1060]: 2026/09/20 12:50:09 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst5994worker1 # [ 161.281883] nix-grpc-daemon[1616]: time=2026-09-20T12:50:09.796Z level=INFO msg="Uploading 1 narinfos"5995lb # [ 161.249866] niks3-server[1060]: 2026/09/20 12:50:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5996lb # [ 161.260335] niks3-server[1060]: 2026/09/20 12:50:09 INFO Signed narinfos id=49 count=15997lb # [ 161.267603] niks3-server[1060]: 2026/09/20 12:50:09 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv5998lb # [ 161.274902] niks3-server[1060]: 2026/09/20 12:50:09 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls5999lb # [ 161.285669] niks3-server[1060]: 2026/09/20 12:50:09 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6000worker1 # [ 161.317891] nix-grpc-daemon[1616]: time=2026-09-20T12:50:09.832Z level=INFO msg="Upload complete. (387ms)"6001lb # [ 161.288770] niks3-server[1060]: 2026/09/20 12:50:09 INFO Completed upload id=496002lb # [ 161.290550] niks3-server[1060]: 2026/09/20 12:50:09 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6003worker1 # [ 161.340880] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=16004worker1 # [ 161.361360] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:09Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16005worker1 # [ 161.372527] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766006worker1 # [ 161.378990] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:09Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06007worker1 # [ 161.384628] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385766008worker1 # [ 161.388553] nix-daemon[836]: reaped child process 1848, status = succeeded6009worker1 # [ 161.391590] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385766010worker1 # [ 161.403399] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6011worker1 # [ 161.628393] nix-daemon[836]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6012worker1 # [ 161.651810] nix-daemon[836]: reaped child process 1877, status = succeeded6013lb # [ 161.716234] envoy[922]: conn peer=[2001:db8:1::4]:57902 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=13762 tx=957 ms=462546014worker1 # [ 161.849583] nix-daemon[836]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)6015worker1 # [ 161.869167] nix-daemon[836]: reaped child process 1882, status = succeeded6016lb # [ 161.851335] niks3-server[1060]: 2026/09/20 12:50:10 INFO Received uploads request method=POST path=/api/pending_closures6017worker1 # [ 161.893091] nix-grpc-daemon[1616]: time=2026-09-20T12:50:10.407Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6018worker1 # [ 161.894639] nix-grpc-daemon[1616]: time=2026-09-20T12:50:10.407Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6019lb # [ 161.887237] niks3-server[1060]: 2026/09/20 12:50:10 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6020worker1 # [ 161.938877] nix-grpc-daemon[1616]: time=2026-09-20T12:50:10.453Z level=INFO msg="Uploading 1 narinfos"6021lb # [ 161.908279] niks3-server[1060]: 2026/09/20 12:50:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6022lb # [ 161.916906] niks3-server[1060]: 2026/09/20 12:50:10 INFO Signed narinfos id=50 count=16023lb # [ 161.927770] niks3-server[1060]: 2026/09/20 12:50:10 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6024lb # [ 161.940440] niks3-server[1060]: 2026/09/20 12:50:10 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6025lb # [ 161.952490] niks3-server[1060]: 2026/09/20 12:50:10 INFO Completed upload id=506026worker1 # [ 161.984698] nix-grpc-daemon[1616]: time=2026-09-20T12:50:10.499Z level=INFO msg="Upload complete. (569ms)"6027lb # [ 161.955426] niks3-server[1060]: 2026/09/20 12:50:10 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6028worker1 # [ 161.997274] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=5126029worker1 # [ 162.016692] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766030worker1 # [ 162.025195] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6031worker1 # [ 162.030970] nix-daemon[836]: reaped child process 1872, status = succeeded6032worker1 # [ 162.044102] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6033worker1 # [ 163.510402] nix-daemon[836]: reaped child process 1886, status = succeeded6034worker1 # [ 163.729522] nix-daemon[836]: accepted connection from pid 1901, user nix-grpc-daemon (trusted)6035worker1 # [ 163.750111] nix-daemon[836]: reaped child process 1904, status = succeeded6036lb # [ 163.731533] niks3-server[1060]: 2026/09/20 12:50:12 INFO Received uploads request method=POST path=/api/pending_closures6037worker1 # [ 163.776487] nix-grpc-daemon[1616]: time=2026-09-20T12:50:12.290Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6038worker1 # [ 163.777969] nix-grpc-daemon[1616]: time=2026-09-20T12:50:12.290Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6039lb # [ 163.775435] niks3-server[1060]: 2026/09/20 12:50:12 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6040lb # [ 163.795707] niks3-server[1060]: 2026/09/20 12:50:12 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6041lb # [ 163.815978] niks3-server[1060]: 2026/09/20 12:50:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6042worker1 # [ 163.854005] nix-grpc-daemon[1616]: time=2026-09-20T12:50:12.368Z level=INFO msg="Uploading 1 narinfos"6043lb # [ 163.826982] niks3-server[1060]: 2026/09/20 12:50:12 INFO Signed narinfos id=51 count=16044lb # [ 163.832453] niks3-server[1060]: 2026/09/20 12:50:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6045lb # [ 163.842601] niks3-server[1060]: 2026/09/20 12:50:12 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6046lb # [ 163.850393] niks3-server[1060]: 2026/09/20 12:50:12 INFO Completed upload id=516047worker1 # [ 163.881994] nix-grpc-daemon[1616]: time=2026-09-20T12:50:12.397Z level=INFO msg="Upload complete. (419ms)"6048lb # [ 163.853140] niks3-server[1060]: 2026/09/20 12:50:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6049worker1 # [ 163.907272] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=16050worker1 # [ 163.924550] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:12Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16051worker1 # [ 163.936386] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766052worker1 # [ 163.946231] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:12Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06053worker1 # [ 163.951752] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385766054worker1 # [ 163.955947] nix-daemon[836]: reaped child process 1884, status = succeeded6055worker1 # [ 163.962329] nix-grpc-daemon[1616]: time=2026-09-20T12:50:12.465Z level=INFO msg="All 1 paths already cached"6056worker1 # [ 163.972241] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385766057worker1 # [ 163.981148] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6058worker1 # [ 164.177825] nix-daemon[836]: accepted connection from pid 1910, user nix-grpc-daemon (trusted)6059worker1 # [ 164.199115] nix-daemon[836]: reaped child process 1913, status = succeeded6060worker1 # [ 164.392091] nix-daemon[836]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6061worker1 # [ 164.413074] nix-daemon[836]: reaped child process 1918, status = succeeded6062lb # [ 164.395052] niks3-server[1060]: 2026/09/20 12:50:12 INFO Received uploads request method=POST path=/api/pending_closures6063worker1 # [ 164.438062] nix-grpc-daemon[1616]: time=2026-09-20T12:50:12.952Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6064worker1 # [ 164.439566] nix-grpc-daemon[1616]: time=2026-09-20T12:50:12.952Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6065lb # [ 164.431443] niks3-server[1060]: 2026/09/20 12:50:12 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6066lb # [ 164.446137] niks3-server[1060]: 2026/09/20 12:50:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6067worker1 # [ 164.485927] nix-grpc-daemon[1616]: time=2026-09-20T12:50:13.000Z level=INFO msg="Uploading 1 narinfos"6068lb # [ 164.456939] niks3-server[1060]: 2026/09/20 12:50:12 INFO Signed narinfos id=52 count=16069lb # [ 164.463713] niks3-server[1060]: 2026/09/20 12:50:12 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6070lb # [ 164.477890] niks3-server[1060]: 2026/09/20 12:50:12 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6071lb # [ 164.484529] niks3-server[1060]: 2026/09/20 12:50:12 INFO Completed upload id=526072worker1 # [ 164.516233] nix-grpc-daemon[1616]: time=2026-09-20T12:50:13.031Z level=INFO msg="Upload complete. (529ms)"6073lb # [ 164.487205] niks3-server[1060]: 2026/09/20 12:50:12 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6074worker1 # [ 164.524326] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=6246075worker1 # [ 164.549397] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766076worker1 # [ 164.554342] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6077worker1 # [ 164.560817] nix-daemon[836]: reaped child process 1908, status = succeeded6078worker1 # [ 164.572707] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6079worker1 # [ 165.428782] nix-daemon[836]: reaped child process 1922, status = succeeded6080worker1 # [ 165.638021] nix-daemon[836]: accepted connection from pid 1937, user nix-grpc-daemon (trusted)6081worker1 # [ 165.658656] nix-daemon[836]: reaped child process 1940, status = succeeded6082lb # [ 165.645438] niks3-server[1060]: 2026/09/20 12:50:14 INFO Received uploads request method=POST path=/api/pending_closures6083worker1 # [ 165.690755] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.205Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6084worker1 # [ 165.696032] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.207Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6085lb # [ 165.691490] niks3-server[1060]: 2026/09/20 12:50:14 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6086lb # [ 165.710562] niks3-server[1060]: 2026/09/20 12:50:14 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6087lb # [ 165.725991] niks3-server[1060]: 2026/09/20 12:50:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6088worker1 # [ 165.763838] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.277Z level=INFO msg="Uploading 1 narinfos"6089lb # [ 165.734713] niks3-server[1060]: 2026/09/20 12:50:14 INFO Signed narinfos id=53 count=16090lb # [ 165.740240] niks3-server[1060]: 2026/09/20 12:50:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6091lb # [ 165.749320] niks3-server[1060]: 2026/09/20 12:50:14 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6092lb # [ 165.756367] niks3-server[1060]: 2026/09/20 12:50:14 INFO Completed upload id=536093worker1 # [ 165.787346] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.302Z level=INFO msg="Upload complete. (400ms)"6094lb # [ 165.758978] niks3-server[1060]: 2026/09/20 12:50:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6095worker1 # [ 165.812181] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=16096worker1 # [ 165.833256] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:14Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16097worker1 # [ 165.838097] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766098worker1 # [ 165.845192] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:14Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06099worker1 # [ 165.851677] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385766100worker1 # [ 165.858457] nix-daemon[836]: reaped child process 1920, status = succeeded6101worker1 # [ 165.860894] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.372Z level=INFO msg="All 1 paths already cached"6102worker1 # [ 165.868683] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385766103worker1 # [ 165.883917] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6104worker1 # [ 166.083488] nix-daemon[836]: accepted connection from pid 1946, user nix-grpc-daemon (trusted)6105worker1 # [ 166.103151] nix-daemon[836]: reaped child process 1949, status = succeeded6106worker1 # [ 166.293095] nix-daemon[836]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6107worker1 # [ 166.312545] nix-daemon[836]: reaped child process 1954, status = succeeded6108lb # [ 166.291916] niks3-server[1060]: 2026/09/20 12:50:14 INFO Received uploads request method=POST path=/api/pending_closures6109worker1 # [ 166.337204] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.851Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6110worker1 # [ 166.338736] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.852Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6111lb # [ 166.329795] niks3-server[1060]: 2026/09/20 12:50:14 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6112lb # [ 166.350211] niks3-server[1060]: 2026/09/20 12:50:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6113worker1 # [ 166.388445] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.903Z level=INFO msg="Uploading 1 narinfos"6114lb # [ 166.362930] niks3-server[1060]: 2026/09/20 12:50:14 INFO Signed narinfos id=54 count=16115lb # [ 166.369746] niks3-server[1060]: 2026/09/20 12:50:14 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6116lb # [ 166.381808] niks3-server[1060]: 2026/09/20 12:50:14 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6117lb # [ 166.386903] niks3-server[1060]: 2026/09/20 12:50:14 INFO Completed upload id=546118worker1 # [ 166.419343] nix-grpc-daemon[1616]: time=2026-09-20T12:50:14.934Z level=INFO msg="Upload complete. (529ms)"6119lb # [ 166.389664] niks3-server[1060]: 2026/09/20 12:50:14 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6120worker1 # [ 166.431352] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=6246121worker1 # [ 166.449387] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766122worker1 # [ 166.461465] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6123worker1 # [ 166.472339] nix-daemon[836]: reaped child process 1944, status = succeeded6124worker1 # [ 166.480938] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6125worker1 # [ 167.709708] nix-daemon[836]: reaped child process 1958, status = succeeded6126worker1 # [ 167.877608] nix-daemon[836]: accepted connection from pid 1973, user nix-grpc-daemon (trusted)6127worker1 # [ 167.894368] nix-daemon[836]: reaped child process 1976, status = succeeded6128lb # [ 167.874621] niks3-server[1060]: 2026/09/20 12:50:16 INFO Received uploads request method=POST path=/api/pending_closures6129worker1 # [ 167.918259] nix-grpc-daemon[1616]: time=2026-09-20T12:50:16.432Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6130worker1 # [ 167.919717] nix-grpc-daemon[1616]: time=2026-09-20T12:50:16.432Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6131lb # [ 167.913010] niks3-server[1060]: 2026/09/20 12:50:16 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6132worker1 # [ 167.958884] nix-grpc-daemon[1616]: time=2026-09-20T12:50:16.473Z level=INFO msg="Uploading 1 narinfos"6133lb # [ 167.928867] niks3-server[1060]: 2026/09/20 12:50:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6134lb # [ 167.936911] niks3-server[1060]: 2026/09/20 12:50:16 INFO Signed narinfos id=55 count=16135lb # [ 167.939185] niks3-server[1060]: 2026/09/20 12:50:16 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6136lb # [ 167.949899] niks3-server[1060]: 2026/09/20 12:50:16 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6137lb # [ 167.954504] niks3-server[1060]: 2026/09/20 12:50:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6138lb # [ 167.957383] niks3-server[1060]: 2026/09/20 12:50:16 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6139lb # [ 167.960878] niks3-server[1060]: 2026/09/20 12:50:16 INFO Completed upload id=556140worker1 # [ 167.993181] nix-grpc-daemon[1616]: time=2026-09-20T12:50:16.508Z level=INFO msg="Upload complete. (302ms)"6141worker1 # [ 168.009576] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=16142worker1 # [ 168.024851] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:16Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16143worker1 # [ 168.031429] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766144worker1 # [ 168.035830] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:16Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06145worker1 # [ 168.039777] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385766146worker1 # [ 168.043853] nix-daemon[836]: reaped child process 1956, status = succeeded6147worker1 # [ 168.046955] nix-grpc-daemon[1616]: time=2026-09-20T12:50:16.554Z level=INFO msg="All 1 paths already cached"6148worker1 # [ 168.059030] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385766149worker1 # [ 168.062942] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6150worker1 # [ 168.238523] nix-daemon[836]: accepted connection from pid 1982, user nix-grpc-daemon (trusted)6151worker1 # [ 168.255707] nix-daemon[836]: reaped child process 1985, status = succeeded6152worker1 # [ 168.428086] nix-daemon[836]: accepted connection from pid 1987, user nix-grpc-daemon (trusted)6153worker1 # [ 168.446014] nix-daemon[836]: reaped child process 1990, status = succeeded6154lb # [ 168.424896] niks3-server[1060]: 2026/09/20 12:50:16 INFO Received uploads request method=POST path=/api/pending_closures6155worker1 # [ 168.467336] nix-grpc-daemon[1616]: time=2026-09-20T12:50:16.982Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6156worker1 # [ 168.468939] nix-grpc-daemon[1616]: time=2026-09-20T12:50:16.982Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6157lb # [ 168.463606] niks3-server[1060]: 2026/09/20 12:50:16 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6158lb # [ 168.477098] niks3-server[1060]: 2026/09/20 12:50:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6159worker1 # [ 168.516025] nix-grpc-daemon[1616]: time=2026-09-20T12:50:17.030Z level=INFO msg="Uploading 1 narinfos"6160lb # [ 168.489439] niks3-server[1060]: 2026/09/20 12:50:16 INFO Signed narinfos id=56 count=16161lb # [ 168.494236] niks3-server[1060]: 2026/09/20 12:50:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6162worker1 # [ 168.533370] nix-grpc-daemon[1616]: time=2026-09-20T12:50:17.048Z level=INFO msg="Upload complete. (467ms)"6163lb # [ 168.502295] niks3-server[1060]: 2026/09/20 12:50:17 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6164lb # [ 168.507469] niks3-server[1060]: 2026/09/20 12:50:17 INFO Completed upload id=566165lb # [ 168.510609] niks3-server[1060]: 2026/09/20 12:50:17 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6166worker1 # [ 168.539610] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=6246167worker1 # [ 168.554661] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766168worker1 # [ 168.560334] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6169worker1 # [ 168.564945] nix-daemon[836]: reaped child process 1980, status = succeeded6170worker1 # [ 168.577604] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6171worker1 # [ 169.196201] nix-daemon[836]: reaped child process 1994, status = succeeded6172worker1 # [ 169.355610] nix-daemon[836]: accepted connection from pid 2009, user nix-grpc-daemon (trusted)6173worker1 # [ 169.371960] nix-daemon[836]: reaped child process 2012, status = succeeded6174lb # [ 169.355873] niks3-server[1060]: 2026/09/20 12:50:17 INFO Received uploads request method=POST path=/api/pending_closures6175worker1 # [ 169.397443] nix-grpc-daemon[1616]: time=2026-09-20T12:50:17.912Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6176worker1 # [ 169.400698] nix-grpc-daemon[1616]: time=2026-09-20T12:50:17.914Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6177lb # [ 169.394831] niks3-server[1060]: 2026/09/20 12:50:17 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6178lb # [ 169.406143] niks3-server[1060]: 2026/09/20 12:50:17 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6179worker1 # [ 169.450010] nix-grpc-daemon[1616]: time=2026-09-20T12:50:17.964Z level=INFO msg="Uploading 1 narinfos"6180lb # [ 169.419534] niks3-server[1060]: 2026/09/20 12:50:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6181lb # [ 169.428870] niks3-server[1060]: 2026/09/20 12:50:17 INFO Signed narinfos id=57 count=16182lb # [ 169.431574] niks3-server[1060]: 2026/09/20 12:50:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6183lb # [ 169.442231] niks3-server[1060]: 2026/09/20 12:50:17 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6184lb # [ 169.447361] niks3-server[1060]: 2026/09/20 12:50:17 INFO Completed upload id=576185worker1 # [ 169.479110] nix-grpc-daemon[1616]: time=2026-09-20T12:50:17.994Z level=INFO msg="Upload complete. (293ms)"6186lb # [ 169.450284] niks3-server[1060]: 2026/09/20 12:50:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6187worker1 # [ 169.492896] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=16188worker1 # [ 169.504358] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:18Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16189worker1 # [ 169.510123] nix-daemon[836]: reaped child process 1992, status = succeeded6190worker1 # [ 169.534003] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766191worker1 # [ 169.543788] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:18Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06192worker1 # [ 169.554595] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385766193worker1 # [ 169.568353] nix-grpc-daemon[1616]: time=2026-09-20T12:50:18.083Z level=INFO msg="All 2 paths already cached"6194worker1 # [ 169.577166] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385766195worker1 # [ 169.589560] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6196worker1 # [ 169.785240] nix-daemon[836]: accepted connection from pid 2018, user nix-grpc-daemon (trusted)6197worker1 # [ 169.804315] nix-daemon[836]: reaped child process 2021, status = succeeded6198worker1 # [ 169.978183] nix-daemon[836]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)6199worker1 # [ 169.995524] nix-daemon[836]: reaped child process 2026, status = succeeded6200lb # [ 169.978258] niks3-server[1060]: 2026/09/20 12:50:18 INFO Received uploads request method=POST path=/api/pending_closures6201worker1 # [ 170.019985] nix-grpc-daemon[1616]: time=2026-09-20T12:50:18.534Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6202worker1 # [ 170.021631] nix-grpc-daemon[1616]: time=2026-09-20T12:50:18.534Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6203lb # [ 170.005567] niks3-server[1060]: 2026/09/20 12:50:18 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6204worker1 # [ 170.051266] nix-grpc-daemon[1616]: time=2026-09-20T12:50:18.566Z level=INFO msg="Uploading 1 narinfos"6205lb # [ 170.019100] niks3-server[1060]: 2026/09/20 12:50:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6206lb # [ 170.028055] niks3-server[1060]: 2026/09/20 12:50:18 INFO Signed narinfos id=58 count=16207lb # [ 170.034031] niks3-server[1060]: 2026/09/20 12:50:18 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6208lb # [ 170.040268] niks3-server[1060]: 2026/09/20 12:50:18 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6209lb # [ 170.048604] niks3-server[1060]: 2026/09/20 12:50:18 INFO Completed upload id=586210worker1 # [ 170.080978] nix-grpc-daemon[1616]: time=2026-09-20T12:50:18.596Z level=INFO msg="Upload complete. (477ms)"6211lb # [ 170.050880] niks3-server[1060]: 2026/09/20 12:50:18 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6212worker1 # [ 170.088141] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=7526213worker1 # [ 170.101976] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766214worker1 # [ 170.107709] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6215worker1 # [ 170.112726] nix-daemon[836]: reaped child process 2016, status = succeeded6216worker1 # [ 170.125436] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6217worker1 # [ 171.545737] nix-daemon[836]: reaped child process 2030, status = succeeded6218worker1 # [ 171.708222] nix-daemon[836]: accepted connection from pid 2045, user nix-grpc-daemon (trusted)6219worker1 # [ 171.725399] nix-daemon[836]: reaped child process 2048, status = succeeded6220lb # [ 171.709934] niks3-server[1060]: 2026/09/20 12:50:20 INFO Received uploads request method=POST path=/api/pending_closures6221worker1 # [ 171.750745] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.264Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6222worker1 # [ 171.752307] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.264Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6223lb # [ 171.737108] niks3-server[1060]: 2026/09/20 12:50:20 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6224lb # [ 171.753758] niks3-server[1060]: 2026/09/20 12:50:20 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6225worker1 # [ 171.799556] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.314Z level=INFO msg="Uploading 1 narinfos"6226lb # [ 171.769920] niks3-server[1060]: 2026/09/20 12:50:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6227lb # [ 171.776733] niks3-server[1060]: 2026/09/20 12:50:20 INFO Signed narinfos id=59 count=16228lb # [ 171.778758] niks3-server[1060]: 2026/09/20 12:50:20 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6229lb # [ 171.785289] niks3-server[1060]: 2026/09/20 12:50:20 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6230lb # [ 171.789436] niks3-server[1060]: 2026/09/20 12:50:20 INFO Completed upload id=596231worker1 # [ 171.821551] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.336Z level=INFO msg="Upload complete. (286ms)"6232lb # [ 171.792031] niks3-server[1060]: 2026/09/20 12:50:20 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6233worker1 # [ 171.832793] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=16234worker1 # [ 171.848581] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:20Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16235worker1 # [ 171.854267] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766236worker1 # [ 171.859605] nix-daemon[836]: reaped child process 2028, status = succeeded6237worker1 # [ 171.863216] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385766238worker1 # [ 171.874359] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.389Z level=INFO msg="All 2 paths already cached"6239worker1 # [ 171.881644] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385766240worker1 # [ 171.892141] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6241worker1 # [ 172.084852] nix-daemon[836]: accepted connection from pid 2052, user nix-grpc-daemon (trusted)6242worker1 # [ 172.102191] nix-daemon[836]: reaped child process 2055, status = succeeded6243worker1 # [ 172.279702] nix-daemon[836]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)6244worker1 # [ 172.297123] nix-daemon[836]: reaped child process 2060, status = succeeded6245lb # [ 172.278838] niks3-server[1060]: 2026/09/20 12:50:20 INFO Received uploads request method=POST path=/api/pending_closures6246worker1 # [ 172.320098] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.835Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6247worker1 # [ 172.321617] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.835Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6248lb # [ 172.308841] niks3-server[1060]: 2026/09/20 12:50:20 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6249lb # [ 172.319858] niks3-server[1060]: 2026/09/20 12:50:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6250worker1 # [ 172.358657] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.873Z level=INFO msg="Uploading 1 narinfos"6251lb # [ 172.330819] niks3-server[1060]: 2026/09/20 12:50:20 INFO Signed narinfos id=60 count=16252lb # [ 172.335046] niks3-server[1060]: 2026/09/20 12:50:20 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6253lb # [ 172.346142] niks3-server[1060]: 2026/09/20 12:50:20 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6254lb # [ 172.352725] niks3-server[1060]: 2026/09/20 12:50:20 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6255lb # [ 172.358170] niks3-server[1060]: 2026/09/20 12:50:20 INFO Completed upload id=606256worker1 # [ 172.390616] nix-grpc-daemon[1616]: time=2026-09-20T12:50:20.905Z level=INFO msg="Upload complete. (485ms)"6257worker1 # [ 172.398880] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=7526258worker1 # [ 172.412410] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766259worker1 # [ 172.419255] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6260worker1 # [ 172.424808] nix-daemon[836]: reaped child process 2050, status = succeeded6261worker1 # [ 172.439170] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6262worker1 # [ 173.672808] nix-daemon[836]: reaped child process 2064, status = succeeded6263worker1 # [ 173.841216] nix-daemon[836]: accepted connection from pid 2079, user nix-grpc-daemon (trusted)6264worker1 # [ 173.860522] nix-daemon[836]: reaped child process 2082, status = succeeded6265lb # [ 173.841287] niks3-server[1060]: 2026/09/20 12:50:22 INFO Received uploads request method=POST path=/api/pending_closures6266worker1 # [ 173.884401] nix-grpc-daemon[1616]: time=2026-09-20T12:50:22.397Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6267worker1 # [ 173.885896] nix-grpc-daemon[1616]: time=2026-09-20T12:50:22.397Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6268lb # [ 173.879716] niks3-server[1060]: 2026/09/20 12:50:22 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6269lb # [ 173.892747] niks3-server[1060]: 2026/09/20 12:50:22 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6270lb # [ 173.901785] niks3-server[1060]: 2026/09/20 12:50:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6271worker1 # [ 173.940175] nix-grpc-daemon[1616]: time=2026-09-20T12:50:22.454Z level=INFO msg="Uploading 1 narinfos"6272lb # [ 173.907706] niks3-server[1060]: 2026/09/20 12:50:22 INFO Signed narinfos id=61 count=16273lb # [ 173.912567] niks3-server[1060]: 2026/09/20 12:50:22 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6274lb # [ 173.921662] niks3-server[1060]: 2026/09/20 12:50:22 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6275lb # [ 173.929997] niks3-server[1060]: 2026/09/20 12:50:22 INFO Completed upload id=616276worker1 # [ 173.961989] nix-grpc-daemon[1616]: time=2026-09-20T12:50:22.477Z level=INFO msg="Upload complete. (312ms)"6277lb # [ 173.932583] niks3-server[1060]: 2026/09/20 12:50:22 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6278worker1 # [ 173.977020] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=16279worker1 # [ 173.988311] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:22Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16280worker1 # [ 173.993232] nix-daemon[836]: reaped child process 2062, status = succeeded6281worker1 # [ 174.016968] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766282worker1 # [ 174.031570] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:385766283worker1 # [ 174.044984] nix-grpc-daemon[1616]: time=2026-09-20T12:50:22.560Z level=INFO msg="All 2 paths already cached"6284worker1 # [ 174.054043] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:385766285worker1 # [ 174.067418] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6286worker1 # [ 174.270102] nix-daemon[836]: accepted connection from pid 2086, user nix-grpc-daemon (trusted)6287worker1 # [ 174.288549] nix-daemon[836]: reaped child process 2089, status = succeeded6288worker1 # [ 174.466187] nix-daemon[836]: accepted connection from pid 2091, user nix-grpc-daemon (trusted)6289worker1 # [ 174.482879] nix-daemon[836]: reaped child process 2094, status = succeeded6290lb # [ 174.461963] niks3-server[1060]: 2026/09/20 12:50:22 INFO Received uploads request method=POST path=/api/pending_closures6291worker1 # [ 174.503303] nix-grpc-daemon[1616]: time=2026-09-20T12:50:23.018Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6292worker1 # [ 174.505211] nix-grpc-daemon[1616]: time=2026-09-20T12:50:23.018Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6293lb # [ 174.487406] niks3-server[1060]: 2026/09/20 12:50:22 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6294worker1 # [ 174.541559] nix-grpc-daemon[1616]: time=2026-09-20T12:50:23.056Z level=INFO msg="Uploading 1 narinfos"6295lb # [ 174.501290] niks3-server[1060]: 2026/09/20 12:50:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6296lb # [ 174.514935] niks3-server[1060]: 2026/09/20 12:50:23 INFO Signed narinfos id=62 count=16297lb # [ 174.522346] niks3-server[1060]: 2026/09/20 12:50:23 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6298lb # [ 174.533048] niks3-server[1060]: 2026/09/20 12:50:23 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6299worker1 # [ 174.570794] nix-grpc-daemon[1616]: time=2026-09-20T12:50:23.085Z level=INFO msg="Upload complete. (488ms)"6300lb # [ 174.539068] niks3-server[1060]: 2026/09/20 12:50:23 INFO Completed upload id=626301lb # [ 174.543635] niks3-server[1060]: 2026/09/20 12:50:23 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6302worker1 # [ 174.580335] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=0 paths=1 nar_bytes_in=7526303worker1 # [ 174.591960] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:385766304worker1 # [ 174.599849] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6305worker1 # [ 174.605297] nix-daemon[836]: reaped child process 2084, status = succeeded6306worker1 # [ 174.619620] nix-daemon[836]: accepted connection from pid 1542, user nix-grpc-daemon (trusted)6307worker1 # [ 176.262022] nix-daemon[836]: reaped child process 2098, status = succeeded6308worker1 # [ 176.445400] nix-daemon[836]: accepted connection from pid 2113, user nix-grpc-daemon (trusted)6309worker1 # [ 176.463456] nix-daemon[836]: reaped child process 2116, status = succeeded6310lb # [ 176.447738] niks3-server[1060]: 2026/09/20 12:50:24 INFO Received uploads request method=POST path=/api/pending_closures6311worker1 # [ 176.489416] nix-grpc-daemon[1616]: time=2026-09-20T12:50:25.004Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6312worker1 # [ 176.491913] nix-grpc-daemon[1616]: time=2026-09-20T12:50:25.006Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6313lb # [ 176.485580] niks3-server[1060]: 2026/09/20 12:50:24 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6314worker1 # [ 176.530602] nix-grpc-daemon[1616]: time=2026-09-20T12:50:25.045Z level=INFO msg="Uploading 1 narinfos"6315lb # [ 176.501746] niks3-server[1060]: 2026/09/20 12:50:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6316lb # [ 176.512314] niks3-server[1060]: 2026/09/20 12:50:25 INFO Signed narinfos id=63 count=16317lb # [ 176.516772] niks3-server[1060]: 2026/09/20 12:50:25 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6318worker1 # [ 176.555931] nix-grpc-daemon[1616]: time=2026-09-20T12:50:25.071Z level=INFO msg="Upload complete. (325ms)"6319lb # [ 176.524369] niks3-server[1060]: 2026/09/20 12:50:25 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6320lb # [ 176.528593] niks3-server[1060]: 2026/09/20 12:50:25 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6321lb # [ 176.532384] niks3-server[1060]: 2026/09/20 12:50:25 INFO Completed upload id=636322lb # [ 176.534290] niks3-server[1060]: 2026/09/20 12:50:25 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6323worker1 # [ 176.565256] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:38576 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=16324worker1 # [ 176.579799] nix-daemon[836]: reaped child process 2096, status = succeeded6325client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 21.02 seconds)6326worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6327worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6328client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16329worker1 # [ 177.014180] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:385766330worker2 # [ 176.994456] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551446331worker1 # [ 177.052993] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:25Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06332lb # [ 181.720280] envoy[922]: conn peer=[2001:db8:1::1]:58896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27230 tx=10830 ms=207006333client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 30.35 seconds)6334(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 51.45 seconds)6335subtest: low disk drains a worker and builds go to the other6336worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6337worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.15 seconds)6338worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6339worker1 # [ 208.889881] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:57Z level=info event=unhealthy reason=min_free6340worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.13 seconds)6341client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26342client # don't know how to build these paths:6343client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6344worker2 # [ 209.633779] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:551446345worker2 # [ 209.644126] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551446346worker1 # [ 209.710310] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:58Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06347worker1 # [ 209.710772] nix-grpc-daemon[1542]: ts=2026-09-20T12:50:58Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16348worker2 # [ 209.724562] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551446349worker2 # [ 209.741180] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551446350client # copying 1 paths to 'lb:50051'...6351client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6352worker2 # [ 209.757113] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551446353worker2 # [ 209.781460] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6354worker2 # [ 209.814930] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:58Z level=info event=niks3_push_started pid=17056355worker2 # [ 210.046507] nix-grpc-daemon[1705]: time=2026-09-20T12:50:58.566Z level=INFO msg="Configuring client TLS" cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.pem key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/worker2.key ca=""6356worker2 # [ 210.230134] nix-daemon[904]: accepted connection from pid 1718, user nix-grpc-daemon (trusted)6357worker2 # [ 210.248725] nix-daemon[904]: reaped child process 1721, status = succeeded6358worker2 # [ 210.419433] nix-daemon[904]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6359worker2 # [ 210.434261] nix-daemon[904]: reaped child process 1726, status = succeeded6360lb # [ 210.455805] niks3-server[1060]: 2026/09/20 12:50:58 INFO Received uploads request method=POST path=/api/pending_closures6361worker2 # [ 210.468074] nix-grpc-daemon[1705]: time=2026-09-20T12:50:58.988Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6362worker2 # [ 210.469553] nix-grpc-daemon[1705]: time=2026-09-20T12:50:58.988Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6363lb # [ 210.499621] niks3-server[1060]: 2026/09/20 12:50:59 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6364lb # [ 210.511625] niks3-server[1060]: 2026/09/20 12:50:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6365worker2 # [ 210.516129] nix-grpc-daemon[1705]: time=2026-09-20T12:50:59.036Z level=INFO msg="Uploading 1 narinfos"6366lb # [ 210.523129] niks3-server[1060]: 2026/09/20 12:50:59 INFO Signed narinfos id=64 count=16367lb # [ 210.529147] niks3-server[1060]: 2026/09/20 12:50:59 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6368lb # [ 210.536086] niks3-server[1060]: 2026/09/20 12:50:59 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6369worker2 # [ 210.542561] nix-grpc-daemon[1705]: time=2026-09-20T12:50:59.063Z level=INFO msg="Upload complete. (491ms)"6370lb # [ 210.544782] niks3-server[1060]: 2026/09/20 12:50:59 INFO Completed upload id=646371lb # [ 210.547715] niks3-server[1060]: 2026/09/20 12:50:59 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6372worker2 # [ 210.552434] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55144 duration_s=0 paths=1 nar_bytes_in=5126373worker2 # [ 210.568389] nix-grpc-daemon[1649]: ts=2026-09-20T12:50:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551446374worker2 # [ 210.574979] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6375worker2 # [ 210.580924] nix-daemon[904]: reaped child process 1703, status = succeeded6376client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6377worker2 # [ 210.592221] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6378client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6379client # farm-a> LOG-farm-a-drain6380lb # [ 211.725537] envoy[922]: conn peer=[2001:db8:1::1]:52762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300636381worker2 # [ 211.834910] nix-daemon[904]: reaped child process 1730, status = succeeded6382worker2 # [ 212.007563] nix-daemon[904]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6383worker2 # [ 212.023351] nix-daemon[904]: reaped child process 1748, status = succeeded6384lb # [ 212.040185] niks3-server[1060]: 2026/09/20 12:51:00 INFO Received uploads request method=POST path=/api/pending_closures6385worker2 # [ 212.052300] nix-grpc-daemon[1705]: time=2026-09-20T12:51:00.570Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6386worker2 # [ 212.053779] nix-grpc-daemon[1705]: time=2026-09-20T12:51:00.570Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6387lb # [ 212.078064] niks3-server[1060]: 2026/09/20 12:51:00 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6388lb # [ 212.089544] niks3-server[1060]: 2026/09/20 12:51:00 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6389lb # [ 212.099877] niks3-server[1060]: 2026/09/20 12:51:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6390worker2 # [ 212.105043] nix-grpc-daemon[1705]: time=2026-09-20T12:51:00.625Z level=INFO msg="Uploading 1 narinfos"6391lb # [ 212.105897] niks3-server[1060]: 2026/09/20 12:51:00 INFO Signed narinfos id=65 count=16392lb # [ 212.110482] niks3-server[1060]: 2026/09/20 12:51:00 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6393lb # [ 212.120282] niks3-server[1060]: 2026/09/20 12:51:00 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6394lb # [ 212.128793] niks3-server[1060]: 2026/09/20 12:51:00 INFO Completed upload id=656395worker2 # [ 212.127653] nix-grpc-daemon[1705]: time=2026-09-20T12:51:00.648Z level=INFO msg="Upload complete. (322ms)"6396lb # [ 212.131478] niks3-server[1060]: 2026/09/20 12:51:00 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6397worker2 # [ 212.140970] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55144 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=16398worker1 # [ 212.178601] nix-grpc-daemon[1542]: ts=2026-09-20T12:51:00Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06399worker1 # [ 212.184722] nix-grpc-daemon[1542]: ts=2026-09-20T12:51:00Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16400worker2 # [ 212.154758] nix-daemon[904]: reaped child process 1728, status = succeeded6401worker2 # [ 212.191793] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551446402worker2 # [ 212.209356] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551446403worker2 # [ 212.223978] nix-grpc-daemon[1705]: time=2026-09-20T12:51:00.745Z level=INFO msg="All 1 paths already cached"6404client # copying 1 paths to 'lb:50051'...6405client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6406worker2 # [ 212.234280] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551446407worker2 # [ 212.250943] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6408worker2 # [ 212.457992] nix-daemon[904]: accepted connection from pid 1752, user nix-grpc-daemon (trusted)6409worker2 # [ 212.476456] nix-daemon[904]: reaped child process 1755, status = succeeded6410worker2 # [ 212.650198] nix-daemon[904]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6411worker2 # [ 212.666988] nix-daemon[904]: reaped child process 1760, status = succeeded6412lb # [ 212.679332] niks3-server[1060]: 2026/09/20 12:51:01 INFO Received uploads request method=POST path=/api/pending_closures6413worker2 # [ 212.686513] nix-grpc-daemon[1705]: time=2026-09-20T12:51:01.207Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6414worker2 # [ 212.687981] nix-grpc-daemon[1705]: time=2026-09-20T12:51:01.207Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6415lb # [ 212.707286] niks3-server[1060]: 2026/09/20 12:51:01 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6416lb # [ 212.717943] niks3-server[1060]: 2026/09/20 12:51:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6417worker2 # [ 212.722767] nix-grpc-daemon[1705]: time=2026-09-20T12:51:01.243Z level=INFO msg="Uploading 1 narinfos"6418lb # [ 212.727991] niks3-server[1060]: 2026/09/20 12:51:01 INFO Signed narinfos id=66 count=16419lb # [ 212.733739] niks3-server[1060]: 2026/09/20 12:51:01 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6420lb # [ 212.740678] niks3-server[1060]: 2026/09/20 12:51:01 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6421lb # [ 212.746128] niks3-server[1060]: 2026/09/20 12:51:01 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6422lb # [ 212.750972] niks3-server[1060]: 2026/09/20 12:51:01 INFO Completed upload id=666423worker2 # [ 212.749736] nix-grpc-daemon[1705]: time=2026-09-20T12:51:01.271Z level=INFO msg="Upload complete. (481ms)"6424worker2 # [ 212.756565] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55144 duration_s=0 paths=1 nar_bytes_in=6326425worker2 # [ 212.767701] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551446426worker2 # [ 212.773998] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6427worker2 # [ 212.778347] nix-daemon[904]: reaped child process 1750, status = succeeded6428client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6429client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6430worker2 # [ 212.791051] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6431client # farm-b> LOG-farm-b-drain6432worker2 # [ 214.434005] nix-daemon[904]: reaped child process 1764, status = succeeded6433worker2 # [ 214.601383] nix-daemon[904]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6434worker2 # [ 214.617661] nix-daemon[904]: reaped child process 1782, status = succeeded6435lb # [ 214.631282] niks3-server[1060]: 2026/09/20 12:51:03 INFO Received uploads request method=POST path=/api/pending_closures6436worker2 # [ 214.639927] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.160Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6437worker2 # [ 214.641559] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.160Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6438lb # [ 214.662409] niks3-server[1060]: 2026/09/20 12:51:03 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6439lb # [ 214.678119] niks3-server[1060]: 2026/09/20 12:51:03 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6440lb # [ 214.684811] niks3-server[1060]: 2026/09/20 12:51:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6441worker2 # [ 214.688480] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.209Z level=INFO msg="Uploading 1 narinfos"6442lb # [ 214.695300] niks3-server[1060]: 2026/09/20 12:51:03 INFO Signed narinfos id=67 count=16443lb # [ 214.700267] niks3-server[1060]: 2026/09/20 12:51:03 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6444lb # [ 214.705922] niks3-server[1060]: 2026/09/20 12:51:03 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6445lb # [ 214.711809] niks3-server[1060]: 2026/09/20 12:51:03 INFO Completed upload id=676446worker2 # [ 214.710143] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.231Z level=INFO msg="Upload complete. (308ms)"6447lb # [ 214.714316] niks3-server[1060]: 2026/09/20 12:51:03 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6448worker2 # [ 214.720991] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55144 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=16449worker1 # [ 214.760881] nix-grpc-daemon[1542]: ts=2026-09-20T12:51:03Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06450worker2 # [ 214.736525] nix-daemon[904]: reaped child process 1762, status = succeeded6451worker1 # [ 214.766497] nix-grpc-daemon[1542]: ts=2026-09-20T12:51:03Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16452worker2 # [ 214.770889] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551446453worker2 # [ 214.786235] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:551446454worker2 # [ 214.799905] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.321Z level=INFO msg="All 2 paths already cached"6455client # copying 1 paths to 'lb:50051'...6456client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6457worker2 # [ 214.809999] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:551446458worker2 # [ 214.824468] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6459worker2 # [ 215.026088] nix-daemon[904]: accepted connection from pid 1786, user nix-grpc-daemon (trusted)6460worker2 # [ 215.042587] nix-daemon[904]: reaped child process 1789, status = succeeded6461worker2 # [ 215.213777] nix-daemon[904]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6462worker2 # [ 215.229750] nix-daemon[904]: reaped child process 1794, status = succeeded6463lb # [ 215.241790] niks3-server[1060]: 2026/09/20 12:51:03 INFO Received uploads request method=POST path=/api/pending_closures6464worker2 # [ 215.249634] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.770Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6465worker2 # [ 215.251119] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.770Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6466lb # [ 215.267138] niks3-server[1060]: 2026/09/20 12:51:03 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6467lb # [ 215.280677] niks3-server[1060]: 2026/09/20 12:51:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6468worker2 # [ 215.287237] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.808Z level=INFO msg="Uploading 1 narinfos"6469lb # [ 215.288669] niks3-server[1060]: 2026/09/20 12:51:03 INFO Signed narinfos id=68 count=16470lb # [ 215.296991] niks3-server[1060]: 2026/09/20 12:51:03 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6471lb # [ 215.304673] niks3-server[1060]: 2026/09/20 12:51:03 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6472worker2 # [ 215.309920] nix-grpc-daemon[1705]: time=2026-09-20T12:51:03.831Z level=INFO msg="Upload complete. (469ms)"6473lb # [ 215.313217] niks3-server[1060]: 2026/09/20 12:51:03 INFO Completed upload id=686474worker2 # [ 215.315821] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55144 duration_s=0 paths=1 nar_bytes_in=7606475lb # [ 215.317169] niks3-server[1060]: 2026/09/20 12:51:03 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6476worker2 # [ 215.327703] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:551446477worker2 # [ 215.333880] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6478worker2 # [ 215.338403] nix-daemon[904]: reaped child process 1784, status = succeeded6479client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6480worker2 # [ 215.350845] nix-daemon[904]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6481client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6482client # farm-top> LOG-farm-top-drain6483worker2 # [ 216.589491] nix-daemon[904]: reaped child process 1798, status = succeeded6484worker2 # [ 216.769425] nix-daemon[904]: accepted connection from pid 1813, user nix-grpc-daemon (trusted)6485worker2 # [ 216.788246] nix-daemon[904]: reaped child process 1816, status = succeeded6486lb # [ 216.802505] niks3-server[1060]: 2026/09/20 12:51:05 INFO Received uploads request method=POST path=/api/pending_closures6487worker2 # [ 216.812382] nix-grpc-daemon[1705]: time=2026-09-20T12:51:05.331Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6488worker2 # [ 216.814082] nix-grpc-daemon[1705]: time=2026-09-20T12:51:05.331Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6489lb # [ 216.833678] niks3-server[1060]: 2026/09/20 12:51:05 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6490lb # [ 216.847252] niks3-server[1060]: 2026/09/20 12:51:05 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6491worker2 # [ 216.858045] nix-grpc-daemon[1705]: time=2026-09-20T12:51:05.378Z level=INFO msg="Uploading 1 narinfos"6492lb # [ 216.860305] niks3-server[1060]: 2026/09/20 12:51:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6493lb # [ 216.863855] niks3-server[1060]: 2026/09/20 12:51:05 INFO Signed narinfos id=69 count=16494lb # [ 216.871979] niks3-server[1060]: 2026/09/20 12:51:05 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6495lb # [ 216.878477] niks3-server[1060]: 2026/09/20 12:51:05 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6496lb # [ 216.882751] niks3-server[1060]: 2026/09/20 12:51:05 INFO Completed upload id=696497worker2 # [ 216.881697] nix-grpc-daemon[1705]: time=2026-09-20T12:51:05.402Z level=INFO msg="Upload complete. (316ms)"6498lb # [ 216.885039] niks3-server[1060]: 2026/09/20 12:51:05 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6499worker2 # [ 216.891170] nix-grpc-daemon[1649]: ts=2026-09-20T12:51:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55144 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=16500worker2 # [ 216.907299] nix-daemon[904]: reaped child process 1796, status = succeeded6501client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ca.pem&client-cert=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.pem&client-key=/nix/store/1k2a3bdhnmd3p212243kr1xfai6n0fph-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 7.58 seconds)6502client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6503client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.23 seconds)6504worker1: must succeed: rm /nix/.rw-store/fill6505worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)6506worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6507worker1 # [ 217.894008] nix-grpc-daemon[1542]: ts=2026-09-20T12:51:06Z level=info event=healthy reason=min_free6508worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.12 seconds)6509(finished: subtest: low disk drains a worker and builds go to the other, in 11.47 seconds)6510(finished: run the VM test script, in 220.19 seconds)6511test script finished in 220.48s6512cleanup6513kill QemuMachine (pid 50)6514client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6515kill QemuMachine (pid 49)6516lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6517kill QemuMachine (pid 52)6518worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6519kill QemuMachine (pid 51)6520worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6521(finished: cleanup, in 1.14 seconds)6522additionally exposed symbols:6523 client, lb, worker1, worker2,6524 vlan1,6525 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh