vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #239
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14client # Disk image does not exist, creating the virtualisation disk image...15worker2 # Formatting '/build/vm-state-worker2/tmp.xJFR4IvDgb', fmt=raw size=107374182416worker1 # Disk image does not exist, creating the virtualisation disk image...17client: QEMU running (pid 50)18lb: QEMU running (pid 49)19worker2: QEMU running (pid 51)20worker1: QEMU running (pid 52)21lb # Formatting '/build/vm-state-lb/tmp.ZgD4FfBz3A', fmt=raw size=107374182422worker2 # mke2fs 1.47.4 (6-Mar-2025)23client # Formatting '/build/vm-state-client/tmp.9XPxGVClCp', fmt=raw size=107374182424worker1 # Formatting '/build/vm-state-worker1/tmp.qy8TJgBgY5', fmt=raw size=107374182425lb # mke2fs 1.47.4 (6-Mar-2025)26client # mke2fs 1.47.4 (6-Mar-2025)27worker1 # mke2fs 1.47.4 (6-Mar-2025)28lb # Discarding device blocks: 0/262144 done29client # Discarding device blocks: 0/262144 done30worker1 # Discarding device blocks: 0/262144 done31lb # Creating filesystem with 262144 4k blocks and 65536 inodes32client # Creating filesystem with 262144 4k blocks and 65536 inodes33worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes34worker1 # Filesystem UUID: 5ca053f9-faf1-4a46-ba9f-e1b2f97ae2ed35lb # Filesystem UUID: 4c362ea9-7add-44cf-9ffb-8a8a2fffb80636(finished: start all VMs, in 0.52 seconds)37client # Filesystem UUID: 0a51723e-468e-497e-80d9-f812fedad80f38worker2 # Discarding device blocks: 0/262144 done39worker1 # Superblock backups stored on blocks:40lb # Superblock backups stored on blocks:41worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes42worker1 # 32768, 98304, 163840, 22937643lb # 32768, 98304, 163840, 22937644worker2 # Filesystem UUID: 74781864-2c89-4177-a820-f0e630c6f5f045worker1 # 46lb # 47worker2 # Superblock backups stored on blocks:48worker1 # Allocating group tables: 0/8 done49lb # Allocating group tables: 0/8 done50worker2 # 32768, 98304, 163840, 22937651worker1 # Writing inode tables: 0/8 done52lb # Writing inode tables: 0/8 done53worker2 # 54worker1 # Creating journal (8192 blocks): done55lb # Creating journal (8192 blocks): done56worker2 # Allocating group tables: 0/8 done57worker1 # Writing superblocks and filesystem accounting information: 0/8 done58lb # Writing superblocks and filesystem accounting information: 0/8 done59worker2 # Writing inode tables: 0/8 done60worker1 # 61lb # 62worker2 # Creating journal (8192 blocks): done63worker1 # Virtualisation disk image created.64lb # Virtualisation disk image created.65worker2 # Writing superblocks and filesystem accounting information: 0/8 done66client: waiting for unit network-addresses-eth1.service67worker2 # 68client: waiting for the VM to finish booting69worker2 # Virtualisation disk image created.70client # Superblock backups stored on blocks:71client # 32768, 98304, 163840, 22937672client # 73client # Allocating group tables: 0/8 done74client # Writing inode tables: 0/8 done75client # Creating journal (8192 blocks): done76client # Writing superblocks and filesystem accounting information: 0/8 done77client # 78client # Virtualisation disk image created.79lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681lb # [ 0.000000] KASLR enabled82lb # [ 0.000000] random: crng init done83lb # [ 0.000000] Machine model: linux,dummy-virt84lb # [ 0.000000] efi: UEFI not found.85worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]86lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT87worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202688lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]89worker1 # [ 0.000000] KASLR enabled90worker1 # [ 0.000000] random: crng init done91lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]92lb # [ 0.000000] Zone ranges:93worker1 # [ 0.000000] Machine model: linux,dummy-virt94worker1 # [ 0.000000] efi: UEFI not found.95lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]96lb # [ 0.000000] DMA32 empty97worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT98lb # [ 0.000000] Normal empty99lb # [ 0.000000] Device empty100worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]101lb # [ 0.000000] Movable zone start for each node102lb # [ 0.000000] Early memory node ranges103worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]104worker1 # [ 0.000000] Zone ranges:105lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]106worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]107lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]108worker1 # [ 0.000000] DMA32 empty109worker1 # [ 0.000000] Normal empty110lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000111worker1 # [ 0.000000] Device empty112worker1 # [ 0.000000] Movable zone start for each node113lb # [ 0.000000] psci: probing for conduit method from DT.114worker1 # [ 0.000000] Early memory node ranges115lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.116worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]117lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs118lb # [ 0.000000] psci: Trusted OS migration not required119worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]120lb # [ 0.000000] psci: SMC Calling Convention v1.1121worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000122worker1 # [ 0.000000] psci: probing for conduit method from DT.123lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)124worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.125lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296126worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs127lb # [ 0.000000] Detected PIPT I-cache on CPU0128worker1 # [ 0.000000] psci: Trusted OS migration not required129worker1 # [ 0.000000] psci: SMC Calling Convention v1.1130lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)131lb # [ 0.000000] CPU features: detected: GICv3 CPU interface132worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)133lb # [ 0.000000] CPU features: detected: Spectre-v4134lb # [ 0.000000] CPU features: detected: Spectre-BHB135worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296136worker1 # [ 0.000000] Detected PIPT I-cache on CPU0137lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38138lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23139worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)140lb # [ 0.000000] alternatives: applying boot alternatives141worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface142worker1 # [ 0.000000] CPU features: detected: Spectre-v4143worker1 # [ 0.000000] CPU features: detected: Spectre-BHB144worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38145worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23146worker1 # [ 0.000000] alternatives: applying boot alternatives147lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ynn2zqc2v8ljbhyrx04lvra7yrxsl31x-nixos-system-lb-test/init regInfo=/nix/store/i5d7jzm2mrpgnjivlqdpqn88fi5a5927-closure-info/registration console=ttyAMA0,115200n8 console=tty0148lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/i5d7jzm2mrpgnjivlqdpqn88fi5a5927-closure-info/registration", will be passed to user space.149lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes150worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/x767bbbgc1gh7w7bmx80ivaysmx3252h-nixos-system-worker1-test/init regInfo=/nix/store/by8fdbb1qz7pg8dasbhia6d79n76s9cx-closure-info/registration console=ttyAMA0,115200n8 console=tty0151lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)152lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)153worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/by8fdbb1qz7pg8dasbhia6d79n76s9cx-closure-info/registration", will be passed to user space.154lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB155lb # [ 0.000000] software IO TLB: area num 1.156worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes157lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB158worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)159lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)160worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)161lb # [ 0.000000] Fallback order for Node 0: 0162worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB163lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216164worker1 # [ 0.000000] software IO TLB: area num 1.165lb # [ 0.000000] Policy zone: DMA166worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB167lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off168lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1169worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)170lb # [ 0.000000] allocated 3145728 bytes of page_ext171worker1 # [ 0.000000] Fallback order for Node 0: 0172lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages173worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216174lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups175worker1 # [ 0.000000] Policy zone: DMA176lb # [ 0.000000] rcu: Hierarchical RCU implementation.177worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off178lb # [ 0.000000] rcu: RCU event tracing is enabled.179worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1180lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.181worker1 # [ 0.000000] allocated 3145728 bytes of page_ext182lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.183worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages184lb # [ 0.000000] Rude variant of Tasks RCU enabled.185worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups186lb # [ 0.000000] Tracing variant of Tasks RCU enabled.187worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.188worker1 # [ 0.000000] rcu: RCU event tracing is enabled.189lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.190worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.191lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1192worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.193lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.194worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.195worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.196lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.197client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]198worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.199lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.200worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1201client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026202lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0203worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]204client # [ 0.000000] KASLR enabled205lb # [ 0.000000] GICv3: 256 SPIs implemented206client # [ 0.000000] random: crng init done207worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.208lb # [ 0.000000] GICv3: 0 Extended SPIs implemented209client # [ 0.000000] Machine model: linux,dummy-virt210client # [ 0.000000] efi: UEFI not found.211lb # [ 0.000000] Root IRQ handler: gic_handle_irq212worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026213worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.214worker2 # [ 0.000000] KASLR enabled215lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI216client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT217worker2 # [ 0.000000] random: crng init done218lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0219worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.220worker2 # [ 0.000000] Machine model: linux,dummy-virt221worker2 # [ 0.000000] efi: UEFI not found.222client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]223worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0224lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000225worker1 # [ 0.000000] GICv3: 256 SPIs implemented226lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]227client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]228worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT229client # [ 0.000000] Zone ranges:230worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented231worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]232worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq233client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]234lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)235client # [ 0.000000] DMA32 empty236worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI237worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]238client # [ 0.000000] Normal empty239worker2 # [ 0.000000] Zone ranges:240client # [ 0.000000] Device empty241worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0242lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)243client # [ 0.000000] Movable zone start for each node244worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]245worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000246lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000247client # [ 0.000000] Early memory node ranges248worker2 # [ 0.000000] DMA32 empty249worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]250worker2 # [ 0.000000] Normal empty251client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]252worker2 # [ 0.000000] Device empty253lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000254worker2 # [ 0.000000] Movable zone start for each node255worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)256client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]257lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.258worker2 # [ 0.000000] Early memory node ranges259client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000260worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]261worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)262client # [ 0.000000] psci: probing for conduit method from DT.263lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns264worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]265client # [ 0.000000] psci: PSCIv1.3 detected in firmware.266worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000267lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).268worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000269client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs270worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000271worker2 # [ 0.000000] psci: probing for conduit method from DT.272client # [ 0.000000] psci: Trusted OS migration not required273lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns274worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.275client # [ 0.000000] psci: SMC Calling Convention v1.1276worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.277worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs278lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns279client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)280worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns281lb # [ 0.000032] arm-pv: using stolen time PV282worker2 # [ 0.000000] psci: Trusted OS migration not required283worker2 # [ 0.000000] psci: SMC Calling Convention v1.1284client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296285worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).286client # [ 0.000000] Detected PIPT I-cache on CPU0287lb # [ 0.000555] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)288worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)289lb # [ 0.000752] Console: colour dummy device 80x25290worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns291client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)292lb # [ 0.000761] printk: legacy console [tty0] enabled293worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296294client # [ 0.000000] CPU features: detected: GICv3 CPU interface295worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns296worker2 # [ 0.000000] Detected PIPT I-cache on CPU0297client # [ 0.000000] CPU features: detected: Spectre-v4298worker1 # [ 0.000033] arm-pv: using stolen time PV299lb # [ 0.000976] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)300client # [ 0.000000] CPU features: detected: Spectre-BHB301worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)302lb # [ 0.000983] pid_max: default: 32768 minimum: 301303worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface304client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38305worker1 # [ 0.000541] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)306lb # [ 0.001057] LSM: initializing lsm=capability,landlock,yama,bpf,ima307worker1 # [ 0.000718] Console: colour dummy device 80x25308worker2 # [ 0.000000] CPU features: detected: Spectre-v4309client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23310lb # [ 0.001207] landlock: Up and running.311worker2 # [ 0.000000] CPU features: detected: Spectre-BHB312worker1 # [ 0.000726] printk: legacy console [tty0] enabled313lb # [ 0.001210] Yama: becoming mindful.314client # [ 0.000000] alternatives: applying boot alternatives315lb # [ 0.001769] LSM support for eBPF active316worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38317worker1 # [ 0.000926] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)318lb # [ 0.001911] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)319worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23320worker1 # [ 0.000933] pid_max: default: 32768 minimum: 301321worker2 # [ 0.000000] alternatives: applying boot alternatives322lb # [ 0.001944] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)323worker1 # [ 0.001003] LSM: initializing lsm=capability,landlock,yama,bpf,ima324worker1 # [ 0.001158] landlock: Up and running.325lb # [ 0.003058] cacheinfo: Unable to detect cache hierarchy for CPU 0326worker1 # [ 0.001160] Yama: becoming mindful.327lb # [ 0.003741] rcu: Hierarchical SRCU implementation.328worker1 # [ 0.001717] LSM support for eBPF active329client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test/init regInfo=/nix/store/0gwqnfilk84qzblipx1jjb3mpyj9v6w4-closure-info/registration console=ttyAMA0,115200n8 console=tty0330lb # [ 0.003745] rcu: Max phase no-delay instances is 1000.331worker1 # [ 0.001880] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)332lb # [ 0.005025] fsl-mc MSI: its@8080000 domain created333lb # [ 0.005126] EFI services will not be available.334worker1 # [ 0.001917] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)335client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0gwqnfilk84qzblipx1jjb3mpyj9v6w4-closure-info/registration", will be passed to user space.336lb # [ 0.005225] smp: Bringing up secondary CPUs ...337worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1bzxa2ywiwhcy35nv8fmxn1c5f1l5q67-nixos-system-worker2-test/init regInfo=/nix/store/466zwlxjh6l88mm7w6pkp2fws9pfxhpl-closure-info/registration console=ttyAMA0,115200n8 console=tty0338worker1 # [ 0.003055] cacheinfo: Unable to detect cache hierarchy for CPU 0339lb # [ 0.005234] smp: Brought up 1 node, 1 CPU340worker1 # [ 0.003820] rcu: Hierarchical SRCU implementation.341lb # [ 0.005238] SMP: Total of 1 processors activated.342client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes343lb # [ 0.005243] CPU: All CPU(s) started at EL1344worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/466zwlxjh6l88mm7w6pkp2fws9pfxhpl-closure-info/registration", will be passed to user space.345worker1 # [ 0.003824] rcu: Max phase no-delay instances is 1000.346worker1 # [ 0.005075] fsl-mc MSI: its@8080000 domain created347lb # [ 0.005253] CPU features: detected: Branch Target Identification348worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes349client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)350worker1 # [ 0.005172] EFI services will not be available.351lb # [ 0.005258] CPU features: detected: ARMv8.4 Translation Table Level352client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)353worker1 # [ 0.005247] smp: Bringing up secondary CPUs ...354worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)355worker1 # [ 0.005257] smp: Brought up 1 node, 1 CPU356lb # [ 0.005261] CPU features: detected: Instruction cache invalidation not required for I/D coherence357client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB358worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)359worker1 # [ 0.005260] SMP: Total of 1 processors activated.360client # [ 0.000000] software IO TLB: area num 1.361worker1 # [ 0.005266] CPU: All CPU(s) started at EL1362lb # [ 0.005265] CPU features: detected: Data cache clean to the PoU not required for I/D coherence363worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB364client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB365worker2 # [ 0.000000] software IO TLB: area num 1.366worker1 # [ 0.005278] CPU features: detected: Branch Target Identification367lb # [ 0.005268] CPU features: detected: Common not Private translations368worker1 # [ 0.005284] CPU features: detected: ARMv8.4 Translation Table Level369worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB370client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)371lb # [ 0.005271] CPU features: detected: CRC32 instructions372client # [ 0.000000] Fallback order for Node 0: 0373worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)374worker1 # [ 0.005287] CPU features: detected: Instruction cache invalidation not required for I/D coherence375lb # [ 0.005274] CPU features: detected: Data cache clean to Point of Deep Persistence376client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216377worker2 # [ 0.000000] Fallback order for Node 0: 0378client # [ 0.000000] Policy zone: DMA379lb # [ 0.005277] CPU features: detected: Data cache clean to Point of Persistence380worker1 # [ 0.005291] CPU features: detected: Data cache clean to the PoU not required for I/D coherence381worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216382client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off383worker2 # [ 0.000000] Policy zone: DMA384lb # [ 0.005280] CPU features: detected: Data independent timing control (DIT)385worker1 # [ 0.005295] CPU features: detected: Common not Private translations386lb # [ 0.005283] CPU features: detected: E0PD387client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1388worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off389worker1 # [ 0.005298] CPU features: detected: CRC32 instructions390client # [ 0.000000] allocated 3145728 bytes of page_ext391lb # [ 0.005286] CPU features: detected: Enhanced Counter Virtualization392worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1393worker1 # [ 0.005301] CPU features: detected: Data cache clean to Point of Deep Persistence394client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages395lb # [ 0.005289] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)396worker2 # [ 0.000000] allocated 3145728 bytes of page_ext397client # [ 0.000000] ftrace: allocated 294 pages with 4 groups398worker1 # [ 0.005304] CPU features: detected: Data cache clean to Point of Persistence399worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages400lb # [ 0.005292] CPU features: detected: Enhanced Virtualization Traps401client # [ 0.000000] rcu: Hierarchical RCU implementation.402worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups403worker1 # [ 0.005308] CPU features: detected: Data independent timing control (DIT)404lb # [ 0.005295] CPU features: detected: Fine Grained Traps405client # [ 0.000000] rcu: RCU event tracing is enabled.406worker1 # [ 0.005311] CPU features: detected: E0PD407worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.408client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.409lb # [ 0.005299] CPU features: detected: Generic authentication (architected QARMA5 algorithm)410worker2 # [ 0.000000] rcu: RCU event tracing is enabled.411worker1 # [ 0.005313] CPU features: detected: Enhanced Counter Virtualization412client # [ 0.000000] Trampoline variant of Tasks RCU enabled.413lb # [ 0.005303] CPU features: detected: RCpc load-acquire (LDAPR)414worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.415worker1 # [ 0.005316] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)416client # [ 0.000000] Rude variant of Tasks RCU enabled.417lb # [ 0.005306] CPU features: detected: LSE atomic instructions418worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.419client # [ 0.000000] Tracing variant of Tasks RCU enabled.420worker1 # [ 0.005320] CPU features: detected: Enhanced Virtualization Traps421lb # [ 0.005309] CPU features: detected: Privileged Access Never422worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.423worker1 # [ 0.005322] CPU features: detected: Fine Grained Traps424lb # [ 0.005312] CPU features: detected: PMUv3425client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.426worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.427lb # [ 0.005314] CPU features: detected: RAS Extension Support428client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1429worker1 # [ 0.005326] CPU features: detected: Generic authentication (architected QARMA5 algorithm)430worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.431lb # [ 0.005317] CPU features: detected: RASv1p1 Extension Support432worker1 # [ 0.005331] CPU features: detected: RCpc load-acquire (LDAPR)433worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1434client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.435lb # [ 0.005319] CPU features: detected: Random Number Generator436worker1 # [ 0.005334] CPU features: detected: LSE atomic instructions437lb # [ 0.005322] CPU features: detected: Speculation barrier (SB)438worker1 # [ 0.005337] CPU features: detected: Privileged Access Never439worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.440client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.441worker1 # [ 0.005340] CPU features: detected: PMUv3442lb # [ 0.005325] CPU features: detected: Stage-2 Force Write-Back443worker1 # [ 0.005342] CPU features: detected: RAS Extension Support444worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.445client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.446lb # [ 0.005328] CPU features: detected: TLB range maintenance instructions447worker1 # [ 0.005345] CPU features: detected: RASv1p1 Extension Support448client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0449lb # [ 0.005331] CPU features: detected: Speculative Store Bypassing Safe (SSBS)450worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.451client # [ 0.000000] GICv3: 256 SPIs implemented452worker1 # [ 0.005348] CPU features: detected: Random Number Generator453lb # [ 0.005372] alternatives: applying system-wide alternatives454client # [ 0.000000] GICv3: 0 Extended SPIs implemented455worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0456client # [ 0.000000] Root IRQ handler: gic_handle_irq457worker2 # [ 0.000000] GICv3: 256 SPIs implemented458lb # [ 0.008539] CPU features: detected: BBM Level 2 without TLB conflict abort459worker1 # [ 0.005350] CPU features: detected: Speculation barrier (SB)460worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented461client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI462worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq463client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0464worker1 # [ 0.005353] CPU features: detected: Stage-2 Force Write-Back465worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI466lb # [ 0.008791] Memory: 1406556K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)467worker1 # [ 0.005356] CPU features: detected: TLB range maintenance instructions468client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000469worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0470lb # [ 0.009230] devtmpfs: initialized471client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]472worker1 # [ 0.005361] CPU features: detected: Speculative Store Bypassing Safe (SSBS)473worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000474lb # [ 0.011034] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)475worker1 # [ 0.005401] alternatives: applying system-wide alternatives476worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]477client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)478worker1 # [ 0.008435] CPU features: detected: BBM Level 2 without TLB conflict abort479worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)480client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)481lb # [ 0.011056] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).482client # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000483lb # [ 0.011253] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL484worker1 # [ 0.008630] Memory: 1406548K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)485worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)486lb # [ 0.011257] 0 pages in range for non-PLT usage487worker1 # [ 0.009024] devtmpfs: initialized488client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000489worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000490lb # [ 0.011258] 508288 pages in range for PLT usage491client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.492worker1 # [ 0.010881] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)493lb # [ 0.011372] pinctrl core: initialized pinctrl subsystem494worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000495lb # [ 0.012135] DMI not present or invalid.496worker1 # [ 0.010909] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).497worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.498client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns499lb # [ 0.015755] NET: Registered PF_NETLINK/PF_ROUTE protocol family500worker1 # [ 0.011096] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL501client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).502worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns503lb # [ 0.018114] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations504worker1 # [ 0.011101] 0 pages in range for non-PLT usage505worker1 # [ 0.011102] 508288 pages in range for PLT usage506worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).507lb # [ 0.018314] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations508client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns509worker1 # [ 0.011213] pinctrl core: initialized pinctrl subsystem510worker1 # [ 0.011973] DMI not present or invalid.511lb # [ 0.018519] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations512client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns513worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns514worker1 # [ 0.015599] NET: Registered PF_NETLINK/PF_ROUTE protocol family515client # [ 0.000030] arm-pv: using stolen time PV516lb # [ 0.018543] audit: initializing netlink subsys (disabled)517worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns518lb # [ 0.019186] thermal_sys: Registered thermal governor 'fair_share'519worker1 # [ 0.017825] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations520worker2 # [ 0.000031] arm-pv: using stolen time PV521client # [ 0.000409] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)522lb # [ 0.019189] thermal_sys: Registered thermal governor 'bang_bang'523worker1 # [ 0.018002] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations524client # [ 0.000585] Console: colour dummy device 80x25525lb # [ 0.019193] thermal_sys: Registered thermal governor 'step_wise'526worker2 # [ 0.000440] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)527client # [ 0.000592] printk: legacy console [tty0] enabled528worker1 # [ 0.018204] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations529worker2 # [ 0.000610] Console: colour dummy device 80x25530lb # [ 0.019200] thermal_sys: Registered thermal governor 'user_space'531worker1 # [ 0.018227] audit: initializing netlink subsys (disabled)532worker2 # [ 0.000618] printk: legacy console [tty0] enabled533client # [ 0.000781] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)534lb # [ 0.019203] thermal_sys: Registered thermal governor 'power_allocator'535worker1 # [ 0.018802] thermal_sys: Registered thermal governor 'fair_share'536client # [ 0.000787] pid_max: default: 32768 minimum: 301537worker2 # [ 0.000812] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)538lb # [ 0.019234] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1539worker1 # [ 0.018804] thermal_sys: Registered thermal governor 'bang_bang'540client # [ 0.000872] LSM: initializing lsm=capability,landlock,yama,bpf,ima541lb # [ 0.019243] cpuidle: using governor ladder542worker2 # [ 0.000819] pid_max: default: 32768 minimum: 301543client # [ 0.001036] landlock: Up and running.544worker1 # [ 0.018807] thermal_sys: Registered thermal governor 'step_wise'545lb # [ 0.019250] cpuidle: using governor menu546client # [ 0.001039] Yama: becoming mindful.547worker2 # [ 0.000898] LSM: initializing lsm=capability,landlock,yama,bpf,ima548worker1 # [ 0.018812] thermal_sys: Registered thermal governor 'user_space'549client # [ 0.001510] LSM support for eBPF active550worker2 # [ 0.001033] landlock: Up and running.551lb # [ 0.019487] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.552worker2 # [ 0.001035] Yama: becoming mindful.553worker1 # [ 0.018815] thermal_sys: Registered thermal governor 'power_allocator'554lb # [ 0.019505] ASID allocator initialised with 65536 entries555client # [ 0.001661] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)556worker2 # [ 0.001541] LSM support for eBPF active557lb # [ 0.020761] Serial: AMBA PL011 UART driver558worker1 # [ 0.018841] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1559client # [ 0.001692] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)560worker2 # [ 0.001667] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)561worker1 # [ 0.018849] cpuidle: using governor ladder562lb # [ 0.026173] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1563client # [ 0.002834] cacheinfo: Unable to detect cache hierarchy for CPU 0564worker1 # [ 0.018855] cpuidle: using governor menu565lb # [ 0.026309] printk: console [ttyAMA0] enabled566worker2 # [ 0.001700] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)567client # [ 0.003552] rcu: Hierarchical SRCU implementation.568worker1 # [ 0.019046] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.569worker2 # [ 0.002776] cacheinfo: Unable to detect cache hierarchy for CPU 0570client # [ 0.003556] rcu: Max phase no-delay instances is 1000.571worker1 # [ 0.019061] ASID allocator initialised with 65536 entries572worker2 # [ 0.003524] rcu: Hierarchical SRCU implementation.573client # [ 0.004797] fsl-mc MSI: its@8080000 domain created574worker1 # [ 0.020316] Serial: AMBA PL011 UART driver575worker2 # [ 0.003528] rcu: Max phase no-delay instances is 1000.576client # [ 0.004891] EFI services will not be available.577worker2 # [ 0.004793] fsl-mc MSI: its@8080000 domain created578client # [ 0.004973] smp: Bringing up secondary CPUs ...579worker1 # [ 0.025668] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1580client # [ 0.004983] smp: Brought up 1 node, 1 CPU581worker2 # [ 0.004887] EFI services will not be available.582worker1 # [ 0.025834] printk: console [ttyAMA0] enabled583client # [ 0.004986] SMP: Total of 1 processors activated.584worker2 # [ 0.004965] smp: Bringing up secondary CPUs ...585client # [ 0.004994] CPU: All CPU(s) started at EL1586worker2 # [ 0.004974] smp: Brought up 1 node, 1 CPU587worker2 # [ 0.004977] SMP: Total of 1 processors activated.588client # [ 0.005006] CPU features: detected: Branch Target Identification589worker2 # [ 0.004982] CPU: All CPU(s) started at EL1590client # [ 0.005011] CPU features: detected: ARMv8.4 Translation Table Level591worker2 # [ 0.004993] CPU features: detected: Branch Target Identification592worker2 # [ 0.004998] CPU features: detected: ARMv8.4 Translation Table Level593client # [ 0.005014] CPU features: detected: Instruction cache invalidation not required for I/D coherence594client # [ 0.005018] CPU features: detected: Data cache clean to the PoU not required for I/D coherence595worker2 # [ 0.005001] CPU features: detected: Instruction cache invalidation not required for I/D coherence596client # [ 0.005021] CPU features: detected: Common not Private translations597worker2 # [ 0.005005] CPU features: detected: Data cache clean to the PoU not required for I/D coherence598client # [ 0.005025] CPU features: detected: CRC32 instructions599worker2 # [ 0.005008] CPU features: detected: Common not Private translations600lb # [ 0.147716] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages601client # [ 0.005027] CPU features: detected: Data cache clean to Point of Deep Persistence602worker1 # [ 0.144420] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages603worker2 # [ 0.005011] CPU features: detected: CRC32 instructions604lb # [ 0.147736] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page605worker1 # [ 0.144438] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page606client # [ 0.005031] CPU features: detected: Data cache clean to Point of Persistence607worker2 # [ 0.005014] CPU features: detected: Data cache clean to Point of Deep Persistence608lb # [ 0.147742] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages609worker1 # [ 0.144443] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages610client # [ 0.005035] CPU features: detected: Data independent timing control (DIT)611lb # [ 0.147747] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page612client # [ 0.005038] CPU features: detected: E0PD613worker1 # [ 0.144447] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page614lb # [ 0.147751] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages615worker2 # [ 0.005018] CPU features: detected: Data cache clean to Point of Persistence616client # [ 0.005040] CPU features: detected: Enhanced Counter Virtualization617worker1 # [ 0.144452] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages618lb # [ 0.147755] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page619worker2 # [ 0.005021] CPU features: detected: Data independent timing control (DIT)620client # [ 0.005043] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)621worker1 # [ 0.144456] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page622worker2 # [ 0.005024] CPU features: detected: E0PD623lb # [ 0.147759] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages624client # [ 0.005046] CPU features: detected: Enhanced Virtualization Traps625worker1 # [ 0.144461] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages626worker2 # [ 0.005026] CPU features: detected: Enhanced Counter Virtualization627lb # [ 0.147764] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page628client # [ 0.005049] CPU features: detected: Fine Grained Traps629worker1 # [ 0.144465] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page630worker2 # [ 0.005029] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)631client # [ 0.005053] CPU features: detected: Generic authentication (architected QARMA5 algorithm)632worker2 # [ 0.005033] CPU features: detected: Enhanced Virtualization Traps633client # [ 0.005058] CPU features: detected: RCpc load-acquire (LDAPR)634worker2 # [ 0.005036] CPU features: detected: Fine Grained Traps635lb # [ 0.154955] fbcon: Taking over console636client # [ 0.005061] CPU features: detected: LSE atomic instructions637lb # [ 0.154970] ACPI: Interpreter disabled.638worker1 # [ 0.151606] fbcon: Taking over console639worker2 # [ 0.005040] CPU features: detected: Generic authentication (architected QARMA5 algorithm)640client # [ 0.005064] CPU features: detected: Privileged Access Never641worker1 # [ 0.151622] ACPI: Interpreter disabled.642worker2 # [ 0.005045] CPU features: detected: RCpc load-acquire (LDAPR)643lb # [ 0.156858] iommu: Default domain type: Translated644client # [ 0.005067] CPU features: detected: PMUv3645worker2 # [ 0.005048] CPU features: detected: LSE atomic instructions646client # [ 0.005069] CPU features: detected: RAS Extension Support647lb # [ 0.156868] iommu: DMA domain TLB invalidation policy: strict mode648worker2 # [ 0.005051] CPU features: detected: Privileged Access Never649lb # [ 0.158498] SCSI subsystem initialized650client # [ 0.005072] CPU features: detected: RASv1p1 Extension Support651worker2 # [ 0.005054] CPU features: detected: PMUv3652client # [ 0.005075] CPU features: detected: Random Number Generator653worker2 # [ 0.005057] CPU features: detected: RAS Extension Support654client # [ 0.005077] CPU features: detected: Speculation barrier (SB)655worker2 # [ 0.005060] CPU features: detected: RASv1p1 Extension Support656worker2 # [ 0.005062] CPU features: detected: Random Number Generator657client # [ 0.005080] CPU features: detected: Stage-2 Force Write-Back658worker2 # [ 0.005065] CPU features: detected: Speculation barrier (SB)659client # [ 0.005083] CPU features: detected: TLB range maintenance instructions660worker2 # [ 0.005068] CPU features: detected: Stage-2 Force Write-Back661client # [ 0.005088] CPU features: detected: Speculative Store Bypassing Safe (SSBS)662worker2 # [ 0.005071] CPU features: detected: TLB range maintenance instructions663client # [ 0.005125] alternatives: applying system-wide alternatives664worker2 # [ 0.005076] CPU features: detected: Speculative Store Bypassing Safe (SSBS)665client # [ 0.008221] CPU features: detected: BBM Level 2 without TLB conflict abort666worker2 # [ 0.005113] alternatives: applying system-wide alternatives667lb # [ 0.163945] usbcore: registered new interface driver usbfs668worker1 # [ 0.160641] iommu: Default domain type: Translated669worker2 # [ 0.008186] CPU features: detected: BBM Level 2 without TLB conflict abort670lb # [ 0.163976] usbcore: registered new interface driver hub671client # [ 0.008441] Memory: 1406532K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)672worker1 # [ 0.160652] iommu: DMA domain TLB invalidation policy: strict mode673lb # [ 0.163994] usbcore: registered new device driver usb674client # [ 0.008809] devtmpfs: initialized675worker1 # [ 0.160995] SCSI subsystem initialized676lb # [ 0.164290] pps_core: LinuxPPS API ver. 1 registered677worker2 # [ 0.008392] Memory: 1406548K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)678client # [ 0.010641] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)679worker2 # [ 0.008793] devtmpfs: initialized680worker1 # [ 0.162881] usbcore: registered new interface driver usbfs681lb # [ 0.164296] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>682lb # [ 0.164306] PTP clock support registered683worker1 # [ 0.162912] usbcore: registered new interface driver hub684worker2 # [ 0.010656] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)685lb # [ 0.164358] EDAC MC: Ver: 3.0.0686worker1 # [ 0.162936] usbcore: registered new device driver usb687worker2 # [ 0.010683] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).688worker1 # [ 0.163196] pps_core: LinuxPPS API ver. 1 registered689lb # [ 0.168885] scmi_core: SCMI protocol bus registered690worker2 # [ 0.010877] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL691worker1 # [ 0.163202] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>692worker2 # [ 0.010881] 0 pages in range for non-PLT usage693lb # [ 0.169858] FPGA manager framework694worker1 # [ 0.163219] PTP clock support registered695lb # [ 0.170835] vgaarb: loaded696worker2 # [ 0.010883] 508288 pages in range for PLT usage697worker1 # [ 0.163268] EDAC MC: Ver: 3.0.0698worker2 # [ 0.010976] pinctrl core: initialized pinctrl subsystem699lb # [ 0.171481] clocksource: Switched to clocksource arch_sys_counter700worker2 # [ 0.011779] DMI not present or invalid.701worker1 # [ 0.167819] scmi_core: SCMI protocol bus registered702worker2 # [ 0.015097] NET: Registered PF_NETLINK/PF_ROUTE protocol family703worker1 # [ 0.168836] FPGA manager framework704worker2 # [ 0.017422] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations705worker1 # [ 0.169792] vgaarb: loaded706worker2 # [ 0.017616] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations707worker1 # [ 0.170440] clocksource: Switched to clocksource arch_sys_counter708worker2 # [ 0.017820] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations709lb # [ 0.175373] VFS: Disk quotas dquot_6.6.0710client # [ 0.010663] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).711worker2 # [ 0.017843] audit: initializing netlink subsys (disabled)712lb # [ 0.175411] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)713client # [ 0.010851] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL714worker2 # [ 0.018481] thermal_sys: Registered thermal governor 'fair_share'715client # [ 0.010856] 0 pages in range for non-PLT usage716worker2 # [ 0.018483] thermal_sys: Registered thermal governor 'bang_bang'717client # [ 0.010857] 508288 pages in range for PLT usage718worker2 # [ 0.018487] thermal_sys: Registered thermal governor 'step_wise'719client # [ 0.010971] pinctrl core: initialized pinctrl subsystem720client # [ 0.011751] DMI not present or invalid.721worker2 # [ 0.018492] thermal_sys: Registered thermal governor 'user_space'722lb # [ 0.179124] netfs: FS-Cache loaded723client # [ 0.014872] NET: Registered PF_NETLINK/PF_ROUTE protocol family724worker2 # [ 0.018494] thermal_sys: Registered thermal governor 'power_allocator'725lb # [ 0.179265] pnp: PnP ACPI: disabled726client # [ 0.017163] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations727worker2 # [ 0.018519] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1728worker1 # [ 0.176589] VFS: Disk quotas dquot_6.6.0729worker2 # [ 0.018528] cpuidle: using governor ladder730client # [ 0.017384] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations731worker1 # [ 0.176640] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)732worker2 # [ 0.018534] cpuidle: using governor menu733worker1 # [ 0.178254] netfs: FS-Cache loaded734client # [ 0.017592] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations735worker2 # [ 0.018729] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.736worker1 # [ 0.178388] pnp: PnP ACPI: disabled737client # [ 0.017613] audit: initializing netlink subsys (disabled)738worker2 # [ 0.018746] ASID allocator initialised with 65536 entries739worker2 # [ 0.020015] Serial: AMBA PL011 UART driver740client # [ 0.018205] thermal_sys: Registered thermal governor 'fair_share'741lb # [ 0.183120] NET: Registered PF_INET protocol family742client # [ 0.018207] thermal_sys: Registered thermal governor 'bang_bang'743worker2 # [ 0.025219] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1744lb # [ 0.183390] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)745worker2 # [ 0.025350] printk: console [ttyAMA0] enabled746client # [ 0.018211] thermal_sys: Registered thermal governor 'step_wise'747client # [ 0.018217] thermal_sys: Registered thermal governor 'user_space'748worker1 # [ 0.182217] NET: Registered PF_INET protocol family749client # [ 0.018219] thermal_sys: Registered thermal governor 'power_allocator'750client # [ 0.018257] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1751client # [ 0.018265] cpuidle: using governor ladder752client # [ 0.018271] cpuidle: using governor menu753client # [ 0.018483] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.754client # [ 0.018498] ASID allocator initialised with 65536 entries755client # [ 0.019684] Serial: AMBA PL011 UART driver756worker1 # [ 0.183137] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)757client # [ 0.024855] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1758client # [ 0.024980] printk: console [ttyAMA0] enabled759client # [ 0.146201] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages760client # [ 0.146223] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page761worker2 # [ 0.145775] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages762client # [ 0.146228] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages763worker2 # [ 0.145795] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page764client # [ 0.146233] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page765worker2 # [ 0.145801] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages766client # [ 0.149408] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages767worker2 # [ 0.145805] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page768client # [ 0.149418] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page769worker2 # [ 0.145810] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages770client # [ 0.149422] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages771worker2 # [ 0.145814] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page772client # [ 0.149426] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page773worker2 # [ 0.145819] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages774worker2 # [ 0.145823] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page775client # [ 0.153882] fbcon: Taking over console776client # [ 0.153899] ACPI: Interpreter disabled.777worker2 # [ 0.153442] fbcon: Taking over console778worker2 # [ 0.153459] ACPI: Interpreter disabled.779worker2 # [ 0.155356] iommu: Default domain type: Translated780worker2 # [ 0.155366] iommu: DMA domain TLB invalidation policy: strict mode781worker2 # [ 0.157036] SCSI subsystem initialized782client # [ 0.162760] iommu: Default domain type: Translated783client # [ 0.162771] iommu: DMA domain TLB invalidation policy: strict mode784worker2 # [ 0.162278] usbcore: registered new interface driver usbfs785client # [ 0.164438] SCSI subsystem initialized786worker2 # [ 0.162311] usbcore: registered new interface driver hub787worker2 # [ 0.162337] usbcore: registered new device driver usb788client # [ 0.165131] usbcore: registered new interface driver usbfs789worker2 # [ 0.162618] pps_core: LinuxPPS API ver. 1 registered790client # [ 0.165172] usbcore: registered new interface driver hub791client # [ 0.165190] usbcore: registered new device driver usb792worker2 # [ 0.162623] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>793worker2 # [ 0.162634] PTP clock support registered794client # [ 0.165471] pps_core: LinuxPPS API ver. 1 registered795worker2 # [ 0.162686] EDAC MC: Ver: 3.0.0796client # [ 0.165477] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>797worker2 # [ 0.167385] scmi_core: SCMI protocol bus registered798client # [ 0.165487] PTP clock support registered799client # [ 0.165530] EDAC MC: Ver: 3.0.0800worker2 # [ 0.168369] FPGA manager framework801worker2 # [ 0.169307] vgaarb: loaded802client # [ 0.170251] scmi_core: SCMI protocol bus registered803worker2 # [ 0.169944] clocksource: Switched to clocksource arch_sys_counter804client # [ 0.171248] FPGA manager framework805client # [ 0.172258] vgaarb: loaded806lb # [ 0.213968] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)807client # [ 0.172879] clocksource: Switched to clocksource arch_sys_counter808lb # [ 0.214021] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)809client # [ 0.173502] VFS: Disk quotas dquot_6.6.0810lb # [ 0.214051] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)811client # [ 0.173528] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)812worker2 # [ 0.174223] VFS: Disk quotas dquot_6.6.0813lb # [ 0.214130] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)814worker2 # [ 0.174259] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)815lb # [ 0.214274] TCP: Hash tables configured (established 16384 bind 16384)816lb # [ 0.214355] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)817client # [ 0.177971] netfs: FS-Cache loaded818client # [ 0.178075] pnp: PnP ACPI: disabled819lb # [ 0.214391] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)820worker2 # [ 0.177937] netfs: FS-Cache loaded821lb # [ 0.214427] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)822worker2 # [ 0.178504] pnp: PnP ACPI: disabled823lb # [ 0.214515] NET: Registered PF_UNIX/PF_LOCAL protocol family824worker1 # [ 0.213523] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)825lb # [ 0.214563] NET: Registered PF_XDP protocol family826worker1 # [ 0.213576] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)827lb # [ 0.214582] PCI: CLS 0 bytes, default 64828lb # [ 0.214820] Trying to unpack rootfs image as initramfs...829worker1 # [ 0.213607] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)830worker1 # [ 0.213691] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)831worker2 # [ 0.181917] NET: Registered PF_INET protocol family832worker1 # [ 0.213833] TCP: Hash tables configured (established 16384 bind 16384)833worker1 # [ 0.213919] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)834lb # [ 0.229222] kvm [1]: HYP mode not available835client # [ 0.181938] NET: Registered PF_INET protocol family836worker1 # [ 0.213956] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)837client # [ 0.182191] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)838worker1 # [ 0.213992] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)839worker1 # [ 0.214090] NET: Registered PF_UNIX/PF_LOCAL protocol family840worker1 # [ 0.214113] NET: Registered PF_XDP protocol family841worker2 # [ 0.182808] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)842worker1 # [ 0.214132] PCI: CLS 0 bytes, default 64843worker1 # [ 0.214388] Trying to unpack rootfs image as initramfs...844worker1 # [ 0.232140] kvm [1]: HYP mode not available845client # [ 0.212493] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)846client # [ 0.212548] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)847client # [ 0.212579] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)848client # [ 0.212663] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849client # [ 0.212808] TCP: Hash tables configured (established 16384 bind 16384)850worker2 # [ 0.213192] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)851worker2 # [ 0.213251] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)852client # [ 0.217900] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)853worker2 # [ 0.213283] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)854client # [ 0.217944] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)855worker2 # [ 0.213365] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)856client # [ 0.217983] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)857worker2 # [ 0.213507] TCP: Hash tables configured (established 16384 bind 16384)858client # [ 0.218097] NET: Registered PF_UNIX/PF_LOCAL protocol family859worker2 # [ 0.213599] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)860client # [ 0.218128] NET: Registered PF_XDP protocol family861client # [ 0.218147] PCI: CLS 0 bytes, default 64862worker2 # [ 0.213636] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)863client # [ 0.218385] Trying to unpack rootfs image as initramfs...864worker2 # [ 0.213672] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)865worker2 # [ 0.213792] NET: Registered PF_UNIX/PF_LOCAL protocol family866worker2 # [ 0.213822] NET: Registered PF_XDP protocol family867worker2 # [ 0.213841] PCI: CLS 0 bytes, default 64868worker2 # [ 0.214025] Trying to unpack rootfs image as initramfs...869client # [ 0.230148] kvm [1]: HYP mode not available870worker2 # [ 0.231831] kvm [1]: HYP mode not available871worker1 # [ 0.332200] Initialise system trusted keyrings872worker1 # [ 0.332974] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker1 # [ 0.334210] squashfs: version 4.0 (2009/01/31) Phillip Lougher874lb # [ 0.338558] Initialise system trusted keyrings875lb # [ 0.339318] workingset: timestamp_bits=42 max_order=19 bucket_order=0876lb # [ 0.340674] squashfs: version 4.0 (2009/01/31) Phillip Lougher877worker1 # [ 0.335039] 9p: Installing v9fs 9p2000 file system support878lb # [ 0.341440] 9p: Installing v9fs 9p2000 file system support879lb # [ 0.361399] Key type asymmetric registered880lb # [ 0.361427] Asymmetric key parser 'x509' registered881lb # [ 0.361507] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)882worker1 # [ 0.363061] Key type asymmetric registered883worker1 # [ 0.363091] Asymmetric key parser 'x509' registered884lb # [ 0.367534] io scheduler mq-deadline registered885lb # [ 0.367554] io scheduler kyber registered886worker1 # [ 0.363173] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)887worker1 # [ 0.365312] io scheduler mq-deadline registered888worker1 # [ 0.365323] io scheduler kyber registered889worker1 # [ 0.374587] pl061_gpio 9030000.pl061: PL061 GPIO chip registered890worker2 # [ 0.331018] Initialise system trusted keyrings891worker2 # [ 0.331787] workingset: timestamp_bits=42 max_order=19 bucket_order=0892client # [ 0.333512] Initialise system trusted keyrings893worker2 # [ 0.333144] squashfs: version 4.0 (2009/01/31) Phillip Lougher894client # [ 0.334319] workingset: timestamp_bits=42 max_order=19 bucket_order=0895worker2 # [ 0.333918] 9p: Installing v9fs 9p2000 file system support896client # [ 0.335601] squashfs: version 4.0 (2009/01/31) Phillip Lougher897client # [ 0.336373] 9p: Installing v9fs 9p2000 file system support898worker1 # [ 0.376063] ledtrig-cpu: registered to indicate activity on CPUs899worker1 # [ 0.376467] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:900lb # [ 0.379654] pl061_gpio 9030000.pl061: PL061 GPIO chip registered901worker1 # [ 0.376487] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000902lb # [ 0.380340] ledtrig-cpu: registered to indicate activity on CPUs903lb # [ 0.380726] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:904worker1 # [ 0.376498] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000905lb # [ 0.380745] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000906worker1 # [ 0.376506] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000907lb # [ 0.380757] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000908worker1 # [ 0.376539] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits909lb # [ 0.380766] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000910worker1 # [ 0.376564] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]911lb # [ 0.380788] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits912worker1 # [ 0.376643] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00913worker1 # [ 0.376652] pci_bus 0000:00: root bus resource [bus 00-ff]914lb # [ 0.380813] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]915worker1 # [ 0.376658] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]916lb # [ 0.380903] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00917worker1 # [ 0.376663] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]918lb # [ 0.380912] pci_bus 0000:00: root bus resource [bus 00-ff]919worker1 # [ 0.376668] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]920lb # [ 0.380918] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]921worker1 # [ 0.376735] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint922lb # [ 0.380924] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]923lb # [ 0.380929] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]924worker1 # [ 0.377205] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint925worker1 # [ 0.377397] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]926lb # [ 0.381022] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint927worker1 # [ 0.377413] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]928lb # [ 0.381461] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint929worker1 # [ 0.377443] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]930lb # [ 0.381646] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]931worker1 # [ 0.377458] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]932lb # [ 0.381662] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]933worker1 # [ 0.377911] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint934lb # [ 0.381692] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]935worker1 # [ 0.378100] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]936lb # [ 0.381708] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]937worker1 # [ 0.378115] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]938lb # [ 0.382163] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint939worker1 # [ 0.378145] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]940lb # [ 0.382346] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]941worker1 # [ 0.378612] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint942lb # [ 0.382362] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]943worker1 # [ 0.378795] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]944lb # [ 0.382391] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]945worker1 # [ 0.378810] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]946client # [ 0.360379] Key type asymmetric registered947lb # [ 0.382835] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint948client # [ 0.360409] Asymmetric key parser 'x509' registered949worker1 # [ 0.378839] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]950lb # [ 0.383016] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]951client # [ 0.360492] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)952worker1 # [ 0.379284] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint953lb # [ 0.383032] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]954worker1 # [ 0.379465] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]955lb # [ 0.383061] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]956worker1 # [ 0.379481] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]957worker2 # [ 0.362610] Key type asymmetric registered958lb # [ 0.383529] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint959worker1 # [ 0.379513] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]960worker2 # [ 0.362648] Asymmetric key parser 'x509' registered961lb # [ 0.383708] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]962worker1 # [ 0.379961] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint963lb # [ 0.383724] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]964worker2 # [ 0.362737] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)965worker1 # [ 0.380143] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]966worker2 # [ 0.364879] io scheduler mq-deadline registered967lb # [ 0.383756] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]968worker1 # [ 0.380158] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]969worker2 # [ 0.364890] io scheduler kyber registered970lb # [ 0.384251] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint971worker1 # [ 0.380187] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]972lb # [ 0.384431] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]973worker1 # [ 0.380640] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint974lb # [ 0.384447] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]975worker1 # [ 0.380822] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]976lb # [ 0.384476] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]977worker1 # [ 0.380837] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]978lb # [ 0.384926] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint979worker1 # [ 0.380866] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]980lb # [ 0.385105] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]981worker1 # [ 0.381358] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint982lb # [ 0.385121] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]983worker1 # [ 0.381541] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]984lb # [ 0.385150] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]985client # [ 0.368932] io scheduler mq-deadline registered986worker1 # [ 0.381557] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]987client # [ 0.368953] io scheduler kyber registered988lb # [ 0.385594] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint989worker1 # [ 0.381585] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990lb # [ 0.385774] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]991worker1 # [ 0.381601] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]992client # [ 0.374145] pl061_gpio 9030000.pl061: PL061 GPIO chip registered993lb # [ 0.385789] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]994worker1 # [ 0.382050] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint995lb # [ 0.385818] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996worker1 # [ 0.382237] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]997lb # [ 0.385834] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]998worker1 # [ 0.382266] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]999lb # [ 0.386283] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1000lb # [ 0.386467] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1001lb # [ 0.386497] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1002lb # [ 0.386941] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1003worker2 # [ 0.378185] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1004lb # [ 0.387126] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1005client # [ 0.375412] ledtrig-cpu: registered to indicate activity on CPUs1006lb # [ 0.387155] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1007client # [ 0.375789] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1008client # [ 0.375806] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001009worker2 # [ 0.379627] ledtrig-cpu: registered to indicate activity on CPUs1010client # [ 0.375819] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001011worker2 # [ 0.380032] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1012client # [ 0.375827] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001013worker2 # [ 0.380049] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001014client # [ 0.375847] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1015worker2 # [ 0.380061] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001016client # [ 0.375876] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1017worker2 # [ 0.380069] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001018worker1 # [ 0.426795] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1019client # [ 0.375955] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001020worker2 # [ 0.380091] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1021worker1 # [ 0.426998] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1022client # [ 0.375965] pci_bus 0000:00: root bus resource [bus 00-ff]1023worker1 # [ 0.427028] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024client # [ 0.375971] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1025worker2 # [ 0.380113] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1026worker1 # [ 0.427418] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1027client # [ 0.375976] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1028worker2 # [ 0.380192] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001029worker1 # [ 0.427600] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1030worker2 # [ 0.380201] pci_bus 0000:00: root bus resource [bus 00-ff]1031client # [ 0.375982] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1032worker2 # [ 0.380208] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1033worker1 # [ 0.427844] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1034client # [ 0.376043] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1035worker1 # [ 0.428140] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1036worker2 # [ 0.380213] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1037lb # [ 0.435629] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1038client # [ 0.376479] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1039worker1 # [ 0.428157] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1040worker2 # [ 0.380217] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1041client # [ 0.376667] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1042lb # [ 0.435828] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1043worker1 # [ 0.428187] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1044client # [ 0.376684] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1045worker2 # [ 0.380306] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1046lb # [ 0.436099] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1047worker1 # [ 0.428661] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1048client # [ 0.376714] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049lb # [ 0.436391] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1050worker2 # [ 0.380744] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1051worker1 # [ 0.428845] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1052client # [ 0.376730] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1053lb # [ 0.436408] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1054worker2 # [ 0.380931] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1055worker1 # [ 0.428862] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1056lb # [ 0.436437] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1057worker2 # [ 0.380948] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1058client # [ 0.377236] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1059worker1 # [ 0.428891] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1060client # [ 0.377424] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1061worker2 # [ 0.380978] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1062lb # [ 0.436890] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1063worker1 # [ 0.429462] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1064client # [ 0.377440] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1065worker2 # [ 0.380994] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1066lb # [ 0.437073] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1067worker1 # [ 0.429473] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1068client # [ 0.377469] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069lb # [ 0.437088] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1070worker2 # [ 0.381456] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1071worker1 # [ 0.429478] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1072worker2 # [ 0.381640] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1073client # [ 0.377922] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1074lb # [ 0.437117] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1075worker2 # [ 0.381656] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1076client # [ 0.378106] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1077worker1 # [ 0.429524] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1078lb # [ 0.437693] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1079client # [ 0.378122] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1080worker2 # [ 0.381686] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081worker1 # [ 0.429570] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1082lb # [ 0.437704] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1083client # [ 0.378151] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1084worker1 # [ 0.429617] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1085lb # [ 0.437709] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1086client # [ 0.378597] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1087worker1 # [ 0.429664] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1088lb # [ 0.437754] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1089client # [ 0.378784] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1090worker2 # [ 0.400816] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1091client # [ 0.378800] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1092worker2 # [ 0.401003] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1093worker1 # [ 0.429711] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1094lb # [ 0.437801] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1095worker2 # [ 0.401019] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1096client # [ 0.378833] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1097worker1 # [ 0.429759] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1098lb # [ 0.437847] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1099worker2 # [ 0.401049] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100client # [ 0.379298] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1101worker1 # [ 0.429807] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1102lb # [ 0.437893] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1103client # [ 0.379481] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1104worker2 # [ 0.401509] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1105worker1 # [ 0.429853] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1106lb # [ 0.437940] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1107client # [ 0.379497] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1108worker2 # [ 0.401691] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1109worker1 # [ 0.429900] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1110worker2 # [ 0.401706] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1111lb # [ 0.437986] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1112client # [ 0.379527] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1113worker2 # [ 0.401738] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114worker1 # [ 0.429963] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1115lb # [ 0.438034] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1116client # [ 0.379979] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1117worker1 # [ 0.430033] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1118client # [ 0.380162] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1119lb # [ 0.438081] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1120client # [ 0.380178] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1121worker1 # [ 0.430055] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1122lb # [ 0.438128] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1123worker1 # [ 0.430076] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1124client # [ 0.380207] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1125lb # [ 0.438188] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1126worker1 # [ 0.430097] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1127client # [ 0.380656] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1128lb # [ 0.438233] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1129worker1 # [ 0.430121] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1130client # [ 0.380840] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1131lb # [ 0.438254] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1132client # [ 0.380856] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1133worker1 # [ 0.430142] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1134lb # [ 0.438275] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1135worker1 # [ 0.430164] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1136lb # [ 0.438297] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1137worker1 # [ 0.430185] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1138lb # [ 0.438321] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1139worker1 # [ 0.430207] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1140lb # [ 0.438343] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1141worker1 # [ 0.430229] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1142lb # [ 0.438365] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1143worker1 # [ 0.430250] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1144worker1 # [ 0.430273] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1145lb # [ 0.438386] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1146worker1 # [ 0.430294] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1147lb # [ 0.438407] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1148worker1 # [ 0.430316] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1149lb # [ 0.438429] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1150worker1 # [ 0.430336] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1151lb # [ 0.438450] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1152worker1 # [ 0.430357] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1153lb # [ 0.438473] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1154worker1 # [ 0.430379] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1155lb # [ 0.438494] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1156worker1 # [ 0.430400] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1157lb # [ 0.438516] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1158worker2 # [ 0.414262] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1159worker1 # [ 0.430421] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1160lb # [ 0.438536] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1161worker2 # [ 0.414465] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1162lb # [ 0.438558] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1163worker2 # [ 0.414482] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1164client # [ 0.420945] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1165lb # [ 0.438578] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1166worker2 # [ 0.414512] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167client # [ 0.420976] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1168lb # [ 0.438600] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1169worker2 # [ 0.414980] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1170client # [ 0.421513] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1171lb # [ 0.438621] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1172worker2 # [ 0.415165] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1173client # [ 0.421712] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1174lb # [ 0.438642] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1175worker2 # [ 0.415180] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1176lb # [ 0.438664] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1177client # [ 0.421742] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1178worker2 # [ 0.415210] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1179lb # [ 0.438689] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1180client # [ 0.422204] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1181lb # [ 0.438699] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1182worker2 # [ 0.415667] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1183client # [ 0.422393] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1184worker2 # [ 0.415853] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1185lb # [ 0.438703] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1186client # [ 0.422423] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1187worker2 # [ 0.415869] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1188client # [ 0.422826] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1189worker2 # [ 0.415898] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1190client # [ 0.423011] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1191worker2 # [ 0.415923] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1192client # [ 0.423261] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1193worker2 # [ 0.416432] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1194client # [ 0.423569] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1195worker2 # [ 0.416622] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1196worker1 # [ 0.474471] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1197client # [ 0.423587] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1198worker1 # [ 0.474532] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1199worker2 # [ 0.416651] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200client # [ 0.423617] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1201worker1 # [ 0.474568] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1202worker2 # [ 0.417113] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1203worker1 # [ 0.474577] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1204client # [ 0.424076] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1205worker2 # [ 0.417301] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1206client # [ 0.424267] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1207worker1 # [ 0.474582] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1208worker2 # [ 0.417330] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1209client # [ 0.424283] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1210worker1 # [ 0.475760] pci 0000:00:0a.0: enabling device (0000 -> 0002)1211worker2 # [ 0.417738] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1212lb # [ 0.483934] pci 0000:00:0a.0: enabling device (0000 -> 0002)1213client # [ 0.424313] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1214worker2 # [ 0.417920] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1215worker2 # [ 0.418185] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1216worker2 # [ 0.418488] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1217worker2 # [ 0.418505] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1218worker2 # [ 0.418534] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1219worker2 # [ 0.419008] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1220worker2 # [ 0.419194] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1221worker2 # [ 0.419210] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1222worker2 # [ 0.419240] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1223worker2 # [ 0.419839] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1224worker2 # [ 0.419850] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1225worker2 # [ 0.419856] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1226client # [ 0.444976] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1227worker2 # [ 0.419901] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1228client # [ 0.444998] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1229worker2 # [ 0.419959] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1230client # [ 0.445004] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1231worker2 # [ 0.420007] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1232client # [ 0.445056] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1233worker2 # [ 0.420053] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1234client # [ 0.445106] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1235worker2 # [ 0.420099] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1236client # [ 0.445154] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1237worker2 # [ 0.420146] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1238client # [ 0.445202] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1239worker2 # [ 0.420193] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1240worker1 # [ 0.496210] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1241client # [ 0.445265] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1242worker2 # [ 0.420239] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1243client # [ 0.445314] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1244worker2 # [ 0.420287] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1245client # [ 0.445363] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1246worker2 # [ 0.420361] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1247client # [ 0.445412] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1248worker2 # [ 0.420407] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1249client # [ 0.445460] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1250worker2 # [ 0.420429] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1251worker2 # [ 0.420450] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1252lb # [ 0.504723] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1253client # [ 0.445574] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1254worker2 # [ 0.420472] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1255lb # [ 0.506878] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1256client # [ 0.445622] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1257worker2 # [ 0.420497] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1258client # [ 0.445644] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1259worker2 # [ 0.420519] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1260client # [ 0.445666] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1261worker2 # [ 0.420542] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1262client # [ 0.445688] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1263worker2 # [ 0.420563] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1264client # [ 0.445714] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1265worker2 # [ 0.420585] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1266client # [ 0.445736] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1267worker2 # [ 0.420608] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1268client # [ 0.445758] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1269worker2 # [ 0.420634] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1270client # [ 0.445780] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1271worker1 # [ 0.506473] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1272worker2 # [ 0.420657] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1273client # [ 0.445802] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1274worker1 # [ 0.510183] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1275worker2 # [ 0.420679] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1276client # [ 0.445825] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1277worker2 # [ 0.420701] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1278client # [ 0.445849] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1279worker2 # [ 0.420722] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1280client # [ 0.445872] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1281worker2 # [ 0.420744] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1282client # [ 0.445893] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1283worker2 # [ 0.420766] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1284client # [ 0.445915] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1285worker2 # [ 0.420787] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1286lb # [ 0.517897] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1287client # [ 0.445937] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1288worker2 # [ 0.420809] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1289client # [ 0.445959] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1290worker2 # [ 0.420831] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1291client # [ 0.445980] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1292worker2 # [ 0.420853] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1293client # [ 0.446002] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1294worker2 # [ 0.420879] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1295client # [ 0.446024] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1296worker2 # [ 0.420889] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1297client # [ 0.446047] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1298worker2 # [ 0.420894] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1299client # [ 0.446069] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1300client # [ 0.446096] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1301client # [ 0.446106] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1302client # [ 0.446110] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1303worker1 # [ 0.519799] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1304client # [ 0.447287] pci 0000:00:0a.0: enabling device (0000 -> 0002)1305worker2 # [ 0.482140] pci 0000:00:0a.0: enabling device (0000 -> 0002)1306lb # [ 0.528162] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1307lb # [ 0.530478] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1308lb # [ 0.532688] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1309worker1 # [ 0.531014] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1310worker1 # [ 0.533310] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1311lb # [ 0.534821] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1312worker1 # [ 0.544562] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1313client # [ 0.503388] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1314worker1 # [ 0.547497] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1315worker2 # [ 0.503229] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1316worker1 # [ 0.549733] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1317worker2 # [ 0.505415] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1318lb # [ 0.553081] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1319lb # [ 0.555297] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1320client # [ 0.509595] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1321lb # [ 0.558803] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1322client # [ 0.511673] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1323lb # [ 0.561895] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1324worker2 # [ 0.516504] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1325worker1 # [ 0.560043] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1326worker1 # [ 0.562428] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1327client # [ 0.522080] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1328client # [ 0.524256] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1329worker2 # [ 0.522646] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1330lb # [ 0.575412] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1331worker2 # [ 0.524673] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1332client # [ 0.535597] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1333worker2 # [ 0.535090] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1334lb # [ 0.584840] msm_serial: driver initialized1335worker2 # [ 0.537525] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1336lb # [ 0.585006] SuperH (H)SCI(F) driver initialized1337lb # [ 0.585060] STM32 USART driver initialized1338client # [ 0.538748] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1339client # [ 0.540812] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1340worker1 # [ 0.579676] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1341worker1 # [ 0.581579] msm_serial: driver initialized1342worker1 # [ 0.581728] SuperH (H)SCI(F) driver initialized1343worker1 # [ 0.581781] STM32 USART driver initialized1344worker2 # [ 0.548970] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1345client # [ 0.550866] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1346client # [ 0.552724] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1347worker2 # [ 0.555546] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1348worker2 # [ 0.557435] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1349client # [ 0.562989] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1350lb # [ 0.614449] loop: module loaded1351lb # [ 0.614650] virtio_blk virtio5: 1/0/0 default/read/poll queues1352lb # [ 0.615430] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1353worker1 # [ 0.612979] loop: module loaded1354worker1 # [ 0.613199] virtio_blk virtio5: 1/0/0 default/read/poll queues1355worker1 # [ 0.613950] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1356worker2 # [ 0.571767] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1357client # [ 0.578136] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1358worker1 # [ 0.619096] megasas: 07.734.00.00-rc11359worker2 # [ 0.577598] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1360worker1 # [ 0.619815] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1361client # [ 0.580790] msm_serial: driver initialized1362worker1 # [ 0.621711] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001363worker1 # [ 0.621737] Intel/Sharp Extended Query Table at 0x00311364client # [ 0.581540] SuperH (H)SCI(F) driver initialized1365client # [ 0.581598] STM32 USART driver initialized1366worker2 # [ 0.583464] msm_serial: driver initialized1367worker2 # [ 0.583627] SuperH (H)SCI(F) driver initialized1368worker2 # [ 0.583681] STM32 USART driver initialized1369lb # [ 0.628230] megasas: 07.734.00.00-rc11370lb # [ 0.628954] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1371lb # [ 0.630936] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001372worker1 # [ 0.631643] Using buffer write method1373lb # [ 0.630961] Intel/Sharp Extended Query Table at 0x00311374lb # [ 0.632728] Using buffer write method1375worker1 # [ 0.631732] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1376worker1 # [ 0.633296] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001377lb # [ 0.632790] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1378worker1 # [ 0.633331] Intel/Sharp Extended Query Table at 0x00311379lb # [ 0.634470] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001380lb # [ 0.634527] Intel/Sharp Extended Query Table at 0x00311381worker1 # [ 0.647080] Using buffer write method1382worker1 # [ 0.647118] Concatenating MTD devices:1383worker1 # [ 0.647122] (0): "0.flash"1384worker1 # [ 0.647127] (1): "0.flash"1385lb # [ 0.652309] Using buffer write method1386worker1 # [ 0.647130] into device "0.flash"1387lb # [ 0.652366] Concatenating MTD devices:1388lb # [ 0.652370] (0): "0.flash"1389lb # [ 0.652375] (1): "0.flash"1390lb # [ 0.652378] into device "0.flash"1391client # [ 0.614044] loop: module loaded1392client # [ 0.614236] virtio_blk virtio5: 1/0/0 default/read/poll queues1393client # [ 0.615282] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1394worker2 # [ 0.614736] loop: module loaded1395worker2 # [ 0.614922] virtio_blk virtio5: 1/0/0 default/read/poll queues1396worker2 # [ 0.615685] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1397client # [ 0.621720] megasas: 07.734.00.00-rc11398client # [ 0.622427] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399client # [ 0.624700] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001400client # [ 0.624727] Intel/Sharp Extended Query Table at 0x00311401worker2 # [ 0.626676] megasas: 07.734.00.00-rc11402worker2 # [ 0.627481] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1403worker2 # [ 0.629888] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001404worker2 # [ 0.629916] Intel/Sharp Extended Query Table at 0x00311405client # [ 0.633932] Using buffer write method1406client # [ 0.634030] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1407worker2 # [ 0.639005] Using buffer write method1408worker2 # [ 0.639088] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1409client # [ 0.644923] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001410client # [ 0.644977] Intel/Sharp Extended Query Table at 0x00311411client # [ 0.646527] Using buffer write method1412client # [ 0.646558] Concatenating MTD devices:1413client # [ 0.646563] (0): "0.flash"1414client # [ 0.646567] (1): "0.flash"1415client # [ 0.646570] into device "0.flash"1416worker2 # [ 0.649984] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417worker2 # [ 0.650024] Intel/Sharp Extended Query Table at 0x00311418worker2 # [ 0.651549] Using buffer write method1419worker2 # [ 0.651579] Concatenating MTD devices:1420worker2 # [ 0.651584] (0): "0.flash"1421worker2 # [ 0.651588] (1): "0.flash"1422worker2 # [ 0.651591] into device "0.flash"1423lb # [ 0.911187] Freeing initrd memory: 26140K1424worker1 # [ 0.909622] Freeing initrd memory: 26148K1425lb # [ 0.917349] tun: Universal TUN/TAP device driver, 1.61426worker1 # [ 0.915830] tun: Universal TUN/TAP device driver, 1.61427lb # [ 0.921208] thunder_xcv, ver 1.01428lb # [ 0.921254] thunder_bgx, ver 1.01429lb # [ 0.921276] nicpf, ver 1.01430lb # [ 0.921818] e1000: Intel(R) PRO/1000 Network Driver1431lb # [ 0.921826] e1000: Copyright (c) 1999-2006 Intel Corporation.1432lb # [ 0.921849] e1000e: Intel(R) PRO/1000 Network Driver1433worker1 # [ 0.919768] thunder_xcv, ver 1.01434worker1 # [ 0.919822] thunder_bgx, ver 1.01435lb # [ 0.921858] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1436worker1 # [ 0.919845] nicpf, ver 1.01437lb # [ 0.921887] igb: Intel(R) Gigabit Ethernet Network Driver1438worker1 # [ 0.920437] e1000: Intel(R) PRO/1000 Network Driver1439lb # [ 0.921893] igb: Copyright (c) 2007-2014 Intel Corporation.1440worker1 # [ 0.920445] e1000: Copyright (c) 1999-2006 Intel Corporation.1441worker1 # [ 0.920468] e1000e: Intel(R) PRO/1000 Network Driver1442lb # [ 0.921916] igbvf: Intel(R) Gigabit Virtual Function Network Driver1443worker1 # [ 0.920478] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1444lb # [ 0.921921] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1445lb # [ 0.922061] sky2: driver version 1.301446worker1 # [ 0.920507] igb: Intel(R) Gigabit Ethernet Network Driver1447lb # [ 0.924260] ehci-pci 0000:00:0a.0: EHCI Host Controller1448worker1 # [ 0.920513] igb: Copyright (c) 2007-2014 Intel Corporation.1449lb # [ 0.924289] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11450worker1 # [ 0.920536] igbvf: Intel(R) Gigabit Virtual Function Network Driver1451lb # [ 0.924550] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001452worker1 # [ 0.920542] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1453worker1 # [ 0.920677] sky2: driver version 1.301454worker1 # [ 0.922284] usbcore: registered new interface driver usb-storage1455worker1 # [ 0.922370] usbcore: registered new interface driver usbserial_generic1456worker1 # [ 0.922385] usbserial: USB Serial support registered for generic1457lb # [ 0.934264] usbcore: registered new interface driver usb-storage1458worker1 # [ 0.923271] ehci-pci 0000:00:0a.0: EHCI Host Controller1459lb # [ 0.934406] usbcore: registered new interface driver usbserial_generic1460lb # [ 0.934420] usbserial: USB Serial support registered for generic1461worker1 # [ 0.923313] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11462lb # [ 0.935014] hv_vmbus: registering driver hyperv_keyboard1463lb # [ 0.935530] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001464worker1 # [ 0.923526] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001465lb # [ 0.935858] hub 1-0:1.0: USB hub found1466worker1 # [ 0.934657] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001467lb # [ 0.935879] hub 1-0:1.0: 6 ports detected1468worker1 # [ 0.935027] hub 1-0:1.0: USB hub found1469worker1 # [ 0.935044] hub 1-0:1.0: 6 ports detected1470lb # [ 0.941214] rtc-pl031 9010000.pl031: registered as rtc01471worker1 # [ 0.938476] hv_vmbus: registering driver hyperv_keyboard1472lb # [ 0.941242] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T08:17:05 UTC (1789892225)1473lb # [ 0.941599] i2c_dev: i2c /dev entries driver1474worker1 # [ 0.940037] rtc-pl031 9010000.pl031: registered as rtc01475worker1 # [ 0.940068] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T08:17:05 UTC (1789892225)1476worker1 # [ 0.940401] i2c_dev: i2c /dev entries driver1477lb # [ 0.946559] sdhci: Secure Digital Host Controller Interface driver1478lb # [ 0.946568] sdhci: Copyright(c) Pierre Ossman1479lb # [ 0.946834] Synopsys Designware Multimedia Card Interface Driver1480lb # [ 0.947203] sdhci-pltfm: SDHCI platform and OF driver helper1481worker1 # [ 0.945317] sdhci: Secure Digital Host Controller Interface driver1482worker1 # [ 0.945327] sdhci: Copyright(c) Pierre Ossman1483worker1 # [ 0.945591] Synopsys Designware Multimedia Card Interface Driver1484worker1 # [ 0.945963] sdhci-pltfm: SDHCI platform and OF driver helper1485lb # [ 0.951568] hid: raw HID events driver (C) Jiri Kosina1486lb # [ 0.951807] usbcore: registered new interface driver usbhid1487lb # [ 0.951813] usbhid: USB HID core driver1488worker1 # [ 0.950335] hid: raw HID events driver (C) Jiri Kosina1489client # [ 0.906210] Freeing initrd memory: 26160K1490lb # [ 0.954745] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1491worker1 # [ 0.951285] usbcore: registered new interface driver usbhid1492worker1 # [ 0.951296] usbhid: USB HID core driver1493lb # [ 0.957364] drop_monitor: Initializing network drop monitor service1494lb # [ 0.957524] NET: Registered PF_INET6 protocol family1495worker1 # [ 0.953543] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1496client # [ 0.912230] tun: Universal TUN/TAP device driver, 1.61497worker2 # [ 0.911116] Freeing initrd memory: 26148K1498lb # [ 0.959462] Segment Routing with IPv61499lb # [ 0.959481] In-situ OAM (IOAM) with IPv61500worker1 # [ 0.956250] drop_monitor: Initializing network drop monitor service1501lb # [ 0.960495] NET: Registered PF_PACKET protocol family1502worker1 # [ 0.956405] NET: Registered PF_INET6 protocol family1503worker1 # [ 0.958363] Segment Routing with IPv61504worker1 # [ 0.958383] In-situ OAM (IOAM) with IPv61505worker1 # [ 0.958413] NET: Registered PF_PACKET protocol family1506client # [ 0.916058] thunder_xcv, ver 1.01507lb # [ 0.961215] 9pnet: Installing 9P2000 support1508client # [ 0.916105] thunder_bgx, ver 1.01509lb # [ 0.963381] Key type dns_resolver registered1510client # [ 0.916128] nicpf, ver 1.01511client # [ 0.916665] e1000: Intel(R) PRO/1000 Network Driver1512worker2 # [ 0.917214] tun: Universal TUN/TAP device driver, 1.61513client # [ 0.916673] e1000: Copyright (c) 1999-2006 Intel Corporation.1514worker1 # [ 0.960264] 9pnet: Installing 9P2000 support1515client # [ 0.916696] e1000e: Intel(R) PRO/1000 Network Driver1516client # [ 0.916707] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1517client # [ 0.916736] igb: Intel(R) Gigabit Ethernet Network Driver1518client # [ 0.916741] igb: Copyright (c) 2007-2014 Intel Corporation.1519client # [ 0.916763] igbvf: Intel(R) Gigabit Virtual Function Network Driver1520client # [ 0.916769] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1521worker2 # [ 0.921044] thunder_xcv, ver 1.01522worker1 # [ 0.963218] Key type dns_resolver registered1523worker2 # [ 0.921094] thunder_bgx, ver 1.01524worker2 # [ 0.921117] nicpf, ver 1.01525worker2 # [ 0.921676] e1000: Intel(R) PRO/1000 Network Driver1526lb # [ 0.970288] registered taskstats version 11527client # [ 0.923700] sky2: driver version 1.301528worker2 # [ 0.921683] e1000: Copyright (c) 1999-2006 Intel Corporation.1529lb # [ 0.970457] Loading compiled-in X.509 certificates1530worker2 # [ 0.921706] e1000e: Intel(R) PRO/1000 Network Driver1531worker2 # [ 0.921715] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1532worker2 # [ 0.921745] igb: Intel(R) Gigabit Ethernet Network Driver1533client # [ 0.925819] usbcore: registered new interface driver usb-storage1534worker1 # [ 0.969770] registered taskstats version 11535worker2 # [ 0.921751] igb: Copyright (c) 2007-2014 Intel Corporation.1536worker1 # [ 0.969937] Loading compiled-in X.509 certificates1537client # [ 0.925876] usbcore: registered new interface driver usbserial_generic1538worker2 # [ 0.921774] igbvf: Intel(R) Gigabit Virtual Function Network Driver1539client # [ 0.925890] usbserial: USB Serial support registered for generic1540worker2 # [ 0.921780] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1541client # [ 0.926474] hv_vmbus: registering driver hyperv_keyboard1542worker2 # [ 0.921916] sky2: driver version 1.301543client # [ 0.930709] ehci-pci 0000:00:0a.0: EHCI Host Controller1544client # [ 0.930741] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11545lb # [ 0.978954] Demotion targets for Node 0: null1546client # [ 0.930985] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001547lb # [ 0.979072] Key type .fscrypt registered1548lb # [ 0.979079] Key type fscrypt-provisioning registered1549lb # [ 0.979177] ima: No TPM chip found, activating TPM-bypass!1550worker2 # [ 0.930711] usbcore: registered new interface driver usb-storage1551client # [ 0.933682] rtc-pl031 9010000.pl031: registered as rtc01552lb # [ 0.979196] ima: Allocated hash algorithm: sha11553worker2 # [ 0.930807] usbcore: registered new interface driver usbserial_generic1554lb # [ 0.979219] ima: No architecture policies found1555client # [ 0.933711] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T08:17:05 UTC (1789892225)1556worker1 # [ 0.978607] Demotion targets for Node 0: null1557worker2 # [ 0.930822] usbserial: USB Serial support registered for generic1558client # [ 0.934051] i2c_dev: i2c /dev entries driver1559worker1 # [ 0.978730] Key type .fscrypt registered1560worker2 # [ 0.931421] hv_vmbus: registering driver hyperv_keyboard1561lb # [ 0.983602] input: gpio-keys as /devices/platform/gpio-keys/input/input01562worker1 # [ 0.978736] Key type fscrypt-provisioning registered1563worker2 # [ 0.932778] ehci-pci 0000:00:0a.0: EHCI Host Controller1564worker1 # [ 0.978840] ima: No TPM chip found, activating TPM-bypass!1565worker1 # [ 0.978861] ima: Allocated hash algorithm: sha11566worker2 # [ 0.932805] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11567worker1 # [ 0.978883] ima: No architecture policies found1568worker2 # [ 0.933049] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001569worker1 # [ 0.983161] input: gpio-keys as /devices/platform/gpio-keys/input/input01570worker2 # [ 0.938590] rtc-pl031 9010000.pl031: registered as rtc01571client # [ 0.938899] sdhci: Secure Digital Host Controller Interface driver1572client # [ 0.938911] sdhci: Copyright(c) Pierre Ossman1573worker2 # [ 0.938620] rtc-pl031 9010000.pl031: setting system clock to 2026-09-20T08:17:05 UTC (1789892225)1574client # [ 0.939172] Synopsys Designware Multimedia Card Interface Driver1575worker2 # [ 0.938977] i2c_dev: i2c /dev entries driver1576client # [ 0.939535] sdhci-pltfm: SDHCI platform and OF driver helper1577client # [ 0.940917] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001578worker2 # [ 0.942037] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001579worker2 # [ 0.942415] hub 1-0:1.0: USB hub found1580worker2 # [ 0.942435] hub 1-0:1.0: 6 ports detected1581client # [ 0.944780] hid: raw HID events driver (C) Jiri Kosina1582client # [ 0.945195] hub 1-0:1.0: USB hub found1583client # [ 0.945222] hub 1-0:1.0: 6 ports detected1584client # [ 0.947327] usbcore: registered new interface driver usbhid1585client # [ 0.947338] usbhid: USB HID core driver1586worker2 # [ 0.946401] sdhci: Secure Digital Host Controller Interface driver1587worker2 # [ 0.946410] sdhci: Copyright(c) Pierre Ossman1588client # [ 0.949679] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1589worker2 # [ 0.946679] Synopsys Designware Multimedia Card Interface Driver1590client # [ 0.951185] drop_monitor: Initializing network drop monitor service1591worker2 # [ 0.947048] sdhci-pltfm: SDHCI platform and OF driver helper1592client # [ 0.951370] NET: Registered PF_INET6 protocol family1593worker2 # [ 0.948825] hid: raw HID events driver (C) Jiri Kosina1594client # [ 0.954480] Segment Routing with IPv61595worker2 # [ 0.949086] usbcore: registered new interface driver usbhid1596client # [ 0.954508] In-situ OAM (IOAM) with IPv61597worker2 # [ 0.949093] usbhid: USB HID core driver1598client # [ 0.954539] NET: Registered PF_PACKET protocol family1599lb # [ 1.002585] clk: Disabling unused clocks1600client # [ 0.956198] 9pnet: Installing 9P2000 support1601lb # [ 1.002626] PM: genpd: Disabling unused power domains1602worker2 # [ 0.954718] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1603worker2 # [ 0.956315] drop_monitor: Initializing network drop monitor service1604worker1 # [ 1.002435] clk: Disabling unused clocks1605worker2 # [ 0.956451] NET: Registered PF_INET6 protocol family1606lb # [ 1.006929] Freeing unused kernel memory: 4736K1607worker2 # [ 0.959569] Segment Routing with IPv61608client # [ 0.959025] Key type dns_resolver registered1609lb # [ 1.007139] Run /init as init process1610worker2 # [ 0.959588] In-situ OAM (IOAM) with IPv61611worker2 # [ 0.959619] NET: Registered PF_PACKET protocol family1612worker1 # [ 1.003058] PM: genpd: Disabling unused power domains1613worker2 # [ 0.961245] 9pnet: Installing 9P2000 support1614worker1 # [ 1.006947] Freeing unused kernel memory: 4736K1615worker1 # [ 1.007171] Run /init as init process1616worker2 # [ 0.964099] Key type dns_resolver registered1617client # [ 0.965390] registered taskstats version 11618client # [ 0.965576] Loading compiled-in X.509 certificates1619client # [ 0.974497] Demotion targets for Node 0: null1620lb # [ 1.021793] systemd[1]: Successfully made /usr/ read-only.1621worker2 # [ 0.970718] registered taskstats version 11622client # [ 0.974611] Key type .fscrypt registered1623worker2 # [ 0.970876] Loading compiled-in X.509 certificates1624client # [ 0.974618] Key type fscrypt-provisioning registered1625client # [ 0.974718] ima: No TPM chip found, activating TPM-bypass!1626client # [ 0.974739] ima: Allocated hash algorithm: sha11627client # [ 0.974761] ima: No architecture policies found1628client # [ 0.978956] input: gpio-keys as /devices/platform/gpio-keys/input/input01629worker2 # [ 0.979532] Demotion targets for Node 0: null1630worker2 # [ 0.979646] Key type .fscrypt registered1631worker1 # [ 1.024769] systemd[1]: Successfully made /usr/ read-only.1632worker2 # [ 0.979652] Key type fscrypt-provisioning registered1633worker2 # [ 0.979790] ima: No TPM chip found, activating TPM-bypass!1634worker2 # [ 0.979810] ima: Allocated hash algorithm: sha11635worker2 # [ 0.979831] ima: No architecture policies found1636worker2 # [ 0.984047] input: gpio-keys as /devices/platform/gpio-keys/input/input01637client # [ 0.997643] clk: Disabling unused clocks1638client # [ 0.997675] PM: genpd: Disabling unused power domains1639client # [ 1.002077] Freeing unused kernel memory: 4736K1640client # [ 1.002285] Run /init as init process1641worker2 # [ 1.002843] clk: Disabling unused clocks1642worker2 # [ 1.002888] PM: genpd: Disabling unused power domains1643worker2 # [ 1.007279] Freeing unused kernel memory: 4736K1644worker2 # [ 1.007495] Run /init as init process1645client # [ 1.019203] systemd[1]: Successfully made /usr/ read-only.1646worker2 # [ 1.024584] systemd[1]: Successfully made /usr/ read-only.1647lb # [ 1.187587] usb 1-1: new high-speed USB device number 2 using ehci-pci1648worker1 # [ 1.182523] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker2 # [ 1.190029] usb 1-1: new high-speed USB device number 2 using ehci-pci1650client # [ 1.192960] usb 1-1: new high-speed USB device number 2 using ehci-pci1651worker1 # [ 1.334903] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11652lb # [ 1.343835] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653lb # [ 1.357156] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1654lb # [ 1.369416] systemd[1]: Detected virtualization qemu.1655lb # [ 1.369519] systemd[1]: Detected architecture arm64.1656worker1 # [ 1.359401] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1657lb # [ 1.369546] systemd[1]: Running in initrd.1658worker1 # [ 1.371693] systemd[1]: Detected virtualization qemu.1659lb # [ 1.370536] systemd[1]: Initializing machine ID from random generator.1660worker1 # [ 1.373731] systemd[1]: Detected architecture arm64.1661lb # [ 1.370822] systemd[1]: Hostname set to <lb>.1662worker1 # [ 1.375725] systemd[1]: Running in initrd.1663worker1 # [ 1.378334] systemd[1]: Initializing machine ID from random generator.1664worker1 # [ 1.381310] systemd[1]: Hostname set to <worker1>.1665worker2 # [ 1.341755] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11666client # [ 1.345080] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11667worker2 # [ 1.359250] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668worker2 # [ 1.363554] systemd[1]: Detected virtualization qemu.1669client # [ 1.354089] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1670worker2 # [ 1.364281] systemd[1]: Detected architecture arm64.1671worker2 # [ 1.364939] systemd[1]: Running in initrd.1672worker2 # [ 1.365920] systemd[1]: Initializing machine ID from random generator.1673client # [ 1.366631] systemd[1]: Detected virtualization qemu.1674worker2 # [ 1.367042] systemd[1]: Hostname set to <worker2>.1675client # [ 1.368733] systemd[1]: Detected architecture arm64.1676client # [ 1.370782] systemd[1]: Running in initrd.1677client # [ 1.373487] systemd[1]: Initializing machine ID from random generator.1678client # [ 1.373777] systemd[1]: Hostname set to <client>.1679lb # [ 1.427783] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01680worker1 # [ 1.430750] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01681client # [ 1.425264] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682worker2 # [ 1.430150] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683lb # [ 1.547581] usb 1-2: new high-speed USB device number 3 using ehci-pci1684worker1 # [ 1.554534] usb 1-2: new high-speed USB device number 3 using ehci-pci1685client # [ 1.548955] usb 1-2: new high-speed USB device number 3 using ehci-pci1686worker2 # [ 1.550112] usb 1-2: new high-speed USB device number 3 using ehci-pci1687lb # [ 1.672770] systemd[1]: bpf-restrict-fs: LSM BPF program attached1688worker2 # [ 1.625867] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689worker1 # [ 1.693655] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690lb # [ 1.701709] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21691lb # [ 1.704953] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01692worker1 # [ 1.716341] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21693worker1 # [ 1.723238] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694client # [ 1.687757] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695worker2 # [ 1.704381] systemd[1]: Queued start job for default target Initrd Default Target.1696lb # [ 1.753615] systemd[1]: Queued start job for default target Initrd Default Target.1697worker2 # [ 1.706112] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21698client # [ 1.707011] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21699worker2 # [ 1.708333] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01700client # [ 1.709897] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01701lb # [ 1.761669] systemd[1]: Created slice Slice /system/modprobe.1702lb # [ 1.762724] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1703lb # [ 1.763914] systemd[1]: Expecting device /dev/disk/by-label/nixos...1704lb # [ 1.764835] systemd[1]: Reached target Path Units.1705worker2 # [ 1.716854] systemd[1]: Created slice Slice /system/modprobe.1706lb # [ 1.765554] systemd[1]: Reached target Slice Units.1707lb # [ 1.766255] systemd[1]: Reached target Swaps.1708lb # [ 1.766960] systemd[1]: Reached target Timer Units.1709worker2 # [ 1.718288] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1710lb # [ 1.767865] systemd[1]: Listening on D-Bus System Message Bus Socket.1711worker2 # [ 1.719666] systemd[1]: Expecting device /dev/disk/by-label/nixos...1712lb # [ 1.768942] systemd[1]: Listening on Journal Socket (/dev/log).1713worker2 # [ 1.720872] systemd[1]: Reached target Path Units.1714lb # [ 1.769931] systemd[1]: Listening on Journal Sockets.1715worker2 # [ 1.721736] systemd[1]: Reached target Slice Units.1716lb # [ 1.770729] systemd[1]: Listening on udev Control Socket.1717worker2 # [ 1.722640] systemd[1]: Reached target Swaps.1718worker2 # [ 1.723424] systemd[1]: Reached target Timer Units.1719lb # [ 1.771678] systemd[1]: Listening on udev Kernel Socket.1720lb # [ 1.772470] systemd[1]: Reached target Socket Units.1721worker2 # [ 1.724563] systemd[1]: Listening on D-Bus System Message Bus Socket.1722worker2 # [ 1.725935] systemd[1]: Listening on Journal Socket (/dev/log).1723lb # [ 1.774689] systemd[1]: Starting Create List of Static Device Nodes...1724worker2 # [ 1.727193] systemd[1]: Listening on Journal Sockets.1725worker2 # [ 1.728267] systemd[1]: Listening on udev Control Socket.1726worker2 # [ 1.729328] systemd[1]: Listening on udev Kernel Socket.1727worker2 # [ 1.730391] systemd[1]: Reached target Socket Units.1728worker2 # [ 1.733026] systemd[1]: Starting Create List of Static Device Nodes...1729lb # [ 1.782740] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1730lb # [ 1.782852] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1731lb # [ 1.791695] systemd[1]: Mounting Kernel Configuration File System...1732worker2 # [ 1.742174] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1733worker2 # [ 1.742283] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1734worker2 # [ 1.750253] systemd[1]: Mounting Kernel Configuration File System...1735worker1 # [ 1.808781] systemd[1]: Queued start job for default target Initrd Default Target.1736lb # [ 1.815095] systemd[1]: Starting Journal Service...1737client # [ 1.767991] systemd[1]: Queued start job for default target Initrd Default Target.1738worker1 # [ 1.816762] systemd[1]: Created slice Slice /system/modprobe.1739worker1 # [ 1.817867] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1740worker1 # [ 1.819152] systemd[1]: Expecting device /dev/disk/by-label/nixos...1741lb # [ 1.823816] systemd[1]: Starting Load Kernel Modules...1742worker1 # [ 1.820116] systemd[1]: Reached target Path Units.1743worker1 # [ 1.820887] systemd[1]: Reached target Slice Units.1744lb # [ 1.824730] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1745worker1 # [ 1.821617] systemd[1]: Reached target Swaps.1746client # [ 1.778998] systemd[1]: Created slice Slice /system/modprobe.1747worker1 # [ 1.822368] systemd[1]: Reached target Timer Units.1748worker1 # [ 1.823345] systemd[1]: Listening on D-Bus System Message Bus Socket.1749client # [ 1.780050] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1750client # [ 1.781353] systemd[1]: Expecting device /dev/disk/by-label/nixos...1751worker1 # [ 1.824439] systemd[1]: Listening on Journal Socket (/dev/log).1752worker2 # [ 1.780432] systemd[1]: Starting Journal Service...1753client # [ 1.782274] systemd[1]: Reached target Path Units.1754worker1 # [ 1.825427] systemd[1]: Listening on Journal Sockets.1755client # [ 1.782972] systemd[1]: Reached target Slice Units.1756worker1 # [ 1.826311] systemd[1]: Listening on udev Control Socket.1757client # [ 1.783688] systemd[1]: Reached target Swaps.1758client # [ 1.784326] systemd[1]: Reached target Timer Units.1759client # [ 1.785281] systemd[1]: Listening on D-Bus System Message Bus Socket.1760worker1 # [ 1.827312] systemd[1]: Listening on udev Kernel Socket.1761worker1 # [ 1.827335] systemd[1]: Reached target Socket Units.1762client # [ 1.786343] systemd[1]: Listening on Journal Socket (/dev/log).1763client # [ 1.787319] systemd[1]: Listening on Journal Sockets.1764worker1 # [ 1.830405] systemd[1]: Starting Create List of Static Device Nodes...1765client # [ 1.788178] systemd[1]: Listening on udev Control Socket.1766worker2 # [ 1.787133] systemd[1]: Starting Load Kernel Modules...1767worker2 # [ 1.788049] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1768client # [ 1.789095] systemd[1]: Listening on udev Kernel Socket.1769client # [ 1.789120] systemd[1]: Reached target Socket Units.1770lb # [ 1.836947] systemd[1]: Starting Coldplug All udev Devices...1771client # [ 1.792172] systemd[1]: Starting Create List of Static Device Nodes...1772worker1 # [ 1.838755] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1773worker1 # [ 1.839753] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1774worker2 # [ 1.798414] systemd[1]: Starting Coldplug All udev Devices...1775client # [ 1.801092] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1776client # [ 1.801178] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1777worker1 # [ 1.852053] systemd[1]: Mounting Kernel Configuration File System...1778client # [ 1.809256] systemd[1]: Mounting Kernel Configuration File System...1779worker2 # [ 1.811681] systemd[1]: Finished Create List of Static Device Nodes.1780worker2 # [ 1.812569] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1781lb # [ 1.860844] systemd[1]: Finished Create List of Static Device Nodes.1782lb # [ 1.862033] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1783lb # [ 1.862348] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1784lb # [ 1.862591] systemd[1]: Mounted Kernel Configuration File System.1785worker2 # [ 1.818244] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1786worker2 # [ 1.818785] systemd[1]: Mounted Kernel Configuration File System.1787lb # [ 1.872474] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1788worker1 # [ 1.870963] systemd[1]: Starting Journal Service...1789worker2 # [ 1.826463] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1790client # [ 1.829146] systemd[1]: Starting Journal Service...1791worker1 # [ 1.878269] systemd[1]: Starting Load Kernel Modules...1792worker1 # [ 1.879386] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1793lb # [ 1.884085] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1794client # [ 1.844209] systemd[1]: Starting Load Kernel Modules...1795client # [ 1.845325] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1796lb # [ 1.895131] systemd-journald[73]: Collecting audit messages is disabled.1797lb # [ 1.899599] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1798client # [ 1.858338] systemd[1]: Starting Coldplug All udev Devices...1799lb # [ 1.912016] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01800lb # [ 1.912292] [drm] features: -virgl +edid -resource_blob -host_visible1801lb # [ 1.912303] [drm] features: -context_init1802lb # [ 1.913013] [drm] number of scanouts: 11803lb # [ 1.913039] [drm] number of cap sets: 01804worker2 # [ 1.865745] systemd-journald[73]: Collecting audit messages is disabled.1805worker2 # [ 1.873431] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1806worker1 # [ 1.918665] systemd[1]: Starting Coldplug All udev Devices...1807worker1 # [ 1.921181] systemd[1]: Finished Create List of Static Device Nodes.1808client # [ 1.881129] systemd[1]: Finished Create List of Static Device Nodes.1809worker1 # [ 1.926859] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1810client # [ 1.882363] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1811client # [ 1.882686] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1812worker1 # [ 1.929244] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1813client # [ 1.882939] systemd[1]: Mounted Kernel Configuration File System.1814lb # [ 1.931876] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1815worker2 # [ 1.887184] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1816lb # [ 1.931912] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01817worker1 # [ 1.933401] systemd-journald[73]: Collecting audit messages is disabled.1818worker2 # [ 1.890180] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1819worker1 # [ 1.934909] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1820worker1 # [ 1.935245] systemd[1]: Mounted Kernel Configuration File System.1821worker2 # [ 1.898223] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01822client # [ 1.897344] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1823worker2 # [ 1.898489] [drm] features: -virgl +edid -resource_blob -host_visible1824worker2 # [ 1.898498] [drm] features: -context_init1825worker2 # [ 1.899234] [drm] number of scanouts: 11826lb # [ 1.948291] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1827worker2 # [ 1.899254] [drm] number of cap sets: 01828worker2 # [ 1.903108] systemd[1]: Starting Create Static Device Nodes in /dev...1829lb # [ 1.954700] systemd[1]: Starting Create Static Device Nodes in /dev...1830worker1 # [ 1.950612] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1831client # [ 1.912356] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1832client # [ 1.917951] systemd-journald[73]: Collecting audit messages is disabled.1833worker1 # [ 1.959930] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01834worker1 # [ 1.960174] [drm] features: -virgl +edid -resource_blob -host_visible1835worker1 # [ 1.960184] [drm] features: -context_init1836worker1 # [ 1.960931] [drm] number of scanouts: 11837worker1 # [ 1.960949] [drm] number of cap sets: 01838worker1 # [ 1.964427] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1839client # [ 1.921405] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1840worker2 # [ 1.919980] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1841worker2 # [ 1.920013] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01842lb # [ 1.967901] Console: switching to colour frame buffer device 160x501843lb # [ 1.974583] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1844client # [ 1.933007] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01845client # [ 1.933268] [drm] features: -virgl +edid -resource_blob -host_visible1846client # [ 1.933280] [drm] features: -context_init1847client # [ 1.933996] [drm] number of scanouts: 11848client # [ 1.934014] [drm] number of cap sets: 01849worker1 # [ 1.979051] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1850worker1 # [ 1.979080] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01851worker1 # [ 1.995188] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1852lb # [ 2.000549] systemd[1]: Finished Load Kernel Modules.1853client # [ 1.953377] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1854client # [ 1.953409] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01855lb # [ 2.004168] systemd[1]: Starting Apply Kernel Variables...1856worker2 # [ 1.950270] Console: switching to colour frame buffer device 160x501857worker1 # [ 2.001558] systemd[1]: Starting Create Static Device Nodes in /dev...1858client # [ 1.973564] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1859worker2 # [ 1.974722] systemd[1]: Finished Create Static Device Nodes in /dev.1860worker2 # [ 1.974947] systemd[1]: Reached target Preparation for Local File Systems.1861worker2 # [ 1.974972] systemd[1]: Reached target Local File Systems.1862worker1 # [ 2.014745] Console: switching to colour frame buffer device 160x501863worker1 # [ 2.021802] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1864worker2 # [ 1.979356] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1865worker2 # [ 1.982745] systemd[1]: Starting Rule-based Manager for Device Events and Files...1866client # [ 1.985396] systemd[1]: Starting Create Static Device Nodes in /dev...1867lb # [ 2.035873] systemd[1]: Finished Create Static Device Nodes in /dev.1868lb # [ 2.036206] systemd[1]: Reached target Preparation for Local File Systems.1869lb # [ 2.036243] systemd[1]: Reached target Local File Systems.1870lb # [ 2.041782] systemd[1]: Starting Rule-based Manager for Device Events and Files...1871worker2 # [ 1.994620] systemd[1]: Finished Load Kernel Modules.1872client # [ 1.990167] Console: switching to colour frame buffer device 160x501873worker1 # [ 2.048915] systemd[1]: Finished Load Kernel Modules.1874worker1 # [ 2.052573] systemd[1]: Starting Apply Kernel Variables...1875client # [ 2.009688] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1876lb # [ 2.071938] systemd[1]: Finished Apply Kernel Variables.1877client # [ 2.026483] systemd[1]: Finished Load Kernel Modules.1878worker2 # [ 2.026190] systemd[1]: Starting Apply Kernel Variables...1879worker1 # [ 2.071518] systemd[1]: Finished Create Static Device Nodes in /dev.1880worker1 # [ 2.074234] systemd[1]: Reached target Preparation for Local File Systems.1881client # [ 2.033479] systemd[1]: Starting Apply Kernel Variables...1882worker1 # [ 2.078580] systemd[1]: Reached target Local File Systems.1883lb # [ 2.072722] systemd-modules-load[74]: Inserted module 'dm_mod'1884lb # [ 2.089353] systemd[1]: Started Journal Service.1885lb # [ 2.077550] systemd-modules-load[74]: Module 'virtio_balloon' is built in1886worker1 # [ 2.088526] systemd[1]: Starting Rule-based Manager for Device Events and Files...1887lb # [ 2.078690] systemd-modules-load[74]: Module 'virtio_console' is built in1888lb # [ 2.079756] systemd-modules-load[74]: Inserted module 'virtio_gpu'1889lb # [ 2.092512] systemd-modules-load[74]: Module 'virtio_rng' is built in1890worker2 # [ 2.060285] systemd[1]: Started Journal Service.1891client # [ 2.061675] systemd[1]: Finished Create Static Device Nodes in /dev.1892lb # [ 2.096342] systemd[1]: Starting Create System Files and Directories...1893client # [ 2.061892] systemd[1]: Reached target Preparation for Local File Systems.1894client # [ 2.061928] systemd[1]: Reached target Local File Systems.1895worker2 # [ 2.052406] systemd-modules-load[74]: Inserted module 'dm_mod'1896worker2 # [ 2.053752] systemd-modules-load[74]: Module 'virtio_balloon' is built in1897worker2 # [ 2.060355] systemd-modules-load[74]: Module 'virtio_console' is built in1898client # [ 2.073490] systemd[1]: Starting Rule-based Manager for Device Events and Files...1899worker2 # [ 2.067630] systemd-modules-load[74]: Inserted module 'virtio_gpu'1900worker2 # [ 2.068840] systemd-modules-load[74]: Module 'virtio_rng' is built in1901worker1 # [ 2.126882] systemd[1]: Finished Apply Kernel Variables.1902worker2 # [ 2.077619] systemd[1]: Starting Create System Files and Directories...1903lb # [ 2.126131] systemd-udevd[80]: Using default interface naming scheme 'v261'.1904worker2 # [ 2.081732] systemd[1]: Finished Apply Kernel Variables.1905worker1 # [ 2.140660] systemd[1]: Started Journal Service.1906worker1 # [ 2.128000] systemd-modules-load[74]: Inserted module 'dm_mod'1907worker1 # [ 2.131395] systemd-modules-load[74]: Module 'virtio_balloon' is built in1908client # [ 2.105397] systemd[1]: Finished Apply Kernel Variables.1909client # [ 2.095185] systemd-modules-load[74]: Inserted module 'dm_mod'1910client # [ 2.096543] systemd-modules-load[74]: Module 'virtio_balloon' is built in1911worker1 # [ 2.140584] systemd-modules-load[74]: Module 'virtio_console' is built in1912worker1 # [ 2.141782] systemd-modules-load[74]: Inserted module 'virtio_gpu'1913client # [ 2.110472] systemd[1]: Started Journal Service.1914worker1 # [ 2.142803] systemd-modules-load[74]: Module 'virtio_rng' is built in1915worker1 # [ 2.143816] systemd[1]: Starting Create System Files and Directories...1916worker2 # [ 2.101193] systemd-udevd[79]: Using default interface naming scheme 'v261'.1917lb # [ 2.149514] systemd[1]: Finished Create System Files and Directories.1918client # [ 2.105608] systemd-modules-load[74]: Module 'virtio_console' is built in1919client # [ 2.112424] systemd-modules-load[74]: Inserted module 'virtio_gpu'1920client # [ 2.117256] systemd-modules-load[74]: Module 'virtio_rng' is built in1921worker2 # [ 2.114959] systemd[1]: Finished Create System Files and Directories.1922client # [ 2.120433] systemd[1]: Starting Create System Files and Directories...1923lb # [ 2.168408] systemd[1]: Started Rule-based Manager for Device Events and Files.1924worker2 # [ 2.138316] systemd[1]: Started Rule-based Manager for Device Events and Files.1925worker1 # [ 2.200751] systemd[1]: Finished Create System Files and Directories.1926worker1 # [ 2.204964] systemd-udevd[81]: Using default interface naming scheme 'v261'.1927client # [ 2.165403] systemd-udevd[81]: Using default interface naming scheme 'v261'.1928client # [ 2.173801] systemd[1]: Finished Create System Files and Directories.1929lb # [ 2.224375] systemd[1]: Starting Virtual Console Setup...1930worker1 # [ 2.235197] systemd[1]: Started Rule-based Manager for Device Events and Files.1931client # [ 2.195192] systemd[1]: Started Rule-based Manager for Device Events and Files.1932worker2 # [ 2.200260] systemd[1]: Starting Virtual Console Setup...1933lb # [ 2.280596] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1934lb # [ 2.284251] systemd[1]: Finished Virtual Console Setup.1935client # [ 2.260164] systemd[1]: Starting Virtual Console Setup...1936worker1 # [ 2.305015] systemd[1]: Starting Virtual Console Setup...1937worker2 # [ 2.260590] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1938worker2 # [ 2.264412] systemd[1]: Finished Virtual Console Setup.1939client # [ 2.312587] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1940client # [ 2.316220] systemd[1]: Finished Virtual Console Setup.1941worker1 # [ 2.364577] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1942worker1 # [ 2.368169] systemd[1]: Finished Virtual Console Setup.1943lb # [ 2.887459] systemd[1]: Finished Coldplug All udev Devices.1944lb # [ 2.888947] systemd[1]: Reached target System Initialization.1945lb # [ 2.892198] systemd[1]: Reached target Basic System.1946worker2 # [ 2.860736] systemd[1]: Finished Coldplug All udev Devices.1947worker2 # [ 2.861791] systemd[1]: Reached target System Initialization.1948worker2 # [ 2.862674] systemd[1]: Reached target Basic System.1949client # [ 2.900777] systemd[1]: Finished Coldplug All udev Devices.1950client # [ 2.901763] systemd[1]: Reached target System Initialization.1951client # [ 2.902588] systemd[1]: Reached target Basic System.1952worker1 # [ 2.977141] systemd[1]: Finished Coldplug All udev Devices.1953worker1 # [ 2.978767] systemd[1]: Reached target System Initialization.1954worker1 # [ 2.979608] systemd[1]: Reached target Basic System.1955worker2 # [ 2.999654] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1956lb # [ 3.059150] (udev-worker)[105]: Network interface NamePolicy= disabled on kernel command line.1957worker2 # [ 3.027300] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1958lb # [ 3.085867] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1959worker2 # [ 3.042828] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1960lb # [ 3.092193] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1961client # [ 3.055265] (udev-worker)[107]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1962client # [ 3.066201] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1963client # [ 3.081533] (udev-worker)[107]: Network interface NamePolicy= disabled on kernel command line.1964worker1 # [ 3.144508] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1965lb # [ 3.149563] systemd[1]: Found device /dev/disk/by-label/nixos.1966lb # [ 3.156082] systemd[1]: Reached target Initrd Root Device.1967lb # [ 3.161574] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1968worker1 # [ 3.159006] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1969worker2 # [ 3.117659] systemd[1]: Found device /dev/disk/by-label/nixos.1970worker2 # [ 3.123568] systemd[1]: Reached target Initrd Root Device.1971worker1 # [ 3.173462] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1972worker2 # [ 3.132085] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1973client # [ 3.144906] systemd[1]: Found device /dev/disk/by-label/nixos.1974client # [ 3.149119] systemd[1]: Reached target Initrd Root Device.1975client # [ 3.152534] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1976lb # [ 3.216357] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1977lb # [ 3.224135] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1978worker2 # [ 3.178356] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1979lb # [ 3.228176] systemd[1]: Mounting /sysroot...1980worker1 # [ 3.231641] systemd[1]: Found device /dev/disk/by-label/nixos.1981worker1 # [ 3.234108] systemd[1]: Reached target Initrd Root Device.1982worker2 # [ 3.191231] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1983worker1 # [ 3.239044] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1984worker2 # [ 3.196315] systemd[1]: Mounting /sysroot...1985client # [ 3.206589] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1986client # [ 3.215304] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1987client # [ 3.220655] systemd[1]: Mounting /sysroot...1988lb # [ 3.281533] EXT4-fs (vda): mounted filesystem 4c362ea9-7add-44cf-9ffb-8a8a2fffb806 r/w with ordered data mode. Quota mode: none.1989lb # [ 3.271797] systemd[1]: Mounted /sysroot.1990lb # [ 3.275378] systemd[1]: Reached target Initrd Root File System.1991worker2 # [ 3.242647] EXT4-fs (vda): mounted filesystem 74781864-2c89-4177-a820-f0e630c6f5f0 r/w with ordered data mode. Quota mode: none.1992lb # [ 3.279335] systemd[1]: Starting Mountpoints Configured in the Real Root...1993worker2 # [ 3.234910] systemd[1]: Mounted /sysroot.1994worker2 # [ 3.238076] systemd[1]: Reached target Initrd Root File System.1995worker2 # [ 3.247084] systemd[1]: Starting Mountpoints Configured in the Real Root...1996worker1 # [ 3.298272] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1997client # [ 3.273170] EXT4-fs (vda): mounted filesystem 0a51723e-468e-497e-80d9-f812fedad80f r/w with ordered data mode. Quota mode: none.1998client # [ 3.265035] systemd[1]: Mounted /sysroot.1999lb # [ 3.311060] systemd-sysroot-fstab-check[119]: /sysroot should be mounted in the initrd, will request daemon-reload.2000worker1 # [ 3.309704] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.
loading 1,591 hidden lines…
3592worker1 # [ 23.199193] nix-grpc-daemon[777]: ts=2026-09-20T08:17:27Z level=info event=scheduler_connected addr=lb:500513593lb # [ 23.200770] envoy[931]: [2026-09-20 08:17:27.739][931][info][config] [source/common/listener_manager/listener_manager_impl.cc:1067] all dependencies initialized. starting workers3594worker2 # [ 23.173035] nix-grpc-daemon[778]: ts=2026-09-20T08:17:27Z level=info event=scheduler_connected addr=lb:500513595worker2 # [ 23.183845] nix-grpc-daemon[778]: ts=2026-09-20T08:17:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3596worker1 # [ 23.227600] nix-grpc-daemon[777]: ts=2026-09-20T08:17:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3597worker2 # [ 23.385558] nix-grpc-daemon[778]: ts=2026-09-20T08:17:27Z level=info event=scheduler_connected addr=lb:500513598worker1 # [ 23.429201] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513599worker1 # [ 23.437397] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3600worker2 # [ 23.393675] nix-grpc-daemon[778]: ts=2026-09-20T08:17:27Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3601worker2 # [ 23.594765] nix-grpc-daemon[778]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513602worker1 # [ 23.638555] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513603worker2 # [ 23.602868] nix-grpc-daemon[778]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3604worker1 # [ 23.647021] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3605worker2 # [ 23.804270] nix-grpc-daemon[778]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513606worker1 # [ 23.848746] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513607worker2 # [ 23.811545] nix-grpc-daemon[778]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3608worker1 # [ 23.855737] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3609worker2 # [ 24.012821] nix-grpc-daemon[778]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513610worker1 # [ 24.061782] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513611worker2 # [ 24.020429] nix-grpc-daemon[778]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3612worker1 # [ 24.066443] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3613worker1 # [ 24.262762] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513614worker2 # [ 24.227263] nix-grpc-daemon[778]: ts=2026-09-20T08:17:28Z level=info event=scheduler_connected addr=lb:500513615worker1 # [ 24.270502] nix-grpc-daemon[777]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3616worker2 # [ 24.231687] nix-grpc-daemon[778]: ts=2026-09-20T08:17:28Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3617worker2 # [ 24.426853] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513618worker1 # [ 24.471691] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513619worker1 # [ 24.479294] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3620worker2 # [ 24.435713] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3621worker2 # [ 24.637102] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513622worker1 # [ 24.681077] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513623worker2 # [ 24.644880] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3624worker1 # [ 24.689327] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3625worker2 # [ 24.846257] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513626worker1 # [ 24.890569] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513627worker1 # [ 24.898004] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3628worker2 # [ 24.854259] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3629worker2 # [ 25.055693] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513630worker1 # [ 25.100855] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513631worker2 # [ 25.064140] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3632worker1 # [ 25.108191] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3633worker1 # [ 25.309424] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513634worker2 # [ 25.268643] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_connected addr=lb:500513635worker1 # [ 25.315727] nix-grpc-daemon[777]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3636worker2 # [ 25.274360] nix-grpc-daemon[778]: ts=2026-09-20T08:17:29Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3637worker1 # [ 25.517046] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513638worker2 # [ 25.475599] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513639worker1 # [ 25.524530] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3640worker2 # [ 25.482974] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3641worker2 # [ 25.683982] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513642worker1 # [ 25.726241] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513643worker2 # [ 25.689077] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3644worker1 # [ 25.733901] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3645worker1 # [ 25.935147] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513646worker2 # [ 25.891015] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513647worker1 # [ 25.942709] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3648worker2 # [ 25.898897] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3649worker2 # [ 26.099557] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513650worker1 # [ 26.146925] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513651worker2 # [ 26.106905] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3652worker1 # [ 26.152836] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3653worker1 # [ 26.354924] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513654worker2 # [ 26.309857] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_connected addr=lb:500513655worker2 # [ 26.318186] nix-grpc-daemon[778]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3656worker1 # [ 26.362240] nix-grpc-daemon[777]: ts=2026-09-20T08:17:30Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3657worker1 # [ 26.562728] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513658worker2 # [ 26.520316] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513659worker1 # [ 26.570617] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3660worker2 # [ 26.527501] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3661worker1 # [ 26.772004] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513662worker2 # [ 26.729570] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513663worker1 # [ 26.780198] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3664worker2 # [ 26.737930] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3665worker1 # [ 26.981631] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513666worker2 # [ 26.939272] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513667worker1 # [ 26.989172] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3668worker2 # [ 26.947060] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3669worker2 # [ 27.149535] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513670worker1 # [ 27.195889] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513671worker1 # [ 27.200523] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3672worker2 # [ 27.156890] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3673worker1 # [ 27.396893] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513674worker2 # [ 27.362256] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_connected addr=lb:500513675worker1 # [ 27.404548] nix-grpc-daemon[777]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3676worker2 # [ 27.366771] nix-grpc-daemon[778]: ts=2026-09-20T08:17:31Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3677worker1 # [ 27.606190] nix-grpc-daemon[777]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513678worker2 # [ 27.562814] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513679worker1 # [ 27.614056] nix-grpc-daemon[777]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3680worker2 # [ 27.570761] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3681worker1 # [ 27.815548] nix-grpc-daemon[777]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513682worker2 # [ 27.773015] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513683worker1 # [ 27.823319] nix-grpc-daemon[777]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3684worker2 # [ 27.780847] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3685worker1 # [ 28.024415] nix-grpc-daemon[777]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513686worker2 # [ 27.982419] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513687worker1 # [ 28.032116] nix-grpc-daemon[777]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3688worker2 # [ 27.989087] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3689worker2 # [ 28.189732] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513690worker1 # [ 28.235185] nix-grpc-daemon[777]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513691worker2 # [ 28.196525] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3692worker1 # [ 28.241035] nix-grpc-daemon[777]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3693worker1 # [ 28.441871] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513694worker2 # [ 28.400680] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_connected addr=lb:500513695worker1 # [ 28.448359] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3696worker2 # [ 28.405989] nix-grpc-daemon[778]: ts=2026-09-20T08:17:32Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3697worker1 # [ 28.648983] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513698worker2 # [ 28.606821] nix-grpc-daemon[778]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513699worker1 # [ 28.655627] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3700worker2 # [ 28.613746] nix-grpc-daemon[778]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3701worker1 # [ 28.856924] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513702worker2 # [ 28.815200] nix-grpc-daemon[778]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513703worker1 # [ 28.864092] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3704worker2 # [ 28.822295] nix-grpc-daemon[778]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3705worker1 # [ 29.064969] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513706worker2 # [ 29.023520] nix-grpc-daemon[778]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513707worker1 # [ 29.069072] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3708worker2 # [ 29.030223] nix-grpc-daemon[778]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3709worker1 # [ 29.270511] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513710worker1 # [ 29.273922] nix-grpc-daemon[777]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3711worker2 # [ 29.231225] nix-grpc-daemon[778]: ts=2026-09-20T08:17:33Z level=info event=scheduler_connected addr=lb:500513712worker2 # [ 29.238170] nix-grpc-daemon[778]: ts=2026-09-20T08:17:33Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3713worker1 # [ 29.474848] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513714worker1 # [ 29.481656] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3715worker2 # [ 29.442204] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513716worker2 # [ 29.446471] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3717worker1 # [ 29.682746] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513718worker2 # [ 29.642956] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513719worker1 # [ 29.689586] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3720worker2 # [ 29.649954] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3721worker1 # [ 29.890855] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513722worker2 # [ 29.851148] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513723worker1 # [ 29.898098] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3724worker2 # [ 29.858265] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3725worker1 # [ 30.099438] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513726worker2 # [ 30.059257] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513727worker1 # [ 30.106595] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3728worker2 # [ 30.065972] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3729worker1 # [ 30.309268] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513730worker2 # [ 30.267014] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_connected addr=lb:500513731worker1 # [ 30.315970] nix-grpc-daemon[777]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3732worker2 # [ 30.275021] nix-grpc-daemon[778]: ts=2026-09-20T08:17:34Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3733worker1 # [ 30.517185] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513734worker2 # [ 30.478960] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513735worker1 # [ 30.524693] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3736worker2 # [ 30.483317] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3737worker2 # [ 30.679657] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513738worker1 # [ 30.725731] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513739worker1 # [ 30.732270] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3740worker2 # [ 30.688751] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3741worker1 # [ 30.932786] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513742worker2 # [ 30.889986] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513743worker1 # [ 30.940299] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3744worker2 # [ 30.896536] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3745worker2 # [ 31.097854] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513746worker1 # [ 31.141599] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513747worker2 # [ 31.104877] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3748worker1 # [ 31.148976] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3749worker2 # [ 31.305911] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513750worker1 # [ 31.353285] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_connected addr=lb:500513751worker2 # [ 31.311872] nix-grpc-daemon[778]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3752worker1 # [ 31.358758] nix-grpc-daemon[777]: ts=2026-09-20T08:17:35Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3753worker2 # [ 31.514056] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513754worker1 # [ 31.560474] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513755worker2 # [ 31.520372] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3756worker1 # [ 31.566346] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3757worker2 # [ 31.721106] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513758worker1 # [ 31.767125] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513759worker2 # [ 31.729970] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3760worker1 # [ 31.773693] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3761worker1 # [ 31.974224] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513762worker2 # [ 31.931226] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513763worker1 # [ 31.981338] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3764worker2 # [ 31.938865] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3765worker1 # [ 32.182307] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513766worker2 # [ 32.140315] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513767worker1 # [ 32.189266] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3768worker2 # [ 32.146887] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3769worker2 # [ 32.349238] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513770worker1 # [ 32.393043] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_connected addr=lb:500513771worker2 # [ 32.354238] nix-grpc-daemon[778]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3772worker1 # [ 32.398869] nix-grpc-daemon[777]: ts=2026-09-20T08:17:36Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3773worker1 # [ 32.599752] nix-grpc-daemon[777]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513774worker2 # [ 32.556334] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513775worker1 # [ 32.607124] nix-grpc-daemon[777]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3776worker2 # [ 32.562563] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3777worker2 # [ 32.763570] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513778worker1 # [ 32.808466] nix-grpc-daemon[777]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513779worker2 # [ 32.769971] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3780worker1 # [ 32.815054] nix-grpc-daemon[777]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3781worker2 # [ 32.970718] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513782worker1 # [ 33.016000] nix-grpc-daemon[777]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513783worker2 # [ 32.978296] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3784worker1 # [ 33.022493] nix-grpc-daemon[777]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3785worker2 # [ 33.179225] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513786worker1 # [ 33.223145] nix-grpc-daemon[777]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513787worker2 # [ 33.186212] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3788worker1 # [ 33.230053] nix-grpc-daemon[777]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3789worker2 # [ 33.387428] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_connected addr=lb:500513790worker1 # [ 33.432709] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513791worker1 # [ 33.438718] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3792worker2 # [ 33.395664] nix-grpc-daemon[778]: ts=2026-09-20T08:17:37Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3793worker1 # [ 33.639295] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513794worker2 # [ 33.598883] nix-grpc-daemon[778]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513795worker1 # [ 33.647585] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3796worker2 # [ 33.604741] nix-grpc-daemon[778]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3797worker1 # [ 33.848958] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513798worker2 # [ 33.805717] nix-grpc-daemon[778]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513799worker2 # [ 33.812698] nix-grpc-daemon[778]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3800worker1 # [ 33.856582] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3801worker1 # [ 34.057770] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513802worker2 # [ 34.013943] nix-grpc-daemon[778]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513803worker2 # [ 34.021193] nix-grpc-daemon[778]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3804worker1 # [ 34.065178] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3805worker1 # [ 34.266280] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513806worker2 # [ 34.222501] nix-grpc-daemon[778]: ts=2026-09-20T08:17:38Z level=info event=scheduler_connected addr=lb:500513807worker2 # [ 34.229803] nix-grpc-daemon[778]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3808worker1 # [ 34.273861] nix-grpc-daemon[777]: ts=2026-09-20T08:17:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3809worker2 # [ 34.431063] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513810worker1 # [ 34.476557] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513811worker1 # [ 34.482545] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3812worker2 # [ 34.438985] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3813worker1 # [ 34.683622] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513814worker2 # [ 34.643704] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513815worker1 # [ 34.690730] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3816worker2 # [ 34.648318] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3817worker2 # [ 34.844920] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513818worker1 # [ 34.891798] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513819worker2 # [ 34.851619] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3820worker1 # [ 34.897602] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3821worker2 # [ 35.052203] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513822worker1 # [ 35.098938] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513823worker2 # [ 35.059214] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3824worker1 # [ 35.104798] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3825worker2 # [ 35.259212] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513826worker1 # [ 35.305477] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_connected addr=lb:500513827worker2 # [ 35.265920] nix-grpc-daemon[778]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3828worker1 # [ 35.311097] nix-grpc-daemon[777]: ts=2026-09-20T08:17:39Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3829worker2 # [ 35.466505] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513830worker1 # [ 35.513313] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513831worker1 # [ 35.519447] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3832worker2 # [ 35.476525] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3833worker1 # [ 35.720283] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513834worker2 # [ 35.680436] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513835worker2 # [ 35.682983] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3836worker1 # [ 35.727524] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3837worker1 # [ 35.928966] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513838worker2 # [ 35.882489] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513839worker1 # [ 35.933192] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3840worker2 # [ 35.889347] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3841worker2 # [ 36.089852] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513842worker1 # [ 36.134424] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513843worker2 # [ 36.097002] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3844worker1 # [ 36.141928] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3845worker2 # [ 36.297366] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513846worker1 # [ 36.343306] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_connected addr=lb:500513847worker1 # [ 36.349495] nix-grpc-daemon[777]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3848worker2 # [ 36.306509] nix-grpc-daemon[778]: ts=2026-09-20T08:17:40Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3849worker2 # [ 36.507768] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513850worker1 # [ 36.553508] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513851worker1 # [ 36.557816] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3852worker2 # [ 36.514774] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3853worker1 # [ 36.754492] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513854worker2 # [ 36.718993] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513855worker1 # [ 36.761481] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3856worker2 # [ 36.723285] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3857worker1 # [ 36.962606] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513858worker2 # [ 36.919924] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513859worker1 # [ 36.969648] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3860worker2 # [ 36.927402] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3861worker1 # [ 37.170748] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513862worker2 # [ 37.128940] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513863worker1 # [ 37.177648] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3864worker2 # [ 37.135714] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3865worker1 # [ 37.378875] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513866worker2 # [ 37.336828] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_connected addr=lb:500513867worker1 # [ 37.385936] nix-grpc-daemon[777]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3868worker2 # [ 37.343582] nix-grpc-daemon[778]: ts=2026-09-20T08:17:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3869worker2 # [ 37.544674] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513870worker1 # [ 37.590208] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513871worker1 # [ 37.594597] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3872worker2 # [ 37.551289] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3873worker1 # [ 37.791193] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513874worker2 # [ 37.754735] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513875worker1 # [ 37.798167] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3876worker2 # [ 37.759721] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3877worker1 # [ 37.999359] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513878worker2 # [ 37.956797] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513879worker1 # [ 38.006830] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3880worker2 # [ 37.963432] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3881worker2 # [ 38.164561] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513882worker1 # [ 38.207744] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513883worker1 # [ 38.214683] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3884worker2 # [ 38.172137] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3885worker1 # [ 38.415577] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513886worker2 # [ 38.373335] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_connected addr=lb:500513887worker1 # [ 38.422355] nix-grpc-daemon[777]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3888worker2 # [ 38.379852] nix-grpc-daemon[778]: ts=2026-09-20T08:17:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3889worker2 # [ 38.580579] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513890worker1 # [ 38.624604] nix-grpc-daemon[777]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513891worker1 # [ 38.631472] nix-grpc-daemon[777]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3892worker2 # [ 38.587862] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3893worker1 # [ 38.832704] nix-grpc-daemon[777]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513894worker2 # [ 38.790188] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513895worker1 # [ 38.839574] nix-grpc-daemon[777]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3896worker2 # [ 38.796963] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3897worker2 # [ 38.997746] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513898worker1 # [ 39.040692] nix-grpc-daemon[777]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513899worker2 # [ 39.001874] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3900worker1 # [ 39.047492] nix-grpc-daemon[777]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3901worker1 # [ 39.248315] nix-grpc-daemon[777]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513902worker2 # [ 39.202852] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513903worker1 # [ 39.254226] nix-grpc-daemon[777]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3904worker2 # [ 39.209308] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3905worker2 # [ 39.410326] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_connected addr=lb:500513906worker1 # [ 39.455585] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513907worker1 # [ 39.461688] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3908worker2 # [ 39.418014] nix-grpc-daemon[778]: ts=2026-09-20T08:17:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3909worker2 # [ 39.619441] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513910worker1 # [ 39.663936] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513911worker1 # [ 39.670519] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3912worker2 # [ 39.626940] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3913worker1 # [ 39.871072] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513914worker2 # [ 39.831174] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513915worker1 # [ 39.878583] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3916worker2 # [ 39.835950] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3917worker2 # [ 40.031717] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513918worker1 # [ 40.079816] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513919worker2 # [ 40.038255] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3920worker1 # [ 40.085380] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3921worker2 # [ 40.239363] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513922worker1 # [ 40.286239] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_connected addr=lb:500513923worker2 # [ 40.245945] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3924worker1 # [ 40.291965] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3925worker1 # [ 40.363056] nix-grpc-daemon[777]: ts=2026-09-20T08:17:44Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"3926worker2 # [ 40.331401] nix-grpc-daemon[778]: ts=2026-09-20T08:17:44Z level=info event=scheduler_lock_lost error="error: http://lb:5751/api/farm/lead: Timeout was reached"3927worker2 # [ 40.446633] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513928worker1 # [ 40.492806] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513929worker1 # [ 40.499120] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3930worker2 # [ 40.456087] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3931worker2 # [ 40.656649] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513932worker1 # [ 40.702079] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513933worker2 # [ 40.662488] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3934worker1 # [ 40.707568] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3935worker1 # [ 40.908485] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513936worker2 # [ 40.865059] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513937worker1 # [ 40.914532] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3938worker2 # [ 40.871860] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3939worker1 # [ 41.115425] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513940worker2 # [ 41.072950] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513941worker1 # [ 41.122329] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3942worker2 # [ 41.079442] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3943worker1 # [ 41.323386] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513944worker2 # [ 41.280628] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_connected addr=lb:500513945worker1 # [ 41.330461] nix-grpc-daemon[777]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3946worker2 # [ 41.287546] nix-grpc-daemon[778]: ts=2026-09-20T08:17:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3947worker1 # [ 41.531610] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513948worker2 # [ 41.488890] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513949worker1 # [ 41.538541] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3950worker2 # [ 41.495622] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3951worker2 # [ 41.697973] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513952worker1 # [ 41.742884] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513953worker2 # [ 41.703080] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3954worker1 # [ 41.747181] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3955worker1 # [ 41.943602] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513956worker2 # [ 41.903346] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513957worker1 # [ 41.950591] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3958worker2 # [ 41.907839] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3959worker2 # [ 42.104552] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513960worker1 # [ 42.151652] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513961worker2 # [ 42.111223] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3962worker1 # [ 42.157249] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3963worker2 # [ 42.311629] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513964worker1 # [ 42.358676] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_connected addr=lb:500513965worker2 # [ 42.318170] nix-grpc-daemon[778]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3966worker1 # [ 42.364603] nix-grpc-daemon[777]: ts=2026-09-20T08:17:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3967worker2 # [ 42.518315] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513968worker1 # [ 42.565327] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513969worker1 # [ 42.569978] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3970worker2 # [ 42.524949] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3971worker2 # [ 42.725590] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513972worker1 # [ 42.772667] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513973worker2 # [ 42.732484] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3974worker1 # [ 42.777894] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3975worker2 # [ 42.934305] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513976worker1 # [ 42.979497] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513977worker2 # [ 42.940721] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3978worker1 # [ 42.985118] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3979worker2 # [ 43.141508] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513980worker1 # [ 43.185982] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513981worker1 # [ 43.192360] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3982worker2 # [ 43.148771] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3983worker1 # [ 43.393206] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513984worker2 # [ 43.349914] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_connected addr=lb:500513985worker1 # [ 43.400299] nix-grpc-daemon[777]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3986worker2 # [ 43.357037] nix-grpc-daemon[778]: ts=2026-09-20T08:17:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3987worker1 # [ 43.601726] nix-grpc-daemon[777]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500513988worker2 # [ 43.558359] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500513989worker2 # [ 43.566112] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3990worker1 # [ 43.610065] nix-grpc-daemon[777]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3991worker2 # [ 43.767216] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500513992worker1 # [ 43.812494] nix-grpc-daemon[777]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500513993worker1 # [ 43.818821] nix-grpc-daemon[777]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3994worker2 # [ 43.776140] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3995worker1 # [ 44.019956] nix-grpc-daemon[777]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500513996worker2 # [ 43.980645] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500513997worker1 # [ 44.027138] nix-grpc-daemon[777]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3998worker2 # [ 43.985050] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"3999worker2 # [ 44.181468] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500514000worker1 # [ 44.227908] nix-grpc-daemon[777]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500514001worker2 # [ 44.188914] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4002worker1 # [ 44.233515] nix-grpc-daemon[777]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4003worker2 # [ 44.389771] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_connected addr=lb:500514004worker1 # [ 44.434369] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514005worker2 # [ 44.397192] nix-grpc-daemon[778]: ts=2026-09-20T08:17:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4006worker1 # [ 44.441009] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4007worker1 # [ 44.641836] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514008worker2 # [ 44.598375] nix-grpc-daemon[778]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514009worker2 # [ 44.605598] nix-grpc-daemon[778]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4010worker1 # [ 44.649511] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4011worker2 # [ 44.806837] nix-grpc-daemon[778]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514012worker1 # [ 44.853130] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514013worker2 # [ 44.813526] nix-grpc-daemon[778]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4014worker1 # [ 44.858483] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4015worker1 # [ 45.059142] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514016worker2 # [ 45.015484] nix-grpc-daemon[778]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514017worker1 # [ 45.066464] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4018worker2 # [ 45.022704] nix-grpc-daemon[778]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4019worker1 # [ 45.267455] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514020worker2 # [ 45.223745] nix-grpc-daemon[778]: ts=2026-09-20T08:17:49Z level=info event=scheduler_connected addr=lb:500514021worker2 # [ 45.230684] nix-grpc-daemon[778]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4022worker1 # [ 45.274607] nix-grpc-daemon[777]: ts=2026-09-20T08:17:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4023worker1 # [ 45.475644] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514024worker2 # [ 45.431853] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514025worker1 # [ 45.482637] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4026worker2 # [ 45.438856] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4027worker2 # [ 45.639601] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514028worker1 # [ 45.683684] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514029worker2 # [ 45.646726] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4030worker1 # [ 45.689802] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4031worker1 # [ 45.891033] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514032worker2 # [ 45.847807] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514033worker1 # [ 45.895652] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4034worker2 # [ 45.854301] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4035worker1 # [ 46.096967] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514036worker2 # [ 46.058174] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514037worker1 # [ 46.105024] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4038worker2 # [ 46.062476] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4039worker2 # [ 46.258629] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514040worker1 # [ 46.305838] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_connected addr=lb:500514041worker2 # [ 46.264928] nix-grpc-daemon[778]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4042worker1 # [ 46.311256] nix-grpc-daemon[777]: ts=2026-09-20T08:17:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4043worker2 # [ 46.465600] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514044worker1 # [ 46.512189] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514045worker2 # [ 46.472093] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4046worker1 # [ 46.517804] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4047worker2 # [ 46.672712] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514048worker1 # [ 46.718706] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514049worker1 # [ 46.724934] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4050worker2 # [ 46.682125] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4051worker2 # [ 46.883457] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514052worker1 # [ 46.927854] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514053worker1 # [ 46.932955] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4054worker2 # [ 46.890594] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4055worker1 # [ 47.129754] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514056worker2 # [ 47.094716] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514057worker1 # [ 47.137400] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4058worker2 # [ 47.099716] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4059worker1 # [ 47.338611] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514060worker2 # [ 47.295727] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_connected addr=lb:500514061worker1 # [ 47.345602] nix-grpc-daemon[777]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4062worker2 # [ 47.302565] nix-grpc-daemon[778]: ts=2026-09-20T08:17:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4063worker1 # [ 47.546828] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514064worker2 # [ 47.503905] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514065worker1 # [ 47.554030] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4066worker2 # [ 47.510547] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4067worker2 # [ 47.711291] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514068worker1 # [ 47.755171] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514069worker1 # [ 47.762392] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4070worker2 # [ 47.718687] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4071lb # [ 47.924918] rustfs-bucket-start[1162]: mb s3://farm4072lb # [ 47.936681] systemd[1]: Finished rustfs-bucket.service.4073lb # [ 47.950251] systemd[1]: Starting niks3 server...4074worker1 # [ 47.965066] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514075worker2 # [ 47.919833] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514076worker2 # [ 47.926794] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4077worker1 # [ 47.971026] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4078lb # [ 48.165519] postgres[1179]: [1179] ERROR: relation "goose_db_version" does not exist at character 364079lb # [ 48.166896] postgres[1179]: [1179] STATEMENT: SELECT version_id, is_applied from goose_db_version ORDER BY id DESC4080worker1 # [ 48.172203] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514081worker2 # [ 48.131004] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514082worker1 # [ 48.178652] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4083worker2 # [ 48.135362] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4084lb # [ 48.199948] niks3-server[1172]: 2026/09/20 08:17:52 OK 20241026095416_initial_model.sql (19.17ms)4085lb # [ 48.242639] niks3-server[1172]: 2026/09/20 08:17:52 OK 20251210153512_drop_unused_gin_index.sql (43.02ms)4086lb # [ 48.250257] niks3-server[1172]: 2026/09/20 08:17:52 OK 20251218171726_add_pins.sql (7.52ms)4087lb # [ 48.257213] niks3-server[1172]: 2026/09/20 08:17:52 OK 20260628120000_add_object_size_and_stats.sql (6.91ms)4088lb # [ 48.264472] niks3-server[1172]: 2026/09/20 08:17:52 OK 20260905000000_add_claims.sql (7.23ms)4089lb # [ 48.270965] niks3-server[1172]: 2026/09/20 08:17:52 OK 20260920000000_drop_claims.sql (6.33ms)4090lb # [ 48.273555] niks3-server[1172]: 2026/09/20 08:17:52 goose: successfully migrated database to version: 202609200000004091lb # [ 48.280891] niks3-server[1172]: 2026/09/20 08:17:52 OK 1_commit_pending_closure.sql (9.91ms)4092lb # [ 48.284444] niks3-server[1172]: 2026/09/20 08:17:52 OK 2_object_stats_trigger.sql (3.37ms)4093lb # [ 48.285890] niks3-server[1172]: 2026/09/20 08:17:52 goose: up to current file version: 24094lb # [ 48.295188] niks3-server[1172]: 2026/09/20 08:17:52 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key4095lb # [ 48.358696] niks3-server[1172]: 2026/09/20 08:17:52 INFO Read proxy enabled — serving cache objects from S34096lb # [ 48.360397] niks3-server[1172]: 2026/09/20 08:17:52 INFO Using socket-activated listener address=0.0.0.0:57514097lb # [ 48.362597] niks3-server[1172]: 2026/09/20 08:17:52 INFO systemd watchdog enabled interval=15s4098lb # [ 48.367565] niks3-server[1172]: 2026/09/20 08:17:52 INFO Starting HTTP server address=0.0.0.0:57514099lb # [ 48.371748] systemd[1]: Started niks3 server.4100lb # [ 48.375634] niks3-server[1172]: 2026/09/20 08:17:52 INFO lead: acquired remote=192.168.1.4:512564101worker2 # [ 48.328997] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_take_over4102lb # [ 48.383457] systemd[1]: Reached target Multi-User System.4103worker2 # [ 48.333900] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514104worker1 # [ 48.379451] nix-grpc-daemon[777]: ts=2026-09-20T08:17:52Z level=info event=scheduler_connected addr=lb:500514105lb # [ 48.387981] systemd[1]: Startup finished in 996ms (kernel) + 5.459s (initrd) + 41.910s (userspace) = 48.366s.4106worker2 # [ 48.340882] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04107worker2 # [ 48.347768] nix-grpc-daemon[778]: ts=2026-09-20T08:17:52Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4108worker2 # [ 48.548573] nix-grpc-daemon[778]: ts=2026-09-20T08:17:53Z level=info event=scheduler_connected addr=lb:500514109worker2 # [ 48.554004] nix-grpc-daemon[778]: ts=2026-09-20T08:17:53Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4110worker2 # [ 48.754772] nix-grpc-daemon[778]: ts=2026-09-20T08:17:53Z level=info event=scheduler_connected addr=lb:500514111worker2 # [ 48.760479] nix-grpc-daemon[778]: ts=2026-09-20T08:17:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04112lb: (finished: waiting for unit niks3.service, in 35.26 seconds)4113lb: waiting for TCP port 5751 on localhost4114lb # Connection to localhost (127.0.0.1) 5751 port [tcp/*] succeeded!4115lb: (finished: waiting for TCP port 5751 on localhost, in 0.09 seconds)4116lb: waiting for unit envoy.service4117lb: (finished: waiting for unit envoy.service, in 0.07 seconds)4118lb: waiting for TCP port 50051 on localhost4119lb # Connection to localhost (::1) 50051 port [tcp/*] succeeded!4120lb # [ 49.695661] envoy[931]: conn peer=[::1]:49250 tls=TLSv1.2 subject="-" sni=- flags=- tls_fail="-" rx=0 tx=0 ms=44121lb: (finished: waiting for TCP port 50051 on localhost, in 0.05 seconds)4122lb: waiting until exactly no endpoints of aarch64-linux are unhealthy4123??? Warning (UserWarning): retry(): The 'timeout_seconds' argument is deprecated. Use 'timeout' instead.4124 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 1284125lb: must succeed: curl -sf localhost:9901/clusters4126lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4127lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.10 seconds)4128subtest: one node schedules and both builders hold a WorkerSession on it4129lb: must succeed: curl -sf localhost:9901/clusters4130lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4131worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true4132worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.11 seconds)4133(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 0.16 seconds)4134subtest: balancer auth: client cert, bearer token, nothing, unknown CN4135worker2 # [ 50.465115] nix-grpc-daemon[778]: ts=2026-09-20T08:17:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:498024136worker2 # [ 50.478913] systemd[1]: Started Nix Daemon.4137worker2 # [ 50.754948] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4138worker2 # [ 50.794686] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 1/5)4139worker2 # [ 50.847564] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 150 ms (attempt 2/5)4140worker2 # [ 50.998127] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 343 ms (attempt 3/5)4141worker2 # [ 51.348894] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 511 ms (attempt 4/5)4142worker2 # [ 51.870880] nix-grpc-daemon[778]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4143worker1 # [ 51.964819] nix-grpc-daemon[777]: ts=2026-09-20T08:17:56Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575324144worker1 # [ 51.976985] systemd[1]: Started Nix Daemon.4145worker1 # [ 52.274798] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4146worker1 # [ 52.304186] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)4147worker1 # [ 52.373918] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 27 ms (attempt 2/5)4148worker1 # [ 52.401297] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 307 ms (attempt 3/5)4149worker1 # [ 52.708813] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 780 ms (attempt 4/5)4150worker1 # [ 53.492892] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4151worker1 # [ 53.508159] nix-grpc-daemon[777]: ts=2026-09-20T08:17:58Z level=debug event=path_info_miss path=h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test error="error: path '/nix/store/h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test' is required, but there is no substituter that can build it"4152worker2 # [ 53.490293] nix-grpc-daemon[778]: ts=2026-09-20T08:17:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:498024153worker2 # [ 53.506085] nix-grpc-daemon[778]: ts=2026-09-20T08:17:58Z level=debug event=path_info_miss path=h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test error="error: path '/nix/store/h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test' is required, but there is no substituter that can build it"4154client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4155client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.12 seconds)4156client: must succeed: install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4157client: (finished: must succeed: install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)4158worker2 # [ 53.968704] nix-grpc-daemon[778]: ts=2026-09-20T08:17:58Z level=info event=oidc_jwks provider=mock keys=14159worker2 # [ 53.976630] nix-grpc-daemon[778]: ts=2026-09-20T08:17:58Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:498024160worker1 # [ 54.043685] nix-grpc-daemon[777]: ts=2026-09-20T08:17:58Z level=info event=oidc_jwks provider=mock keys=14161worker1 # [ 54.050549] nix-grpc-daemon[777]: ts=2026-09-20T08:17:58Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:575324162worker1 # [ 54.059320] nix-grpc-daemon[777]: ts=2026-09-20T08:17:58Z level=debug event=path_info_miss path=h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test error="error: path '/nix/store/h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test' is required, but there is no substituter that can build it"4163worker1 # [ 54.074557] nix-grpc-daemon[777]: ts=2026-09-20T08:17:58Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:575324164worker1 # [ 54.082879] nix-grpc-daemon[777]: ts=2026-09-20T08:17:58Z level=debug event=path_info_miss path=h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test error="error: path '/nix/store/h2bir90alm3n3apkaasn6dz8i6s91c1s-nixos-system-client-test' is required, but there is no substituter that can build it"4165client: must succeed: rm -r /var/lib/nix-grpc-store4166client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.04 seconds)4167worker2 # [ 54.342914] nix-grpc-daemon[778]: ts=2026-09-20T08:17:58Z level=info event=denied method=QueryMissing cn=- role=none4168worker1 # [ 54.637772] nix-grpc-daemon[777]: ts=2026-09-20T08:17:59Z level=info event=denied method=QueryMissing cn=stranger role=none4169(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 4.52 seconds)4170subtest: DAG build is scheduled across workers and lands in the cache4171client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag t1 >&24172worker1 # [ 54.903458] nix-grpc-daemon[777]: ts=2026-09-20T08:17:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575324173client # don't know how to build these paths:4174client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4175worker2 # [ 54.892704] nix-grpc-daemon[778]: ts=2026-09-20T08:17:59Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04176worker2 # [ 54.898765] nix-grpc-daemon[778]: ts=2026-09-20T08:17:59Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=14177worker1 # [ 54.989084] nix-grpc-daemon[777]: ts=2026-09-20T08:17:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324178worker1 # [ 55.009932] nix-grpc-daemon[777]: ts=2026-09-20T08:17:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324179client # copying 1 paths to 'lb:50051'...4180worker1 # [ 55.025712] nix-grpc-daemon[777]: ts=2026-09-20T08:17:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575324181client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4182worker1 # [ 55.057145] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4183worker1 # [ 55.091752] nix-grpc-daemon[777]: ts=2026-09-20T08:17:59Z level=info event=niks3_push_started pid=8494184worker1 # [ 55.565589] nix-daemon[840]: accepted connection from pid 862, user nix-grpc-daemon (trusted)4185worker1 # [ 55.584969] nix-daemon[840]: reaped child process 865, status = succeeded4186worker1 # [ 55.761118] nix-daemon[840]: accepted connection from pid 867, user nix-grpc-daemon (trusted)4187worker1 # [ 55.778855] nix-daemon[840]: reaped child process 870, status = succeeded4188lb # [ 55.794142] niks3-server[1172]: 2026/09/20 08:18:00 INFO Received uploads request method=POST path=/api/pending_closures4189worker1 # [ 55.816784] nix-grpc-daemon[849]: time=2026-09-20T08:18:00.390Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4190worker1 # [ 55.820064] nix-grpc-daemon[849]: time=2026-09-20T08:18:00.393Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4191lb # [ 55.862288] niks3-server[1172]: 2026/09/20 08:18:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4192lb # [ 55.876890] niks3-server[1172]: 2026/09/20 08:18:00 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4193worker1 # [ 55.884540] nix-grpc-daemon[849]: time=2026-09-20T08:18:00.458Z level=INFO msg="Uploading 1 narinfos"4194lb # [ 55.886593] niks3-server[1172]: 2026/09/20 08:18:00 INFO Signed narinfos id=1 count=14195lb # [ 55.903924] niks3-server[1172]: 2026/09/20 08:18:00 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4196lb # [ 55.921313] niks3-server[1172]: 2026/09/20 08:18:00 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4197lb # [ 55.927595] niks3-server[1172]: 2026/09/20 08:18:00 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4198lb # [ 55.936088] niks3-server[1172]: 2026/09/20 08:18:00 INFO Completed upload id=14199worker1 # [ 55.933941] nix-grpc-daemon[849]: time=2026-09-20T08:18:00.508Z level=INFO msg="Upload complete. (556ms)"4200worker1 # [ 55.949473] nix-grpc-daemon[777]: ts=2026-09-20T08:18:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_in=5044201worker1 # [ 55.964377] nix-grpc-daemon[777]: ts=2026-09-20T08:18:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324202worker1 # [ 55.974456] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4203worker1 # [ 55.984574] nix-daemon[840]: reaped child process 847, status = succeeded4204client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4205client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4206worker1 # [ 55.995272] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4207client # farm-a> LOG-farm-a-t14208worker1 # [ 57.231342] nix-daemon[840]: reaped child process 874, status = succeeded4209worker1 # [ 57.403521] nix-daemon[840]: accepted connection from pid 889, user nix-grpc-daemon (trusted)4210worker1 # [ 57.421448] nix-daemon[840]: reaped child process 892, status = succeeded4211lb # [ 57.439195] niks3-server[1172]: 2026/09/20 08:18:02 INFO Received uploads request method=POST path=/api/pending_closures4212worker1 # [ 57.451754] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.024Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4213worker1 # [ 57.454168] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.024Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4214lb # [ 57.498578] niks3-server[1172]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4215lb # [ 57.508784] niks3-server[1172]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4216lb # [ 57.516435] niks3-server[1172]: 2026/09/20 08:18:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4217lb # [ 57.522040] niks3-server[1172]: 2026/09/20 08:18:02 INFO Signed narinfos id=2 count=14218worker1 # [ 57.520243] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.092Z level=INFO msg="Uploading 1 narinfos"4219lb # [ 57.525062] niks3-server[1172]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4220lb # [ 57.534795] niks3-server[1172]: 2026/09/20 08:18:02 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4221lb # [ 57.544964] niks3-server[1172]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4222lb # [ 57.550945] niks3-server[1172]: 2026/09/20 08:18:02 INFO Completed upload id=24223worker1 # [ 57.548586] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.122Z level=INFO msg="Upload complete. (340ms)"4224worker1 # [ 57.561266] nix-grpc-daemon[777]: ts=2026-09-20T08:18:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14225worker2 # [ 57.529292] nix-grpc-daemon[778]: ts=2026-09-20T08:18:02Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04226worker1 # [ 57.579405] nix-daemon[840]: reaped child process 872, status = succeeded4227worker2 # [ 57.535846] nix-grpc-daemon[778]: ts=2026-09-20T08:18:02Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14228worker1 # [ 57.626375] nix-grpc-daemon[777]: ts=2026-09-20T08:18:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324229worker1 # [ 57.642728] nix-grpc-daemon[777]: ts=2026-09-20T08:18:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324230worker1 # [ 57.658239] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.232Z level=INFO msg="All 1 paths already cached"4231client # copying 1 paths to 'lb:50051'...4232worker1 # [ 57.668628] nix-grpc-daemon[777]: ts=2026-09-20T08:18:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575324233client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4234worker1 # [ 57.688332] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4235worker1 # [ 57.902693] nix-daemon[840]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4236worker1 # [ 57.922086] nix-daemon[840]: reaped child process 899, status = succeeded4237worker1 # [ 58.096316] nix-daemon[840]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4238worker1 # [ 58.114058] nix-daemon[840]: reaped child process 904, status = succeeded4239lb # [ 58.130181] niks3-server[1172]: 2026/09/20 08:18:02 INFO Received uploads request method=POST path=/api/pending_closures4240worker1 # [ 58.140649] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.714Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4241worker1 # [ 58.142162] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.714Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4242lb # [ 58.162865] niks3-server[1172]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4243lb # [ 58.181404] niks3-server[1172]: 2026/09/20 08:18:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4244worker1 # [ 58.186297] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.760Z level=INFO msg="Uploading 1 narinfos"4245lb # [ 58.193476] niks3-server[1172]: 2026/09/20 08:18:02 INFO Signed narinfos id=3 count=14246lb # [ 58.200332] niks3-server[1172]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4247lb # [ 58.214019] niks3-server[1172]: 2026/09/20 08:18:02 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4248worker1 # [ 58.217078] nix-grpc-daemon[849]: time=2026-09-20T08:18:02.791Z level=INFO msg="Upload complete. (507ms)"4249lb # [ 58.220604] niks3-server[1172]: 2026/09/20 08:18:02 INFO Completed upload id=34250lb # [ 58.223359] niks3-server[1172]: 2026/09/20 08:18:02 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4251worker1 # [ 58.224152] nix-grpc-daemon[777]: ts=2026-09-20T08:18:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_in=6244252worker1 # [ 58.240139] nix-grpc-daemon[777]: ts=2026-09-20T08:18:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324253worker1 # [ 58.247204] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4254worker1 # [ 58.252802] nix-daemon[840]: reaped child process 894, status = succeeded4255client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4256worker1 # [ 58.264966] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4257client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4258worker1 # [ 59.491765] nix-daemon[840]: reaped child process 908, status = succeeded4259client # farm-b> LOG-farm-b-t14260worker1 # [ 59.652997] nix-daemon[840]: accepted connection from pid 923, user nix-grpc-daemon (trusted)4261worker1 # [ 59.674378] nix-daemon[840]: reaped child process 926, status = succeeded4262lb # [ 59.691503] niks3-server[1172]: 2026/09/20 08:18:04 INFO Received uploads request method=POST path=/api/pending_closures4263worker1 # [ 59.861332] nix-grpc-daemon[849]: time=2026-09-20T08:18:04.432Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4264worker1 # [ 59.865781] nix-grpc-daemon[849]: time=2026-09-20T08:18:04.433Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4265lb # [ 59.896311] niks3-server[1172]: 2026/09/20 08:18:04 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4266worker1 # [ 59.908742] nix-grpc-daemon[849]: time=2026-09-20T08:18:04.482Z level=INFO msg="Uploading 1 narinfos"4267lb # [ 59.910655] niks3-server[1172]: 2026/09/20 08:18:04 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4268lb # [ 59.922741] niks3-server[1172]: 2026/09/20 08:18:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4269lb # [ 59.930359] niks3-server[1172]: 2026/09/20 08:18:04 INFO Signed narinfos id=4 count=14270lb # [ 59.939420] niks3-server[1172]: 2026/09/20 08:18:04 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4271worker1 # [ 59.941660] nix-grpc-daemon[849]: time=2026-09-20T08:18:04.515Z level=INFO msg="Upload complete. (463ms)"4272lb # [ 59.945023] niks3-server[1172]: 2026/09/20 08:18:04 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4273lb # [ 59.951156] niks3-server[1172]: 2026/09/20 08:18:04 INFO Completed upload id=44274lb # [ 59.954422] niks3-server[1172]: 2026/09/20 08:18:04 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4275worker1 # [ 59.953189] nix-grpc-daemon[777]: ts=2026-09-20T08:18:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14276worker2 # [ 59.917277] nix-grpc-daemon[778]: ts=2026-09-20T08:18:04Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04277worker2 # [ 59.923501] nix-grpc-daemon[778]: ts=2026-09-20T08:18:04Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14278worker1 # [ 59.974995] nix-daemon[840]: reaped child process 906, status = succeeded4279worker1 # [ 60.013915] nix-grpc-daemon[777]: ts=2026-09-20T08:18:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324280worker1 # [ 60.029965] nix-grpc-daemon[777]: ts=2026-09-20T08:18:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324281worker1 # [ 60.045318] nix-grpc-daemon[849]: time=2026-09-20T08:18:04.619Z level=INFO msg="All 2 paths already cached"4282client # copying 1 paths to 'lb:50051'...4283worker1 # [ 60.055756] nix-grpc-daemon[777]: ts=2026-09-20T08:18:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575324284client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4285worker1 # [ 60.084893] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4286worker1 # [ 60.306319] nix-daemon[840]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4287worker1 # [ 60.324780] nix-daemon[840]: reaped child process 933, status = succeeded4288worker1 # [ 60.506075] nix-daemon[840]: accepted connection from pid 935, user nix-grpc-daemon (trusted)4289worker1 # [ 60.524868] nix-daemon[840]: reaped child process 938, status = succeeded4290lb # [ 60.538583] niks3-server[1172]: 2026/09/20 08:18:05 INFO Received uploads request method=POST path=/api/pending_closures4291worker1 # [ 60.545672] nix-grpc-daemon[849]: time=2026-09-20T08:18:05.119Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4292worker1 # [ 60.547200] nix-grpc-daemon[849]: time=2026-09-20T08:18:05.119Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4293lb # [ 60.568617] niks3-server[1172]: 2026/09/20 08:18:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4294worker1 # [ 60.575770] nix-grpc-daemon[849]: time=2026-09-20T08:18:05.149Z level=INFO msg="Uploading 1 narinfos"4295lb # [ 60.584387] niks3-server[1172]: 2026/09/20 08:18:05 INFO Signed narinfos id=5 count=14296lb # [ 60.587555] niks3-server[1172]: 2026/09/20 08:18:05 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4297worker1 # [ 60.603470] nix-grpc-daemon[849]: time=2026-09-20T08:18:05.177Z level=INFO msg="Upload complete. (499ms)"4298lb # [ 60.606886] niks3-server[1172]: 2026/09/20 08:18:05 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4299lb # [ 60.612320] niks3-server[1172]: 2026/09/20 08:18:05 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4300worker1 # [ 60.610284] nix-grpc-daemon[777]: ts=2026-09-20T08:18:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_in=7524301lb # [ 60.616823] niks3-server[1172]: 2026/09/20 08:18:05 INFO Completed upload id=54302lb # [ 60.618842] niks3-server[1172]: 2026/09/20 08:18:05 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4303worker1 # [ 60.623402] nix-grpc-daemon[777]: ts=2026-09-20T08:18:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324304worker1 # [ 60.630236] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4305worker1 # [ 60.634887] nix-daemon[840]: reaped child process 928, status = succeeded4306client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4307client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4308worker1 # [ 60.647666] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4309client # farm-top> LOG-farm-top-t14310worker1 # [ 62.298419] nix-daemon[840]: reaped child process 942, status = succeeded4311worker1 # [ 62.495349] nix-daemon[840]: accepted connection from pid 957, user nix-grpc-daemon (trusted)4312worker1 # [ 62.521534] nix-daemon[840]: reaped child process 960, status = succeeded4313lb # [ 62.537725] niks3-server[1172]: 2026/09/20 08:18:07 INFO Received uploads request method=POST path=/api/pending_closures4314worker1 # [ 62.694620] nix-grpc-daemon[849]: time=2026-09-20T08:18:07.267Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4315worker1 # [ 62.703503] nix-grpc-daemon[849]: time=2026-09-20T08:18:07.267Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4316lb # [ 62.739837] niks3-server[1172]: 2026/09/20 08:18:07 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4317worker1 # [ 62.761379] nix-grpc-daemon[849]: time=2026-09-20T08:18:07.335Z level=INFO msg="Uploading 1 narinfos"4318lb # [ 62.754481] niks3-server[1172]: 2026/09/20 08:18:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4319lb # [ 62.769669] niks3-server[1172]: 2026/09/20 08:18:07 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4320lb # [ 62.782572] niks3-server[1172]: 2026/09/20 08:18:07 INFO Signed narinfos id=6 count=14321lb # [ 62.798228] envoy[931]: conn peer=[2001:db8:1::1]:42580 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=30884322lb # [ 62.804641] envoy[931]: conn peer=[2001:db8:1::1]:37834 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=984323worker1 # [ 62.805966] nix-grpc-daemon[849]: time=2026-09-20T08:18:07.380Z level=INFO msg="Upload complete. (546ms)"4324lb # [ 62.809766] envoy[931]: conn peer=[2001:db8:1::1]:37838 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=134325lb # [ 62.812852] envoy[931]: conn peer=[2001:db8:1::1]:37852 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134326lb # [ 62.818818] niks3-server[1172]: 2026/09/20 08:18:07 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4327lb # [ 62.822663] niks3-server[1172]: 2026/09/20 08:18:07 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4328worker1 # [ 62.817312] nix-grpc-daemon[777]: ts=2026-09-20T08:18:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14329lb # [ 62.826503] niks3-server[1172]: 2026/09/20 08:18:07 INFO Completed upload id=64330lb # [ 62.829342] niks3-server[1172]: 2026/09/20 08:18:07 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4331worker1 # [ 62.839254] nix-daemon[840]: reaped child process 940, status = succeeded4332client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag t1 >&2, in 8.22 seconds)4333client: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag t1 outPath4334client: (finished: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag t1 outPath, in 0.24 seconds)4335client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4336client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4337client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4338client # copying 3 paths...4339client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4340client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4341client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4342client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.31 seconds)4343(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.93 seconds)4344subtest: repeat is answered Cached by the scheduler without building4345worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4346worker1 # finding garbage collector roots...4347worker1 # deleting specified paths...4348worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4349worker1 # deleting unused links...4350worker1 # note: hard linking is currently saving 0.0 KiB4351worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.28 seconds)4352worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4353worker2 # finding garbage collector roots...4354worker2 # deleting specified paths...4355worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4356worker2 # deleting unused links...4357worker2 # note: hard linking is currently saving 0.0 KiB4358worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.30 seconds)4359client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag t1 >&24360worker1 # [ 64.467190] nix-grpc-daemon[777]: ts=2026-09-20T08:18:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575324361client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4362client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4363worker2 # [ 64.469470] nix-grpc-daemon[778]: ts=2026-09-20T08:18:09Z level=debug event=want client=2 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=14364worker2 # [ 64.486770] nix-grpc-daemon[778]: ts=2026-09-20T08:18:09Z level=debug event=want client=2 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=14365worker2 # [ 64.503123] nix-grpc-daemon[778]: ts=2026-09-20T08:18:09Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14366client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag t1 >&2, in 0.43 seconds)4367client: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag t1 outPath4368client: (finished: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag t1 outPath, in 0.27 seconds)4369lb: must succeed: curl -sf localhost:9901/clusters4370lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.12 seconds)4371lb: must succeed: curl -sf localhost:9901/clusters4372lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4373worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true4374worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="cached"} ' || true, in 0.11 seconds)4375(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.72 seconds)4376subtest: two clients wanting the same drv build it once4377client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4378client # [ 65.366145] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4379client # Running as unit: dedup1.service; invocation ID: 97f083a158d14540a2f413dedb710f3b4380client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.12 seconds)4381client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4382client # [ 65.638431] nix[864]: don't know how to build these paths:4383client # [ 65.639447] nix[864]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4384worker1 # [ 65.679394] nix-grpc-daemon[777]: ts=2026-09-20T08:18:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575324385worker2 # [ 65.653259] nix-grpc-daemon[778]: ts=2026-09-20T08:18:10Z level=debug event=want client=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04386worker2 # [ 65.659171] nix-grpc-daemon[778]: ts=2026-09-20T08:18:10Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14387worker2 # [ 65.665752] nix-grpc-daemon[778]: ts=2026-09-20T08:18:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498024388worker2 # [ 65.687689] nix-grpc-daemon[778]: ts=2026-09-20T08:18:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498024389client # [ 65.702600] nix[864]: copying 1 paths to 'lb:50051'...4390client # [ 65.707253] nix[864]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4391worker2 # [ 65.705256] nix-grpc-daemon[778]: ts=2026-09-20T08:18:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:498024392worker2 # [ 65.721312] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4393worker2 # [ 65.749157] nix-grpc-daemon[778]: ts=2026-09-20T08:18:10Z level=info event=niks3_push_started pid=8874394worker2 # [ 66.215068] nix-daemon[846]: accepted connection from pid 900, user nix-grpc-daemon (trusted)4395worker2 # [ 66.236478] nix-daemon[846]: reaped child process 903, status = succeeded4396worker2 # [ 66.415018] nix-daemon[846]: accepted connection from pid 905, user nix-grpc-daemon (trusted)4397worker2 # [ 66.433235] nix-daemon[846]: reaped child process 908, status = succeeded4398lb # [ 66.493070] niks3-server[1172]: 2026/09/20 08:18:11 INFO Received uploads request method=POST path=/api/pending_closures4399worker2 # [ 66.459991] nix-grpc-daemon[887]: time=2026-09-20T08:18:11.033Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4400worker2 # [ 66.461615] nix-grpc-daemon[887]: time=2026-09-20T08:18:11.033Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4401lb # [ 66.535212] niks3-server[1172]: 2026/09/20 08:18:11 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4402worker2 # [ 66.502422] nix-grpc-daemon[887]: time=2026-09-20T08:18:11.077Z level=INFO msg="Uploading 1 narinfos"4403lb # [ 66.548326] niks3-server[1172]: 2026/09/20 08:18:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4404client # [ 66.501911] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4405lb # [ 66.556394] niks3-server[1172]: 2026/09/20 08:18:11 INFO Signed narinfos id=7 count=14406client # Running as unit: dedup2.service; invocation ID: b7a4ba8a671c4a86a5cd49cd128a394f4407lb # [ 66.565827] niks3-server[1172]: 2026/09/20 08:18:11 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4408lb # [ 66.576783] niks3-server[1172]: 2026/09/20 08:18:11 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4409client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4410lb # [ 66.790865] niks3-server[1172]: 2026/09/20 08:18:11 INFO Completed upload id=74411worker2 # [ 66.744983] nix-grpc-daemon[887]: time=2026-09-20T08:18:11.320Z level=INFO msg="Upload complete. (720ms)"4412lb # [ 66.799113] niks3-server[1172]: 2026/09/20 08:18:11 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4413worker2 # [ 66.757256] nix-grpc-daemon[778]: ts=2026-09-20T08:18:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49802 duration_s=1 paths=1 nar_bytes_in=4324414worker2 # [ 66.776729] nix-grpc-daemon[778]: ts=2026-09-20T08:18:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498024415worker2 # [ 66.790432] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4416worker2 # [ 66.801035] nix-daemon[846]: reaped child process 885, status = succeeded4417client # [ 66.805106] nix[864]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4418worker2 # [ 66.808303] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4419worker2 # [ 66.844918] nix-grpc-daemon[778]: ts=2026-09-20T08:18:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:498024420client # [ 66.859164] nix[879]: this derivation will be built:4421worker1 # [ 66.905800] nix-grpc-daemon[777]: ts=2026-09-20T08:18:11Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575324422worker1 # [ 66.920386] nix-grpc-daemon[777]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' from 'http://lb:5751'...4423client # [ 66.903889] nix[879]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4424worker2 # [ 66.915071] nix-grpc-daemon[778]: ts=2026-09-20T08:18:11Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04425worker2 # [ 66.924902] nix-grpc-daemon[778]: ts=2026-09-20T08:18:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498024426client # [ 66.934472] nix[879]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4427client: must succeed: sleep 24428client: (finished: must succeed: sleep 2, in 2.07 seconds)4429??? Warning (UserWarning): wait_until_succeeds(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4430 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394431client: waiting for success: ! systemctl is-active dedup1 dedup24432??? Warning (UserWarning): execute(): passing a bare int/float as a duration is deprecated. Use datetime.timedelta instead.4433 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/duration.py", line 394434worker2 # [ 71.444248] nix-daemon[846]: reaped child process 916, status = succeeded4435worker2 # [ 71.616574] nix-daemon[846]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4436worker2 # [ 71.635907] nix-daemon[846]: reaped child process 950, status = succeeded4437lb # [ 71.695517] niks3-server[1172]: 2026/09/20 08:18:16 INFO Received uploads request method=POST path=/api/pending_closures4438worker2 # [ 71.812187] nix-grpc-daemon[887]: time=2026-09-20T08:18:16.380Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4439worker2 # [ 71.821870] nix-grpc-daemon[887]: time=2026-09-20T08:18:16.380Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4440lb # [ 71.903429] niks3-server[1172]: 2026/09/20 08:18:16 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4441worker2 # [ 71.871487] nix-grpc-daemon[887]: time=2026-09-20T08:18:16.444Z level=INFO msg="Uploading 1 narinfos"4442lb # [ 71.919204] niks3-server[1172]: 2026/09/20 08:18:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4443lb # [ 71.928528] niks3-server[1172]: 2026/09/20 08:18:16 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4444lb # [ 71.938463] niks3-server[1172]: 2026/09/20 08:18:16 INFO Signed narinfos id=8 count=14445lb # [ 71.946422] niks3-server[1172]: 2026/09/20 08:18:16 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4446lb # [ 71.950940] niks3-server[1172]: 2026/09/20 08:18:16 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4447lb # [ 71.956928] niks3-server[1172]: 2026/09/20 08:18:16 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4448lb # [ 71.961688] niks3-server[1172]: 2026/09/20 08:18:16 INFO Completed upload id=84449worker2 # [ 71.915133] nix-grpc-daemon[887]: time=2026-09-20T08:18:16.490Z level=INFO msg="Upload complete. (489ms)"4450worker2 # [ 71.929257] nix-grpc-daemon[778]: ts=2026-09-20T08:18:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49802 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=4 outputs=14451worker2 # [ 71.948449] nix-grpc-daemon[778]: ts=2026-09-20T08:18:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49802 duration_s=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=4 outputs=14452worker2 # [ 71.956949] nix-daemon[846]: reaped child process 914, status = succeeded4453client # [ 71.974934] systemd[1]: dedup2.service: Deactivated successfully.4454client # [ 71.981647] systemd[1]: dedup2.service: Consumed 192ms CPU time over 5.477s wall clock time, 28.9M memory peak, 7.8K incoming IP traffic, 6.5K outgoing IP traffic.4455client # [ 72.016567] systemd[1]: dedup1.service: Deactivated successfully.4456client # [ 72.022558] systemd[1]: dedup1.service: Consumed 163ms CPU time over 6.656s wall clock time, 37.2M memory peak, 8.7K incoming IP traffic, 9K outgoing IP traffic.4457client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.15 seconds)4458client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4459client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.08 seconds)4460worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4461worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4462worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4463worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4464worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4465worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.09 seconds)4466worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true4467lb # [ 72.764844] envoy[931]: conn peer=[2001:db8:1::1]:37858 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9427 tx=3805 ms=79604468lb # [ 72.769830] envoy[931]: conn peer=[2001:db8:1::1]:56886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1162 tx=481 ms=1034469lb # [ 72.774618] envoy[931]: conn peer=[2001:db8:1::1]:56902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1592 tx=1487 ms=50984470lb # [ 72.779555] envoy[931]: conn peer=[2001:db8:1::1]:56900 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1397 ms=63484471worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_events_total{kind="attached"} ' || true, in 0.08 seconds)4472(finished: subtest: two clients wanting the same drv build it once, in 7.52 seconds)4473subtest: build hook: nix-daemon with builders = grpc://lb4474client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag hook 2>/tmp/hook.log4475client # [ 73.024798] systemd[1]: Started Nix Daemon.4476client # [ 73.219706] nix-daemon[920]: accepted connection from pid 917, user root (trusted)4477worker2 # [ 73.421550] nix-grpc-daemon[778]: ts=2026-09-20T08:18:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:498024478worker1 # [ 73.472578] nix-grpc-daemon[777]: ts=2026-09-20T08:18:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324479worker2 # [ 73.440153] nix-grpc-daemon[778]: ts=2026-09-20T08:18:18Z level=debug event=want client=5 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04480worker2 # [ 73.446264] nix-grpc-daemon[778]: ts=2026-09-20T08:18:18Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14481worker1 # [ 73.493021] nix-grpc-daemon[777]: ts=2026-09-20T08:18:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324482worker1 # [ 73.508994] nix-grpc-daemon[777]: ts=2026-09-20T08:18:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324483worker1 # [ 73.523782] nix-grpc-daemon[777]: ts=2026-09-20T08:18:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575324484worker1 # [ 73.539686] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4485worker1 # [ 73.760450] nix-daemon[840]: accepted connection from pid 1005, user nix-grpc-daemon (trusted)4486worker1 # [ 73.778438] nix-daemon[840]: reaped child process 1008, status = succeeded4487worker1 # [ 73.958380] nix-daemon[840]: accepted connection from pid 1010, user nix-grpc-daemon (trusted)4488worker1 # [ 73.975341] nix-daemon[840]: reaped child process 1013, status = succeeded4489lb # [ 73.990929] niks3-server[1172]: 2026/09/20 08:18:18 INFO Received uploads request method=POST path=/api/pending_closures4490worker1 # [ 73.998121] nix-grpc-daemon[849]: time=2026-09-20T08:18:18.571Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4491worker1 # [ 73.999791] nix-grpc-daemon[849]: time=2026-09-20T08:18:18.571Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4492lb # [ 74.025384] niks3-server[1172]: 2026/09/20 08:18:18 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4493worker1 # [ 74.036721] nix-grpc-daemon[849]: time=2026-09-20T08:18:18.610Z level=INFO msg="Uploading 1 narinfos"4494lb # [ 74.040433] niks3-server[1172]: 2026/09/20 08:18:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4495lb # [ 74.051393] niks3-server[1172]: 2026/09/20 08:18:18 INFO Signed narinfos id=9 count=14496lb # [ 74.056512] niks3-server[1172]: 2026/09/20 08:18:18 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4497worker1 # [ 74.062389] nix-grpc-daemon[849]: time=2026-09-20T08:18:18.636Z level=INFO msg="Upload complete. (504ms)"4498lb # [ 74.065061] niks3-server[1172]: 2026/09/20 08:18:18 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4499lb # [ 74.069207] niks3-server[1172]: 2026/09/20 08:18:18 INFO Completed upload id=94500lb # [ 74.072196] niks3-server[1172]: 2026/09/20 08:18:18 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4501worker1 # [ 74.072263] nix-grpc-daemon[777]: ts=2026-09-20T08:18:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_in=5124502worker1 # [ 74.083267] nix-grpc-daemon[777]: ts=2026-09-20T08:18:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324503worker1 # [ 74.090613] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4504worker1 # [ 74.095438] nix-daemon[840]: reaped child process 1003, status = succeeded4505worker1 # [ 74.107086] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4506worker1 # [ 75.156410] nix-daemon[840]: reaped child process 1017, status = succeeded4507worker1 # [ 75.331442] nix-daemon[840]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4508worker1 # [ 75.347863] nix-daemon[840]: reaped child process 1035, status = succeeded4509lb # [ 75.366491] niks3-server[1172]: 2026/09/20 08:18:19 INFO Received uploads request method=POST path=/api/pending_closures4510worker1 # [ 75.376447] nix-grpc-daemon[849]: time=2026-09-20T08:18:19.947Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4511worker1 # [ 75.378072] nix-grpc-daemon[849]: time=2026-09-20T08:18:19.947Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4512lb # [ 75.410150] niks3-server[1172]: 2026/09/20 08:18:19 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4513worker1 # [ 75.425334] nix-grpc-daemon[849]: time=2026-09-20T08:18:19.999Z level=INFO msg="Uploading 1 narinfos"4514lb # [ 75.421685] niks3-server[1172]: 2026/09/20 08:18:19 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4515lb # [ 75.432397] niks3-server[1172]: 2026/09/20 08:18:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4516lb # [ 75.439888] niks3-server[1172]: 2026/09/20 08:18:20 INFO Signed narinfos id=10 count=14517lb # [ 75.446617] niks3-server[1172]: 2026/09/20 08:18:20 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4518worker1 # [ 75.443623] nix-grpc-daemon[849]: time=2026-09-20T08:18:20.017Z level=INFO msg="Upload complete. (323ms)"4519lb # [ 75.449843] niks3-server[1172]: 2026/09/20 08:18:20 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4520lb # [ 75.454564] niks3-server[1172]: 2026/09/20 08:18:20 INFO Completed upload id=104521lb # [ 75.456968] niks3-server[1172]: 2026/09/20 08:18:20 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4522worker1 # [ 75.457108] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=5 outputs=14523worker1 # [ 75.470986] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575324524worker1 # [ 75.475462] nix-daemon[840]: reaped child process 1015, status = succeeded4525worker1 # [ 75.491827] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:575324526worker1 # [ 75.498209] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_out=1284527worker2 # [ 75.764771] nix-grpc-daemon[778]: ts=2026-09-20T08:18:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:498024528worker2 # [ 75.772545] nix-grpc-daemon[778]: ts=2026-09-20T08:18:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498024529worker2 # [ 75.793260] nix-grpc-daemon[778]: ts=2026-09-20T08:18:20Z level=debug event=want client=6 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04530worker2 # [ 75.799853] nix-grpc-daemon[778]: ts=2026-09-20T08:18:20Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14531worker1 # [ 75.846948] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324532worker1 # [ 75.861971] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324533worker1 # [ 75.876606] nix-grpc-daemon[849]: time=2026-09-20T08:18:20.450Z level=INFO msg="All 1 paths already cached"4534worker1 # [ 75.886048] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575324535worker1 # [ 75.900123] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4536worker1 # [ 76.106334] nix-daemon[840]: accepted connection from pid 1039, user nix-grpc-daemon (trusted)4537worker1 # [ 76.126411] nix-daemon[840]: reaped child process 1042, status = succeeded4538worker1 # [ 76.302317] nix-daemon[840]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)4539worker1 # [ 76.319502] nix-daemon[840]: reaped child process 1047, status = succeeded4540lb # [ 76.333565] niks3-server[1172]: 2026/09/20 08:18:20 INFO Received uploads request method=POST path=/api/pending_closures4541worker1 # [ 76.340893] nix-grpc-daemon[849]: time=2026-09-20T08:18:20.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4542worker1 # [ 76.342407] nix-grpc-daemon[849]: time=2026-09-20T08:18:20.914Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4543lb # [ 76.361888] niks3-server[1172]: 2026/09/20 08:18:20 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4544worker1 # [ 76.374207] nix-grpc-daemon[849]: time=2026-09-20T08:18:20.948Z level=INFO msg="Uploading 1 narinfos"4545lb # [ 76.378040] niks3-server[1172]: 2026/09/20 08:18:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4546lb # [ 76.390373] niks3-server[1172]: 2026/09/20 08:18:20 INFO Signed narinfos id=11 count=14547lb # [ 76.401354] niks3-server[1172]: 2026/09/20 08:18:20 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4548worker1 # [ 76.403784] nix-grpc-daemon[849]: time=2026-09-20T08:18:20.977Z level=INFO msg="Upload complete. (485ms)"4549lb # [ 76.405872] niks3-server[1172]: 2026/09/20 08:18:20 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4550lb # [ 76.410749] niks3-server[1172]: 2026/09/20 08:18:20 INFO Completed upload id=114551lb # [ 76.414164] niks3-server[1172]: 2026/09/20 08:18:20 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4552worker1 # [ 76.411810] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_in=6244553worker1 # [ 76.423747] nix-grpc-daemon[777]: ts=2026-09-20T08:18:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324554worker1 # [ 76.429996] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4555worker1 # [ 76.434690] nix-daemon[840]: reaped child process 1037, status = succeeded4556worker1 # [ 76.444823] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4557worker1 # [ 77.688751] nix-daemon[840]: reaped child process 1051, status = succeeded4558worker1 # [ 77.862897] nix-daemon[840]: accepted connection from pid 1066, user nix-grpc-daemon (trusted)4559worker1 # [ 77.881089] nix-daemon[840]: reaped child process 1069, status = succeeded4560lb # [ 77.897952] niks3-server[1172]: 2026/09/20 08:18:22 INFO Received uploads request method=POST path=/api/pending_closures4561worker1 # [ 77.904530] nix-grpc-daemon[849]: time=2026-09-20T08:18:22.478Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4562worker1 # [ 77.910359] nix-grpc-daemon[849]: time=2026-09-20T08:18:22.478Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4563lb # [ 77.941139] niks3-server[1172]: 2026/09/20 08:18:22 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4564lb # [ 77.954166] niks3-server[1172]: 2026/09/20 08:18:22 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4565lb # [ 77.964223] niks3-server[1172]: 2026/09/20 08:18:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4566worker1 # [ 77.966862] nix-grpc-daemon[849]: time=2026-09-20T08:18:22.540Z level=INFO msg="Uploading 1 narinfos"4567lb # [ 77.972236] niks3-server[1172]: 2026/09/20 08:18:22 INFO Signed narinfos id=12 count=14568lb # [ 77.974303] niks3-server[1172]: 2026/09/20 08:18:22 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4569lb # [ 77.984113] niks3-server[1172]: 2026/09/20 08:18:22 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4570lb # [ 77.989733] niks3-server[1172]: 2026/09/20 08:18:22 INFO Completed upload id=124571worker1 # [ 77.987299] nix-grpc-daemon[849]: time=2026-09-20T08:18:22.561Z level=INFO msg="Upload complete. (329ms)"4572lb # [ 77.992435] niks3-server[1172]: 2026/09/20 08:18:22 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4573worker1 # [ 78.001317] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=6 outputs=14574worker1 # [ 78.014807] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575324575worker1 # [ 78.018968] nix-daemon[840]: reaped child process 1049, status = succeeded4576worker1 # [ 78.033995] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:575324577worker1 # [ 78.039443] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_out=1764578worker1 # [ 78.345090] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:575324579worker1 # [ 78.352923] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324580worker1 # [ 78.359626] nix-grpc-daemon[849]: time=2026-09-20T08:18:22.932Z level=INFO msg="All 2 paths already cached"4581worker2 # [ 78.329339] nix-grpc-daemon[778]: ts=2026-09-20T08:18:22Z level=debug event=want client=7 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04582worker2 # [ 78.335355] nix-grpc-daemon[778]: ts=2026-09-20T08:18:22Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14583worker1 # [ 78.382024] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324584worker1 # [ 78.397009] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324585worker1 # [ 78.410615] nix-grpc-daemon[849]: time=2026-09-20T08:18:22.984Z level=INFO msg="All 2 paths already cached"4586worker1 # [ 78.418985] nix-grpc-daemon[777]: ts=2026-09-20T08:18:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575324587worker1 # [ 78.432773] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4588worker1 # [ 78.636269] nix-daemon[840]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)4589worker1 # [ 78.655964] nix-daemon[840]: reaped child process 1076, status = succeeded4590worker1 # [ 78.825154] nix-daemon[840]: accepted connection from pid 1078, user nix-grpc-daemon (trusted)4591worker1 # [ 78.843889] nix-daemon[840]: reaped child process 1081, status = succeeded4592lb # [ 78.858176] niks3-server[1172]: 2026/09/20 08:18:23 INFO Received uploads request method=POST path=/api/pending_closures4593worker1 # [ 78.868634] nix-grpc-daemon[849]: time=2026-09-20T08:18:23.442Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4594worker1 # [ 78.870133] nix-grpc-daemon[849]: time=2026-09-20T08:18:23.442Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4595lb # [ 78.889013] niks3-server[1172]: 2026/09/20 08:18:23 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4596worker1 # [ 78.902711] nix-grpc-daemon[849]: time=2026-09-20T08:18:23.476Z level=INFO msg="Uploading 1 narinfos"4597lb # [ 78.905960] niks3-server[1172]: 2026/09/20 08:18:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4598lb # [ 78.918938] niks3-server[1172]: 2026/09/20 08:18:23 INFO Signed narinfos id=13 count=14599lb # [ 78.923695] niks3-server[1172]: 2026/09/20 08:18:23 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4600lb # [ 78.932603] niks3-server[1172]: 2026/09/20 08:18:23 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4601lb # [ 78.937724] niks3-server[1172]: 2026/09/20 08:18:23 INFO Completed upload id=134602worker1 # [ 78.935238] nix-grpc-daemon[849]: time=2026-09-20T08:18:23.508Z level=INFO msg="Upload complete. (485ms)"4603lb # [ 78.941029] niks3-server[1172]: 2026/09/20 08:18:23 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4604worker1 # [ 78.944391] nix-grpc-daemon[777]: ts=2026-09-20T08:18:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_in=7524605worker1 # [ 78.958246] nix-grpc-daemon[777]: ts=2026-09-20T08:18:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324606worker1 # [ 78.964776] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4607worker1 # [ 78.970002] nix-daemon[840]: reaped child process 1071, status = succeeded4608worker1 # [ 78.984253] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4609worker1 # [ 80.230895] nix-daemon[840]: reaped child process 1085, status = succeeded4610worker1 # [ 80.409422] nix-daemon[840]: accepted connection from pid 1100, user nix-grpc-daemon (trusted)4611worker1 # [ 80.429462] nix-daemon[840]: reaped child process 1103, status = succeeded4612lb # [ 80.445563] niks3-server[1172]: 2026/09/20 08:18:25 INFO Received uploads request method=POST path=/api/pending_closures4613worker1 # [ 80.452416] nix-grpc-daemon[849]: time=2026-09-20T08:18:25.025Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4614worker1 # [ 80.453925] nix-grpc-daemon[849]: time=2026-09-20T08:18:25.025Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4615lb # [ 80.486926] niks3-server[1172]: 2026/09/20 08:18:25 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4616lb # [ 80.498227] niks3-server[1172]: 2026/09/20 08:18:25 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4617worker1 # [ 80.506502] nix-grpc-daemon[849]: time=2026-09-20T08:18:25.080Z level=INFO msg="Uploading 1 narinfos"4618lb # [ 80.511270] niks3-server[1172]: 2026/09/20 08:18:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4619lb # [ 80.519080] niks3-server[1172]: 2026/09/20 08:18:25 INFO Signed narinfos id=14 count=14620lb # [ 80.522092] niks3-server[1172]: 2026/09/20 08:18:25 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4621lb # [ 80.529575] niks3-server[1172]: 2026/09/20 08:18:25 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4622worker1 # [ 80.526783] nix-grpc-daemon[849]: time=2026-09-20T08:18:25.101Z level=INFO msg="Upload complete. (331ms)"4623lb # [ 80.532913] niks3-server[1172]: 2026/09/20 08:18:25 INFO Completed upload id=144624lb # [ 80.535656] niks3-server[1172]: 2026/09/20 08:18:25 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4625worker1 # [ 80.537148] nix-grpc-daemon[777]: ts=2026-09-20T08:18:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=7 outputs=14626worker2 # [ 80.499235] nix-grpc-daemon[778]: ts=2026-09-20T08:18:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:498024627worker1 # [ 80.553434] nix-daemon[840]: reaped child process 1083, status = succeeded4628worker2 # [ 80.516759] nix-grpc-daemon[778]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4629worker2 # [ 80.538594] nix-grpc-daemon[778]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4630worker2 # [ 80.552992] nix-grpc-daemon[778]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4631worker1 # [ 80.636090] nix-grpc-daemon[777]: ts=2026-09-20T08:18:25Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:575324632worker1 # [ 80.642896] nix-grpc-daemon[777]: ts=2026-09-20T08:18:25Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_out=2324633client # [ 80.690750] nix-daemon[920]: reaped child process 922, status = succeeded4634client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.92 seconds)4635client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4636client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4637client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4638client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4639client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4640client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4641(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.05 seconds)4642subtest: an input only one worker has still reaches the builder4643client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4644client # this derivation will be built:4645client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4646client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4647client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.44 seconds)4648client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4649client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.20 seconds)4650worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4651worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.25 seconds)4652worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4653worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4654client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24655client # [ 81.981720] nix-daemon[920]: accepted connection from pid 1000, user root (trusted)4656client # this derivation will be built:4657client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4658worker2 # [ 82.183315] nix-grpc-daemon[778]: ts=2026-09-20T08:18:26Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:498024659client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4660worker1 # [ 82.231131] nix-grpc-daemon[777]: ts=2026-09-20T08:18:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324661worker1 # [ 82.434614] nix-daemon[840]: accepted connection from pid 1110, user nix-grpc-daemon (trusted)4662worker1 # [ 82.452146] nix-daemon[840]: reaped child process 1113, status = succeeded4663lb # [ 82.467035] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received uploads request method=POST path=/api/pending_closures4664worker1 # [ 82.473762] nix-grpc-daemon[849]: time=2026-09-20T08:18:27.047Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4665worker1 # [ 82.475225] nix-grpc-daemon[849]: time=2026-09-20T08:18:27.047Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4666lb # [ 82.494836] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4667lb # [ 82.509437] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4668worker1 # [ 82.512569] nix-grpc-daemon[849]: time=2026-09-20T08:18:27.086Z level=INFO msg="Uploading 1 narinfos"4669lb # [ 82.521690] niks3-server[1172]: 2026/09/20 08:18:27 INFO Signed narinfos id=15 count=14670lb # [ 82.527064] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4671lb # [ 82.534771] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4672lb # [ 82.540884] niks3-server[1172]: 2026/09/20 08:18:27 INFO Completed upload id=154673worker1 # [ 82.537850] nix-grpc-daemon[849]: time=2026-09-20T08:18:27.112Z level=INFO msg="Upload complete. (300ms)"4674lb # [ 82.543863] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4675worker2 # [ 82.519344] nix-grpc-daemon[778]: ts=2026-09-20T08:18:27Z level=debug event=want client=8 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04676worker2 # [ 82.527206] nix-grpc-daemon[778]: ts=2026-09-20T08:18:27Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14677worker2 # [ 82.537391] nix-grpc-daemon[778]: ts=2026-09-20T08:18:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498024678worker2 # [ 82.549444] nix-grpc-daemon[778]: ts=2026-09-20T08:18:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498024679client # copying 2 paths to 'lb:50051'...4680client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4681worker2 # [ 82.568971] nix-grpc-daemon[778]: ts=2026-09-20T08:18:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:498024682client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4683worker2 # [ 82.589873] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4684lb # [ 82.762093] envoy[931]: conn peer=[2001:db8:1::1]:43720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3429 tx=1660 ms=20944685lb # [ 82.767160] envoy[931]: conn peer=[2001:db8:1::1]:43728 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=724686lb # [ 82.771908] envoy[931]: conn peer=[2001:db8:1::1]:43744 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3901 tx=1876 ms=22934687lb # [ 82.776801] envoy[931]: conn peer=[2001:db8:1::1]:43760 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=714688lb # [ 82.781366] envoy[931]: conn peer=[2001:db8:1::1]:43774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4352 tx=2222 ms=23174689lb # [ 82.786894] envoy[931]: conn peer=[2001:db8:1::1]:43786 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=294690worker2 # [ 82.806313] nix-daemon[846]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4691worker2 # [ 82.825864] nix-daemon[846]: reaped child process 982, status = succeeded4692worker2 # [ 82.999810] nix-daemon[846]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4693worker2 # [ 83.019116] nix-daemon[846]: reaped child process 987, status = succeeded4694lb # [ 83.077087] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received uploads request method=POST path=/api/pending_closures4695lb # [ 83.086005] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received uploads request method=POST path=/api/pending_closures4696worker2 # [ 83.048750] nix-grpc-daemon[887]: time=2026-09-20T08:18:27.623Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4697worker2 # [ 83.050241] nix-grpc-daemon[887]: time=2026-09-20T08:18:27.623Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4698worker2 # [ 83.051946] nix-grpc-daemon[887]: time=2026-09-20T08:18:27.623Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4699lb # [ 83.116749] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4700lb # [ 83.139664] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4701lb # [ 83.150717] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4702lb # [ 83.156355] niks3-server[1172]: 2026/09/20 08:18:27 INFO Signed narinfos id=16 count=14703worker2 # [ 83.110460] nix-grpc-daemon[887]: time=2026-09-20T08:18:27.684Z level=INFO msg="Uploading 3 narinfos"4704lb # [ 83.159804] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4705lb # [ 83.166131] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4706lb # [ 83.172851] niks3-server[1172]: 2026/09/20 08:18:27 INFO Signed narinfos id=17 count=24707lb # [ 83.176673] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4708lb # [ 83.190038] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4709lb # [ 83.200167] niks3-server[1172]: 2026/09/20 08:18:27 INFO Completed upload id=164710lb # [ 83.201642] niks3-server[1172]: 2026/09/20 08:18:27 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4711lb # [ 83.205648] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4712lb # [ 83.209258] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4713lb # [ 83.212403] niks3-server[1172]: 2026/09/20 08:18:27 INFO Completed upload id=174714worker2 # [ 83.165809] nix-grpc-daemon[887]: time=2026-09-20T08:18:27.740Z level=INFO msg="Upload complete. (555ms)"4715lb # [ 83.214206] niks3-server[1172]: 2026/09/20 08:18:27 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4716worker2 # [ 83.176374] nix-grpc-daemon[778]: ts=2026-09-20T08:18:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49802 duration_s=0 paths=2 nar_bytes_in=9844717worker2 # [ 83.193557] nix-grpc-daemon[778]: ts=2026-09-20T08:18:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498024718worker2 # [ 83.205104] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4719worker2 # [ 83.209876] nix-daemon[846]: reaped child process 977, status = succeeded4720worker2 # [ 83.224395] nix-grpc-daemon[778]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4721client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4722worker2 # [ 83.242264] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4723client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4724worker2 # [ 84.486537] nix-daemon[846]: reaped child process 992, status = succeeded4725worker2 # [ 84.655082] nix-daemon[846]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)4726worker2 # [ 84.674623] nix-daemon[846]: reaped child process 1010, status = succeeded4727lb # [ 84.734036] niks3-server[1172]: 2026/09/20 08:18:29 INFO Received uploads request method=POST path=/api/pending_closures4728worker2 # [ 84.697306] nix-grpc-daemon[887]: time=2026-09-20T08:18:29.272Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4729lb # [ 84.766127] niks3-server[1172]: 2026/09/20 08:18:29 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4730worker2 # [ 84.728951] nix-grpc-daemon[887]: time=2026-09-20T08:18:29.303Z level=INFO msg="Uploading 1 narinfos"4731lb # [ 84.779061] niks3-server[1172]: 2026/09/20 08:18:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4732lb # [ 84.790178] niks3-server[1172]: 2026/09/20 08:18:29 INFO Signed narinfos id=18 count=14733lb # [ 84.795491] niks3-server[1172]: 2026/09/20 08:18:29 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4734lb # [ 84.803327] niks3-server[1172]: 2026/09/20 08:18:29 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4735worker2 # [ 84.761856] nix-grpc-daemon[887]: time=2026-09-20T08:18:29.336Z level=INFO msg="Upload complete. (301ms)"4736lb # [ 84.808956] niks3-server[1172]: 2026/09/20 08:18:29 INFO Completed upload id=184737lb # [ 84.811841] niks3-server[1172]: 2026/09/20 08:18:29 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4738worker2 # [ 84.771109] nix-grpc-daemon[778]: ts=2026-09-20T08:18:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49802 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=8 outputs=14739worker2 # [ 84.787396] nix-daemon[846]: reaped child process 989, status = succeeded4740client # copying 1 paths...4741worker2 # [ 84.818915] nix-grpc-daemon[778]: ts=2026-09-20T08:18:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:498024742client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4743worker1 # [ 84.898483] nix-grpc-daemon[777]: ts=2026-09-20T08:18:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:575324744worker1 # [ 84.915974] nix-grpc-daemon[777]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4745worker1 # [ 84.934212] nix-grpc-daemon[777]: ts=2026-09-20T08:18:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_out=1204746client # [ 85.023954] nix-daemon[920]: reaped child process 1003, status = succeeded4747client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.27 seconds)4748client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24749client # [ 85.243984] nix-daemon[920]: accepted connection from pid 1021, user root (trusted)4750client # this derivation will be built:4751client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4752worker1 # [ 85.480174] nix-grpc-daemon[777]: ts=2026-09-20T08:18:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:575324753client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4754worker1 # [ 85.487931] nix-grpc-daemon[777]: ts=2026-09-20T08:18:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324755worker1 # [ 85.495777] nix-grpc-daemon[849]: time=2026-09-20T08:18:30.067Z level=INFO msg="All 1 paths already cached"4756worker2 # [ 85.463996] nix-grpc-daemon[778]: ts=2026-09-20T08:18:30Z level=debug event=want client=9 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04757worker2 # [ 85.470715] nix-grpc-daemon[778]: ts=2026-09-20T08:18:30Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14758worker1 # [ 85.517330] nix-grpc-daemon[777]: ts=2026-09-20T08:18:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324759worker1 # [ 85.532736] nix-grpc-daemon[777]: ts=2026-09-20T08:18:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575324760client # copying 1 paths to 'lb:50051'...4761client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4762worker1 # [ 85.554890] nix-grpc-daemon[777]: ts=2026-09-20T08:18:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575324763worker1 # [ 85.570255] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4764worker1 # [ 85.592131] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)4765worker1 # [ 85.604856] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 2/5)4766worker1 # [ 85.796187] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 395 ms (attempt 3/5)4767worker1 # [ 86.200148] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 4/5)4768worker1 # [ 86.283930] nix-grpc-daemon[777]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4769worker1 # [ 86.294202] nix-grpc-daemon[777]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4770worker1 # [ 86.516647] nix-daemon[840]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4771worker1 # [ 86.536159] nix-daemon[840]: reaped child process 1125, status = succeeded4772worker1 # [ 86.710855] nix-daemon[840]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)4773worker1 # [ 86.731772] nix-daemon[840]: reaped child process 1130, status = succeeded4774lb # [ 86.746566] niks3-server[1172]: 2026/09/20 08:18:31 INFO Received uploads request method=POST path=/api/pending_closures4775worker1 # [ 86.754276] nix-grpc-daemon[849]: time=2026-09-20T08:18:31.327Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4776worker1 # [ 86.755756] nix-grpc-daemon[849]: time=2026-09-20T08:18:31.327Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4777lb # [ 86.773215] niks3-server[1172]: 2026/09/20 08:18:31 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4778lb # [ 86.788937] niks3-server[1172]: 2026/09/20 08:18:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4779worker1 # [ 86.792808] nix-grpc-daemon[849]: time=2026-09-20T08:18:31.366Z level=INFO msg="Uploading 1 narinfos"4780lb # [ 86.801254] niks3-server[1172]: 2026/09/20 08:18:31 INFO Signed narinfos id=19 count=14781lb # [ 86.811204] niks3-server[1172]: 2026/09/20 08:18:31 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4782lb # [ 86.817022] niks3-server[1172]: 2026/09/20 08:18:31 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4783worker1 # [ 86.813961] nix-grpc-daemon[849]: time=2026-09-20T08:18:31.388Z level=INFO msg="Upload complete. (497ms)"4784lb # [ 86.821878] niks3-server[1172]: 2026/09/20 08:18:31 INFO Completed upload id=194785worker1 # [ 86.820448] nix-grpc-daemon[777]: ts=2026-09-20T08:18:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=1 paths=1 nar_bytes_in=5684786lb # [ 86.825044] niks3-server[1172]: 2026/09/20 08:18:31 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4787worker1 # [ 86.831876] nix-grpc-daemon[777]: ts=2026-09-20T08:18:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575324788worker1 # [ 86.842882] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4789worker1 # [ 86.847562] nix-daemon[840]: reaped child process 1116, status = succeeded4790client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4791worker1 # [ 86.860289] nix-daemon[840]: accepted connection from pid 777, user nix-grpc-daemon (trusted)4792client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4793worker1 # [ 87.686785] nix-daemon[840]: reaped child process 1134, status = succeeded4794worker1 # [ 87.849220] nix-daemon[840]: accepted connection from pid 1149, user nix-grpc-daemon (trusted)4795worker1 # [ 87.868529] nix-daemon[840]: reaped child process 1152, status = succeeded4796lb # [ 87.883171] niks3-server[1172]: 2026/09/20 08:18:32 INFO Received uploads request method=POST path=/api/pending_closures4797worker1 # [ 87.892388] nix-grpc-daemon[849]: time=2026-09-20T08:18:32.464Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4798lb # [ 87.915370] niks3-server[1172]: 2026/09/20 08:18:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4799worker1 # [ 87.921517] nix-grpc-daemon[849]: time=2026-09-20T08:18:32.495Z level=INFO msg="Uploading 1 narinfos"4800lb # [ 87.928510] niks3-server[1172]: 2026/09/20 08:18:32 INFO Signed narinfos id=20 count=14801lb # [ 87.935446] niks3-server[1172]: 2026/09/20 08:18:32 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4802lb # [ 87.941751] niks3-server[1172]: 2026/09/20 08:18:32 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4803lb # [ 87.950738] niks3-server[1172]: 2026/09/20 08:18:32 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4804worker1 # [ 87.953458] nix-grpc-daemon[849]: time=2026-09-20T08:18:32.527Z level=INFO msg="Upload complete. (282ms)"4805lb # [ 87.956023] niks3-server[1172]: 2026/09/20 08:18:32 INFO Completed upload id=204806lb # [ 87.958857] niks3-server[1172]: 2026/09/20 08:18:32 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4807worker1 # [ 87.960794] nix-grpc-daemon[777]: ts=2026-09-20T08:18:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=9 outputs=14808client # copying 1 paths...4809worker1 # [ 87.973080] nix-daemon[840]: reaped child process 1132, status = succeeded4810worker2 # [ 87.927775] nix-grpc-daemon[778]: ts=2026-09-20T08:18:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:498024811worker2 # [ 87.941215] nix-grpc-daemon[778]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4812client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4813worker2 # [ 87.987906] nix-grpc-daemon[778]: ts=2026-09-20T08:18:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:498024814worker2 # [ 87.994707] nix-grpc-daemon[778]: ts=2026-09-20T08:18:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49802 duration_s=0 paths=1 nar_bytes_out=1204815client # [ 88.123475] nix-daemon[920]: reaped child process 1024, status = succeeded4816client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.10 seconds)4817client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24818client # [ 88.338222] nix-daemon[920]: accepted connection from pid 1041, user root (trusted)4819client # this derivation will be built:4820client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4821worker2 # [ 88.530683] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:498024822client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux'...4823worker2 # [ 88.539080] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498024824worker2 # [ 88.550026] nix-grpc-daemon[887]: time=2026-09-20T08:18:33.124Z level=INFO msg="All 1 paths already cached"4825worker2 # [ 88.566960] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=debug event=want client=10 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04826worker2 # [ 88.574582] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14827worker2 # [ 88.583633] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498024828worker2 # [ 88.593934] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498024829worker2 # [ 88.609389] nix-grpc-daemon[887]: time=2026-09-20T08:18:33.184Z level=INFO msg="All 1 paths already cached"4830client # copying 1 paths to 'lb:50051'...4831client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4832worker2 # [ 88.620559] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:498024833worker2 # [ 88.635453] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4834worker2 # [ 88.837354] nix-daemon[846]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4835worker2 # [ 88.856363] nix-daemon[846]: reaped child process 1022, status = succeeded4836worker2 # [ 89.032525] nix-daemon[846]: accepted connection from pid 1024, user nix-grpc-daemon (trusted)4837worker2 # [ 89.050051] nix-daemon[846]: reaped child process 1027, status = succeeded4838lb # [ 89.111706] niks3-server[1172]: 2026/09/20 08:18:33 INFO Received uploads request method=POST path=/api/pending_closures4839worker2 # [ 89.075098] nix-grpc-daemon[887]: time=2026-09-20T08:18:33.650Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4840worker2 # [ 89.076676] nix-grpc-daemon[887]: time=2026-09-20T08:18:33.650Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4841lb # [ 89.140378] niks3-server[1172]: 2026/09/20 08:18:33 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4842lb # [ 89.153672] niks3-server[1172]: 2026/09/20 08:18:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4843worker2 # [ 89.112889] nix-grpc-daemon[887]: time=2026-09-20T08:18:33.687Z level=INFO msg="Uploading 1 narinfos"4844lb # [ 89.165584] niks3-server[1172]: 2026/09/20 08:18:33 INFO Signed narinfos id=21 count=14845lb # [ 89.176641] niks3-server[1172]: 2026/09/20 08:18:33 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4846lb # [ 89.180132] niks3-server[1172]: 2026/09/20 08:18:33 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4847worker2 # [ 89.137573] nix-grpc-daemon[887]: time=2026-09-20T08:18:33.712Z level=INFO msg="Upload complete. (484ms)"4848lb # [ 89.184484] niks3-server[1172]: 2026/09/20 08:18:33 INFO Completed upload id=214849lb # [ 89.187022] niks3-server[1172]: 2026/09/20 08:18:33 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4850worker2 # [ 89.144735] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49802 duration_s=0 paths=1 nar_bytes_in=5684851worker2 # [ 89.156993] nix-grpc-daemon[778]: ts=2026-09-20T08:18:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498024852worker2 # [ 89.161195] nix-daemon[846]: reaped child process 1017, status = succeeded4853worker2 # [ 89.168224] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4854client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4855worker2 # [ 89.182155] nix-daemon[846]: accepted connection from pid 778, user nix-grpc-daemon (trusted)4856client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4857worker2 # [ 90.801218] nix-daemon[846]: reaped child process 1031, status = succeeded4858worker2 # [ 90.965812] nix-daemon[846]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4859worker2 # [ 90.983219] nix-daemon[846]: reaped child process 1049, status = succeeded4860lb # [ 91.042348] niks3-server[1172]: 2026/09/20 08:18:35 INFO Received uploads request method=POST path=/api/pending_closures4861worker2 # [ 91.007816] nix-grpc-daemon[887]: time=2026-09-20T08:18:35.581Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4862lb # [ 91.069530] niks3-server[1172]: 2026/09/20 08:18:35 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4863lb # [ 91.082698] niks3-server[1172]: 2026/09/20 08:18:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4864worker2 # [ 91.043331] nix-grpc-daemon[887]: time=2026-09-20T08:18:35.618Z level=INFO msg="Uploading 1 narinfos"4865lb # [ 91.096206] niks3-server[1172]: 2026/09/20 08:18:35 INFO Signed narinfos id=22 count=14866lb # [ 91.099450] niks3-server[1172]: 2026/09/20 08:18:35 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4867lb # [ 91.113881] niks3-server[1172]: 2026/09/20 08:18:35 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4868worker2 # [ 91.072775] nix-grpc-daemon[887]: time=2026-09-20T08:18:35.647Z level=INFO msg="Upload complete. (277ms)"4869lb # [ 91.119692] niks3-server[1172]: 2026/09/20 08:18:35 INFO Completed upload id=224870lb # [ 91.124093] niks3-server[1172]: 2026/09/20 08:18:35 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4871worker2 # [ 91.080722] nix-grpc-daemon[778]: ts=2026-09-20T08:18:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49802 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=10 outputs=14872client # copying 1 paths...4873worker2 # [ 91.095142] nix-daemon[846]: reaped child process 1029, status = succeeded4874worker1 # [ 91.139890] nix-grpc-daemon[777]: ts=2026-09-20T08:18:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575324875worker1 # [ 91.153707] nix-grpc-daemon[777]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4876client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4877worker1 # [ 91.199646] nix-grpc-daemon[777]: ts=2026-09-20T08:18:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:575324878worker1 # [ 91.204957] nix-grpc-daemon[777]: ts=2026-09-20T08:18:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:57532 duration_s=0 paths=1 nar_bytes_out=1204879client # [ 91.286952] nix-daemon[920]: reaped child process 1044, status = succeeded4880client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 3.16 seconds)4881(finished: subtest: an input only one worker has still reaches the builder, in 10.46 seconds)4882subtest: upload whose reference the worker lacks is completed from the cache4883client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache4884client # this derivation will be built:4885client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4886client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4887client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.46 seconds)4888client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache4889client # this derivation will be built:4890client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4891client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4892client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.44 seconds)4893client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4894client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.20 seconds)4895worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4896worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.24 seconds)4897worker2 # [ 92.705955] systemd[1]: Stopping Nix worker-protocol over gRPC...4898lb # [ 92.764282] envoy[931]: conn peer=[2001:db8:1::1]:46886 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3863 tx=1767 ms=27684899lb # [ 92.764628] envoy[931]: conn peer=[2001:db8:1::1]:46890 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1014900lb # [ 92.764742] envoy[931]: conn peer=[2001:db8:1::1]:46896 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3712 tx=1917 ms=26174901lb # [ 92.764862] envoy[931]: conn peer=[2001:db8:1::1]:46908 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=714902lb # [ 92.765012] envoy[931]: conn peer=[2001:db8:1::1]:46918 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3713 tx=1918 ms=26824903lb # [ 92.765115] envoy[931]: conn peer=[2001:db8:1::1]:56672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=634904worker2 # [ 93.121756] nix-grpc-daemon[778]: ts=2026-09-20T08:18:37Z level=info event=signal_exit4905worker2 # [ 93.322296] nix-grpc-daemon[778]: ts=2026-09-20T08:18:37Z level=info event=scheduler_yield4906worker2 # [ 93.327606] nix-grpc-daemon[778]: ts=2026-09-20T08:18:37Z level=info event=scheduler_restarting addr=lb:500514907lb # [ 93.376418] niks3-server[1172]: 2026/09/20 08:18:37 INFO lead: released remote=192.168.1.4:512564908worker1 # [ 93.373863] nix-grpc-daemon[777]: ts=2026-09-20T08:18:37Z level=info event=scheduler_take_over4909worker2 # [ 93.332635] nix-grpc-daemon[778]: ts=2026-09-20T08:18:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:476884910worker1 # [ 93.378725] nix-grpc-daemon[777]: ts=2026-09-20T08:18:37Z level=info event=scheduler_restarting addr=lb:500514911worker2 # [ 93.337393] nix-grpc-daemon[778]: ts=2026-09-20T08:18:37Z level=info event=worker_gone addr=192.168.1.4:500514912lb # [ 93.387049] niks3-server[1172]: 2026/09/20 08:18:37 INFO lead: acquired remote=192.168.1.3:363544913worker2 # [ 93.343970] nix-grpc-daemon[778]: ts=2026-09-20T08:18:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:476884914worker2 # [ 93.348784] nix-grpc-daemon[778]: ts=2026-09-20T08:18:37Z level=info event=worker_gone addr=192.168.1.3:500514915worker2 # [ 93.528555] nix-grpc-daemon[778]: ts=2026-09-20T08:18:38Z level=info event=scheduler_connected addr=lb:500514916worker1 # [ 93.574892] nix-grpc-daemon[777]: ts=2026-09-20T08:18:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04917worker1 # [ 93.581785] nix-grpc-daemon[777]: ts=2026-09-20T08:18:38Z level=info event=scheduler_connected addr=lb:500514918worker1 # [ 93.599684] nix-grpc-daemon[777]: ts=2026-09-20T08:18:38Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4919worker1 # [ 93.800783] nix-grpc-daemon[777]: ts=2026-09-20T08:18:38Z level=info event=scheduler_connected addr=lb:500514920worker1 # [ 93.806109] nix-grpc-daemon[777]: ts=2026-09-20T08:18:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04921worker1 # [ 94.379691] nix-grpc-daemon[777]: ts=2026-09-20T08:18:38Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:537804922worker1 # [ 94.385170] nix-grpc-daemon[777]: ts=2026-09-20T08:18:38Z level=info event=worker_gone addr=192.168.1.4:500514923worker2 # [ 94.350700] nix-grpc-daemon[778]: ts=2026-09-20T08:18:38Z level=info event=scheduler_restarting4924worker2 # [ 94.365520] nix-grpc-daemon[778]: ts=2026-09-20T08:18:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4925worker2 # [ 94.393545] nix-daemon[846]: reaped child process 848, status = succeeded4926worker2 # [ 94.860868] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4927worker2 # [ 94.863903] systemd[1]: Stopped Nix worker-protocol over gRPC.4928worker2 # [ 94.873337] systemd[1]: nix-grpc-daemon.service: Consumed 2.161s CPU time over 1min 12.134s wall clock time, 54.5M memory peak, 69.5K incoming IP traffic, 125.4K outgoing IP traffic.4929worker2 # [ 94.884664] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4930worker2 # [ 94.886600] systemd[1]: Closed Nix worker-protocol over gRPC.4931worker2 # [ 94.891277] systemd[1]: nix-grpc-daemon.socket: Consumed 651us CPU time over 1min 24.103s wall clock time, 256K memory peak, 123.7K incoming IP traffic, 102.5K outgoing IP traffic.4932lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy4933lb: must succeed: curl -sf localhost:9901/clusters4934lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4935lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4936client: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&24937worker1 # [ 95.273862] nix-grpc-daemon[777]: ts=2026-09-20T08:18:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575324938worker1 # [ 95.283393] nix-grpc-daemon[777]: ts=2026-09-20T08:18:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:575324939client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4940client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.29 seconds)4941worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache4942worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)4943worker2 # [ 95.377937] systemd[1]: Listening on Nix worker-protocol over gRPC.4944worker1 # [ 95.502083] systemd[1]: Stopping Nix worker-protocol over gRPC...4945worker1 # [ 96.167210] nix-grpc-daemon[777]: ts=2026-09-20T08:18:40Z level=info event=signal_exit4946worker2 # [ 96.295514] systemd[1]: Starting Nix worker-protocol over gRPC...4947worker2 # [ 96.512699] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc4948worker2 # [ 96.515850] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=14949worker2 # [ 96.524095] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket4950worker2 # [ 96.525929] systemd[1]: Started Nix worker-protocol over gRPC.4951worker2 # [ 96.567312] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_connected addr=lb:500514952worker2 # [ 96.573288] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4953worker1 # [ 96.627176] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=scheduler_yield4954lb # [ 96.633530] niks3-server[1172]: 2026/09/20 08:18:41 INFO lead: released remote=192.168.1.3:363544955worker1 # [ 96.632919] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=scheduler_restarting addr=lb:500514956worker1 # [ 96.639111] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:537804957worker1 # [ 96.643847] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=worker_gone addr=192.168.1.3:500514958worker2 # [ 96.774540] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_connected addr=lb:500514959worker1 # [ 96.835706] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=scheduler_connected addr=lb:500514960worker1 # [ 96.843001] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4961worker2 # [ 96.799351] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4962worker1 # [ 97.042641] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=scheduler_connected addr=lb:500514963worker2 # [ 97.000349] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_connected addr=lb:500514964worker1 # [ 97.050081] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4965worker2 # [ 97.007449] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4966worker1 # [ 97.252805] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=scheduler_connected addr=lb:500514967worker2 # [ 97.208570] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_connected addr=lb:500514968worker1 # [ 97.258975] nix-grpc-daemon[777]: ts=2026-09-20T08:18:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4969worker2 # [ 97.215814] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4970worker1 # [ 97.460051] nix-grpc-daemon[777]: ts=2026-09-20T08:18:42Z level=info event=scheduler_connected addr=lb:500514971worker2 # [ 97.416880] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_connected addr=lb:500514972worker2 # [ 97.423305] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:41Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4973worker1 # [ 97.467083] nix-grpc-daemon[777]: ts=2026-09-20T08:18:42Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4974worker1 # [ 97.632181] nix-grpc-daemon[777]: ts=2026-09-20T08:18:42Z level=info event=scheduler_restarting4975worker2 # [ 97.625466] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:42Z level=info event=scheduler_connected addr=lb:500514976worker1 # [ 97.679894] nix-daemon[840]: reaped child process 842, status = succeeded4977worker1 # [ 97.695562] nix-grpc-daemon[777]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR4978worker1 # [ 97.701395] nix-grpc-daemon[777]: E0000 00:00:1789892322.246829 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 11, started true, shutdown true4979worker1 # [ 97.706163] nix-grpc-daemon[777]: ts=2026-09-20T08:18:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4980worker1 # [ 98.276874] nix-grpc-daemon[777]: E0000 00:00:1789892322.849199 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true4981worker1 # [ 98.332484] nix-grpc-daemon[777]: E0000 00:00:1789892322.906649 820 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true4982worker2 # [ 98.751365] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"4983worker1 # [ 98.813735] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4984worker1 # [ 98.821234] systemd[1]: Stopped Nix worker-protocol over gRPC.4985worker1 # [ 98.823826] systemd[1]: nix-grpc-daemon.service: Consumed 3.698s CPU time over 1min 16.048s wall clock time, 62M memory peak, 94.7K incoming IP traffic, 172.3K outgoing IP traffic.4986worker1 # [ 98.846757] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4987worker1 # [ 98.849346] systemd[1]: Closed Nix worker-protocol over gRPC.4988lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy4989lb: must succeed: curl -sf localhost:9901/clusters4990worker1 # [ 98.852585] systemd[1]: nix-grpc-daemon.socket: Consumed 576us CPU time over 1min 28.020s wall clock time, 256K memory peak, 139.2K incoming IP traffic, 114.5K outgoing IP traffic.4991lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4992lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)4993client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&24994worker2 # [ 98.951301] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500514995worker2 # [ 98.959784] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"4996worker2 # [ 99.160679] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500514997worker2 # [ 99.165932] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"4998worker2 # [ 99.183888] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362864999worker2 # [ 99.200937] nix-daemon[846]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)5000worker2 # [ 99.226471] nix-grpc-daemon[1070]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5)5001worker2 # [ 99.324174] nix-grpc-daemon[1070]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 109 ms (attempt 2/5)5002worker2 # [ 99.367229] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:43Z level=info event=scheduler_connected addr=lb:500515003worker2 # [ 99.375033] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:43Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5004worker2 # [ 99.441032] nix-grpc-daemon[1070]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 283 ms (attempt 3/5)5005worker2 # [ 99.576126] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_connected addr=lb:500515006worker2 # [ 99.581758] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5007worker2 # [ 99.724871] nix-grpc-daemon[1070]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 665 ms (attempt 4/5)5008worker2 # [ 99.784422] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_connected addr=lb:500515009worker2 # [ 99.791774] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5010worker2 # [ 99.993514] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_connected addr=lb:500515011worker2 # [ 99.999439] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5012worker2 # [ 100.200569] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_connected addr=lb:500515013worker2 # [ 100.207889] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5014worker2 # [ 100.391361] nix-grpc-daemon[1070]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5015worker2 # [ 100.409959] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_connected addr=lb:500515016worker2 # [ 100.417672] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:44Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5017client # copying 2 paths...5018client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5019worker2 # [ 100.438849] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362865020client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5021worker2 # [ 100.463491] nix-daemon[846]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)5022worker2 # [ 100.505136] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=niks3_push_started pid=11265023worker2 # [ 100.618100] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=scheduler_connected addr=lb:500515024worker2 # [ 100.624522] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5025worker2 # [ 100.825967] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=scheduler_connected addr=lb:500515026worker2 # [ 100.829360] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5027worker2 # [ 100.982652] nix-daemon[846]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)5028worker2 # [ 101.003247] nix-daemon[846]: reaped child process 1142, status = succeeded5029lb # [ 101.067880] niks3-server[1172]: 2026/09/20 08:18:45 INFO Received uploads request method=POST path=/api/pending_closures5030worker2 # [ 101.031128] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=scheduler_connected addr=lb:500515031worker2 # [ 101.037990] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5032lb # [ 101.090171] niks3-server[1172]: 2026/09/20 08:18:45 INFO Received uploads request method=POST path=/api/pending_closures5033worker2 # [ 101.056548] nix-grpc-daemon[1126]: time=2026-09-20T08:18:45.629Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5034worker2 # [ 101.059382] nix-grpc-daemon[1126]: time=2026-09-20T08:18:45.629Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5035worker2 # [ 101.063491] nix-grpc-daemon[1126]: time=2026-09-20T08:18:45.629Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5036lb # [ 101.179380] niks3-server[1172]: 2026/09/20 08:18:45 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5037lb # [ 101.192549] niks3-server[1172]: 2026/09/20 08:18:45 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5038lb # [ 101.212323] niks3-server[1172]: 2026/09/20 08:18:45 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5039lb # [ 101.220103] niks3-server[1172]: 2026/09/20 08:18:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5040lb # [ 101.227545] niks3-server[1172]: 2026/09/20 08:18:45 INFO Signed narinfos id=23 count=15041worker2 # [ 101.184220] nix-grpc-daemon[1126]: time=2026-09-20T08:18:45.756Z level=INFO msg="Uploading 3 narinfos"5042lb # [ 101.234640] niks3-server[1172]: 2026/09/20 08:18:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5043lb # [ 101.241089] niks3-server[1172]: 2026/09/20 08:18:45 INFO Signed narinfos id=24 count=25044lb # [ 101.243085] niks3-server[1172]: 2026/09/20 08:18:45 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5045lb # [ 101.264408] niks3-server[1172]: 2026/09/20 08:18:45 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5046lb # [ 101.276180] niks3-server[1172]: 2026/09/20 08:18:45 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5047lb # [ 101.283366] niks3-server[1172]: 2026/09/20 08:18:45 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5048worker2 # [ 101.239479] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=scheduler_connected addr=lb:500515049lb # [ 101.294578] niks3-server[1172]: 2026/09/20 08:18:45 INFO Completed upload id=235050lb # [ 101.296658] niks3-server[1172]: 2026/09/20 08:18:45 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5051lb # [ 101.301824] niks3-server[1172]: 2026/09/20 08:18:45 INFO Completed upload id=245052lb # [ 101.303939] niks3-server[1172]: 2026/09/20 08:18:45 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5053worker2 # [ 101.247735] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5054worker2 # [ 101.268753] nix-grpc-daemon[1126]: time=2026-09-20T08:18:45.833Z level=INFO msg="Upload complete. (482ms)"5055worker2 # [ 101.276818] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36286 duration_s=0 paths=2 nar_bytes_in=3125056worker2 # [ 101.297883] nix-daemon[846]: reaped child process 1124, status = succeeded5057client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 2.43 seconds)5058worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5059worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.05 seconds)5060worker2 # [ 101.449266] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:46Z level=info event=scheduler_connected addr=lb:500515061worker2 # [ 101.457394] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5062worker1 # [ 101.529443] systemd[1]: Listening on Nix worker-protocol over gRPC.5063worker1 # [ 101.538748] systemd[1]: Starting Nix worker-protocol over gRPC...5064lb # [ 101.584452] niks3-server[1172]: 2026/09/20 08:18:46 INFO lead: acquired remote=192.168.1.4:479345065worker2 # [ 101.538932] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:46Z level=info event=scheduler_take_over5066worker2 # [ 101.658421] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:46Z level=info event=scheduler_connected addr=lb:500515067worker2 # [ 101.664509] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05068worker1 # [ 101.779041] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5069worker1 # [ 101.782241] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15070worker1 # [ 101.792486] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5071worker1 # [ 101.794323] systemd[1]: Started Nix worker-protocol over gRPC.5072lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5073lb: must succeed: curl -sf localhost:9901/clusters5074worker1 # [ 101.858157] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:46Z level=info event=scheduler_connected addr=lb:500515075worker2 # [ 101.818561] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05076lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.14 seconds)5077lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.14 seconds)5078(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.63 seconds)5079subtest: scheduler down: the other node takes the lock and keeps it5080lb: must succeed: curl -sf localhost:9901/clusters5081lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5082lb: must succeed: curl -sf localhost:9901/clusters5083lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5084lb: must succeed: curl -sf localhost:9901/clusters5085lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5086lb: must succeed: curl -sf localhost:9901/clusters5087lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5088worker2 # [ 102.234990] systemd[1]: Stopping Nix worker-protocol over gRPC...5089worker2 # [ 102.529733] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=signal_exit5090lb # [ 102.764265] envoy[931]: conn peer=[2001:db8:1::4]:47226 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=17244 tx=4543 ms=711855091lb # [ 102.764795] envoy[931]: conn peer=[2001:db8:1::1]:56674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=295092lb # [ 102.764967] envoy[931]: conn peer=[2001:db8:1::3]:53438 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=19058 tx=5061 ms=744885093lb # [ 102.765187] envoy[931]: conn peer=[2001:db8:1::1]:56684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1008 tx=282 ms=21035094worker2 # [ 102.743383] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=scheduler_yield5095lb # [ 102.795309] niks3-server[1172]: 2026/09/20 08:18:47 INFO lead: released remote=192.168.1.4:479345096worker1 # [ 102.795007] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:47Z level=info event=scheduler_restarting addr=lb:500515097worker2 # [ 102.755876] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:362825098worker2 # [ 102.761879] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=worker_gone addr=192.168.1.3:500515099worker2 # [ 102.770138] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=scheduler_restarting addr=lb:500515100worker2 # [ 102.774488] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:362825101worker2 # [ 102.779085] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=worker_gone addr=192.168.1.4:500515102worker1 # [ 102.993098] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:47Z level=info event=scheduler_connected addr=lb:500515103worker1 # [ 103.007605] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5104worker2 # [ 102.969245] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=scheduler_connected addr=lb:500515105worker2 # [ 102.994178] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5106worker1 # [ 103.209111] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:47Z level=info event=scheduler_connected addr=lb:500515107worker1 # [ 103.222333] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5108worker2 # [ 103.195351] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=scheduler_connected addr=lb:500515109worker2 # [ 103.203658] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5110worker1 # [ 103.427087] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:47Z level=info event=scheduler_connected addr=lb:500515111worker1 # [ 103.434031] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5112worker2 # [ 103.405390] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=scheduler_connected addr=lb:500515113worker2 # [ 103.414201] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:47Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5114worker1 # [ 103.634638] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:48Z level=info event=scheduler_connected addr=lb:500515115worker1 # [ 103.641833] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5116worker2 # [ 103.617753] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:48Z level=info event=scheduler_connected addr=lb:500515117worker2 # [ 103.622050] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5118worker2 # [ 103.744748] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:48Z level=info event=scheduler_restarting5119worker2 # [ 103.780994] nix-daemon[846]: reaped child process 1119, status = succeeded5120worker1 # [ 103.845435] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:48Z level=info event=scheduler_connected addr=lb:500515121worker1 # [ 103.849607] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:48Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5122worker2 # [ 103.839740] nix-grpc-daemon[1070]: ts=2026-09-20T08:18:48Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5123worker1 # [ 104.046359] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:48Z level=info event=scheduler_connected addr=lb:500515124worker2 # [ 104.005555] nix-grpc-daemon[1070]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5125worker2 # [ 104.010288] nix-grpc-daemon[1070]: E0000 00:00:1789892328.580362 1113 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5126worker1 # [ 104.589057] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5127worker2 # [ 104.564995] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5128worker2 # [ 104.577174] systemd[1]: Stopped Nix worker-protocol over gRPC.5129worker2 # [ 104.579805] systemd[1]: nix-grpc-daemon.service: Consumed 625ms CPU time over 8.271s wall clock time, 33.3M memory peak, 20.9K incoming IP traffic, 28.6K outgoing IP traffic.5130worker2 # [ 104.594669] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5131worker2 # [ 104.601884] systemd[1]: Closed Nix worker-protocol over gRPC.5132worker2 # [ 104.603863] systemd[1]: nix-grpc-daemon.socket: Consumed 781us CPU time over 9.224s wall clock time, 256K memory peak, 30.7K incoming IP traffic, 24.3K outgoing IP traffic.5133worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5134worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5135worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5136worker1 # [ 104.789926] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_connected addr=lb:500515137worker1 # [ 104.795024] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5138worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.06 seconds)5139worker1 # [ 104.997578] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_connected addr=lb:500515140worker1 # [ 105.002723] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5141worker1 # [ 105.203564] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_connected addr=lb:500515142worker1 # [ 105.210775] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5143worker1 # [ 105.411812] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_connected addr=lb:500515144worker1 # [ 105.417311] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:49Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5145worker1 # [ 105.617819] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:50Z level=info event=scheduler_connected addr=lb:500515146worker1 # [ 105.624906] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5147worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5148worker1 # [ 105.826468] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:50Z level=info event=scheduler_connected addr=lb:500515149worker1 # [ 105.839801] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:50Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5150worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.10 seconds)5151worker1 # [ 106.035507] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:50Z level=info event=scheduler_connected addr=lb:500515152worker1 # [ 106.040736] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:50Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5153worker1 # [ 106.241574] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:50Z level=info event=scheduler_connected addr=lb:500515154worker1 # [ 106.246920] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:50Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5155worker1 # [ 106.447420] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=info event=scheduler_connected addr=lb:500515156worker1 # [ 106.454190] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5157worker1 # [ 106.655219] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=info event=scheduler_connected addr=lb:500515158worker1 # [ 106.660514] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5159lb # [ 106.810727] niks3-server[1172]: 2026/09/20 08:18:51 INFO lead: acquired remote=192.168.1.3:469745160worker1 # [ 106.813957] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=info event=scheduler_take_over5161worker1 # [ 106.861191] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=info event=scheduler_connected addr=lb:500515162worker1 # [ 106.866555] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05163worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5164worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.08 seconds)5165client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag on-standby >&25166worker1 # [ 107.260327] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461345167worker1 # [ 107.274732] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5168client # don't know how to build these paths:5169client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5170worker1 # [ 107.323691] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05171worker1 # [ 107.330842] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15172worker1 # [ 107.370303] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461345173worker1 # [ 107.387621] nix-grpc-daemon[1176]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 38 ms (attempt 1/5)5174worker1 # [ 107.433444] nix-grpc-daemon[1176]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 105 ms (attempt 2/5)5175worker1 # [ 107.546112] nix-grpc-daemon[1176]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 282 ms (attempt 3/5)5176worker1 # [ 107.828657] nix-grpc-daemon[1176]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 533 ms (attempt 4/5)5177worker1 # [ 108.363954] nix-grpc-daemon[1176]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5178worker1 # [ 108.390003] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461345179client # copying 1 paths to 'lb:50051'...5180client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5181worker1 # [ 108.406120] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461345182worker1 # [ 108.424421] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5183worker1 # [ 108.453392] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:53Z level=info event=niks3_push_started pid=12555184worker1 # [ 108.859633] nix-daemon[840]: accepted connection from pid 1268, user nix-grpc-daemon (trusted)5185worker1 # [ 108.877130] nix-daemon[840]: reaped child process 1271, status = succeeded5186worker1 # [ 109.056305] nix-daemon[840]: accepted connection from pid 1273, user nix-grpc-daemon (trusted)5187worker1 # [ 109.073264] nix-daemon[840]: reaped child process 1276, status = succeeded5188lb # [ 109.092291] niks3-server[1172]: 2026/09/20 08:18:53 INFO Received uploads request method=POST path=/api/pending_closures5189worker1 # [ 109.103786] nix-grpc-daemon[1255]: time=2026-09-20T08:18:53.676Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5190worker1 # [ 109.105748] nix-grpc-daemon[1255]: time=2026-09-20T08:18:53.676Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5191lb # [ 109.144579] niks3-server[1172]: 2026/09/20 08:18:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5192lb # [ 109.166481] niks3-server[1172]: 2026/09/20 08:18:53 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5193lb # [ 109.179788] niks3-server[1172]: 2026/09/20 08:18:53 INFO Signed narinfos id=25 count=15194worker1 # [ 109.177373] nix-grpc-daemon[1255]: time=2026-09-20T08:18:53.750Z level=INFO msg="Uploading 1 narinfos"5195lb # [ 109.184402] niks3-server[1172]: 2026/09/20 08:18:53 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5196lb # [ 109.191745] niks3-server[1172]: 2026/09/20 08:18:53 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5197lb # [ 109.203451] niks3-server[1172]: 2026/09/20 08:18:53 INFO Completed upload id=255198worker1 # [ 109.201259] nix-grpc-daemon[1255]: time=2026-09-20T08:18:53.775Z level=INFO msg="Upload complete. (521ms)"5199lb # [ 109.211941] niks3-server[1172]: 2026/09/20 08:18:53 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5200worker1 # [ 109.212357] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46134 duration_s=0 paths=1 nar_bytes_in=5205201worker1 # [ 109.231305] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461345202worker1 # [ 109.241692] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5203worker1 # [ 109.245044] nix-daemon[840]: reaped child process 1253, status = succeeded5204client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5205client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5206worker1 # [ 109.259842] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5207client # farm-a> LOG-farm-a-on-standby5208worker1 # [ 110.307318] nix-daemon[840]: reaped child process 1280, status = succeeded5209worker1 # [ 110.493718] nix-daemon[840]: accepted connection from pid 1295, user nix-grpc-daemon (trusted)5210worker1 # [ 110.511184] nix-daemon[840]: reaped child process 1298, status = succeeded5211lb # [ 110.526070] niks3-server[1172]: 2026/09/20 08:18:55 INFO Received uploads request method=POST path=/api/pending_closures5212worker1 # [ 110.540576] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.111Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5213worker1 # [ 110.542736] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.111Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5214lb # [ 110.573972] niks3-server[1172]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5215lb # [ 110.587958] niks3-server[1172]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5216lb # [ 110.600276] niks3-server[1172]: 2026/09/20 08:18:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5217lb # [ 110.606683] niks3-server[1172]: 2026/09/20 08:18:55 INFO Signed narinfos id=26 count=15218worker1 # [ 110.604231] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.177Z level=INFO msg="Uploading 1 narinfos"5219lb # [ 110.610010] niks3-server[1172]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5220lb # [ 110.624099] niks3-server[1172]: 2026/09/20 08:18:55 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5221lb # [ 110.629758] niks3-server[1172]: 2026/09/20 08:18:55 INFO Completed upload id=265222worker1 # [ 110.627277] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.201Z level=INFO msg="Upload complete. (355ms)"5223lb # [ 110.632744] niks3-server[1172]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5224worker1 # [ 110.653016] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46134 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15225worker1 # [ 110.668395] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:55Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05226worker1 # [ 110.671875] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:55Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15227worker1 # [ 110.676715] nix-daemon[840]: reaped child process 1278, status = succeeded5228worker1 # [ 110.695468] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461345229worker1 # [ 110.710776] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461345230worker1 # [ 110.726496] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.300Z level=INFO msg="All 1 paths already cached"5231client # copying 1 paths to 'lb:50051'...5232client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5233worker1 # [ 110.735516] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461345234worker1 # [ 110.750657] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5235worker1 # [ 110.952798] nix-daemon[840]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5236worker1 # [ 110.973138] nix-daemon[840]: reaped child process 1307, status = succeeded5237worker1 # [ 111.145477] nix-daemon[840]: accepted connection from pid 1309, user nix-grpc-daemon (trusted)5238worker1 # [ 111.164332] nix-daemon[840]: reaped child process 1312, status = succeeded5239lb # [ 111.179564] niks3-server[1172]: 2026/09/20 08:18:55 INFO Received uploads request method=POST path=/api/pending_closures5240worker1 # [ 111.187144] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.760Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5241worker1 # [ 111.189047] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.761Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5242lb # [ 111.212080] niks3-server[1172]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5243lb # [ 111.223634] niks3-server[1172]: 2026/09/20 08:18:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5244worker1 # [ 111.228740] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.802Z level=INFO msg="Uploading 1 narinfos"5245lb # [ 111.233129] niks3-server[1172]: 2026/09/20 08:18:55 INFO Signed narinfos id=27 count=15246lb # [ 111.239190] niks3-server[1172]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5247lb # [ 111.248659] niks3-server[1172]: 2026/09/20 08:18:55 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5248lb # [ 111.257099] niks3-server[1172]: 2026/09/20 08:18:55 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5249lb # [ 111.261896] niks3-server[1172]: 2026/09/20 08:18:55 INFO Completed upload id=275250worker1 # [ 111.259436] nix-grpc-daemon[1255]: time=2026-09-20T08:18:55.833Z level=INFO msg="Upload complete. (490ms)"5251worker1 # [ 111.267552] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46134 duration_s=0 paths=1 nar_bytes_in=6405252worker1 # [ 111.283111] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461345253worker1 # [ 111.289667] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5254worker1 # [ 111.294446] nix-daemon[840]: reaped child process 1302, status = succeeded5255client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5256worker1 # [ 111.306182] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5257client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5258client # farm-b> LOG-farm-b-on-standby5259worker1 # [ 111.931089] nix-daemon[840]: reaped child process 1316, status = succeeded5260worker1 # [ 112.089636] nix-daemon[840]: accepted connection from pid 1331, user nix-grpc-daemon (trusted)5261worker1 # [ 112.108431] nix-daemon[840]: reaped child process 1334, status = succeeded5262lb # [ 112.123553] niks3-server[1172]: 2026/09/20 08:18:56 INFO Received uploads request method=POST path=/api/pending_closures5263worker1 # [ 112.133826] nix-grpc-daemon[1255]: time=2026-09-20T08:18:56.707Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5264worker1 # [ 112.135334] nix-grpc-daemon[1255]: time=2026-09-20T08:18:56.707Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5265lb # [ 112.160406] niks3-server[1172]: 2026/09/20 08:18:56 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5266lb # [ 112.171079] niks3-server[1172]: 2026/09/20 08:18:56 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5267lb # [ 112.181232] niks3-server[1172]: 2026/09/20 08:18:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5268worker1 # [ 112.184046] nix-grpc-daemon[1255]: time=2026-09-20T08:18:56.757Z level=INFO msg="Uploading 1 narinfos"5269lb # [ 112.190435] niks3-server[1172]: 2026/09/20 08:18:56 INFO Signed narinfos id=28 count=15270lb # [ 112.195176] niks3-server[1172]: 2026/09/20 08:18:56 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5271lb # [ 112.201198] niks3-server[1172]: 2026/09/20 08:18:56 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5272lb # [ 112.210644] niks3-server[1172]: 2026/09/20 08:18:56 INFO Completed upload id=285273worker1 # [ 112.208295] nix-grpc-daemon[1255]: time=2026-09-20T08:18:56.782Z level=INFO msg="Upload complete. (290ms)"5274lb # [ 112.213032] niks3-server[1172]: 2026/09/20 08:18:56 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5275worker1 # [ 112.222556] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46134 duration_s=0 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15276worker1 # [ 112.237319] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:56Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05277worker1 # [ 112.241109] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:56Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15278worker1 # [ 112.244738] nix-daemon[840]: reaped child process 1314, status = succeeded5279worker1 # [ 112.265561] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461345280worker1 # [ 112.279634] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461345281client # copying 1 paths to 'lb:50051'...5282worker1 # [ 112.294350] nix-grpc-daemon[1255]: time=2026-09-20T08:18:56.868Z level=INFO msg="All 2 paths already cached"5283client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5284worker1 # [ 112.301127] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461345285worker1 # [ 112.314879] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5286worker1 # [ 112.513689] nix-daemon[840]: accepted connection from pid 1340, user nix-grpc-daemon (trusted)5287worker1 # [ 112.533642] nix-daemon[840]: reaped child process 1343, status = succeeded5288worker1 # [ 112.707881] nix-daemon[840]: accepted connection from pid 1345, user nix-grpc-daemon (trusted)5289worker1 # [ 112.727662] nix-daemon[840]: reaped child process 1348, status = succeeded5290lb # [ 112.741349] niks3-server[1172]: 2026/09/20 08:18:57 INFO Received uploads request method=POST path=/api/pending_closures5291worker1 # [ 112.748651] nix-grpc-daemon[1255]: time=2026-09-20T08:18:57.322Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5292worker1 # [ 112.750177] nix-grpc-daemon[1255]: time=2026-09-20T08:18:57.322Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5293lb # [ 112.765489] envoy[931]: conn peer=[2001:db8:1::4]:48852 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3473 tx=1102 ms=72605294lb # [ 112.777960] niks3-server[1172]: 2026/09/20 08:18:57 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5295lb # [ 112.790270] niks3-server[1172]: 2026/09/20 08:18:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5296worker1 # [ 112.794942] nix-grpc-daemon[1255]: time=2026-09-20T08:18:57.368Z level=INFO msg="Uploading 1 narinfos"5297lb # [ 112.803263] niks3-server[1172]: 2026/09/20 08:18:57 INFO Signed narinfos id=29 count=15298lb # [ 112.810255] niks3-server[1172]: 2026/09/20 08:18:57 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5299lb # [ 112.820515] niks3-server[1172]: 2026/09/20 08:18:57 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5300lb # [ 112.825315] niks3-server[1172]: 2026/09/20 08:18:57 INFO Completed upload id=295301worker1 # [ 112.822855] nix-grpc-daemon[1255]: time=2026-09-20T08:18:57.397Z level=INFO msg="Upload complete. (493ms)"5302lb # [ 112.828132] niks3-server[1172]: 2026/09/20 08:18:57 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5303worker1 # [ 112.832890] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46134 duration_s=0 paths=1 nar_bytes_in=7685304worker1 # [ 112.840326] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461345305worker1 # [ 112.850192] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5306worker1 # [ 112.854441] nix-daemon[840]: reaped child process 1338, status = succeeded5307client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5308worker1 # [ 112.866989] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5309client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5310client # farm-top> LOG-farm-top-on-standby5311worker1 # [ 113.710233] nix-daemon[840]: reaped child process 1352, status = succeeded5312worker1 # [ 113.897020] nix-daemon[840]: accepted connection from pid 1367, user nix-grpc-daemon (trusted)5313worker1 # [ 113.915290] nix-daemon[840]: reaped child process 1370, status = succeeded5314lb # [ 113.930135] niks3-server[1172]: 2026/09/20 08:18:58 INFO Received uploads request method=POST path=/api/pending_closures5315worker1 # [ 113.940062] nix-grpc-daemon[1255]: time=2026-09-20T08:18:58.512Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5316worker1 # [ 113.941698] nix-grpc-daemon[1255]: time=2026-09-20T08:18:58.512Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5317lb # [ 113.971211] niks3-server[1172]: 2026/09/20 08:18:58 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5318worker1 # [ 113.981340] nix-grpc-daemon[1255]: time=2026-09-20T08:18:58.555Z level=INFO msg="Uploading 1 narinfos"5319lb # [ 113.982044] niks3-server[1172]: 2026/09/20 08:18:58 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5320lb # [ 113.995257] niks3-server[1172]: 2026/09/20 08:18:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5321lb # [ 114.000659] niks3-server[1172]: 2026/09/20 08:18:58 INFO Signed narinfos id=30 count=15322lb # [ 114.006912] niks3-server[1172]: 2026/09/20 08:18:58 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5323lb # [ 114.010488] niks3-server[1172]: 2026/09/20 08:18:58 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5324lb # [ 114.015016] niks3-server[1172]: 2026/09/20 08:18:58 INFO Completed upload id=305325lb # [ 114.017396] niks3-server[1172]: 2026/09/20 08:18:58 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5326worker1 # [ 114.012498] nix-grpc-daemon[1255]: time=2026-09-20T08:18:58.586Z level=INFO msg="Upload complete. (333ms)"5327worker1 # [ 114.029308] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46134 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15328worker1 # [ 114.042003] nix-daemon[840]: reaped child process 1350, status = succeeded5329client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag on-standby >&2, in 7.09 seconds)5330client: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag on-standby outPath5331client: (finished: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag on-standby outPath, in 0.24 seconds)5332worker2 # [ 114.358867] systemd[1]: Listening on Nix worker-protocol over gRPC.5333worker2 # [ 114.370133] systemd[1]: Starting Nix worker-protocol over gRPC...5334worker2 # [ 114.569935] nix-grpc-daemon[1163]: ts=2026-09-20T08:18:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5335worker2 # [ 114.574007] nix-grpc-daemon[1163]: ts=2026-09-20T08:18:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15336worker2 # [ 114.581936] nix-grpc-daemon[1163]: ts=2026-09-20T08:18:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5337worker2 # [ 114.583674] systemd[1]: Started Nix worker-protocol over gRPC.5338lb: must succeed: curl -sf localhost:9901/clusters5339worker2 # [ 114.640321] nix-grpc-daemon[1163]: ts=2026-09-20T08:18:59Z level=info event=scheduler_connected addr=lb:500515340worker1 # [ 114.687493] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5342worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5343worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)5344lb: must succeed: curl -sf localhost:9901/clusters5345lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5346lb: must succeed: curl -sf localhost:9901/clusters5347lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5348worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5349worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)5350(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.10 seconds)5351subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5352client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5353client # [ 115.100560] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5354client # Running as unit: rs.service; invocation ID: 839f471731954a4492a59fa0b8c3f02b5355client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds)5356client # [ 115.356523] nix[1143]: don't know how to build these paths:5357client # [ 115.357648] nix[1143]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5358worker1 # [ 115.396153] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461345359worker1 # [ 115.409884] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:59Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05360worker1 # [ 115.416989] nix-grpc-daemon[1176]: ts=2026-09-20T08:18:59Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15361worker1 # [ 115.457691] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461345362worker1 # [ 115.479015] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461345363client # [ 115.456174] nix[1143]: copying 1 paths to 'lb:50051'...5364worker1 # [ 115.497691] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461345365client # [ 115.458609] nix[1143]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5366worker1 # [ 115.515474] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5367worker1 # [ 115.723391] nix-daemon[840]: accepted connection from pid 1385, user nix-grpc-daemon (trusted)5368worker1 # [ 115.740553] nix-daemon[840]: reaped child process 1388, status = succeeded5369worker1 # [ 115.919227] nix-daemon[840]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5370worker1 # [ 115.935721] nix-daemon[840]: reaped child process 1393, status = succeeded5371lb # [ 115.954145] niks3-server[1172]: 2026/09/20 08:19:00 INFO Received uploads request method=POST path=/api/pending_closures5372worker1 # [ 115.961841] nix-grpc-daemon[1255]: time=2026-09-20T08:19:00.535Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5373worker1 # [ 115.963311] nix-grpc-daemon[1255]: time=2026-09-20T08:19:00.535Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5374lb # [ 115.982878] niks3-server[1172]: 2026/09/20 08:19:00 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5375lb # [ 115.995474] niks3-server[1172]: 2026/09/20 08:19:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5376worker1 # [ 115.999812] nix-grpc-daemon[1255]: time=2026-09-20T08:19:00.573Z level=INFO msg="Uploading 1 narinfos"5377lb # [ 116.006171] niks3-server[1172]: 2026/09/20 08:19:00 INFO Signed narinfos id=31 count=15378lb # [ 116.013512] niks3-server[1172]: 2026/09/20 08:19:00 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5379lb # [ 116.024196] niks3-server[1172]: 2026/09/20 08:19:00 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5380lb # [ 116.030010] niks3-server[1172]: 2026/09/20 08:19:00 INFO Completed upload id=315381worker1 # [ 116.027023] nix-grpc-daemon[1255]: time=2026-09-20T08:19:00.601Z level=INFO msg="Upload complete. (494ms)"5382lb # [ 116.032523] niks3-server[1172]: 2026/09/20 08:19:00 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5383worker1 # [ 116.036929] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46134 duration_s=0 paths=1 nar_bytes_in=4165384worker1 # [ 116.044644] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461345385worker1 # [ 116.055501] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5386worker1 # [ 116.060961] nix-daemon[840]: reaped child process 1383, status = succeeded5387worker1 # [ 116.071745] nix-daemon[840]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5388client # [ 116.029256] nix[1143]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5389lb: must succeed: curl -sf localhost:9901/clusters5390lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.12 seconds)5391lb: must succeed: curl -sf localhost:9901/clusters5392lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5393lb: must succeed: curl -sf localhost:9901/clusters5394lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5395lb: must succeed: curl -sf localhost:9901/clusters5396lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5397worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5398worker2 # [ 118.063818] systemd-journald[267]: Received client request to rotate journal, rotating.5399worker2 # [ 118.084395] systemd-journald[267]: Vacuuming done, freed 0B of archived journals from /var/log/journal/de48d5f8b7fc48bba7a4754c5966c64c.5400worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5401worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5402worker1 # [ 118.186683] systemd[1]: Stopping Nix worker-protocol over gRPC...5403worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5404??? Warning (UserWarning): sleep(): The 'secs' argument is deprecated. Use 'duration' instead.5405 File "/nix/store/sqpbfna43hn5hr5aj1l73jgim4hg0gwn-nixos-test-driver-1.1/lib/python3.14/site-packages/test_driver/machine/__init__.py", line 5735406client: must succeed: sleep 5.05407worker1 # [ 118.803876] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:03Z level=info event=draining5408lb # [ 122.764821] envoy[931]: conn peer=[2001:db8:1::1]:43006 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9484 tx=3837 ms=67865409client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5410worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5411worker1 # [ 123.727236] nix-daemon[840]: reaped child process 1397, status = succeeded5412worker1 # [ 123.904986] nix-daemon[840]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)5413worker1 # [ 123.924996] nix-daemon[840]: reaped child process 1440, status = succeeded5414lb # [ 123.939477] niks3-server[1172]: 2026/09/20 08:19:08 INFO Received uploads request method=POST path=/api/pending_closures5415worker1 # [ 123.948385] nix-grpc-daemon[1255]: time=2026-09-20T08:19:08.521Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5416worker1 # [ 123.949835] nix-grpc-daemon[1255]: time=2026-09-20T08:19:08.521Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5417lb # [ 123.979778] niks3-server[1172]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5418lb # [ 123.995026] niks3-server[1172]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5419lb # [ 124.009430] niks3-server[1172]: 2026/09/20 08:19:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5420worker1 # [ 124.011934] nix-grpc-daemon[1255]: time=2026-09-20T08:19:08.585Z level=INFO msg="Uploading 1 narinfos"5421lb # [ 124.017441] niks3-server[1172]: 2026/09/20 08:19:08 INFO Signed narinfos id=32 count=15422lb # [ 124.022823] niks3-server[1172]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5423lb # [ 124.031176] niks3-server[1172]: 2026/09/20 08:19:08 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5424lb # [ 124.038204] niks3-server[1172]: 2026/09/20 08:19:08 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5425lb # [ 124.042909] niks3-server[1172]: 2026/09/20 08:19:08 INFO Completed upload id=325426worker1 # [ 124.040415] nix-grpc-daemon[1255]: time=2026-09-20T08:19:08.614Z level=INFO msg="Upload complete. (348ms)"5427worker1 # [ 124.053178] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46134 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15428worker1 # [ 124.071745] nix-daemon[840]: reaped child process 1395, status = succeeded5429client # [ 124.054007] systemd[1]: rs.service: Deactivated successfully.5430client # [ 124.058390] systemd[1]: rs.service: Consumed 169ms CPU time over 8.957s wall clock time, 36.9M memory peak, 8.3K incoming IP traffic, 9K outgoing IP traffic.5431worker1 # [ 124.804839] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=signal_exit5432worker1 # [ 125.035277] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=scheduler_yield5433lb # [ 125.041370] niks3-server[1172]: 2026/09/20 08:19:09 INFO lead: released remote=192.168.1.3:469745434worker2 # [ 124.998070] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:09Z level=info event=scheduler_restarting addr=lb:500515435worker1 # [ 125.051129] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:461245436worker1 # [ 125.059231] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=worker_gone addr=192.168.1.4:500515437worker1 # [ 125.063549] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=scheduler_restarting addr=lb:500515438worker1 # [ 125.067602] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:461245439worker1 # [ 125.072142] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=worker_gone addr=192.168.1.3:500515440worker2 # [ 125.198138] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:09Z level=info event=scheduler_connected addr=lb:500515441worker1 # [ 125.257642] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=scheduler_connected addr=lb:500515442worker2 # [ 125.215979] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5443worker1 # [ 125.279576] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5444worker2 # [ 125.416994] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:09Z level=info event=scheduler_connected addr=lb:500515445worker2 # [ 125.422356] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:09Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5446worker1 # [ 125.480452] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:10Z level=info event=scheduler_connected addr=lb:500515447worker1 # [ 125.485746] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5448worker2 # [ 125.623134] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:10Z level=info event=scheduler_connected addr=lb:500515449worker2 # [ 125.628570] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5450worker1 # [ 125.686374] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:10Z level=info event=scheduler_connected addr=lb:500515451worker1 # [ 125.692224] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5452worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.20 seconds)5453client: waiting for success: ! systemctl is-active rs5454client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5455client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5456worker2 # [ 125.829626] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:10Z level=info event=scheduler_connected addr=lb:500515457worker2 # [ 125.835076] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5458client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5459client: must fail: journalctl -u rs | grep -q 'reconnecting'5460worker1 # [ 125.892305] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:10Z level=info event=scheduler_connected addr=lb:500515461worker1 # [ 125.897391] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5462client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5463lb: must succeed: curl -sf localhost:9901/clusters5464lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5465worker1 # [ 126.041100] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:10Z level=info event=scheduler_restarting5466worker1 # [ 126.073348] nix-daemon[840]: reaped child process 1246, status = succeeded5467worker2 # [ 126.037262] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:10Z level=info event=scheduler_connected addr=lb:500515468worker1 # [ 126.082607] nix-grpc-daemon[1176]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5469worker1 # [ 126.087214] nix-grpc-daemon[1176]: E0000 00:00:1789892350.655583 1219 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5470worker1 # [ 126.120492] nix-grpc-daemon[1176]: ts=2026-09-20T08:19:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5471worker2 # [ 126.824783] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:11Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: remote connection failure"5472worker1 # [ 126.890999] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5473worker1 # [ 126.894156] systemd[1]: Stopped Nix worker-protocol over gRPC.5474worker1 # [ 126.899610] systemd[1]: nix-grpc-daemon.service: Consumed 1.882s CPU time over 25.349s wall clock time, 36.8M memory peak, 47.6K incoming IP traffic, 89.5K outgoing IP traffic.5475worker1 # [ 126.917632] systemd[1]: Starting Nix worker-protocol over gRPC...5476lb: must succeed: curl -sf localhost:9901/clusters5477worker2 # [ 127.025821] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:11Z level=info event=scheduler_connected addr=lb:500515478worker2 # [ 127.033091] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:11Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5479lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5480worker1 # [ 127.114234] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:11Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5481worker1 # [ 127.116736] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:11Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15482worker1 # [ 127.124548] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:11Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5483worker1 # [ 127.126291] systemd[1]: Started Nix worker-protocol over gRPC.5484worker1 # [ 127.139745] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:11Z level=info event=scheduler_take_over5485lb # [ 127.143691] niks3-server[1172]: 2026/09/20 08:19:11 INFO lead: acquired remote=192.168.1.3:460225486worker1 # [ 127.167126] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:11Z level=info event=scheduler_connected addr=lb:500515487worker1 # [ 127.170444] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:11Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05488worker2 # [ 127.235425] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:11Z level=info event=scheduler_connected addr=lb:500515489worker1 # [ 127.279888] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05490lb: must succeed: curl -sf localhost:9901/clusters5491lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5492worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5493worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)5494(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.19 seconds)5495subtest: scheduler restart mid-build: build finishes, no double build5496client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5497client # [ 128.294903] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5498client # Running as unit: mid.service; invocation ID: 05f50d9ad6654d5298f2f3ad7d2c25d05499client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.10 seconds)5500worker2 # [ 128.553257] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:482765501worker2 # [ 128.568856] nix-daemon[846]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5502client # [ 128.590403] nix[1178]: don't know how to build these paths:5503client # [ 128.592075] nix[1178]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5504worker1 # [ 128.647600] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:13Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05505worker1 # [ 128.654509] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:13Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15506worker1 # [ 128.708499] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557145507worker1 # [ 128.723068] nix-daemon[840]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5508worker1 # [ 128.746043] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 97 ms (attempt 1/5)5509worker1 # [ 128.849784] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 2/5)5510worker1 # [ 128.997362] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 255 ms (attempt 3/5)5511worker1 # [ 129.260267] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 650 ms (attempt 4/5)5512worker1 # [ 129.913387] nix-grpc-daemon[1447]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5513worker1 # [ 129.933433] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:557145514client # [ 129.906634] nix[1178]: copying 1 paths to 'lb:50051'...5515worker1 # [ 129.951598] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:557145516client # [ 129.912438] nix[1178]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5517worker1 # [ 129.965810] nix-daemon[840]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5518worker1 # [ 129.998154] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:14Z level=info event=niks3_push_started pid=15185519worker1 # [ 130.382172] nix-daemon[840]: accepted connection from pid 1531, user nix-grpc-daemon (trusted)5520worker1 # [ 130.398973] nix-daemon[840]: reaped child process 1534, status = succeeded5521worker1 # [ 130.571208] nix-daemon[840]: accepted connection from pid 1536, user nix-grpc-daemon (trusted)5522worker1 # [ 130.588472] nix-daemon[840]: reaped child process 1539, status = succeeded5523lb # [ 130.603939] niks3-server[1172]: 2026/09/20 08:19:15 INFO Received uploads request method=POST path=/api/pending_closures5524worker1 # [ 130.612425] nix-grpc-daemon[1518]: time=2026-09-20T08:19:15.186Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5525worker1 # [ 130.613932] nix-grpc-daemon[1518]: time=2026-09-20T08:19:15.186Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5526lb # [ 130.642802] niks3-server[1172]: 2026/09/20 08:19:15 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5527worker1 # [ 130.652693] nix-grpc-daemon[1518]: time=2026-09-20T08:19:15.226Z level=INFO msg="Uploading 1 narinfos"5528lb # [ 130.656393] niks3-server[1172]: 2026/09/20 08:19:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5529lb # [ 130.664232] niks3-server[1172]: 2026/09/20 08:19:15 INFO Signed narinfos id=33 count=15530lb # [ 130.675899] niks3-server[1172]: 2026/09/20 08:19:15 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5531lb # [ 130.691250] niks3-server[1172]: 2026/09/20 08:19:15 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5532worker1 # [ 130.698584] nix-grpc-daemon[1518]: time=2026-09-20T08:19:15.272Z level=INFO msg="Upload complete. (487ms)"5533lb # [ 130.704258] niks3-server[1172]: 2026/09/20 08:19:15 INFO Completed upload id=335534worker1 # [ 130.711862] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55714 duration_s=0 paths=1 nar_bytes_in=4245535worker1 # [ 130.728785] nix-grpc-daemon[1447]: ts=2026-09-20T08:19:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:557145536worker1 # [ 130.731994] nix-grpc-daemon[1518]: time=2026-09-20T08:19:15.294Z level=WARN msg="Failed to register uploaded object" key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo error="server returned 404: object is not pending upload\n"5537worker1 # [ 130.754702] nix-daemon[840]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5538worker1 # [ 130.760604] nix-daemon[840]: reaped child process 1516, status = succeeded5539client # [ 130.722667] nix[1178]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5540worker1 # [ 130.769226] nix-daemon[840]: accepted connection from pid 1447, user nix-grpc-daemon (trusted)5541lb: must succeed: curl -sf localhost:9901/clusters5542lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5543lb: must succeed: curl -sf localhost:9901/clusters5544lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5545worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5546worker1 # [ 132.372931] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1447 (nix-grpc-daemon) on client request.5547worker1 # [ 132.376206] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5548client # [ 132.348756] nix[1178]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5549lb # [ 132.393312] niks3-server[1172]: 2026/09/20 08:19:16 INFO lead: released remote=192.168.1.3:460225550worker2 # [ 132.346159] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:16Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5551client # [ 132.355283] nix[1178]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5552worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.08 seconds)5553client: must succeed: sleep 5.05554worker1 # [ 132.424174] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5555worker1 # [ 132.425765] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5556worker1 # [ 132.426988] systemd[1]: nix-grpc-daemon.service: Consumed 499ms CPU time over 5.504s wall clock time, 34M memory peak, 12K incoming IP traffic, 14.8K outgoing IP traffic.5557worker1 # [ 132.437815] nix-daemon[840]: reaped child process 1507, status = succeeded5558worker1 # [ 132.439007] nix-daemon[840]: reaped child process 1544, status = succeeded5559worker1 # [ 132.448305] nix-daemon[1547]: unexpected Nix daemon error: error: interrupted by the user5560worker1 # [ 132.452346] nix-daemon[840]: reaped child process 1547, status = failed with exit code 15561worker2 # [ 132.554958] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:17Z level=info event=scheduler_connected addr=lb:500515562worker1 # [ 132.602774] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.5563worker1 # [ 132.615095] systemd[1]: Starting Nix worker-protocol over gRPC...5564lb # [ 132.767528] envoy[931]: conn peer=[2001:db8:1::1]:44784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2940 tx=1349 ms=86715565lb # [ 132.772837] envoy[931]: conn peer=[2001:db8:1::3]:35816 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4147 tx=1375 ms=242645566lb # [ 132.777829] envoy[931]: conn peer=[2001:db8:1::3]:56780 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=475 tx=239 ms=52285567worker1 # [ 132.829658] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5568worker1 # [ 132.831949] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15569worker1 # [ 132.839577] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5570worker1 # [ 132.847826] systemd[1]: Started Nix worker-protocol over gRPC.5571worker1 # [ 132.854102] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=info event=scheduler_take_over5572lb # [ 132.856671] niks3-server[1172]: 2026/09/20 08:19:17 INFO lead: acquired remote=192.168.1.3:563645573worker1 # [ 132.868052] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05574worker1 # [ 132.881940] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=info event=scheduler_connected addr=lb:500515575worker1 # [ 132.883764] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05576worker1 # [ 132.897150] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05577worker1 # [ 132.899193] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:17Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15578worker2 # [ 132.898490] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482765579worker2 # [ 132.917089] nix-grpc-daemon[1163]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)5580worker2 # [ 132.990983] nix-grpc-daemon[1163]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 165 ms (attempt 2/5)5581worker2 # [ 133.163613] nix-grpc-daemon[1163]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 275 ms (attempt 3/5)5582worker2 # [ 133.447565] nix-grpc-daemon[1163]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 82 ms (attempt 4/5)5583worker2 # [ 133.529489] nix-grpc-daemon[1163]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5584worker2 # [ 133.548840] nix-grpc-daemon[1163]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5585worker2 # [ 133.578677] nix-daemon[846]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5586client # [ 133.598501] nix[1178]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5587worker2 # [ 133.600995] nix-daemon[846]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5588client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5589client: waiting for success: ! systemctl is-active mid5590worker2 # [ 137.822928] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:22Z level=info event=niks3_push_started pid=13165591worker2 # [ 137.852935] nix-daemon[846]: reaped child process 1292, status = succeeded5592worker2 # [ 138.207441] nix-daemon[846]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5593worker2 # [ 138.225861] nix-daemon[846]: reaped child process 1332, status = succeeded5594lb # [ 138.289932] niks3-server[1172]: 2026/09/20 08:19:22 INFO Received uploads request method=POST path=/api/pending_closures5595worker2 # [ 138.259269] nix-grpc-daemon[1316]: time=2026-09-20T08:19:22.831Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5596worker2 # [ 138.263100] nix-grpc-daemon[1316]: time=2026-09-20T08:19:22.831Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5597lb # [ 138.350643] niks3-server[1172]: 2026/09/20 08:19:22 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5598worker2 # [ 138.329536] nix-grpc-daemon[1316]: time=2026-09-20T08:19:22.904Z level=INFO msg="Uploading 1 narinfos"5599lb # [ 138.375291] niks3-server[1172]: 2026/09/20 08:19:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5600lb # [ 138.390586] niks3-server[1172]: 2026/09/20 08:19:22 INFO Signed narinfos id=34 count=15601lb # [ 138.395383] niks3-server[1172]: 2026/09/20 08:19:22 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5602lb # [ 138.408192] niks3-server[1172]: 2026/09/20 08:19:22 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5603lb # [ 138.420324] niks3-server[1172]: 2026/09/20 08:19:22 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5604lb # [ 138.430772] niks3-server[1172]: 2026/09/20 08:19:23 INFO Completed upload id=345605worker2 # [ 138.383967] nix-grpc-daemon[1316]: time=2026-09-20T08:19:22.958Z level=INFO msg="Upload complete. (348ms)"5606lb # [ 138.439679] niks3-server[1172]: 2026/09/20 08:19:23 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5607worker2 # [ 138.396463] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48276 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15608worker2 # [ 138.423467] nix-daemon[846]: reaped child process 1290, status = succeeded5609client # [ 138.454010] systemd[1]: mid.service: Deactivated successfully.5610client # [ 138.459107] systemd[1]: mid.service: Consumed 181ms CPU time over 10.163s wall clock time, 37M memory peak, 9.8K incoming IP traffic, 10.6K outgoing IP traffic.5611client: (finished: waiting for success: ! systemctl is-active mid, in 1.17 seconds)5612client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5613client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.07 seconds)5614(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.76 seconds)5615subtest: interrupting the client stops the build on the worker5616client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr5617client # [ 139.030304] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.5618client # Running as unit: intr.service; invocation ID: 152c16d68d3d4798bce3fd169004f49a5619client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.07 seconds)5620worker1 # [ 139.343872] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502805621worker1 # [ 139.360792] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5622client # [ 139.344250] nix[1213]: don't know how to build these paths:5623client # [ 139.345814] nix[1213]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5624worker1 # [ 139.406927] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:23Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05625worker1 # [ 139.416167] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:23Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15626worker1 # [ 139.455245] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805627worker1 # [ 139.479679] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)5628worker1 # [ 139.583573] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 2/5)5629worker1 # [ 139.722444] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 3/5)5630worker1 # [ 139.921966] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 254 ms (attempt 4/5)5631worker1 # [ 140.180742] nix-grpc-daemon[1579]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5632worker1 # [ 140.214563] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502805633client # [ 140.188227] nix[1213]: copying 1 paths to 'lb:50051'...5634client # [ 140.193192] nix[1213]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5635worker1 # [ 140.237402] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502805636worker1 # [ 140.256333] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5637worker1 # [ 140.292894] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:24Z level=info event=niks3_push_started pid=16515638worker1 # [ 140.824729] nix-daemon[840]: accepted connection from pid 1668, user nix-grpc-daemon (trusted)5639worker1 # [ 140.846958] nix-daemon[840]: reaped child process 1671, status = succeeded5640worker1 # [ 141.058182] nix-daemon[840]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5641worker1 # [ 141.082078] nix-daemon[840]: reaped child process 1676, status = succeeded5642lb # [ 141.099698] niks3-server[1172]: 2026/09/20 08:19:25 INFO Received uploads request method=POST path=/api/pending_closures5643worker1 # [ 141.112329] nix-grpc-daemon[1651]: time=2026-09-20T08:19:25.684Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5644worker1 # [ 141.113929] nix-grpc-daemon[1651]: time=2026-09-20T08:19:25.684Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5645lb # [ 141.151039] niks3-server[1172]: 2026/09/20 08:19:25 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5646lb # [ 141.165623] niks3-server[1172]: 2026/09/20 08:19:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5647worker1 # [ 141.169858] nix-grpc-daemon[1651]: time=2026-09-20T08:19:25.743Z level=INFO msg="Uploading 1 narinfos"5648lb # [ 141.177185] niks3-server[1172]: 2026/09/20 08:19:25 INFO Signed narinfos id=35 count=15649lb # [ 141.184908] niks3-server[1172]: 2026/09/20 08:19:25 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5650lb # [ 141.196261] niks3-server[1172]: 2026/09/20 08:19:25 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5651lb # [ 141.201620] niks3-server[1172]: 2026/09/20 08:19:25 INFO Completed upload id=355652worker1 # [ 141.199211] nix-grpc-daemon[1651]: time=2026-09-20T08:19:25.773Z level=INFO msg="Upload complete. (594ms)"5653worker1 # [ 141.211872] nix-grpc-daemon[1651]: time=2026-09-20T08:19:25.773Z level=WARN msg="Failed to register uploaded object" key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo error="server returned 404: object is not pending upload\n"5654worker1 # [ 141.226854] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=4245655worker1 # [ 141.242408] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805656worker1 # [ 141.253436] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5657worker1 # [ 141.258460] nix-daemon[840]: reaped child process 1649, status = succeeded5658client # [ 141.232608] nix[1213]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5659worker1 # [ 141.277796] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5660lb # [ 142.768685] envoy[931]: conn peer=[2001:db8:1::1]:59310 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3854 tx=1684 ms=98775661client: must succeed: systemctl kill -s INT intr5662client # [ 143.027758] systemd[1]: intr.service: Sent signal SIGINT to main process 1213 (nix) on client request.5663client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5664client # [ 143.034694] nix[1213]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5665client # [ 143.043414] nix[1213]: error: interrupted by the user5666worker1 # [ 143.121443] nix-daemon[1680]: unexpected Nix daemon error: error: interrupted by the user5667client # [ 143.081480] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5668worker1 # [ 143.125664] nix-daemon[840]: reaped child process 1678, status = succeeded5669client # [ 143.086212] systemd[1]: intr.service: Failed with result 'exit-code'.5670client # [ 143.088799] systemd[1]: intr.service: Consumed 181ms CPU time over 4.054s wall clock time, 37.2M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.5671worker1 # [ 143.136269] nix-daemon[840]: reaped child process 1680, status = failed with exit code 15672(finished: subtest: interrupting the client stops the build on the worker, in 4.22 seconds)5673subtest: a deploy mid-build drains: the build finishes, the new generation takes over5674client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw5675client # [ 143.279402] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.5676client # Running as unit: sw.service; invocation ID: 7804f54d33df4a92b8a68ddfd7a48e8e5677client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.10 seconds)5678client # [ 143.545389] nix[1233]: don't know how to build these paths:5679client # [ 143.546997] nix[1233]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5680worker1 # [ 143.583981] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502805681worker1 # [ 143.600132] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:28Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05682worker1 # [ 143.605867] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:28Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15683worker2 # [ 143.606576] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482765684worker2 # [ 143.624394] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482765685client # [ 143.640331] nix[1233]: copying 1 paths to 'lb:50051'...5686worker2 # [ 143.639602] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:28Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482765687client # [ 143.644160] nix[1233]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5688worker2 # [ 143.659565] nix-daemon[846]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5689worker2 # [ 143.886594] nix-daemon[846]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5690worker2 # [ 143.905675] nix-daemon[846]: reaped child process 1363, status = succeeded5691worker2 # [ 144.079061] nix-daemon[846]: accepted connection from pid 1365, user nix-grpc-daemon (trusted)5692worker2 # [ 144.096805] nix-daemon[846]: reaped child process 1368, status = succeeded5693lb # [ 144.155512] niks3-server[1172]: 2026/09/20 08:19:28 INFO Received uploads request method=POST path=/api/pending_closures5694worker2 # [ 144.120497] nix-grpc-daemon[1316]: time=2026-09-20T08:19:28.695Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5695worker2 # [ 144.122579] nix-grpc-daemon[1316]: time=2026-09-20T08:19:28.695Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5696lb # [ 144.191079] niks3-server[1172]: 2026/09/20 08:19:28 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5697worker2 # [ 144.158620] nix-grpc-daemon[1316]: time=2026-09-20T08:19:28.733Z level=INFO msg="Uploading 1 narinfos"5698lb # [ 144.205073] niks3-server[1172]: 2026/09/20 08:19:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5699lb # [ 144.215806] niks3-server[1172]: 2026/09/20 08:19:28 INFO Signed narinfos id=36 count=15700lb # [ 144.222721] niks3-server[1172]: 2026/09/20 08:19:28 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5701lb # [ 144.228918] niks3-server[1172]: 2026/09/20 08:19:28 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5702lb # [ 144.237312] niks3-server[1172]: 2026/09/20 08:19:28 INFO Completed upload id=365703worker2 # [ 144.190456] nix-grpc-daemon[1316]: time=2026-09-20T08:19:28.765Z level=INFO msg="Upload complete. (511ms)"5704lb # [ 144.239579] niks3-server[1172]: 2026/09/20 08:19:28 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5705worker2 # [ 144.200625] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48276 duration_s=0 paths=1 nar_bytes_in=4165706worker2 # [ 144.212120] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482765707worker2 # [ 144.219034] nix-daemon[846]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5708worker2 # [ 144.223987] nix-daemon[846]: reaped child process 1358, status = succeeded5709client # [ 144.236168] nix[1233]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5710worker2 # [ 144.237598] nix-daemon[846]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)5711worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&25712worker2 # Checking switch inhibitors... done5713worker2 # [ 146.746873] nixos[1403]: switching to system configuration /nix/store/6imafpnhdjd2x47q8iv8mjv4g785bhfv-nixos-system-worker2-test5714worker2 # activating the configuration...5715worker2 # [ 146.755014] systemd[1]: Stopped target Local File Systems.5716worker2 # [ 146.755844] systemd[1]: Stopped target Remote File Systems.5717worker2 # setting up /etc...5718worker2 # [ 147.533183] systemd[1]: Reload requested from client PID 1403 ('.switch-to-conf') (unit backdoor.service)...5719worker2 # [ 147.534941] systemd[1]: Reloading...5720worker2 # [ 148.429487] systemd[1]: Reloading finished in 891 ms.5721worker2 # restarting sysinit-reactivation.target5722worker2 # [ 148.518105] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....5723worker2 # [ 148.671275] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..5724worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service5725worker2 # [ 148.676301] systemd[1]: Reached target Reactivate sysinit units.5726worker2 # [ 148.679887] systemd[1]: Reloading D-Bus System Message Bus...5727worker2 # [ 148.684641] dbus-broker-launch[493]: Caught SIGHUP, trigger reload.5728worker2 # [ 148.688512] systemd[1]: Reloading Nix worker-protocol over gRPC...5729worker2 # [ 148.714111] dbus-broker-launch[493]: Looking up NSS user entry for 'systemd-timesync'...5730worker2 # [ 148.723513] dbus-broker-launch[493]: NSS returned no entry for 'systemd-timesync'5731worker2 # [ 148.730841] dbus-broker-launch[493]: Invalid user-name in /nix/store/c8a4kacp5lnfwrcxrjmwczbl8ba32m4a-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"5732worker2 # [ 148.746073] systemd[1]: Reloaded Nix worker-protocol over gRPC.5733worker2 # [ 148.772564] systemd[1]: Reloaded D-Bus System Message Bus.5734worker2 # [ 148.779045] systemd[1]: Reached target Local File Systems.5735worker2 # [ 148.779896] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore5736worker2 # [ 148.787064] systemd[1]: Starting Create SUID/SGID Wrappers...5737worker2 # [ 148.789602] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.5738worker2 # [ 148.790815] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc675739worker2 # [ 148.804860] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore5740worker2 # [ 148.807160] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5741worker2 # [ 148.812505] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os5742worker2 # [ 148.816177] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os5743worker2 # [ 148.819248] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki5744worker2 # [ 148.824418] systemd[1]: Reached target Remote File Systems.5745worker2 # [ 149.308399] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.5746worker2 # [ 149.310017] systemd[1]: Finished Create SUID/SGID Wrappers.5747worker2 # [ 149.313257] systemd[1]: Starting linger-users.service...5748worker2 # [ 149.373925] systemd[1]: linger-users.service: Deactivated successfully.5749worker2 # [ 149.374888] systemd[1]: Finished linger-users.service.5750worker2 # [ 149.604626] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:34Z level=info event=draining5751worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service5752worker2 # [ 149.629771] nixos[1403]: finished switching to system configuration /nix/store/6imafpnhdjd2x47q8iv8mjv4g785bhfv-nixos-system-worker2-test5753worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 3.71 seconds)5754worker2: must succeed: pgrep -f 'read -t [9]0 x'5755worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)5756client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag during-drain >&25757client # don't know how to build these paths:5758client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5759worker1 # [ 149.997054] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502805760worker1 # [ 150.010282] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:34Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05761worker1 # [ 150.016896] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:34Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=15762worker1 # [ 150.060715] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805763worker1 # [ 150.076361] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502805764client # copying 1 paths to 'lb:50051'...5765worker1 # [ 150.091755] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502805766client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5767worker1 # [ 150.115853] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5768worker1 # [ 150.322587] nix-daemon[840]: accepted connection from pid 1730, user nix-grpc-daemon (trusted)5769worker1 # [ 150.342085] nix-daemon[840]: reaped child process 1733, status = succeeded5770worker1 # [ 150.516848] nix-daemon[840]: accepted connection from pid 1735, user nix-grpc-daemon (trusted)5771worker1 # [ 150.533384] nix-daemon[840]: reaped child process 1738, status = succeeded5772lb # [ 150.550051] niks3-server[1172]: 2026/09/20 08:19:35 INFO Received uploads request method=POST path=/api/pending_closures5773worker1 # [ 150.557911] nix-grpc-daemon[1651]: time=2026-09-20T08:19:35.131Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5774worker1 # [ 150.559363] nix-grpc-daemon[1651]: time=2026-09-20T08:19:35.131Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5775lb # [ 150.577786] niks3-server[1172]: 2026/09/20 08:19:35 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5776lb # [ 150.593125] niks3-server[1172]: 2026/09/20 08:19:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5777worker1 # [ 150.596338] nix-grpc-daemon[1651]: time=2026-09-20T08:19:35.170Z level=INFO msg="Uploading 1 narinfos"5778lb # [ 150.605320] niks3-server[1172]: 2026/09/20 08:19:35 INFO Signed narinfos id=37 count=15779lb # [ 150.611366] niks3-server[1172]: 2026/09/20 08:19:35 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5780worker1 # [ 150.617933] nix-grpc-daemon[1651]: time=2026-09-20T08:19:35.192Z level=INFO msg="Upload complete. (484ms)"5781lb # [ 150.621447] niks3-server[1172]: 2026/09/20 08:19:35 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5782worker1 # [ 150.624266] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=5285783lb # [ 150.629010] niks3-server[1172]: 2026/09/20 08:19:35 INFO Completed upload id=375784lb # [ 150.631353] niks3-server[1172]: 2026/09/20 08:19:35 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5785worker1 # [ 150.636784] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805786worker1 # [ 150.640060] nix-daemon[840]: reaped child process 1728, status = succeeded5787worker1 # [ 150.647568] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5788client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...5789client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5790worker1 # [ 150.661337] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5791client # farm-a> LOG-farm-a-during-drain5792worker1 # [ 151.887455] nix-daemon[840]: reaped child process 1742, status = succeeded5793worker1 # [ 152.051111] nix-daemon[840]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)5794worker1 # [ 152.069468] nix-daemon[840]: reaped child process 1760, status = succeeded5795lb # [ 152.085781] niks3-server[1172]: 2026/09/20 08:19:36 INFO Received uploads request method=POST path=/api/pending_closures5796worker1 # [ 152.093698] nix-grpc-daemon[1651]: time=2026-09-20T08:19:36.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5797worker1 # [ 152.095194] nix-grpc-daemon[1651]: time=2026-09-20T08:19:36.666Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"5798lb # [ 152.118850] niks3-server[1172]: 2026/09/20 08:19:36 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv5799lb # [ 152.136452] niks3-server[1172]: 2026/09/20 08:19:36 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst5800worker1 # [ 152.148136] nix-grpc-daemon[1651]: time=2026-09-20T08:19:36.721Z level=INFO msg="Uploading 1 narinfos"5801lb # [ 152.150006] niks3-server[1172]: 2026/09/20 08:19:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5802lb # [ 152.158368] niks3-server[1172]: 2026/09/20 08:19:36 INFO Signed narinfos id=38 count=15803lb # [ 152.163632] niks3-server[1172]: 2026/09/20 08:19:36 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls5804lb # [ 152.170355] niks3-server[1172]: 2026/09/20 08:19:36 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5805lb # [ 152.177389] niks3-server[1172]: 2026/09/20 08:19:36 INFO Completed upload id=385806worker1 # [ 152.174265] nix-grpc-daemon[1651]: time=2026-09-20T08:19:36.748Z level=INFO msg="Upload complete. (303ms)"5807lb # [ 152.180197] niks3-server[1172]: 2026/09/20 08:19:36 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo5808worker1 # [ 152.189630] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=15809worker1 # [ 152.206260] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:36Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=05810worker1 # [ 152.210331] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:36Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=15811worker1 # [ 152.214530] nix-daemon[840]: reaped child process 1740, status = succeeded5812worker1 # [ 152.243686] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805813worker1 # [ 152.258816] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502805814worker1 # [ 152.272366] nix-grpc-daemon[1651]: time=2026-09-20T08:19:36.846Z level=INFO msg="All 1 paths already cached"5815client # copying 1 paths to 'lb:50051'...5816worker1 # [ 152.282184] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502805817client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...5818worker1 # [ 152.305472] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5819worker1 # [ 152.512618] nix-daemon[840]: accepted connection from pid 1766, user nix-grpc-daemon (trusted)5820worker1 # [ 152.531588] nix-daemon[840]: reaped child process 1769, status = succeeded5821worker1 # [ 152.704092] nix-daemon[840]: accepted connection from pid 1771, user nix-grpc-daemon (trusted)5822worker1 # [ 152.720780] nix-daemon[840]: reaped child process 1774, status = succeeded5823lb # [ 152.737099] niks3-server[1172]: 2026/09/20 08:19:37 INFO Received uploads request method=POST path=/api/pending_closures5824worker1 # [ 152.744066] nix-grpc-daemon[1651]: time=2026-09-20T08:19:37.317Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5825worker1 # [ 152.745556] nix-grpc-daemon[1651]: time=2026-09-20T08:19:37.317Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"5826lb # [ 152.763525] niks3-server[1172]: 2026/09/20 08:19:37 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst5827worker1 # [ 152.774867] nix-grpc-daemon[1651]: time=2026-09-20T08:19:37.348Z level=INFO msg="Uploading 1 narinfos"5828lb # [ 152.781465] niks3-server[1172]: 2026/09/20 08:19:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5829lb # [ 152.790996] niks3-server[1172]: 2026/09/20 08:19:37 INFO Signed narinfos id=39 count=15830worker1 # [ 152.798299] nix-grpc-daemon[1651]: time=2026-09-20T08:19:37.372Z level=INFO msg="Upload complete. (475ms)"5831lb # [ 152.802094] envoy[931]: conn peer=[2001:db8:1::1]:55492 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2931 tx=755 ms=37705832lb # [ 152.806352] niks3-server[1172]: 2026/09/20 08:19:37 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls5833worker1 # [ 152.804262] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=6405834lb # [ 152.810550] niks3-server[1172]: 2026/09/20 08:19:37 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5835lb # [ 152.814612] niks3-server[1172]: 2026/09/20 08:19:37 INFO Completed upload id=395836lb # [ 152.816815] niks3-server[1172]: 2026/09/20 08:19:37 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo5837worker1 # [ 152.815667] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805838worker1 # [ 152.818966] nix-daemon[840]: reaped child process 1764, status = succeeded5839worker1 # [ 152.825952] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5840client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...5841worker1 # [ 152.838707] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5842client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5843client # farm-b> LOG-farm-b-during-drain5844worker1 # [ 154.475910] nix-daemon[840]: reaped child process 1778, status = succeeded5845worker1 # [ 154.659812] nix-daemon[840]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)5846worker1 # [ 154.676149] nix-daemon[840]: reaped child process 1796, status = succeeded5847lb # [ 154.693913] niks3-server[1172]: 2026/09/20 08:19:39 INFO Received uploads request method=POST path=/api/pending_closures5848worker1 # [ 154.701256] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.274Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5849worker1 # [ 154.702721] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.274Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"5850lb # [ 154.723689] niks3-server[1172]: 2026/09/20 08:19:39 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst5851lb # [ 154.739646] niks3-server[1172]: 2026/09/20 08:19:39 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv5852lb # [ 154.750959] niks3-server[1172]: 2026/09/20 08:19:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5853worker1 # [ 154.753195] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.327Z level=INFO msg="Uploading 1 narinfos"5854lb # [ 154.760668] niks3-server[1172]: 2026/09/20 08:19:39 INFO Signed narinfos id=40 count=15855lb # [ 154.767221] niks3-server[1172]: 2026/09/20 08:19:39 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls5856lb # [ 154.775664] niks3-server[1172]: 2026/09/20 08:19:39 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5857lb # [ 154.781755] niks3-server[1172]: 2026/09/20 08:19:39 INFO Completed upload id=405858worker1 # [ 154.778051] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.352Z level=INFO msg="Upload complete. (324ms)"5859lb # [ 154.784630] niks3-server[1172]: 2026/09/20 08:19:39 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo5860worker1 # [ 154.792461] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=15861worker1 # [ 154.810119] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:39Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=05862worker1 # [ 154.814533] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:39Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=15863worker1 # [ 154.819451] nix-daemon[840]: reaped child process 1776, status = succeeded5864worker1 # [ 154.841870] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805865worker1 # [ 154.857397] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502805866worker1 # [ 154.873395] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.447Z level=INFO msg="All 2 paths already cached"5867client # copying 1 paths to 'lb:50051'...5868worker1 # [ 154.883699] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502805869client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...5870worker1 # [ 154.907247] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5871worker1 # [ 155.109969] nix-daemon[840]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)5872worker1 # [ 155.128527] nix-daemon[840]: reaped child process 1805, status = succeeded5873worker1 # [ 155.306299] nix-daemon[840]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)5874worker1 # [ 155.322731] nix-daemon[840]: reaped child process 1810, status = succeeded5875lb # [ 155.336813] niks3-server[1172]: 2026/09/20 08:19:39 INFO Received uploads request method=POST path=/api/pending_closures5876worker1 # [ 155.344206] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.918Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5877worker1 # [ 155.345675] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.918Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"5878lb # [ 155.371521] niks3-server[1172]: 2026/09/20 08:19:39 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst5879worker1 # [ 155.378798] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.952Z level=INFO msg="Uploading 1 narinfos"5880lb # [ 155.383427] niks3-server[1172]: 2026/09/20 08:19:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5881lb # [ 155.393249] niks3-server[1172]: 2026/09/20 08:19:39 INFO Signed narinfos id=41 count=15882lb # [ 155.399347] niks3-server[1172]: 2026/09/20 08:19:39 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls5883lb # [ 155.409843] niks3-server[1172]: 2026/09/20 08:19:39 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5884lb # [ 155.415402] niks3-server[1172]: 2026/09/20 08:19:39 INFO Completed upload id=415885worker1 # [ 155.412040] nix-grpc-daemon[1651]: time=2026-09-20T08:19:39.986Z level=INFO msg="Upload complete. (487ms)"5886lb # [ 155.418137] niks3-server[1172]: 2026/09/20 08:19:39 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo5887worker1 # [ 155.420149] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=7685888worker1 # [ 155.434136] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805889worker1 # [ 155.440651] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5890worker1 # [ 155.445763] nix-daemon[840]: reaped child process 1800, status = succeeded5891client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...5892worker1 # [ 155.457542] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5893client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5894client # farm-top> LOG-farm-top-during-drain5895worker1 # [ 156.506821] nix-daemon[840]: reaped child process 1814, status = succeeded5896worker1 # [ 156.687344] nix-daemon[840]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)5897worker1 # [ 156.703640] nix-daemon[840]: reaped child process 1832, status = succeeded5898lb # [ 156.722621] niks3-server[1172]: 2026/09/20 08:19:41 INFO Received uploads request method=POST path=/api/pending_closures5899worker1 # [ 156.730972] nix-grpc-daemon[1651]: time=2026-09-20T08:19:41.303Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5900worker1 # [ 156.733005] nix-grpc-daemon[1651]: time=2026-09-20T08:19:41.303Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"5901lb # [ 156.759216] niks3-server[1172]: 2026/09/20 08:19:41 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst5902lb # [ 156.771887] niks3-server[1172]: 2026/09/20 08:19:41 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5903worker1 # [ 156.784083] nix-grpc-daemon[1651]: time=2026-09-20T08:19:41.357Z level=INFO msg="Uploading 1 narinfos"5904lb # [ 156.780904] niks3-server[1172]: 2026/09/20 08:19:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5905lb # [ 156.790641] niks3-server[1172]: 2026/09/20 08:19:41 INFO Signed narinfos id=42 count=15906lb # [ 156.796223] niks3-server[1172]: 2026/09/20 08:19:41 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls5907lb # [ 156.801828] niks3-server[1172]: 2026/09/20 08:19:41 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5908lb # [ 156.811859] niks3-server[1172]: 2026/09/20 08:19:41 INFO Completed upload id=425909worker1 # [ 156.809414] nix-grpc-daemon[1651]: time=2026-09-20T08:19:41.383Z level=INFO msg="Upload complete. (338ms)"5910lb # [ 156.814980] niks3-server[1172]: 2026/09/20 08:19:41 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo5911worker1 # [ 156.825149] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=15912worker1 # [ 156.842515] nix-daemon[840]: reaped child process 1812, status = succeeded5913client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag during-drain >&2, in 7.13 seconds)5914client: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag during-drain outPath5915client: (finished: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag during-drain outPath, in 0.23 seconds)5916worker2: must succeed: pkill -f 'read -t [9]0 x'5917worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)5918client: waiting for success: ! systemctl is-active sw5919worker2 # [ 157.297647] nix-daemon[846]: reaped child process 1372, status = succeeded5920worker2 # [ 157.471671] nix-daemon[846]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5921worker2 # [ 157.488446] nix-daemon[846]: reaped child process 1546, status = succeeded5922lb # [ 157.549111] niks3-server[1172]: 2026/09/20 08:19:42 INFO Received uploads request method=POST path=/api/pending_closures5923worker2 # [ 157.508916] nix-grpc-daemon[1316]: time=2026-09-20T08:19:42.083Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5924worker2 # [ 157.510363] nix-grpc-daemon[1316]: time=2026-09-20T08:19:42.083Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"5925lb # [ 157.571638] niks3-server[1172]: 2026/09/20 08:19:42 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5926lb # [ 157.585632] niks3-server[1172]: 2026/09/20 08:19:42 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst5927lb # [ 157.597408] niks3-server[1172]: 2026/09/20 08:19:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5928worker2 # [ 157.557437] nix-grpc-daemon[1316]: time=2026-09-20T08:19:42.131Z level=INFO msg="Uploading 1 narinfos"5929lb # [ 157.603070] niks3-server[1172]: 2026/09/20 08:19:42 INFO Signed narinfos id=43 count=15930lb # [ 157.607610] niks3-server[1172]: 2026/09/20 08:19:42 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls5931lb # [ 157.620061] niks3-server[1172]: 2026/09/20 08:19:42 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5932lb # [ 157.627960] niks3-server[1172]: 2026/09/20 08:19:42 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo5933lb # [ 157.633538] niks3-server[1172]: 2026/09/20 08:19:42 INFO Completed upload id=435934worker2 # [ 157.587290] nix-grpc-daemon[1316]: time=2026-09-20T08:19:42.162Z level=INFO msg="Upload complete. (326ms)"5935worker2 # [ 157.597907] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48276 duration_s=13 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=15936worker2 # [ 157.612743] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:42Z level=info event=signal_exit5937worker2 # [ 157.615837] nix-daemon[846]: reaped child process 1370, status = succeeded5938client # [ 157.642742] systemd[1]: sw.service: Deactivated successfully.5939client # [ 157.647370] systemd[1]: sw.service: Consumed 178ms CPU time over 14.366s wall clock time, 38.9M memory peak, 8.4K incoming IP traffic, 8.8K outgoing IP traffic.5940client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)5941client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }5942client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)5943worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service5944worker2 # [ 158.843805] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:43Z level=info event=scheduler_restarting5945worker1 # [ 158.886939] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:502585946worker1 # [ 158.887222] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:43Z level=info event=worker_gone addr=192.168.1.4:500515947worker2 # [ 158.868804] nix-grpc-daemon[1163]: ts=2026-09-20T08:19:43Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5948worker2 # [ 158.894385] nix-daemon[846]: reaped child process 1268, status = succeeded5949worker2 # [ 159.106189] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5950worker2 # [ 159.112776] systemd[1]: nix-grpc-daemon.service: Consumed 964ms CPU time over 44.734s wall clock time, 33.3M memory peak, 23K incoming IP traffic, 31.8K outgoing IP traffic.5951worker2 # [ 159.130464] systemd[1]: Starting Nix worker-protocol over gRPC...5952worker2 # [ 159.325381] nix-grpc-daemon[1560]: ts=2026-09-20T08:19:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5953worker2 # [ 159.327889] nix-grpc-daemon[1560]: ts=2026-09-20T08:19:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15954worker2 # [ 159.335856] nix-grpc-daemon[1560]: ts=2026-09-20T08:19:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5955worker2 # [ 159.343786] systemd[1]: Started Nix worker-protocol over gRPC.5956worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.08 seconds)5957lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5958lb: must succeed: curl -sf localhost:9901/clusters5959worker2 # [ 159.381610] nix-grpc-daemon[1560]: ts=2026-09-20T08:19:43Z level=info event=scheduler_connected addr=lb:500515960worker1 # [ 159.427531] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:44Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05961lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5962lb: must succeed: curl -sf localhost:9901/clusters5963lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5964lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.17 seconds)5965lb: must succeed: curl -sf localhost:9901/clusters5966lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5967worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true5968worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -F 'nix_grpc_sched{kind="workers"} ' || true, in 0.09 seconds)5969(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.48 seconds)5970subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does5971client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&15972worker1 # [ 160.973639] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:502805973worker1 # [ 160.993592] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:45Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=05974worker1 # [ 161.000991] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:45Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=15975worker1 # [ 161.007103] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:45Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=05976worker1 # [ 161.012149] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:45Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=05977worker1 # [ 161.041957] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805978worker1 # [ 161.059410] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502805979worker1 # [ 161.074192] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502805980worker1 # [ 161.089786] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)5981worker1 # [ 161.291755] nix-daemon[840]: accepted connection from pid 1843, user nix-grpc-daemon (trusted)5982worker1 # [ 161.311387] nix-daemon[840]: reaped child process 1846, status = succeeded5983worker1 # [ 161.485533] nix-daemon[840]: accepted connection from pid 1848, user nix-grpc-daemon (trusted)5984worker1 # [ 161.503744] nix-daemon[840]: reaped child process 1851, status = succeeded5985lb # [ 161.522256] niks3-server[1172]: 2026/09/20 08:19:46 INFO Received uploads request method=POST path=/api/pending_closures5986worker1 # [ 161.532606] nix-grpc-daemon[1651]: time=2026-09-20T08:19:46.106Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5987worker1 # [ 161.534064] nix-grpc-daemon[1651]: time=2026-09-20T08:19:46.106Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"5988lb # [ 161.554246] niks3-server[1172]: 2026/09/20 08:19:46 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst5989worker1 # [ 161.565097] nix-grpc-daemon[1651]: time=2026-09-20T08:19:46.139Z level=INFO msg="Uploading 1 narinfos"5990lb # [ 161.569317] niks3-server[1172]: 2026/09/20 08:19:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5991lb # [ 161.579681] niks3-server[1172]: 2026/09/20 08:19:46 INFO Signed narinfos id=44 count=15992lb # [ 161.585972] niks3-server[1172]: 2026/09/20 08:19:46 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls5993lb # [ 161.592824] niks3-server[1172]: 2026/09/20 08:19:46 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5994lb # [ 161.600846] niks3-server[1172]: 2026/09/20 08:19:46 INFO Completed upload id=445995worker1 # [ 161.598320] nix-grpc-daemon[1651]: time=2026-09-20T08:19:46.172Z level=INFO msg="Upload complete. (493ms)"5996lb # [ 161.603670] niks3-server[1172]: 2026/09/20 08:19:46 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo5997worker1 # [ 161.604260] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=5125998worker1 # [ 161.618062] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502805999worker1 # [ 161.625629] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6000worker1 # [ 161.631179] nix-daemon[840]: reaped child process 1841, status = succeeded6001worker1 # [ 161.642288] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6002lb # [ 162.770543] envoy[931]: conn peer=[2001:db8:1::1]:58168 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9473 tx=3819 ms=68456003lb # [ 162.775688] envoy[931]: conn peer=[2001:db8:1::1]:58164 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2939 tx=1328 ms=140706004lb # [ 162.780660] envoy[931]: conn peer=[2001:db8:1::4]:52668 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1942 tx=833 ms=442316005worker1 # [ 163.071457] nix-daemon[840]: reaped child process 1855, status = succeeded6006worker1 # [ 163.235928] nix-daemon[840]: accepted connection from pid 1870, user nix-grpc-daemon (trusted)6007worker1 # [ 163.255670] nix-daemon[840]: reaped child process 1873, status = succeeded6008lb # [ 163.271266] niks3-server[1172]: 2026/09/20 08:19:47 INFO Received uploads request method=POST path=/api/pending_closures6009worker1 # [ 163.280425] nix-grpc-daemon[1651]: time=2026-09-20T08:19:47.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6010worker1 # [ 163.281999] nix-grpc-daemon[1651]: time=2026-09-20T08:19:47.853Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6011lb # [ 163.312690] niks3-server[1172]: 2026/09/20 08:19:47 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6012lb # [ 163.327351] niks3-server[1172]: 2026/09/20 08:19:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6013worker1 # [ 163.330744] nix-grpc-daemon[1651]: time=2026-09-20T08:19:47.904Z level=INFO msg="Uploading 1 narinfos"6014lb # [ 163.340201] niks3-server[1172]: 2026/09/20 08:19:47 INFO Signed narinfos id=45 count=16015lb # [ 163.347704] niks3-server[1172]: 2026/09/20 08:19:47 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6016lb # [ 163.359523] niks3-server[1172]: 2026/09/20 08:19:47 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6017lb # [ 163.364380] niks3-server[1172]: 2026/09/20 08:19:47 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6018worker1 # [ 163.366445] nix-grpc-daemon[1651]: time=2026-09-20T08:19:47.940Z level=INFO msg="Upload complete. (312ms)"6019lb # [ 163.369916] niks3-server[1172]: 2026/09/20 08:19:47 INFO Completed upload id=456020lb # [ 163.373148] niks3-server[1172]: 2026/09/20 08:19:47 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6021worker1 # [ 163.381134] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16022worker1 # [ 163.396937] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:47Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16023worker1 # [ 163.406866] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806024worker1 # [ 163.412328] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:47Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06025worker1 # [ 163.417016] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502806026worker1 # [ 163.421611] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502806027worker1 # [ 163.426688] nix-daemon[840]: reaped child process 1853, status = succeeded6028worker1 # [ 163.434842] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6029worker1 # [ 163.614046] nix-daemon[840]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)6030worker1 # [ 163.633430] nix-daemon[840]: reaped child process 1882, status = succeeded6031worker1 # [ 163.804444] nix-daemon[840]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6032worker1 # [ 163.823356] nix-daemon[840]: reaped child process 1887, status = succeeded6033lb # [ 163.837642] niks3-server[1172]: 2026/09/20 08:19:48 INFO Received uploads request method=POST path=/api/pending_closures6034worker1 # [ 163.850205] nix-grpc-daemon[1651]: time=2026-09-20T08:19:48.424Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6035worker1 # [ 163.851760] nix-grpc-daemon[1651]: time=2026-09-20T08:19:48.424Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6036lb # [ 163.871528] niks3-server[1172]: 2026/09/20 08:19:48 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6037lb # [ 163.885155] niks3-server[1172]: 2026/09/20 08:19:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6038worker1 # [ 163.889853] nix-grpc-daemon[1651]: time=2026-09-20T08:19:48.463Z level=INFO msg="Uploading 1 narinfos"6039lb # [ 163.892526] niks3-server[1172]: 2026/09/20 08:19:48 INFO Signed narinfos id=46 count=16040lb # [ 163.898279] niks3-server[1172]: 2026/09/20 08:19:48 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6041lb # [ 163.912240] niks3-server[1172]: 2026/09/20 08:19:48 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6042worker1 # [ 163.916090] nix-grpc-daemon[1651]: time=2026-09-20T08:19:48.490Z level=INFO msg="Upload complete. (471ms)"6043lb # [ 163.919316] niks3-server[1172]: 2026/09/20 08:19:48 INFO Completed upload id=466044lb # [ 163.923673] niks3-server[1172]: 2026/09/20 08:19:48 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6045worker1 # [ 163.924131] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=5126046worker1 # [ 163.932678] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806047worker1 # [ 163.943102] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6048worker1 # [ 163.947757] nix-daemon[840]: reaped child process 1877, status = succeeded6049worker1 # [ 163.958217] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6050worker1 # [ 165.405549] nix-daemon[840]: reaped child process 1891, status = succeeded6051worker1 # [ 165.584119] nix-daemon[840]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6052worker1 # [ 165.603733] nix-daemon[840]: reaped child process 1909, status = succeeded6053lb # [ 165.619447] niks3-server[1172]: 2026/09/20 08:19:50 INFO Received uploads request method=POST path=/api/pending_closures6054worker1 # [ 165.640405] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.210Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6055worker1 # [ 165.642622] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.210Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6056lb # [ 165.674447] niks3-server[1172]: 2026/09/20 08:19:50 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6057worker1 # [ 165.682561] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.256Z level=INFO msg="Uploading 1 narinfos"6058lb # [ 165.688262] niks3-server[1172]: 2026/09/20 08:19:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6059lb # [ 165.691900] niks3-server[1172]: 2026/09/20 08:19:50 INFO Signed narinfos id=47 count=16060lb # [ 165.705903] niks3-server[1172]: 2026/09/20 08:19:50 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6061lb # [ 165.711061] niks3-server[1172]: 2026/09/20 08:19:50 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6062worker1 # [ 165.711870] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.286Z level=INFO msg="Upload complete. (340ms)"6063lb # [ 165.724326] niks3-server[1172]: 2026/09/20 08:19:50 INFO Completed upload id=476064lb # [ 165.725802] niks3-server[1172]: 2026/09/20 08:19:50 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6065worker1 # [ 165.725254] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16066worker1 # [ 165.751955] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:50Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16067worker1 # [ 165.757689] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.295Z level=WARN msg="Failed to register uploaded object" key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls error="server returned 404: object is not pending upload\n"6068worker1 # [ 165.764264] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806069worker1 # [ 165.769330] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:50Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06070worker1 # [ 165.774093] nix-daemon[840]: reaped child process 1889, status = succeeded6071worker1 # [ 165.776101] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502806072worker1 # [ 165.781437] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502806073worker1 # [ 165.791285] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6074worker1 # [ 165.981393] nix-daemon[840]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6075worker1 # [ 165.998821] nix-daemon[840]: reaped child process 1918, status = succeeded6076worker1 # [ 166.170687] nix-daemon[840]: accepted connection from pid 1920, user nix-grpc-daemon (trusted)6077worker1 # [ 166.187108] nix-daemon[840]: reaped child process 1923, status = succeeded6078lb # [ 166.201258] niks3-server[1172]: 2026/09/20 08:19:50 INFO Received uploads request method=POST path=/api/pending_closures6079worker1 # [ 166.216634] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.790Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6080worker1 # [ 166.218159] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.790Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6081lb # [ 166.251475] niks3-server[1172]: 2026/09/20 08:19:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6082worker1 # [ 166.255726] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.829Z level=INFO msg="Uploading 1 narinfos"6083lb # [ 166.263867] niks3-server[1172]: 2026/09/20 08:19:50 INFO Signed narinfos id=48 count=16084lb # [ 166.270022] niks3-server[1172]: 2026/09/20 08:19:50 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6085lb # [ 166.275713] niks3-server[1172]: 2026/09/20 08:19:50 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6086worker1 # [ 166.280463] nix-grpc-daemon[1651]: time=2026-09-20T08:19:50.854Z level=INFO msg="Upload complete. (479ms)"6087lb # [ 166.283597] niks3-server[1172]: 2026/09/20 08:19:50 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6088lb # [ 166.286967] niks3-server[1172]: 2026/09/20 08:19:50 INFO Completed upload id=486089lb # [ 166.290303] niks3-server[1172]: 2026/09/20 08:19:50 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6090worker1 # [ 166.288643] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=5126091worker1 # [ 166.303115] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806092worker1 # [ 166.312421] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6093worker1 # [ 166.316877] nix-daemon[840]: reaped child process 1913, status = succeeded6094worker1 # [ 166.328751] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6095worker1 # [ 167.358130] nix-daemon[840]: reaped child process 1927, status = succeeded6096worker1 # [ 167.524204] nix-daemon[840]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6097worker1 # [ 167.543025] nix-daemon[840]: reaped child process 1945, status = succeeded6098lb # [ 167.558972] niks3-server[1172]: 2026/09/20 08:19:52 INFO Received uploads request method=POST path=/api/pending_closures6099worker1 # [ 167.566619] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.140Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6100worker1 # [ 167.569227] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.142Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6101lb # [ 167.593845] niks3-server[1172]: 2026/09/20 08:19:52 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6102lb # [ 167.607383] niks3-server[1172]: 2026/09/20 08:19:52 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6103lb # [ 167.622882] niks3-server[1172]: 2026/09/20 08:19:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6104worker1 # [ 167.625692] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.199Z level=INFO msg="Uploading 1 narinfos"6105lb # [ 167.631357] niks3-server[1172]: 2026/09/20 08:19:52 INFO Signed narinfos id=49 count=16106lb # [ 167.639497] niks3-server[1172]: 2026/09/20 08:19:52 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6107lb # [ 167.650227] niks3-server[1172]: 2026/09/20 08:19:52 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6108lb # [ 167.655299] niks3-server[1172]: 2026/09/20 08:19:52 INFO Completed upload id=496109worker1 # [ 167.652231] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.226Z level=INFO msg="Upload complete. (311ms)"6110lb # [ 167.658375] niks3-server[1172]: 2026/09/20 08:19:52 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6111worker1 # [ 167.663431] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16112worker1 # [ 167.680418] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:52Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16113worker1 # [ 167.685556] nix-daemon[840]: reaped child process 1925, status = succeeded6114worker1 # [ 167.707229] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806115worker1 # [ 167.717552] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:52Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06116worker1 # [ 167.729078] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502806117worker1 # [ 167.743538] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.317Z level=INFO msg="All 1 paths already cached"6118worker1 # [ 167.752122] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502806119worker1 # [ 167.765948] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6120worker1 # [ 167.962683] nix-daemon[840]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6121worker1 # [ 167.979517] nix-daemon[840]: reaped child process 1954, status = succeeded6122worker1 # [ 168.151359] nix-daemon[840]: accepted connection from pid 1956, user nix-grpc-daemon (trusted)6123worker1 # [ 168.167742] nix-daemon[840]: reaped child process 1959, status = succeeded6124lb # [ 168.182099] niks3-server[1172]: 2026/09/20 08:19:52 INFO Received uploads request method=POST path=/api/pending_closures6125worker1 # [ 168.188825] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.762Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6126worker1 # [ 168.190330] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.762Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6127lb # [ 168.208103] niks3-server[1172]: 2026/09/20 08:19:52 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6128worker1 # [ 168.218857] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.792Z level=INFO msg="Uploading 1 narinfos"6129lb # [ 168.223535] niks3-server[1172]: 2026/09/20 08:19:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6130lb # [ 168.236352] niks3-server[1172]: 2026/09/20 08:19:52 INFO Signed narinfos id=50 count=16131lb # [ 168.243638] niks3-server[1172]: 2026/09/20 08:19:52 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6132lb # [ 168.255301] niks3-server[1172]: 2026/09/20 08:19:52 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6133worker1 # [ 168.258953] nix-grpc-daemon[1651]: time=2026-09-20T08:19:52.833Z level=INFO msg="Upload complete. (478ms)"6134lb # [ 168.261495] niks3-server[1172]: 2026/09/20 08:19:52 INFO Completed upload id=506135lb # [ 168.265000] niks3-server[1172]: 2026/09/20 08:19:52 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6136worker1 # [ 168.266839] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=6246137worker1 # [ 168.280191] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806138worker1 # [ 168.287643] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6139worker1 # [ 168.292895] nix-daemon[840]: reaped child process 1949, status = succeeded6140worker1 # [ 168.306632] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6141worker1 # [ 169.738425] nix-daemon[840]: reaped child process 1963, status = succeeded6142worker1 # [ 169.910291] nix-daemon[840]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)6143worker1 # [ 169.927755] nix-daemon[840]: reaped child process 1981, status = succeeded6144lb # [ 169.946505] niks3-server[1172]: 2026/09/20 08:19:54 INFO Received uploads request method=POST path=/api/pending_closures6145worker1 # [ 169.956541] nix-grpc-daemon[1651]: time=2026-09-20T08:19:54.527Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6146worker1 # [ 169.958131] nix-grpc-daemon[1651]: time=2026-09-20T08:19:54.527Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6147lb # [ 169.984902] niks3-server[1172]: 2026/09/20 08:19:54 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6148worker1 # [ 169.999131] nix-grpc-daemon[1651]: time=2026-09-20T08:19:54.573Z level=INFO msg="Uploading 1 narinfos"6149lb # [ 170.001342] niks3-server[1172]: 2026/09/20 08:19:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6150lb # [ 170.014176] niks3-server[1172]: 2026/09/20 08:19:54 INFO Signed narinfos id=51 count=16151lb # [ 170.020981] niks3-server[1172]: 2026/09/20 08:19:54 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6152lb # [ 170.027818] niks3-server[1172]: 2026/09/20 08:19:54 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6153lb # [ 170.038449] niks3-server[1172]: 2026/09/20 08:19:54 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6154lb # [ 170.042788] niks3-server[1172]: 2026/09/20 08:19:54 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6155lb # [ 170.047863] niks3-server[1172]: 2026/09/20 08:19:54 INFO Completed upload id=516156worker1 # [ 170.045083] nix-grpc-daemon[1651]: time=2026-09-20T08:19:54.619Z level=INFO msg="Upload complete. (323ms)"6157worker1 # [ 170.063191] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16158worker1 # [ 170.080955] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:54Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16159worker1 # [ 170.091123] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806160worker1 # [ 170.097792] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:54Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06161worker1 # [ 170.103225] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502806162worker1 # [ 170.108498] nix-daemon[840]: reaped child process 1961, status = succeeded6163worker1 # [ 170.110729] nix-grpc-daemon[1651]: time=2026-09-20T08:19:54.676Z level=INFO msg="All 1 paths already cached"6164worker1 # [ 170.114919] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502806165worker1 # [ 170.128995] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6166worker1 # [ 170.316739] nix-daemon[840]: accepted connection from pid 1987, user nix-grpc-daemon (trusted)6167worker1 # [ 170.335994] nix-daemon[840]: reaped child process 1990, status = succeeded6168worker1 # [ 170.506873] nix-daemon[840]: accepted connection from pid 1992, user nix-grpc-daemon (trusted)6169worker1 # [ 170.523130] nix-daemon[840]: reaped child process 1995, status = succeeded6170lb # [ 170.537899] niks3-server[1172]: 2026/09/20 08:19:55 INFO Received uploads request method=POST path=/api/pending_closures6171worker1 # [ 170.545004] nix-grpc-daemon[1651]: time=2026-09-20T08:19:55.119Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6172worker1 # [ 170.546493] nix-grpc-daemon[1651]: time=2026-09-20T08:19:55.119Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6173lb # [ 170.565297] niks3-server[1172]: 2026/09/20 08:19:55 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6174lb # [ 170.579224] niks3-server[1172]: 2026/09/20 08:19:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6175worker1 # [ 170.582633] nix-grpc-daemon[1651]: time=2026-09-20T08:19:55.156Z level=INFO msg="Uploading 1 narinfos"6176lb # [ 170.592391] niks3-server[1172]: 2026/09/20 08:19:55 INFO Signed narinfos id=52 count=16177lb # [ 170.595331] niks3-server[1172]: 2026/09/20 08:19:55 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6178lb # [ 170.604295] niks3-server[1172]: 2026/09/20 08:19:55 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6179lb # [ 170.610964] niks3-server[1172]: 2026/09/20 08:19:55 INFO Completed upload id=526180worker1 # [ 170.608382] nix-grpc-daemon[1651]: time=2026-09-20T08:19:55.182Z level=INFO msg="Upload complete. (469ms)"6181lb # [ 170.613861] niks3-server[1172]: 2026/09/20 08:19:55 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6182worker1 # [ 170.616155] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=6246183worker1 # [ 170.628405] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806184worker1 # [ 170.634912] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6185worker1 # [ 170.639852] nix-daemon[840]: reaped child process 1985, status = succeeded6186worker1 # [ 170.650875] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6187worker1 # [ 172.293363] nix-daemon[840]: reaped child process 1999, status = succeeded6188worker1 # [ 172.477490] nix-daemon[840]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6189worker1 # [ 172.494984] nix-daemon[840]: reaped child process 2017, status = succeeded6190lb # [ 172.510367] niks3-server[1172]: 2026/09/20 08:19:57 INFO Received uploads request method=POST path=/api/pending_closures6191worker1 # [ 172.519993] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.093Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6192worker1 # [ 172.521927] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.093Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6193lb # [ 172.550613] niks3-server[1172]: 2026/09/20 08:19:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6194worker1 # [ 172.559028] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.132Z level=INFO msg="Uploading 1 narinfos"6195lb # [ 172.568093] niks3-server[1172]: 2026/09/20 08:19:57 INFO Signed narinfos id=53 count=16196lb # [ 172.571336] niks3-server[1172]: 2026/09/20 08:19:57 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6197lb # [ 172.582781] niks3-server[1172]: 2026/09/20 08:19:57 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6198lb # [ 172.592617] niks3-server[1172]: 2026/09/20 08:19:57 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6199worker1 # [ 172.595082] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.169Z level=INFO msg="Upload complete. (330ms)"6200lb # [ 172.597687] niks3-server[1172]: 2026/09/20 08:19:57 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6201lb # [ 172.602685] niks3-server[1172]: 2026/09/20 08:19:57 INFO Completed upload id=536202lb # [ 172.606582] niks3-server[1172]: 2026/09/20 08:19:57 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6203worker1 # [ 172.614406] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16204worker1 # [ 172.637017] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:57Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16205worker1 # [ 172.642595] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806206worker1 # [ 172.647701] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:57Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06207worker1 # [ 172.653065] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502806208worker1 # [ 172.657663] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.231Z level=INFO msg="All 1 paths already cached"6209worker1 # [ 172.660478] nix-daemon[840]: reaped child process 1997, status = succeeded6210worker1 # [ 172.664501] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502806211worker1 # [ 172.673520] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6212worker1 # [ 172.860874] nix-daemon[840]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)6213worker1 # [ 172.879954] nix-daemon[840]: reaped child process 2026, status = succeeded6214worker1 # [ 173.054142] nix-daemon[840]: accepted connection from pid 2028, user nix-grpc-daemon (trusted)6215worker1 # [ 173.071757] nix-daemon[840]: reaped child process 2031, status = succeeded6216lb # [ 173.089277] niks3-server[1172]: 2026/09/20 08:19:57 INFO Received uploads request method=POST path=/api/pending_closures6217worker1 # [ 173.096113] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.670Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6218worker1 # [ 173.097606] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.670Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6219lb # [ 173.117705] niks3-server[1172]: 2026/09/20 08:19:57 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6220worker1 # [ 173.131903] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.706Z level=INFO msg="Uploading 1 narinfos"6221lb # [ 173.135140] niks3-server[1172]: 2026/09/20 08:19:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6222lb # [ 173.146502] niks3-server[1172]: 2026/09/20 08:19:57 INFO Signed narinfos id=54 count=16223lb # [ 173.153054] niks3-server[1172]: 2026/09/20 08:19:57 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6224lb # [ 173.161261] niks3-server[1172]: 2026/09/20 08:19:57 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6225lb # [ 173.168645] niks3-server[1172]: 2026/09/20 08:19:57 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6226lb # [ 173.173260] niks3-server[1172]: 2026/09/20 08:19:57 INFO Completed upload id=546227worker1 # [ 173.170785] nix-grpc-daemon[1651]: time=2026-09-20T08:19:57.745Z level=INFO msg="Upload complete. (485ms)"6228worker1 # [ 173.180274] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=6246229worker1 # [ 173.195130] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806230worker1 # [ 173.201519] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6231worker1 # [ 173.206720] nix-daemon[840]: reaped child process 2021, status = succeeded6232worker1 # [ 173.218394] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6233worker1 # [ 174.647984] nix-daemon[840]: reaped child process 2035, status = succeeded6234worker1 # [ 174.816446] nix-daemon[840]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6235worker1 # [ 174.835487] nix-daemon[840]: reaped child process 2053, status = succeeded6236lb # [ 174.851376] niks3-server[1172]: 2026/09/20 08:19:59 INFO Received uploads request method=POST path=/api/pending_closures6237worker1 # [ 174.860449] nix-grpc-daemon[1651]: time=2026-09-20T08:19:59.432Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6238worker1 # [ 174.862020] nix-grpc-daemon[1651]: time=2026-09-20T08:19:59.432Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6239lb # [ 174.895324] niks3-server[1172]: 2026/09/20 08:19:59 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6240lb # [ 174.906475] niks3-server[1172]: 2026/09/20 08:19:59 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6241lb # [ 174.915876] niks3-server[1172]: 2026/09/20 08:19:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6242worker1 # [ 174.918254] nix-grpc-daemon[1651]: time=2026-09-20T08:19:59.491Z level=INFO msg="Uploading 1 narinfos"6243lb # [ 174.925727] niks3-server[1172]: 2026/09/20 08:19:59 INFO Signed narinfos id=55 count=16244lb # [ 174.931381] niks3-server[1172]: 2026/09/20 08:19:59 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6245lb # [ 174.938587] niks3-server[1172]: 2026/09/20 08:19:59 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6246lb # [ 174.946550] niks3-server[1172]: 2026/09/20 08:19:59 INFO Completed upload id=556247worker1 # [ 174.944194] nix-grpc-daemon[1651]: time=2026-09-20T08:19:59.518Z level=INFO msg="Upload complete. (312ms)"6248lb # [ 174.949808] niks3-server[1172]: 2026/09/20 08:19:59 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6249worker1 # [ 174.960760] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16250worker1 # [ 174.976880] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:59Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16251worker1 # [ 174.984323] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806252worker1 # [ 174.987262] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:59Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06253worker1 # [ 174.992260] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502806254worker1 # [ 174.995747] nix-grpc-daemon[1651]: time=2026-09-20T08:19:59.569Z level=INFO msg="All 2 paths already cached"6255worker1 # [ 174.998253] nix-daemon[840]: reaped child process 2033, status = succeeded6256worker1 # [ 175.001260] nix-grpc-daemon[1579]: ts=2026-09-20T08:19:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502806257worker1 # [ 175.009465] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6258worker1 # [ 175.191245] nix-daemon[840]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)6259worker1 # [ 175.210801] nix-daemon[840]: reaped child process 2062, status = succeeded6260worker1 # [ 175.382337] nix-daemon[840]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)6261worker1 # [ 175.400508] nix-daemon[840]: reaped child process 2067, status = succeeded6262lb # [ 175.417248] niks3-server[1172]: 2026/09/20 08:19:59 INFO Received uploads request method=POST path=/api/pending_closures6263worker1 # [ 175.423967] nix-grpc-daemon[1651]: time=2026-09-20T08:19:59.997Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6264worker1 # [ 175.425593] nix-grpc-daemon[1651]: time=2026-09-20T08:19:59.998Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6265lb # [ 175.446247] niks3-server[1172]: 2026/09/20 08:20:00 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6266worker1 # [ 175.454262] nix-grpc-daemon[1651]: time=2026-09-20T08:20:00.028Z level=INFO msg="Uploading 1 narinfos"6267lb # [ 175.462003] niks3-server[1172]: 2026/09/20 08:20:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6268lb # [ 175.472288] niks3-server[1172]: 2026/09/20 08:20:00 INFO Signed narinfos id=56 count=16269worker1 # [ 175.477043] nix-grpc-daemon[1651]: time=2026-09-20T08:20:00.051Z level=INFO msg="Upload complete. (458ms)"6270lb # [ 175.480848] niks3-server[1172]: 2026/09/20 08:20:00 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6271worker1 # [ 175.482987] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=7526272lb # [ 175.485720] niks3-server[1172]: 2026/09/20 08:20:00 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6273lb # [ 175.493264] niks3-server[1172]: 2026/09/20 08:20:00 INFO Completed upload id=566274lb # [ 175.495588] niks3-server[1172]: 2026/09/20 08:20:00 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6275worker1 # [ 175.494803] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806276worker1 # [ 175.500461] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6277worker1 # [ 175.504447] nix-daemon[840]: reaped child process 2057, status = succeeded6278worker1 # [ 175.514388] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6279worker1 # [ 176.765853] nix-daemon[840]: reaped child process 2071, status = succeeded6280worker1 # [ 176.947576] nix-daemon[840]: accepted connection from pid 2086, user nix-grpc-daemon (trusted)6281worker1 # [ 176.966297] nix-daemon[840]: reaped child process 2089, status = succeeded6282lb # [ 176.983508] niks3-server[1172]: 2026/09/20 08:20:01 INFO Received uploads request method=POST path=/api/pending_closures6283worker1 # [ 176.994422] nix-grpc-daemon[1651]: time=2026-09-20T08:20:01.567Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6284worker1 # [ 176.995928] nix-grpc-daemon[1651]: time=2026-09-20T08:20:01.567Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6285lb # [ 177.022132] niks3-server[1172]: 2026/09/20 08:20:01 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6286lb # [ 177.036539] niks3-server[1172]: 2026/09/20 08:20:01 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6287lb # [ 177.047491] niks3-server[1172]: 2026/09/20 08:20:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6288worker1 # [ 177.049053] nix-grpc-daemon[1651]: time=2026-09-20T08:20:01.622Z level=INFO msg="Uploading 1 narinfos"6289lb # [ 177.055494] niks3-server[1172]: 2026/09/20 08:20:01 INFO Signed narinfos id=57 count=16290lb # [ 177.060178] niks3-server[1172]: 2026/09/20 08:20:01 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6291lb # [ 177.067731] niks3-server[1172]: 2026/09/20 08:20:01 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6292lb # [ 177.073495] niks3-server[1172]: 2026/09/20 08:20:01 INFO Completed upload id=576293worker1 # [ 177.071046] nix-grpc-daemon[1651]: time=2026-09-20T08:20:01.645Z level=INFO msg="Upload complete. (343ms)"6294lb # [ 177.075819] niks3-server[1172]: 2026/09/20 08:20:01 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6295worker1 # [ 177.085275] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16296worker1 # [ 177.102898] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:01Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16297worker1 # [ 177.108511] nix-daemon[840]: reaped child process 2069, status = succeeded6298worker1 # [ 177.126352] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806299worker1 # [ 177.141817] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502806300worker1 # [ 177.159622] nix-grpc-daemon[1651]: time=2026-09-20T08:20:01.733Z level=INFO msg="All 2 paths already cached"6301worker1 # [ 177.167944] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502806302worker1 # [ 177.178741] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6303worker1 # [ 177.377001] nix-daemon[840]: accepted connection from pid 2093, user nix-grpc-daemon (trusted)6304worker1 # [ 177.395444] nix-daemon[840]: reaped child process 2096, status = succeeded6305worker1 # [ 177.570524] nix-daemon[840]: accepted connection from pid 2098, user nix-grpc-daemon (trusted)6306worker1 # [ 177.588310] nix-daemon[840]: reaped child process 2101, status = succeeded6307lb # [ 177.602329] niks3-server[1172]: 2026/09/20 08:20:02 INFO Received uploads request method=POST path=/api/pending_closures6308worker1 # [ 177.609616] nix-grpc-daemon[1651]: time=2026-09-20T08:20:02.183Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6309worker1 # [ 177.611104] nix-grpc-daemon[1651]: time=2026-09-20T08:20:02.183Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6310lb # [ 177.630564] niks3-server[1172]: 2026/09/20 08:20:02 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6311lb # [ 177.642727] niks3-server[1172]: 2026/09/20 08:20:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6312worker1 # [ 177.646065] nix-grpc-daemon[1651]: time=2026-09-20T08:20:02.220Z level=INFO msg="Uploading 1 narinfos"6313lb # [ 177.653920] niks3-server[1172]: 2026/09/20 08:20:02 INFO Signed narinfos id=58 count=16314lb # [ 177.661341] niks3-server[1172]: 2026/09/20 08:20:02 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6315lb # [ 177.671277] niks3-server[1172]: 2026/09/20 08:20:02 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6316worker1 # [ 177.675385] nix-grpc-daemon[1651]: time=2026-09-20T08:20:02.249Z level=INFO msg="Upload complete. (483ms)"6317lb # [ 177.678590] niks3-server[1172]: 2026/09/20 08:20:02 INFO Completed upload id=586318lb # [ 177.682421] niks3-server[1172]: 2026/09/20 08:20:02 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6319worker1 # [ 177.681267] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=7526320worker1 # [ 177.694074] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806321worker1 # [ 177.699252] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6322worker1 # [ 177.703555] nix-daemon[840]: reaped child process 2091, status = succeeded6323worker1 # [ 177.714445] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6324worker1 # [ 178.962114] nix-daemon[840]: reaped child process 2105, status = succeeded6325worker1 # [ 179.146330] nix-daemon[840]: accepted connection from pid 2120, user nix-grpc-daemon (trusted)6326worker1 # [ 179.165354] nix-daemon[840]: reaped child process 2123, status = succeeded6327lb # [ 179.182682] niks3-server[1172]: 2026/09/20 08:20:03 INFO Received uploads request method=POST path=/api/pending_closures6328worker1 # [ 179.190014] nix-grpc-daemon[1651]: time=2026-09-20T08:20:03.764Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6329worker1 # [ 179.193592] nix-grpc-daemon[1651]: time=2026-09-20T08:20:03.766Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6330lb # [ 179.221096] niks3-server[1172]: 2026/09/20 08:20:03 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6331lb # [ 179.236527] niks3-server[1172]: 2026/09/20 08:20:03 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6332worker1 # [ 179.239924] nix-grpc-daemon[1651]: time=2026-09-20T08:20:03.813Z level=INFO msg="Uploading 1 narinfos"6333lb # [ 179.245526] niks3-server[1172]: 2026/09/20 08:20:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6334lb # [ 179.254152] niks3-server[1172]: 2026/09/20 08:20:03 INFO Signed narinfos id=59 count=16335lb # [ 179.257583] niks3-server[1172]: 2026/09/20 08:20:03 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6336worker1 # [ 179.262540] nix-grpc-daemon[1651]: time=2026-09-20T08:20:03.836Z level=INFO msg="Upload complete. (336ms)"6337lb # [ 179.265478] niks3-server[1172]: 2026/09/20 08:20:03 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6338lb # [ 179.268947] niks3-server[1172]: 2026/09/20 08:20:03 INFO Completed upload id=596339lb # [ 179.272616] niks3-server[1172]: 2026/09/20 08:20:03 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6340worker1 # [ 179.272992] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16341worker1 # [ 179.289605] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:03Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16342worker1 # [ 179.294291] nix-daemon[840]: reaped child process 2103, status = succeeded6343worker1 # [ 179.317408] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806344worker1 # [ 179.329471] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:502806345worker1 # [ 179.340600] nix-grpc-daemon[1651]: time=2026-09-20T08:20:03.914Z level=INFO msg="All 2 paths already cached"6346worker1 # [ 179.347717] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:502806347worker1 # [ 179.357602] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6348worker1 # [ 179.541879] nix-daemon[840]: accepted connection from pid 2127, user nix-grpc-daemon (trusted)6349worker1 # [ 179.561806] nix-daemon[840]: reaped child process 2130, status = succeeded6350worker1 # [ 179.739756] nix-daemon[840]: accepted connection from pid 2132, user nix-grpc-daemon (trusted)6351worker1 # [ 179.757439] nix-daemon[840]: reaped child process 2135, status = succeeded6352lb # [ 179.773731] niks3-server[1172]: 2026/09/20 08:20:04 INFO Received uploads request method=POST path=/api/pending_closures6353worker1 # [ 179.780261] nix-grpc-daemon[1651]: time=2026-09-20T08:20:04.354Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6354worker1 # [ 179.781740] nix-grpc-daemon[1651]: time=2026-09-20T08:20:04.354Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6355lb # [ 179.803567] niks3-server[1172]: 2026/09/20 08:20:04 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6356worker1 # [ 179.814949] nix-grpc-daemon[1651]: time=2026-09-20T08:20:04.389Z level=INFO msg="Uploading 1 narinfos"6357lb # [ 179.820430] niks3-server[1172]: 2026/09/20 08:20:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6358lb # [ 179.831027] niks3-server[1172]: 2026/09/20 08:20:04 INFO Signed narinfos id=60 count=16359lb # [ 179.835181] niks3-server[1172]: 2026/09/20 08:20:04 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6360lb # [ 179.844337] niks3-server[1172]: 2026/09/20 08:20:04 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6361worker1 # [ 179.846566] nix-grpc-daemon[1651]: time=2026-09-20T08:20:04.420Z level=INFO msg="Upload complete. (477ms)"6362lb # [ 179.851089] niks3-server[1172]: 2026/09/20 08:20:04 INFO Completed upload id=606363worker1 # [ 179.852230] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=0 paths=1 nar_bytes_in=7526364lb # [ 179.854843] niks3-server[1172]: 2026/09/20 08:20:04 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6365worker1 # [ 179.862783] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:502806366worker1 # [ 179.869023] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6367worker1 # [ 179.873798] nix-daemon[840]: reaped child process 2125, status = succeeded6368worker1 # [ 179.885140] nix-daemon[840]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6369worker1 # [ 181.129799] nix-daemon[840]: reaped child process 2139, status = succeeded6370worker1 # [ 181.311804] nix-daemon[840]: accepted connection from pid 2154, user nix-grpc-daemon (trusted)6371worker1 # [ 181.330051] nix-daemon[840]: reaped child process 2157, status = succeeded6372lb # [ 181.347875] niks3-server[1172]: 2026/09/20 08:20:05 INFO Received uploads request method=POST path=/api/pending_closures6373worker1 # [ 181.356101] nix-grpc-daemon[1651]: time=2026-09-20T08:20:05.929Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6374worker1 # [ 181.357589] nix-grpc-daemon[1651]: time=2026-09-20T08:20:05.929Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6375lb # [ 181.377276] niks3-server[1172]: 2026/09/20 08:20:05 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6376lb # [ 181.391604] niks3-server[1172]: 2026/09/20 08:20:05 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6377lb # [ 181.403760] niks3-server[1172]: 2026/09/20 08:20:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6378worker1 # [ 181.406166] nix-grpc-daemon[1651]: time=2026-09-20T08:20:05.980Z level=INFO msg="Uploading 1 narinfos"6379lb # [ 181.415104] niks3-server[1172]: 2026/09/20 08:20:05 INFO Signed narinfos id=61 count=16380lb # [ 181.420263] niks3-server[1172]: 2026/09/20 08:20:05 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6381lb # [ 181.426880] niks3-server[1172]: 2026/09/20 08:20:05 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6382lb # [ 181.433711] niks3-server[1172]: 2026/09/20 08:20:06 INFO Completed upload id=616383worker1 # [ 181.430537] nix-grpc-daemon[1651]: time=2026-09-20T08:20:06.004Z level=INFO msg="Upload complete. (335ms)"6384lb # [ 181.436169] niks3-server[1172]: 2026/09/20 08:20:06 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6385worker1 # [ 181.445242] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50280 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16386worker1 # [ 181.461317] nix-daemon[840]: reaped child process 2137, status = succeeded6387client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 20.80 seconds)6388worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1'6389worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics | grep -E 'nix_grpc_build_info\{.*features="[^"]*vip[^"]*".*worker="node-a"\} 1', in 0.09 seconds)6390client: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&16391worker2 # [ 181.821956] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426766392worker2 # [ 181.836956] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6393worker1 # [ 181.908197] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:06Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06394client: (finished: must fail: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag f5 --arg features '["gpu"]' 2>&1, in 0.34 seconds)6395(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 21.23 seconds)6396subtest: low disk drains a worker and builds go to the other6397worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6398lb # [ 182.774879] envoy[931]: conn peer=[2001:db8:1::1]:36712 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=26884 tx=10729 ms=204926399lb # [ 182.780106] envoy[931]: conn peer=[2001:db8:1::1]:55092 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=710 tx=440 ms=726400worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.11 seconds)6401worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6402worker1 # [ 183.877654] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:08Z level=info event=unhealthy reason=min_free6403worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.13 seconds)6404client: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag drain >&26405worker2 # [ 184.393285] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:426766406client # don't know how to build these paths:6407client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6408worker1 # [ 184.458037] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:09Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06409worker1 # [ 184.464230] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:09Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16410worker2 # [ 184.462727] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426766411worker2 # [ 184.479359] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)6412worker2 # [ 184.503547] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5)6413worker2 # [ 184.537150] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 226 ms (attempt 3/5)6414worker2 # [ 184.763546] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 737 ms (attempt 4/5)6415worker2 # [ 185.509742] nix-grpc-daemon[1560]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6416worker2 # [ 185.532262] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426766417client # copying 1 paths to 'lb:50051'...6418client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6419worker2 # [ 185.547816] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426766420worker2 # [ 185.564472] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6421worker2 # [ 185.594133] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:10Z level=info event=niks3_push_started pid=16166422worker2 # [ 185.976134] nix-daemon[846]: accepted connection from pid 1629, user nix-grpc-daemon (trusted)6423worker2 # [ 185.993941] nix-daemon[846]: reaped child process 1632, status = succeeded6424worker2 # [ 186.175403] nix-daemon[846]: accepted connection from pid 1634, user nix-grpc-daemon (trusted)6425worker2 # [ 186.191433] nix-daemon[846]: reaped child process 1637, status = succeeded6426lb # [ 186.253726] niks3-server[1172]: 2026/09/20 08:20:10 INFO Received uploads request method=POST path=/api/pending_closures6427worker2 # [ 186.220460] nix-grpc-daemon[1616]: time=2026-09-20T08:20:10.794Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6428worker2 # [ 186.221949] nix-grpc-daemon[1616]: time=2026-09-20T08:20:10.794Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6429lb # [ 186.291018] niks3-server[1172]: 2026/09/20 08:20:10 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6430worker2 # [ 186.257464] nix-grpc-daemon[1616]: time=2026-09-20T08:20:10.832Z level=INFO msg="Uploading 1 narinfos"6431lb # [ 186.304372] niks3-server[1172]: 2026/09/20 08:20:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6432lb # [ 186.318249] niks3-server[1172]: 2026/09/20 08:20:10 INFO Signed narinfos id=62 count=16433lb # [ 186.325515] niks3-server[1172]: 2026/09/20 08:20:10 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6434lb # [ 186.333314] niks3-server[1172]: 2026/09/20 08:20:10 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6435lb # [ 186.339645] niks3-server[1172]: 2026/09/20 08:20:10 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6436lb # [ 186.344902] niks3-server[1172]: 2026/09/20 08:20:10 INFO Completed upload id=626437worker2 # [ 186.298785] nix-grpc-daemon[1616]: time=2026-09-20T08:20:10.873Z level=INFO msg="Upload complete. (494ms)"6438worker2 # [ 186.307589] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42676 duration_s=0 paths=1 nar_bytes_in=5126439worker2 # [ 186.321643] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426766440worker2 # [ 186.328094] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6441worker2 # [ 186.333021] nix-daemon[846]: reaped child process 1614, status = succeeded6442client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6443client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6444worker2 # [ 186.345775] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6445client # farm-a> LOG-farm-a-drain6446worker2 # [ 187.589355] nix-daemon[846]: reaped child process 1641, status = succeeded6447worker2 # [ 187.760779] nix-daemon[846]: accepted connection from pid 1656, user nix-grpc-daemon (trusted)6448worker2 # [ 187.777820] nix-daemon[846]: reaped child process 1659, status = succeeded6449lb # [ 187.838574] niks3-server[1172]: 2026/09/20 08:20:12 INFO Received uploads request method=POST path=/api/pending_closures6450worker2 # [ 187.804448] nix-grpc-daemon[1616]: time=2026-09-20T08:20:12.377Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6451worker2 # [ 187.805928] nix-grpc-daemon[1616]: time=2026-09-20T08:20:12.377Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6452lb # [ 187.879719] niks3-server[1172]: 2026/09/20 08:20:12 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6453lb # [ 187.891902] niks3-server[1172]: 2026/09/20 08:20:12 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6454lb # [ 187.902245] niks3-server[1172]: 2026/09/20 08:20:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6455lb # [ 187.908219] niks3-server[1172]: 2026/09/20 08:20:12 INFO Signed narinfos id=63 count=16456worker2 # [ 187.863439] nix-grpc-daemon[1616]: time=2026-09-20T08:20:12.436Z level=INFO msg="Uploading 1 narinfos"6457lb # [ 187.913618] niks3-server[1172]: 2026/09/20 08:20:12 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6458lb # [ 187.925246] niks3-server[1172]: 2026/09/20 08:20:12 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6459lb # [ 187.931140] niks3-server[1172]: 2026/09/20 08:20:12 INFO Completed upload id=636460worker2 # [ 187.884446] nix-grpc-daemon[1616]: time=2026-09-20T08:20:12.459Z level=INFO msg="Upload complete. (327ms)"6461lb # [ 187.934314] niks3-server[1172]: 2026/09/20 08:20:12 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6462worker1 # [ 187.946335] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:12Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06463worker2 # [ 187.897025] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42676 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16464worker2 # [ 187.914868] nix-daemon[846]: reaped child process 1639, status = succeeded6465worker2 # [ 187.946164] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426766466worker2 # [ 187.961516] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426766467worker2 # [ 187.975495] nix-grpc-daemon[1616]: time=2026-09-20T08:20:12.550Z level=INFO msg="All 1 paths already cached"6468client # copying 1 paths to 'lb:50051'...6469client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6470worker2 # [ 187.986093] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426766471worker2 # [ 188.000303] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6472worker2 # [ 188.200721] nix-daemon[846]: accepted connection from pid 1663, user nix-grpc-daemon (trusted)6473worker2 # [ 188.218151] nix-daemon[846]: reaped child process 1666, status = succeeded6474worker2 # [ 188.397303] nix-daemon[846]: accepted connection from pid 1668, user nix-grpc-daemon (trusted)6475worker2 # [ 188.414585] nix-daemon[846]: reaped child process 1671, status = succeeded6476lb # [ 188.472320] niks3-server[1172]: 2026/09/20 08:20:13 INFO Received uploads request method=POST path=/api/pending_closures6477worker2 # [ 188.435032] nix-grpc-daemon[1616]: time=2026-09-20T08:20:13.009Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6478worker2 # [ 188.436651] nix-grpc-daemon[1616]: time=2026-09-20T08:20:13.009Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6479lb # [ 188.498769] niks3-server[1172]: 2026/09/20 08:20:13 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6480worker2 # [ 188.464650] nix-grpc-daemon[1616]: time=2026-09-20T08:20:13.039Z level=INFO msg="Uploading 1 narinfos"6481lb # [ 188.516679] niks3-server[1172]: 2026/09/20 08:20:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6482lb # [ 188.528062] niks3-server[1172]: 2026/09/20 08:20:13 INFO Signed narinfos id=64 count=16483lb # [ 188.533134] niks3-server[1172]: 2026/09/20 08:20:13 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6484worker2 # [ 188.496439] nix-grpc-daemon[1616]: time=2026-09-20T08:20:13.071Z level=INFO msg="Upload complete. (478ms)"6485lb # [ 188.543033] niks3-server[1172]: 2026/09/20 08:20:13 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6486lb # [ 188.548384] niks3-server[1172]: 2026/09/20 08:20:13 INFO Completed upload id=646487lb # [ 188.551628] niks3-server[1172]: 2026/09/20 08:20:13 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6488worker2 # [ 188.504112] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42676 duration_s=0 paths=1 nar_bytes_in=6326489worker2 # [ 188.515892] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426766490worker2 # [ 188.522198] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6491worker2 # [ 188.526848] nix-daemon[846]: reaped child process 1661, status = succeeded6492client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6493worker2 # [ 188.538036] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6494worker1 # [ 187.953213] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:12Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16495client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6496worker2 # [ 189.965357] nix-daemon[846]: reaped child process 1675, status = succeeded6497client # farm-b> LOG-farm-b-drain6498worker2 # [ 190.127436] nix-daemon[846]: accepted connection from pid 1690, user nix-grpc-daemon (trusted)6499worker2 # [ 190.146720] nix-daemon[846]: reaped child process 1693, status = succeeded6500lb # [ 190.205741] niks3-server[1172]: 2026/09/20 08:20:14 INFO Received uploads request method=POST path=/api/pending_closures6501worker2 # [ 190.169889] nix-grpc-daemon[1616]: time=2026-09-20T08:20:14.744Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6502worker2 # [ 190.171378] nix-grpc-daemon[1616]: time=2026-09-20T08:20:14.744Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6503lb # [ 190.241662] niks3-server[1172]: 2026/09/20 08:20:14 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6504lb # [ 190.253370] niks3-server[1172]: 2026/09/20 08:20:14 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6505worker2 # [ 190.222587] nix-grpc-daemon[1616]: time=2026-09-20T08:20:14.797Z level=INFO msg="Uploading 1 narinfos"6506lb # [ 190.263510] niks3-server[1172]: 2026/09/20 08:20:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6507lb # [ 190.272532] niks3-server[1172]: 2026/09/20 08:20:14 INFO Signed narinfos id=65 count=16508lb # [ 190.275008] niks3-server[1172]: 2026/09/20 08:20:14 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6509lb # [ 190.284476] niks3-server[1172]: 2026/09/20 08:20:14 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6510lb # [ 190.292271] niks3-server[1172]: 2026/09/20 08:20:14 INFO Completed upload id=656511worker2 # [ 190.246008] nix-grpc-daemon[1616]: time=2026-09-20T08:20:14.821Z level=INFO msg="Upload complete. (296ms)"6512lb # [ 190.295252] niks3-server[1172]: 2026/09/20 08:20:14 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6513worker2 # [ 190.256971] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42676 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16514worker1 # [ 190.309223] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:14Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06515worker2 # [ 190.273562] nix-daemon[846]: reaped child process 1673, status = succeeded6516worker1 # [ 190.309601] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:14Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16517worker2 # [ 190.309928] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426766518worker2 # [ 190.325481] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:426766519worker2 # [ 190.339170] nix-grpc-daemon[1616]: time=2026-09-20T08:20:14.914Z level=INFO msg="All 2 paths already cached"6520client # copying 1 paths to 'lb:50051'...6521client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6522worker2 # [ 190.349769] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:426766523worker2 # [ 190.364164] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6524worker2 # [ 190.564740] nix-daemon[846]: accepted connection from pid 1697, user nix-grpc-daemon (trusted)6525worker2 # [ 190.584878] nix-daemon[846]: reaped child process 1700, status = succeeded6526worker2 # [ 190.757797] nix-daemon[846]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6527worker2 # [ 190.775817] nix-daemon[846]: reaped child process 1705, status = succeeded6528lb # [ 190.836246] niks3-server[1172]: 2026/09/20 08:20:15 INFO Received uploads request method=POST path=/api/pending_closures6529worker2 # [ 190.798915] nix-grpc-daemon[1616]: time=2026-09-20T08:20:15.373Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6530worker2 # [ 190.800462] nix-grpc-daemon[1616]: time=2026-09-20T08:20:15.373Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6531lb # [ 190.862626] niks3-server[1172]: 2026/09/20 08:20:15 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6532worker2 # [ 190.839527] nix-grpc-daemon[1616]: time=2026-09-20T08:20:15.414Z level=INFO msg="Uploading 1 narinfos"6533lb # [ 190.877920] niks3-server[1172]: 2026/09/20 08:20:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6534lb # [ 190.891882] niks3-server[1172]: 2026/09/20 08:20:15 INFO Signed narinfos id=66 count=16535lb # [ 190.899242] niks3-server[1172]: 2026/09/20 08:20:15 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6536lb # [ 190.908118] niks3-server[1172]: 2026/09/20 08:20:15 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6537lb # [ 190.914086] niks3-server[1172]: 2026/09/20 08:20:15 INFO Completed upload id=666538worker2 # [ 190.867742] nix-grpc-daemon[1616]: time=2026-09-20T08:20:15.442Z level=INFO msg="Upload complete. (486ms)"6539lb # [ 190.916784] niks3-server[1172]: 2026/09/20 08:20:15 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6540worker2 # [ 190.877163] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42676 duration_s=0 paths=1 nar_bytes_in=7606541worker2 # [ 190.889920] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:426766542worker2 # [ 190.895875] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6543worker2 # [ 190.900867] nix-daemon[846]: reaped child process 1695, status = succeeded6544client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6545worker2 # [ 190.913839] nix-daemon[846]: accepted connection from pid 1560, user nix-grpc-daemon (trusted)6546client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6547client # farm-top> LOG-farm-top-drain6548worker2 # [ 191.943731] nix-daemon[846]: reaped child process 1709, status = succeeded6549worker2 # [ 192.109096] nix-daemon[846]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6550worker2 # [ 192.129008] nix-daemon[846]: reaped child process 1727, status = succeeded6551lb # [ 192.189907] niks3-server[1172]: 2026/09/20 08:20:16 INFO Received uploads request method=POST path=/api/pending_closures6552worker2 # [ 192.152403] nix-grpc-daemon[1616]: time=2026-09-20T08:20:16.726Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6553worker2 # [ 192.153849] nix-grpc-daemon[1616]: time=2026-09-20T08:20:16.726Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6554lb # [ 192.217917] niks3-server[1172]: 2026/09/20 08:20:16 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6555lb # [ 192.234000] niks3-server[1172]: 2026/09/20 08:20:16 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6556lb # [ 192.245563] niks3-server[1172]: 2026/09/20 08:20:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6557worker2 # [ 192.204669] nix-grpc-daemon[1616]: time=2026-09-20T08:20:16.779Z level=INFO msg="Uploading 1 narinfos"6558lb # [ 192.252367] niks3-server[1172]: 2026/09/20 08:20:16 INFO Signed narinfos id=67 count=16559lb # [ 192.258594] niks3-server[1172]: 2026/09/20 08:20:16 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6560lb # [ 192.268674] niks3-server[1172]: 2026/09/20 08:20:16 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6561lb # [ 192.275636] niks3-server[1172]: 2026/09/20 08:20:16 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6562lb # [ 192.280307] niks3-server[1172]: 2026/09/20 08:20:16 INFO Completed upload id=676563worker2 # [ 192.233722] nix-grpc-daemon[1616]: time=2026-09-20T08:20:16.808Z level=INFO msg="Upload complete. (310ms)"6564worker2 # [ 192.248125] nix-grpc-daemon[1560]: ts=2026-09-20T08:20:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42676 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16565worker2 # [ 192.262975] nix-daemon[846]: reaped child process 1707, status = succeeded6566client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?ca-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ca.pem&client-cert=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.pem&client-key=/nix/store/qnr217kg3abnw0irxipxjmff230hy3hd-farm-certs/ci.key' --eval-store auto -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag drain >&2, in 8.15 seconds)6567client: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag drain outPath6568client: (finished: must succeed: nix eval --raw -f /nix/store/3c79s1mhyz00j9rig3m7fz3did05wl5h-job.nix --argstr tag drain outPath, in 0.24 seconds)6569worker1: must succeed: rm /nix/.rw-store/fill6570lb # [ 192.775292] envoy[931]: conn peer=[2001:db8:1::1]:55102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9380 tx=3843 ms=78676571worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.17 seconds)6572worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'6573worker1 # [ 192.888302] nix-grpc-daemon[1579]: ts=2026-09-20T08:20:17Z level=info event=healthy reason=min_free6574worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 0.05 seconds)6575(finished: subtest: low disk drains a worker and builds go to the other, in 10.95 seconds)6576(finished: run the VM test script, in 193.80 seconds)6577test script finished in 194.09s6578cleanup6579kill QemuMachine (pid 50)6580client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6581kill QemuMachine (pid 49)6582lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6583kill QemuMachine (pid 52)6584worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6585kill QemuMachine (pid 51)6586worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)6587(finished: cleanup, in 1.19 seconds)6588additionally exposed symbols:6589 client, lb, worker1, worker2,6590 vlan1,6591 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh