vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #491
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.3xAIyrXiW2', fmt=raw size=107374182414worker2: QEMU running (pid 51)15worker1 # Disk image does not exist, creating the virtualisation disk image...16client # Disk image does not exist, creating the virtualisation disk image...17worker1: QEMU running (pid 52)18lb: QEMU running (pid 49)19worker1 # Formatting '/build/vm-state-worker1/tmp.VLiHE7XKgL', fmt=raw size=107374182420client # Formatting '/build/vm-state-client/tmp.UvZ4kungZW', fmt=raw size=107374182421client: QEMU running (pid 50)22worker1 # mke2fs 1.47.4 (6-Mar-2025)23client # mke2fs 1.47.4 (6-Mar-2025)24worker2 # mke2fs 1.47.4 (6-Mar-2025)25lb # Disk image does not exist, creating the virtualisation disk image...26worker1 # Discarding device blocks: 0/262144 done27client # Discarding device blocks: 0/262144 done28lb # Formatting '/build/vm-state-lb/tmp.YgQPNU2BHZ', fmt=raw size=107374182429worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes30client # Creating filesystem with 262144 4k blocks and 65536 inodes31lb # mke2fs 1.47.4 (6-Mar-2025)32worker1 # Filesystem UUID: f4aed186-e9ce-4b8b-aecf-d37ccfbd1eaa33client # Filesystem UUID: 0e706931-f17b-4b37-b6a0-448fe9e8d98d34lb # Discarding device blocks: 0/262144 done35worker1 # Superblock backups stored on blocks:36client # Superblock backups stored on blocks:37lb # Creating filesystem with 262144 4k blocks and 65536 inodes38worker1 # 32768, 98304, 163840, 22937639client # 32768, 98304, 163840, 22937640lb # Filesystem UUID: 24b38147-c974-4e0c-aab1-7a7726444a7f41worker1 # 42client # 43lb # Superblock backups stored on blocks:44worker1 # Allocating group tables: 0/8 done45client # Allocating group tables: 0/8 done46lb # 32768, 98304, 163840, 22937647worker1 # Writing inode tables: 0/8 done48client # Writing inode tables: 0/8 done49lb # 50worker1 # Creating journal (8192 blocks): done51client # Creating journal (8192 blocks): done52lb # Allocating group tables: 0/8 done53worker1 # Writing superblocks and filesystem accounting information: 0/8 done54client # Writing superblocks and filesystem accounting information: 0/8 done55lb # Writing inode tables: 0/8 done56worker1 # 57client # 58lb # Creating journal (8192 blocks): done59worker1 # Virtualisation disk image created.60client # Virtualisation disk image created.61lb # Writing superblocks and filesystem accounting information: 0/8 done62worker2 # Discarding device blocks: 0/262144 done63lb # 64worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes65lb # Virtualisation disk image created.66worker2 # Filesystem UUID: 13079bea-20af-432e-9217-d176ab4ef00467(finished: start all VMs, in 0.56 seconds)68worker2 # Superblock backups stored on blocks:69client: waiting for unit network-addresses-eth1.service70worker2 # 32768, 98304, 163840, 22937671client: waiting for the VM to finish booting72worker2 # 73worker2 # Allocating group tables: 0/8 done74worker2 # Writing inode tables: 0/8 done75worker2 # Creating journal (8192 blocks): done76worker2 # Writing superblocks and filesystem accounting information: 0/8 done77worker2 # 78worker2 # Virtualisation disk image created.79worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]80worker2 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 202681worker2 # [ 0.000000] KASLR enabled82worker2 # [ 0.000000] random: crng init done83worker2 # [ 0.000000] Machine model: linux,dummy-virt84worker2 # [ 0.000000] efi: UEFI not found.85worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT86worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]87worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]88worker2 # [ 0.000000] Zone ranges:89worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]90worker2 # [ 0.000000] DMA32 empty91worker2 # [ 0.000000] Normal empty92worker2 # [ 0.000000] Device empty93worker2 # [ 0.000000] Movable zone start for each node94worker2 # [ 0.000000] Early memory node ranges95worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]96worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]97worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c40000098worker2 # [ 0.000000] psci: probing for conduit method from DT.99worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.100worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs101worker2 # [ 0.000000] psci: Trusted OS migration not required102worker2 # [ 0.000000] psci: SMC Calling Convention v1.1103worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)104worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296105worker2 # [ 0.000000] Detected PIPT I-cache on CPU0106worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)107worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface108worker2 # [ 0.000000] CPU features: detected: Spectre-v4109worker2 # [ 0.000000] CPU features: detected: Spectre-BHB110worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38111worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23112worker2 # [ 0.000000] alternatives: applying boot alternatives113worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9yhq1hw85db6ijlh0ly649xic3c3g3k7-nixos-system-worker2-test/init regInfo=/nix/store/0bd0gkm9gs3d3zqk1f81npm79q2nx4w1-closure-info/registration console=ttyAMA0,115200n8 console=tty0114worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0bd0gkm9gs3d3zqk1f81npm79q2nx4w1-closure-info/registration", will be passed to user space.115worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes116worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)117worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)118worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB119worker2 # [ 0.000000] software IO TLB: area num 1.120worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB121worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)122worker2 # [ 0.000000] Fallback order for Node 0: 0123worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216124worker2 # [ 0.000000] Policy zone: DMA125worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off126worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1127worker2 # [ 0.000000] allocated 3145728 bytes of page_ext128worker2 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages129worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups130worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.131worker2 # [ 0.000000] rcu: RCU event tracing is enabled.132worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.133worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.134worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.135worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.136worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.137worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1138worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]139worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.140worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.141worker1 # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026142worker1 # [ 0.000000] KASLR enabled143worker1 # [ 0.000000] random: crng init done144worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.145worker1 # [ 0.000000] Machine model: linux,dummy-virt146worker1 # [ 0.000000] efi: UEFI not found.147worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0148worker2 # [ 0.000000] GICv3: 256 SPIs implemented149worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT150worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented151worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq152worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]153worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI154worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]155worker1 # [ 0.000000] Zone ranges:156worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0157worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]158worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000159worker1 # [ 0.000000] DMA32 empty160worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]161worker1 # [ 0.000000] Normal empty162worker1 # [ 0.000000] Device empty163worker1 # [ 0.000000] Movable zone start for each node164worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)165worker1 # [ 0.000000] Early memory node ranges166worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]167worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)168worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]169worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000170worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000171worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000172worker1 # [ 0.000000] psci: probing for conduit method from DT.173worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.174worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.175worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs176worker1 # [ 0.000000] psci: Trusted OS migration not required177worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns178worker1 # [ 0.000000] psci: SMC Calling Convention v1.1179worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).180worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)181worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns182worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296183worker1 # [ 0.000000] Detected PIPT I-cache on CPU0184worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns185worker2 # [ 0.000038] arm-pv: using stolen time PV186worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)187worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface188worker2 # [ 0.000773] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)189worker1 # [ 0.000000] CPU features: detected: Spectre-v4190worker2 # [ 0.000948] Console: colour dummy device 80x25191worker1 # [ 0.000000] CPU features: detected: Spectre-BHB192worker2 # [ 0.000957] printk: legacy console [tty0] enabled193worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38194worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23195worker2 # [ 0.001156] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)196worker1 # [ 0.000000] alternatives: applying boot alternatives197worker2 # [ 0.001163] pid_max: default: 32768 minimum: 301198worker2 # [ 0.001236] LSM: initializing lsm=capability,landlock,yama,bpf,ima199worker2 # [ 0.001421] landlock: Up and running.200worker2 # [ 0.001424] Yama: becoming mindful.201worker2 # [ 0.002212] LSM support for eBPF active202worker2 # [ 0.002394] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)203worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qzg8b7m9hrv2jd9ryfd2msjn8536iryk-nixos-system-worker1-test/init regInfo=/nix/store/a2dzz2r4hng404hn2rp3mikhdpnz0daa-closure-info/registration console=ttyAMA0,115200n8 console=tty0204worker2 # [ 0.002432] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)205worker2 # [ 0.004882] cacheinfo: Unable to detect cache hierarchy for CPU 0206worker2 # [ 0.006142] rcu: Hierarchical SRCU implementation.207worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/a2dzz2r4hng404hn2rp3mikhdpnz0daa-closure-info/registration", will be passed to user space.208worker2 # [ 0.006149] rcu: Max phase no-delay instances is 1000.209worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes210worker2 # [ 0.007352] fsl-mc MSI: its@8080000 domain created211worker2 # [ 0.007442] EFI services will not be available.212worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)213worker2 # [ 0.007693] smp: Bringing up secondary CPUs ...214worker2 # [ 0.007705] smp: Brought up 1 node, 1 CPU215worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)216worker2 # [ 0.007708] SMP: Total of 1 processors activated.217worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB218worker2 # [ 0.007714] CPU: All CPU(s) started at EL1219worker1 # [ 0.000000] software IO TLB: area num 1.220worker2 # [ 0.007725] CPU features: detected: Branch Target Identification221worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB222worker2 # [ 0.007730] CPU features: detected: ARMv8.4 Translation Table Level223worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)224worker2 # [ 0.007733] CPU features: detected: Instruction cache invalidation not required for I/D coherence225worker1 # [ 0.000000] Fallback order for Node 0: 0226worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216227worker2 # [ 0.007737] CPU features: detected: Data cache clean to the PoU not required for I/D coherence228worker1 # [ 0.000000] Policy zone: DMA229worker2 # [ 0.007740] CPU features: detected: Common not Private translations230worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off231worker2 # [ 0.007743] CPU features: detected: CRC32 instructions232worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1233worker2 # [ 0.007746] CPU features: detected: Data cache clean to Point of Deep Persistence234worker1 # [ 0.000000] allocated 3145728 bytes of page_ext235worker1 # [ 0.000000] ftrace: allocating 74885 entries in 294 pages236worker2 # [ 0.007750] CPU features: detected: Data cache clean to Point of Persistence237worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups238client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]239worker2 # [ 0.007753] CPU features: detected: Data independent timing control (DIT)240worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.241worker2 # [ 0.007757] CPU features: detected: E0PD242worker1 # [ 0.000000] rcu: RCU event tracing is enabled.243worker2 # [ 0.007759] CPU features: detected: Enhanced Counter Virtualization244client # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026245worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.246client # [ 0.000000] KASLR enabled247client # [ 0.000000] random: crng init done248worker2 # [ 0.007762] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)249worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.250client # [ 0.000000] Machine model: linux,dummy-virt251worker2 # [ 0.007766] CPU features: detected: Enhanced Virtualization Traps252worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.253client # [ 0.000000] efi: UEFI not found.254worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.255worker2 # [ 0.007769] CPU features: detected: Fine Grained Traps256client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT257worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.258worker2 # [ 0.007772] CPU features: detected: Generic authentication (architected QARMA5 algorithm)259client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]260worker2 # [ 0.007777] CPU features: detected: RCpc load-acquire (LDAPR)261worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1262client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]263client # [ 0.000000] Zone ranges:264worker2 # [ 0.007780] CPU features: detected: LSE atomic instructions265worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.266worker2 # [ 0.007783] CPU features: detected: Privileged Access Never267client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]268client # [ 0.000000] DMA32 empty269worker2 # [ 0.007785] CPU features: detected: PMUv3270client # [ 0.000000] Normal empty271worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.272worker2 # [ 0.007788] CPU features: detected: RAS Extension Support273client # [ 0.000000] Device empty274client # [ 0.000000] Movable zone start for each node275worker2 # [ 0.007791] CPU features: detected: RASv1p1 Extension Support276worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.277client # [ 0.000000] Early memory node ranges278worker2 # [ 0.007793] CPU features: detected: Random Number Generator279worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0280client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]281worker1 # [ 0.000000] GICv3: 256 SPIs implemented282worker2 # [ 0.007796] CPU features: detected: Speculation barrier (SB)283worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented284client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]285worker2 # [ 0.007799] CPU features: detected: Stage-2 Force Write-Back286worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq287client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000288worker2 # [ 0.007802] CPU features: detected: TLB range maintenance instructions289worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI290client # [ 0.000000] psci: probing for conduit method from DT.291worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0292worker2 # [ 0.007806] CPU features: detected: Speculative Store Bypassing Safe (SSBS)293client # [ 0.000000] psci: PSCIv1.3 detected in firmware.294client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs295worker2 # [ 0.007849] alternatives: applying system-wide alternatives296worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000297worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]298client # [ 0.000000] psci: Trusted OS migration not required299worker2 # [ 0.010935] CPU features: detected: BBM Level 2 without TLB conflict abort300client # [ 0.000000] psci: SMC Calling Convention v1.1301worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)302lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]303client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)304worker2 # [ 0.011137] Memory: 1406796K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)305worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)306client # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296307worker2 # [ 0.011637] devtmpfs: initialized308client # [ 0.000000] Detected PIPT I-cache on CPU0309lb # [ 0.000000] Linux version 6.18.50 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 7 15:23:00 UTC 2026310worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000311lb # [ 0.000000] KASLR enabled312worker2 # [ 0.013589] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)313lb # [ 0.000000] random: crng init done314worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000315client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)316lb # [ 0.000000] Machine model: linux,dummy-virt317worker2 # [ 0.013612] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).318client # [ 0.000000] CPU features: detected: GICv3 CPU interface319lb # [ 0.000000] efi: UEFI not found.320worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.321client # [ 0.000000] CPU features: detected: Spectre-v4322worker2 # [ 0.013818] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL323lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT324client # [ 0.000000] CPU features: detected: Spectre-BHB325worker2 # [ 0.013822] 0 pages in range for non-PLT usage326worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns327worker2 # [ 0.013823] 508288 pages in range for PLT usage328lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]329client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38330worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).331worker2 # [ 0.013968] pinctrl core: initialized pinctrl subsystem332lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]333client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23334worker2 # [ 0.014860] DMI not present or invalid.335lb # [ 0.000000] Zone ranges:336client # [ 0.000000] alternatives: applying boot alternatives337worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns338worker2 # [ 0.018718] NET: Registered PF_NETLINK/PF_ROUTE protocol family339lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]340lb # [ 0.000000] DMA32 empty341worker2 # [ 0.021081] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations342lb # [ 0.000000] Normal empty343worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns344lb # [ 0.000000] Device empty345worker1 # [ 0.000038] arm-pv: using stolen time PV346lb # [ 0.000000] Movable zone start for each node347worker2 # [ 0.021267] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations348lb # [ 0.000000] Early memory node ranges349worker1 # [ 0.000800] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)350worker2 # [ 0.021480] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations351lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]352worker1 # [ 0.000989] Console: colour dummy device 80x25353worker2 # [ 0.021502] audit: initializing netlink subsys (disabled)354client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test/init regInfo=/nix/store/0alpn248jpxcl2rq85257rx5nb3drlqq-closure-info/registration console=ttyAMA0,115200n8 console=tty0355worker1 # [ 0.000997] printk: legacy console [tty0] enabled356lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]357worker2 # [ 0.022110] thermal_sys: Registered thermal governor 'fair_share'358lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000359worker2 # [ 0.022112] thermal_sys: Registered thermal governor 'bang_bang'360worker1 # [ 0.001215] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)361lb # [ 0.000000] psci: probing for conduit method from DT.362client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/0alpn248jpxcl2rq85257rx5nb3drlqq-closure-info/registration", will be passed to user space.363worker1 # [ 0.001222] pid_max: default: 32768 minimum: 301364worker2 # [ 0.022116] thermal_sys: Registered thermal governor 'step_wise'365lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.366client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes367worker1 # [ 0.001294] LSM: initializing lsm=capability,landlock,yama,bpf,ima368lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs369worker2 # [ 0.022122] thermal_sys: Registered thermal governor 'user_space'370worker1 # [ 0.001516] landlock: Up and running.371lb # [ 0.000000] psci: Trusted OS migration not required372worker2 # [ 0.022124] thermal_sys: Registered thermal governor 'power_allocator'373client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)374worker1 # [ 0.001519] Yama: becoming mindful.375lb # [ 0.000000] psci: SMC Calling Convention v1.1376worker1 # [ 0.002205] LSM support for eBPF active377worker2 # [ 0.022149] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1378client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)379worker1 # [ 0.002379] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)380worker2 # [ 0.022160] cpuidle: using governor ladder381lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)382client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB383worker2 # [ 0.022167] cpuidle: using governor menu384lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186648 r8192 d116456 u311296385client # [ 0.000000] software IO TLB: area num 1.386worker1 # [ 0.002420] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)387lb # [ 0.000000] Detected PIPT I-cache on CPU0388worker2 # [ 0.022364] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.389worker1 # [ 0.003613] cacheinfo: Unable to detect cache hierarchy for CPU 0390client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB391worker2 # [ 0.022382] ASID allocator initialised with 65536 entries392worker1 # [ 0.004439] rcu: Hierarchical SRCU implementation.393lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)394client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)395worker1 # [ 0.004444] rcu: Max phase no-delay instances is 1000.396lb # [ 0.000000] CPU features: detected: GICv3 CPU interface397client # [ 0.000000] Fallback order for Node 0: 0398worker2 # [ 0.023694] Serial: AMBA PL011 UART driver399worker1 # [ 0.005656] fsl-mc MSI: its@8080000 domain created400lb # [ 0.000000] CPU features: detected: Spectre-v4401client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216402lb # [ 0.000000] CPU features: detected: Spectre-BHB403worker1 # [ 0.005748] EFI services will not be available.404worker2 # [ 0.029229] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1405client # [ 0.000000] Policy zone: DMA406worker1 # [ 0.005821] smp: Bringing up secondary CPUs ...407worker2 # [ 0.029376] printk: console [ttyAMA0] enabled408lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38409worker1 # [ 0.005830] smp: Brought up 1 node, 1 CPU410client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off411worker1 # [ 0.005833] SMP: Total of 1 processors activated.412lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23413client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1414worker1 # [ 0.005839] CPU: All CPU(s) started at EL1415lb # [ 0.000000] alternatives: applying boot alternatives416client # [ 0.000000] allocated 3145728 bytes of page_ext417worker1 # [ 0.005850] CPU features: detected: Branch Target Identification418client # [ 0.000000] ftrace: allocating 74885 entries in 294 pages419worker1 # [ 0.005856] CPU features: detected: ARMv8.4 Translation Table Level420client # [ 0.000000] ftrace: allocated 294 pages with 4 groups421client # [ 0.000000] rcu: Hierarchical RCU implementation.422worker1 # [ 0.005859] CPU features: detected: Instruction cache invalidation not required for I/D coherence423client # [ 0.000000] rcu: RCU event tracing is enabled.424worker1 # [ 0.005863] CPU features: detected: Data cache clean to the PoU not required for I/D coherence425client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.426lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/a9n40kyqx09p2mx9kz6ck1ddzmmfmfqv-nixos-system-lb-test/init regInfo=/nix/store/kxfc7nc3yvm950aydaljrdz4dxr491hv-closure-info/registration console=ttyAMA0,115200n8 console=tty0427client # [ 0.000000] Trampoline variant of Tasks RCU enabled.428worker1 # [ 0.005866] CPU features: detected: Common not Private translations429client # [ 0.000000] Rude variant of Tasks RCU enabled.430worker1 # [ 0.005870] CPU features: detected: CRC32 instructions431client # [ 0.000000] Tracing variant of Tasks RCU enabled.432worker2 # [ 0.155392] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages433lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/kxfc7nc3yvm950aydaljrdz4dxr491hv-closure-info/registration", will be passed to user space.434worker1 # [ 0.005873] CPU features: detected: Data cache clean to Point of Deep Persistence435client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.436worker2 # [ 0.155412] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page437lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes438worker1 # [ 0.005876] CPU features: detected: Data cache clean to Point of Persistence439worker2 # [ 0.155417] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages440client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1441lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)442worker1 # [ 0.005879] CPU features: detected: Data independent timing control (DIT)443worker2 # [ 0.155422] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page444worker1 # [ 0.005882] CPU features: detected: E0PD445client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.446lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)447worker1 # [ 0.005885] CPU features: detected: Enhanced Counter Virtualization448worker2 # [ 0.155426] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages449lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB450client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.451worker2 # [ 0.155431] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page452worker1 # [ 0.005888] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)453lb # [ 0.000000] software IO TLB: area num 1.454worker2 # [ 0.155435] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages455worker1 # [ 0.005891] CPU features: detected: Enhanced Virtualization Traps456client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.457lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB458worker1 # [ 0.005894] CPU features: detected: Fine Grained Traps459worker2 # [ 0.155439] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page460client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0461lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)462client # [ 0.000000] GICv3: 256 SPIs implemented463worker1 # [ 0.005898] CPU features: detected: Generic authentication (architected QARMA5 algorithm)464lb # [ 0.000000] Fallback order for Node 0: 0465client # [ 0.000000] GICv3: 0 Extended SPIs implemented466worker1 # [ 0.005903] CPU features: detected: RCpc load-acquire (LDAPR)467client # [ 0.000000] Root IRQ handler: gic_handle_irq468lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216469lb # [ 0.000000] Policy zone: DMA470client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI471worker1 # [ 0.005906] CPU features: detected: LSE atomic instructions472worker2 # [ 0.162979] fbcon: Taking over console473worker2 # [ 0.162996] ACPI: Interpreter disabled.474client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0475worker1 # [ 0.005908] CPU features: detected: Privileged Access Never476lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off477worker1 # [ 0.005911] CPU features: detected: PMUv3478client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000479lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1480client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]481lb # [ 0.000000] allocated 3145728 bytes of page_ext482worker1 # [ 0.005914] CPU features: detected: RAS Extension Support483lb # [ 0.000000] ftrace: allocating 74885 entries in 294 pages484client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)485worker1 # [ 0.005916] CPU features: detected: RASv1p1 Extension Support486lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups487worker1 # [ 0.005919] CPU features: detected: Random Number Generator488lb # [ 0.000000] rcu: Hierarchical RCU implementation.489client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)490lb # [ 0.000000] rcu: RCU event tracing is enabled.491worker1 # [ 0.005922] CPU features: detected: Speculation barrier (SB)492client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000493worker1 # [ 0.005925] CPU features: detected: Stage-2 Force Write-Back494lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.495client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000496lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.497worker1 # [ 0.005928] CPU features: detected: TLB range maintenance instructions498lb # [ 0.000000] Rude variant of Tasks RCU enabled.499client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.500worker1 # [ 0.005932] CPU features: detected: Speculative Store Bypassing Safe (SSBS)501lb # [ 0.000000] Tracing variant of Tasks RCU enabled.502worker1 # [ 0.005973] alternatives: applying system-wide alternatives503client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns504lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.505worker1 # [ 0.009124] CPU features: detected: BBM Level 2 without TLB conflict abort506worker2 # [ 0.172013] iommu: Default domain type: Translated507client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).508lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1509worker2 # [ 0.172025] iommu: DMA domain TLB invalidation policy: strict mode510lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.511client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns512worker1 # [ 0.009381] Memory: 1406528K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123772K reserved, 32768K cma-reserved)513worker2 # [ 0.173723] SCSI subsystem initialized514worker1 # [ 0.009836] devtmpfs: initialized515client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns516lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.517worker1 # [ 0.011726] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)518client # [ 0.000032] arm-pv: using stolen time PV519worker2 # [ 0.174381] usbcore: registered new interface driver usbfs520lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.521worker2 # [ 0.174413] usbcore: registered new interface driver hub522worker1 # [ 0.011749] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).523client # [ 0.000530] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)524worker2 # [ 0.174431] usbcore: registered new device driver usb525lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0526worker1 # [ 0.011952] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL527client # [ 0.000712] Console: colour dummy device 80x25528lb # [ 0.000000] GICv3: 256 SPIs implemented529worker2 # [ 0.174728] pps_core: LinuxPPS API ver. 1 registered530lb # [ 0.000000] GICv3: 0 Extended SPIs implemented531client # [ 0.000722] printk: legacy console [tty0] enabled532worker1 # [ 0.011957] 0 pages in range for non-PLT usage533worker1 # [ 0.011958] 508288 pages in range for PLT usage534lb # [ 0.000000] Root IRQ handler: gic_handle_irq535worker2 # [ 0.174734] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>536worker1 # [ 0.012076] pinctrl core: initialized pinctrl subsystem537lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI538client # [ 0.000915] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)539worker2 # [ 0.174744] PTP clock support registered540worker1 # [ 0.012923] DMI not present or invalid.541worker2 # [ 0.174791] EDAC MC: Ver: 3.0.0542lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0543client # [ 0.000922] pid_max: default: 32768 minimum: 301544worker1 # [ 0.016787] NET: Registered PF_NETLINK/PF_ROUTE protocol family545lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000546worker2 # [ 0.179458] scmi_core: SCMI protocol bus registered547client # [ 0.000995] LSM: initializing lsm=capability,landlock,yama,bpf,ima548client # [ 0.001140] landlock: Up and running.549lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]550worker1 # [ 0.019650] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations551client # [ 0.001142] Yama: becoming mindful.552worker2 # [ 0.180468] FPGA manager framework553client # [ 0.001740] LSM support for eBPF active554worker1 # [ 0.019860] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations555worker2 # [ 0.181455] vgaarb: loaded556lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)557client # [ 0.001891] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)558worker1 # [ 0.020073] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations559worker2 # [ 0.182113] clocksource: Switched to clocksource arch_sys_counter560lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)561worker1 # [ 0.020098] audit: initializing netlink subsys (disabled)562client # [ 0.001924] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)563lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000564worker1 # [ 0.020719] thermal_sys: Registered thermal governor 'fair_share'565client # [ 0.003177] cacheinfo: Unable to detect cache hierarchy for CPU 0566lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000567worker1 # [ 0.020721] thermal_sys: Registered thermal governor 'bang_bang'568client # [ 0.003956] rcu: Hierarchical SRCU implementation.569worker1 # [ 0.020725] thermal_sys: Registered thermal governor 'step_wise'570client # [ 0.003961] rcu: Max phase no-delay instances is 1000.571lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.572client # [ 0.005207] fsl-mc MSI: its@8080000 domain created573worker1 # [ 0.020731] thermal_sys: Registered thermal governor 'user_space'574client # [ 0.005299] EFI services will not be available.575lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns576worker1 # [ 0.020733] thermal_sys: Registered thermal governor 'power_allocator'577client # [ 0.005367] smp: Bringing up secondary CPUs ...578lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).579client # [ 0.005376] smp: Brought up 1 node, 1 CPU580worker1 # [ 0.020760] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1581client # [ 0.005379] SMP: Total of 1 processors activated.582worker1 # [ 0.020769] cpuidle: using governor ladder583client # [ 0.005385] CPU: All CPU(s) started at EL1584lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns585worker1 # [ 0.020774] cpuidle: using governor menu586client # [ 0.005396] CPU features: detected: Branch Target Identification587worker1 # [ 0.020997] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.588worker2 # [ 0.189009] VFS: Disk quotas dquot_6.6.0589lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns590lb # [ 0.000032] arm-pv: using stolen time PV591client # [ 0.005402] CPU features: detected: ARMv8.4 Translation Table Level592worker1 # [ 0.021014] ASID allocator initialised with 65536 entries593worker2 # [ 0.189060] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)594worker1 # [ 0.022264] Serial: AMBA PL011 UART driver595client # [ 0.005405] CPU features: detected: Instruction cache invalidation not required for I/D coherence596worker2 # [ 0.190778] netfs: FS-Cache loaded597lb # [ 0.000470] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)598worker1 # [ 0.027758] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1599lb # [ 0.000688] Console: colour dummy device 80x25600worker2 # [ 0.190909] pnp: PnP ACPI: disabled601client # [ 0.005408] CPU features: detected: Data cache clean to the PoU not required for I/D coherence602worker1 # [ 0.027945] printk: console [ttyAMA0] enabled603lb # [ 0.000696] printk: legacy console [tty0] enabled604client # [ 0.005412] CPU features: detected: Common not Private translations605client # [ 0.005415] CPU features: detected: CRC32 instructions606lb # [ 0.000926] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)607lb # [ 0.000933] pid_max: default: 32768 minimum: 301608client # [ 0.005418] CPU features: detected: Data cache clean to Point of Deep Persistence609lb # [ 0.001015] LSM: initializing lsm=capability,landlock,yama,bpf,ima610client # [ 0.005421] CPU features: detected: Data cache clean to Point of Persistence611lb # [ 0.001165] landlock: Up and running.612lb # [ 0.001169] Yama: becoming mindful.613client # [ 0.005424] CPU features: detected: Data independent timing control (DIT)614lb # [ 0.001641] LSM support for eBPF active615client # [ 0.005427] CPU features: detected: E0PD616lb # [ 0.001792] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)617client # [ 0.005430] CPU features: detected: Enhanced Counter Virtualization618lb # [ 0.001827] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)619client # [ 0.005433] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)620worker2 # [ 0.194824] NET: Registered PF_INET protocol family621lb # [ 0.003015] cacheinfo: Unable to detect cache hierarchy for CPU 0622client # [ 0.005436] CPU features: detected: Enhanced Virtualization Traps623worker2 # [ 0.195113] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)624lb # [ 0.003813] rcu: Hierarchical SRCU implementation.625client # [ 0.005439] CPU features: detected: Fine Grained Traps626lb # [ 0.003818] rcu: Max phase no-delay instances is 1000.627client # [ 0.005443] CPU features: detected: Generic authentication (architected QARMA5 algorithm)628lb # [ 0.005120] fsl-mc MSI: its@8080000 domain created629worker1 # [ 0.153064] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages630lb # [ 0.005221] EFI services will not be available.631client # [ 0.005448] CPU features: detected: RCpc load-acquire (LDAPR)632worker1 # [ 0.153084] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page633lb # [ 0.005308] smp: Bringing up secondary CPUs ...634client # [ 0.005451] CPU features: detected: LSE atomic instructions635lb # [ 0.005317] smp: Brought up 1 node, 1 CPU636worker1 # [ 0.153090] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages637client # [ 0.005454] CPU features: detected: Privileged Access Never638lb # [ 0.005320] SMP: Total of 1 processors activated.639client # [ 0.005457] CPU features: detected: PMUv3640worker1 # [ 0.153094] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page641lb # [ 0.005326] CPU: All CPU(s) started at EL1642client # [ 0.005459] CPU features: detected: RAS Extension Support643lb # [ 0.005337] CPU features: detected: Branch Target Identification644worker1 # [ 0.153098] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages645client # [ 0.005462] CPU features: detected: RASv1p1 Extension Support646lb # [ 0.005342] CPU features: detected: ARMv8.4 Translation Table Level647worker1 # [ 0.153103] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page648client # [ 0.005465] CPU features: detected: Random Number Generator649worker1 # [ 0.153107] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages650client # [ 0.005467] CPU features: detected: Speculation barrier (SB)651lb # [ 0.005346] CPU features: detected: Instruction cache invalidation not required for I/D coherence652worker1 # [ 0.153111] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page653client # [ 0.005470] CPU features: detected: Stage-2 Force Write-Back654lb # [ 0.005349] CPU features: detected: Data cache clean to the PoU not required for I/D coherence655client # [ 0.005473] CPU features: detected: TLB range maintenance instructions656lb # [ 0.005353] CPU features: detected: Common not Private translations657client # [ 0.005478] CPU features: detected: Speculative Store Bypassing Safe (SSBS)658lb # [ 0.005356] CPU features: detected: CRC32 instructions659worker1 # [ 0.160727] fbcon: Taking over console660client # [ 0.005516] alternatives: applying system-wide alternatives661worker1 # [ 0.160744] ACPI: Interpreter disabled.662lb # [ 0.005359] CPU features: detected: Data cache clean to Point of Deep Persistence663client # [ 0.008542] CPU features: detected: BBM Level 2 without TLB conflict abort664lb # [ 0.005363] CPU features: detected: Data cache clean to Point of Persistence665lb # [ 0.005366] CPU features: detected: Data independent timing control (DIT)666lb # [ 0.005369] CPU features: detected: E0PD667client # [ 0.008716] Memory: 1406524K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123784K reserved, 32768K cma-reserved)668client # [ 0.009107] devtmpfs: initialized669lb # [ 0.005372] CPU features: detected: Enhanced Counter Virtualization670client # [ 0.011009] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)671lb # [ 0.005375] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)672lb # [ 0.005379] CPU features: detected: Enhanced Virtualization Traps673client # [ 0.011032] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).674lb # [ 0.005381] CPU features: detected: Fine Grained Traps675client # [ 0.011232] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL676client # [ 0.011236] 0 pages in range for non-PLT usage677lb # [ 0.005385] CPU features: detected: Generic authentication (architected QARMA5 algorithm)678client # [ 0.011237] 508288 pages in range for PLT usage679lb # [ 0.005391] CPU features: detected: RCpc load-acquire (LDAPR)680client # [ 0.011387] pinctrl core: initialized pinctrl subsystem681lb # [ 0.005394] CPU features: detected: LSE atomic instructions682client # [ 0.012180] DMI not present or invalid.683worker1 # [ 0.169400] iommu: Default domain type: Translated684lb # [ 0.005397] CPU features: detected: Privileged Access Never685client # [ 0.015527] NET: Registered PF_NETLINK/PF_ROUTE protocol family686worker1 # [ 0.169411] iommu: DMA domain TLB invalidation policy: strict mode687worker1 # [ 0.169801] SCSI subsystem initialized688client # [ 0.018331] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations689lb # [ 0.005400] CPU features: detected: PMUv3690lb # [ 0.005402] CPU features: detected: RAS Extension Support691client # [ 0.018534] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations692worker1 # [ 0.171871] usbcore: registered new interface driver usbfs693lb # [ 0.005405] CPU features: detected: RASv1p1 Extension Support694worker1 # [ 0.171908] usbcore: registered new interface driver hub695client # [ 0.018737] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations696lb # [ 0.005408] CPU features: detected: Random Number Generator697worker1 # [ 0.171926] usbcore: registered new device driver usb698client # [ 0.018761] audit: initializing netlink subsys (disabled)699lb # [ 0.005410] CPU features: detected: Speculation barrier (SB)700worker1 # [ 0.172196] pps_core: LinuxPPS API ver. 1 registered701client # [ 0.019394] thermal_sys: Registered thermal governor 'fair_share'702lb # [ 0.005413] CPU features: detected: Stage-2 Force Write-Back703client # [ 0.019397] thermal_sys: Registered thermal governor 'bang_bang'704worker1 # [ 0.172203] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>705lb # [ 0.005416] CPU features: detected: TLB range maintenance instructions706worker1 # [ 0.172213] PTP clock support registered707client # [ 0.019400] thermal_sys: Registered thermal governor 'step_wise'708worker1 # [ 0.172263] EDAC MC: Ver: 3.0.0709lb # [ 0.005421] CPU features: detected: Speculative Store Bypassing Safe (SSBS)710client # [ 0.019405] thermal_sys: Registered thermal governor 'user_space'711lb # [ 0.005460] alternatives: applying system-wide alternatives712worker1 # [ 0.176948] scmi_core: SCMI protocol bus registered713client # [ 0.019408] thermal_sys: Registered thermal governor 'power_allocator'714lb # [ 0.008518] CPU features: detected: BBM Level 2 without TLB conflict abort715worker1 # [ 0.177933] FPGA manager framework716client # [ 0.019435] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1717worker1 # [ 0.178970] vgaarb: loaded718client # [ 0.019443] cpuidle: using governor ladder719client # [ 0.019449] cpuidle: using governor menu720lb # [ 0.008726] Memory: 1406560K/1572864K available (24448K kernel code, 7090K rwdata, 26348K rodata, 4736K init, 1107K bss, 123764K reserved, 32768K cma-reserved)721worker1 # [ 0.179633] clocksource: Switched to clocksource arch_sys_counter722lb # [ 0.009109] devtmpfs: initialized723client # [ 0.019639] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.724lb # [ 0.010971] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)725client # [ 0.019656] ASID allocator initialised with 65536 entries726client # [ 0.020953] Serial: AMBA PL011 UART driver727lb # [ 0.010995] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).728lb # [ 0.011207] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL729lb # [ 0.011211] 0 pages in range for non-PLT usage730client # [ 0.026202] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1731lb # [ 0.011212] 508288 pages in range for PLT usage732client # [ 0.026368] printk: console [ttyAMA0] enabled733worker1 # [ 0.184109] VFS: Disk quotas dquot_6.6.0734worker1 # [ 0.184159] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)735lb # [ 0.011328] pinctrl core: initialized pinctrl subsystem736lb # [ 0.012108] DMI not present or invalid.737lb # [ 0.015397] NET: Registered PF_NETLINK/PF_ROUTE protocol family738worker1 # [ 0.185898] netfs: FS-Cache loaded739worker1 # [ 0.186038] pnp: PnP ACPI: disabled740lb # [ 0.018230] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations741worker2 # [ 0.225327] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)742lb # [ 0.018419] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations743worker2 # [ 0.225380] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)744lb # [ 0.018627] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations745lb # [ 0.018650] audit: initializing netlink subsys (disabled)746worker2 # [ 0.225408] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)747lb # [ 0.019300] thermal_sys: Registered thermal governor 'fair_share'748worker2 # [ 0.225491] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)749lb # [ 0.019302] thermal_sys: Registered thermal governor 'bang_bang'750client # [ 0.149671] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages751worker2 # [ 0.225638] TCP: Hash tables configured (established 16384 bind 16384)752lb # [ 0.019305] thermal_sys: Registered thermal governor 'step_wise'753worker1 # [ 0.190084] NET: Registered PF_INET protocol family754client # [ 0.149688] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page755worker2 # [ 0.225732] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)756lb # [ 0.019311] thermal_sys: Registered thermal governor 'user_space'757worker1 # [ 0.190369] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)758client # [ 0.149693] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages759worker2 # [ 0.225774] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)760lb # [ 0.019314] thermal_sys: Registered thermal governor 'power_allocator'761client # [ 0.149698] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page762worker2 # [ 0.225813] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)763lb # [ 0.019341] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1764client # [ 0.149702] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages765worker2 # [ 0.225926] NET: Registered PF_UNIX/PF_LOCAL protocol family766lb # [ 0.019350] cpuidle: using governor ladder767worker2 # [ 0.225995] NET: Registered PF_XDP protocol family768lb # [ 0.019356] cpuidle: using governor menu769client # [ 0.149706] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page770worker2 # [ 0.226015] PCI: CLS 0 bytes, default 64771lb # [ 0.019545] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.772client # [ 0.149711] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages773worker2 # [ 0.226200] Trying to unpack rootfs image as initramfs...774lb # [ 0.019561] ASID allocator initialised with 65536 entries775client # [ 0.149715] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page776lb # [ 0.020813] Serial: AMBA PL011 UART driver777lb # [ 0.026200] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1778lb # [ 0.026379] printk: console [ttyAMA0] enabled779client # [ 0.157240] fbcon: Taking over console780client # [ 0.157257] ACPI: Interpreter disabled.781worker2 # [ 0.244137] kvm [1]: HYP mode not available782lb # [ 0.151392] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages783client # [ 0.166274] iommu: Default domain type: Translated784lb # [ 0.151415] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page785client # [ 0.166286] iommu: DMA domain TLB invalidation policy: strict mode786client # [ 0.166647] SCSI subsystem initialized787lb # [ 0.151421] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages788lb # [ 0.151426] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page789client # [ 0.168755] usbcore: registered new interface driver usbfs790client # [ 0.168786] usbcore: registered new interface driver hub791lb # [ 0.151430] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages792client # [ 0.168804] usbcore: registered new device driver usb793lb # [ 0.151434] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page794client # [ 0.169072] pps_core: LinuxPPS API ver. 1 registered795lb # [ 0.151439] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages796client # [ 0.169079] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>797lb # [ 0.151443] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page798client # [ 0.169088] PTP clock support registered799client # [ 0.169139] EDAC MC: Ver: 3.0.0800client # [ 0.173993] scmi_core: SCMI protocol bus registered801lb # [ 0.159516] fbcon: Taking over console802lb # [ 0.159534] ACPI: Interpreter disabled.803client # [ 0.174985] FPGA manager framework804client # [ 0.176057] vgaarb: loaded805client # [ 0.176700] clocksource: Switched to clocksource arch_sys_counter806client # [ 0.182969] VFS: Disk quotas dquot_6.6.0807lb # [ 0.168014] iommu: Default domain type: Translated808client # [ 0.183021] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)809lb # [ 0.168025] iommu: DMA domain TLB invalidation policy: strict mode810lb # [ 0.168424] SCSI subsystem initialized811client # [ 0.184791] netfs: FS-Cache loaded812client # [ 0.184930] pnp: PnP ACPI: disabled813lb # [ 0.170547] usbcore: registered new interface driver usbfs814lb # [ 0.170586] usbcore: registered new interface driver hub815worker1 # [ 0.220778] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)816lb # [ 0.170604] usbcore: registered new device driver usb817lb # [ 0.170899] pps_core: LinuxPPS API ver. 1 registered818worker1 # [ 0.220833] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)819lb # [ 0.170907] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>820worker1 # [ 0.220867] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)821lb # [ 0.170926] PTP clock support registered822lb # [ 0.170978] EDAC MC: Ver: 3.0.0823worker1 # [ 0.220947] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)824lb # [ 0.175880] scmi_core: SCMI protocol bus registered825worker1 # [ 0.221092] TCP: Hash tables configured (established 16384 bind 16384)826worker1 # [ 0.221182] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)827lb # [ 0.176928] FPGA manager framework828lb # [ 0.177896] vgaarb: loaded829worker1 # [ 0.221234] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)830client # [ 0.188863] NET: Registered PF_INET protocol family831worker1 # [ 0.221270] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)832lb # [ 0.178612] clocksource: Switched to clocksource arch_sys_counter833client # [ 0.189143] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)834worker1 # [ 0.221362] NET: Registered PF_UNIX/PF_LOCAL protocol family835worker1 # [ 0.221420] NET: Registered PF_XDP protocol family836worker1 # [ 0.221440] PCI: CLS 0 bytes, default 64837worker1 # [ 0.221708] Trying to unpack rootfs image as initramfs...838lb # [ 0.181408] VFS: Disk quotas dquot_6.6.0839lb # [ 0.181447] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)840lb # [ 0.185378] netfs: FS-Cache loaded841lb # [ 0.185520] pnp: PnP ACPI: disabled842worker1 # [ 0.237664] kvm [1]: HYP mode not available843lb # [ 0.189593] NET: Registered PF_INET protocol family844lb # [ 0.189882] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)845client # [ 0.219512] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)846client # [ 0.219569] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)847client # [ 0.219599] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)848client # [ 0.219677] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)849client # [ 0.219823] TCP: Hash tables configured (established 16384 bind 16384)850client # [ 0.219915] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851client # [ 0.219952] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)852client # [ 0.219988] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)853client # [ 0.220124] NET: Registered PF_UNIX/PF_LOCAL protocol family854client # [ 0.220149] NET: Registered PF_XDP protocol family855client # [ 0.220168] PCI: CLS 0 bytes, default 64856client # [ 0.220431] Trying to unpack rootfs image as initramfs...857client # [ 0.238755] kvm [1]: HYP mode not available858lb # [ 0.221450] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)859lb # [ 0.221508] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)860lb # [ 0.221544] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)861lb # [ 0.221639] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)862lb # [ 0.221784] TCP: Hash tables configured (established 16384 bind 16384)863lb # [ 0.221880] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)864lb # [ 0.221920] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)865lb # [ 0.221957] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)866lb # [ 0.222087] NET: Registered PF_UNIX/PF_LOCAL protocol family867lb # [ 0.222154] NET: Registered PF_XDP protocol family868lb # [ 0.222174] PCI: CLS 0 bytes, default 64869lb # [ 0.222434] Trying to unpack rootfs image as initramfs...870lb # [ 0.241476] kvm [1]: HYP mode not available871worker2 # [ 0.350728] Initialise system trusted keyrings872worker2 # [ 0.351537] workingset: timestamp_bits=42 max_order=19 bucket_order=0873worker2 # [ 0.352896] squashfs: version 4.0 (2009/01/31) Phillip Lougher874worker2 # [ 0.353698] 9p: Installing v9fs 9p2000 file system support875worker2 # [ 0.382419] Key type asymmetric registered876worker2 # [ 0.382451] Asymmetric key parser 'x509' registered877worker2 # [ 0.382535] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)878worker2 # [ 0.384821] io scheduler mq-deadline registered879worker2 # [ 0.384833] io scheduler kyber registered880worker1 # [ 0.344217] Initialise system trusted keyrings881worker1 # [ 0.345014] workingset: timestamp_bits=42 max_order=19 bucket_order=0882worker1 # [ 0.346358] squashfs: version 4.0 (2009/01/31) Phillip Lougher883worker1 # [ 0.347163] 9p: Installing v9fs 9p2000 file system support884worker2 # [ 0.394294] pl061_gpio 9030000.pl061: PL061 GPIO chip registered885worker2 # [ 0.395829] ledtrig-cpu: registered to indicate activity on CPUs886worker2 # [ 0.396250] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:887worker2 # [ 0.396268] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000888worker2 # [ 0.396288] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000889worker2 # [ 0.396298] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000890worker2 # [ 0.396321] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits891worker2 # [ 0.396346] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]892worker2 # [ 0.396425] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00893worker2 # [ 0.396435] pci_bus 0000:00: root bus resource [bus 00-ff]894worker2 # [ 0.396441] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]895worker2 # [ 0.396446] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]896worker2 # [ 0.396452] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]897worker2 # [ 0.396512] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint898worker2 # [ 0.396955] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint899worker2 # [ 0.397144] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]900worker2 # [ 0.397162] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]901worker2 # [ 0.397193] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]902worker2 # [ 0.397208] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]903worker2 # [ 0.397674] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint904worker2 # [ 0.397862] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]905worker2 # [ 0.397878] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]906worker2 # [ 0.397908] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]907worker1 # [ 0.375913] Key type asymmetric registered908worker1 # [ 0.375944] Asymmetric key parser 'x509' registered909client # [ 0.335831] Initialise system trusted keyrings910worker1 # [ 0.376028] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)911client # [ 0.336601] workingset: timestamp_bits=42 max_order=19 bucket_order=0912worker1 # [ 0.378263] io scheduler mq-deadline registered913worker1 # [ 0.378274] io scheduler kyber registered914client # [ 0.338306] squashfs: version 4.0 (2009/01/31) Phillip Lougher915client # [ 0.339104] 9p: Installing v9fs 9p2000 file system support916worker2 # [ 0.426514] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint917worker2 # [ 0.426734] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]918worker1 # [ 0.387811] pl061_gpio 9030000.pl061: PL061 GPIO chip registered919worker2 # [ 0.426753] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]920worker2 # [ 0.426783] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]921worker2 # [ 0.427255] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint922worker2 # [ 0.427443] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]923worker1 # [ 0.389380] ledtrig-cpu: registered to indicate activity on CPUs924worker2 # [ 0.427460] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]925worker1 # [ 0.389814] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:926worker2 # [ 0.427510] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]927worker1 # [ 0.389833] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000928worker2 # [ 0.427981] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint929worker2 # [ 0.428165] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]930worker1 # [ 0.389846] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000931lb # [ 0.339234] Initialise system trusted keyrings932worker2 # [ 0.428181] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]933worker1 # [ 0.389855] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000934worker2 # [ 0.428211] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]935worker1 # [ 0.389874] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits936worker2 # [ 0.428670] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint937worker2 # [ 0.428862] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]938worker1 # [ 0.389901] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]939worker2 # [ 0.428878] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]940worker1 # [ 0.389983] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00941worker2 # [ 0.428908] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]942worker1 # [ 0.389992] pci_bus 0000:00: root bus resource [bus 00-ff]943worker2 # [ 0.429408] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint944worker1 # [ 0.389998] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]945worker2 # [ 0.429595] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]946worker1 # [ 0.390003] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]947worker2 # [ 0.429611] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]948worker1 # [ 0.390008] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]949worker2 # [ 0.429640] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]950lb # [ 0.346696] workingset: timestamp_bits=42 max_order=19 bucket_order=0951worker1 # [ 0.390069] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint952worker2 # [ 0.429657] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]953lb # [ 0.348251] squashfs: version 4.0 (2009/01/31) Phillip Lougher954client # [ 0.363081] Key type asymmetric registered955worker1 # [ 0.390531] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint956worker2 # [ 0.430135] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint957client # [ 0.363115] Asymmetric key parser 'x509' registered958worker1 # [ 0.390724] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]959worker2 # [ 0.430322] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]960client # [ 0.363208] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)961worker1 # [ 0.390742] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]962worker2 # [ 0.430353] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]963client # [ 0.365493] io scheduler mq-deadline registered964lb # [ 0.349085] 9p: Installing v9fs 9p2000 file system support965worker1 # [ 0.390772] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]966client # [ 0.365502] io scheduler kyber registered967worker2 # [ 0.430808] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint968worker1 # [ 0.390788] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]969worker2 # [ 0.430999] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]970worker1 # [ 0.391268] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint971worker2 # [ 0.431031] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]972worker1 # [ 0.391462] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]973worker2 # [ 0.431431] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint974worker1 # [ 0.391478] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]975worker2 # [ 0.431628] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]976worker1 # [ 0.391511] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]977worker2 # [ 0.431882] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint978worker2 # [ 0.432190] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]979worker2 # [ 0.432209] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]980worker2 # [ 0.432240] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]981worker2 # [ 0.432704] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint982worker2 # [ 0.432890] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]983worker1 # [ 0.411773] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint984worker2 # [ 0.432906] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]985worker1 # [ 0.411967] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]986worker2 # [ 0.432936] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]987worker1 # [ 0.411983] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]988worker2 # [ 0.433566] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned989worker1 # [ 0.412015] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]990worker2 # [ 0.433578] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned991worker1 # [ 0.412522] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint992worker1 # [ 0.412715] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]993worker2 # [ 0.433584] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned994worker1 # [ 0.412730] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]995worker2 # [ 0.433629] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned996worker1 # [ 0.412763] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]997worker2 # [ 0.433677] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned998worker1 # [ 0.413253] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint999worker2 # [ 0.433723] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1000worker1 # [ 0.413441] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1001worker1 # [ 0.413457] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1002worker2 # [ 0.433769] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1003worker1 # [ 0.413486] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1004worker2 # [ 0.433816] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1005lb # [ 0.369732] Key type asymmetric registered1006worker1 # [ 0.413935] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1007worker2 # [ 0.433865] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1008lb # [ 0.369767] Asymmetric key parser 'x509' registered1009worker1 # [ 0.414118] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1010lb # [ 0.369860] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1011worker2 # [ 0.433912] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1012worker1 # [ 0.414134] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1013worker2 # [ 0.433960] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1014worker1 # [ 0.414163] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1015worker2 # [ 0.434008] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1016worker1 # [ 0.414637] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1017client # [ 0.380891] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1018worker1 # [ 0.414828] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1019worker2 # [ 0.434096] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1020client # [ 0.381584] ledtrig-cpu: registered to indicate activity on CPUs1021worker1 # [ 0.414844] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1022client # [ 0.382009] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1023worker1 # [ 0.414877] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024worker1 # [ 0.414893] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1025client # [ 0.382027] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001026worker1 # [ 0.415364] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1027client # [ 0.382041] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001028worker1 # [ 0.415553] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1029lb # [ 0.378687] io scheduler mq-deadline registered1030client # [ 0.382049] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001031worker1 # [ 0.415583] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1032lb # [ 0.378711] io scheduler kyber registered1033client # [ 0.382072] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1034client # [ 0.382100] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1035client # [ 0.382178] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001036client # [ 0.382187] pci_bus 0000:00: root bus resource [bus 00-ff]1037worker2 # [ 0.482191] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1038client # [ 0.382193] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1039worker2 # [ 0.482224] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1040client # [ 0.382198] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1041worker2 # [ 0.482247] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1042lb # [ 0.384251] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1043client # [ 0.382204] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1044worker2 # [ 0.482271] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1045worker2 # [ 0.482300] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1046client # [ 0.382281] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1047lb # [ 0.385597] ledtrig-cpu: registered to indicate activity on CPUs1048worker2 # [ 0.482323] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1049client # [ 0.382738] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1050lb # [ 0.385992] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1051client # [ 0.382934] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1052worker2 # [ 0.482346] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1053lb # [ 0.386010] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001054client # [ 0.382950] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1055worker2 # [ 0.482370] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1056lb # [ 0.386024] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001057client # [ 0.382981] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1058worker2 # [ 0.482393] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1059client # [ 0.382997] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1060worker2 # [ 0.482419] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1061lb # [ 0.386032] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001062client # [ 0.383501] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1063worker2 # [ 0.482442] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1064lb # [ 0.386054] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1065client # [ 0.383690] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1066worker2 # [ 0.482466] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1067client # [ 0.383706] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1068lb # [ 0.386076] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1069worker2 # [ 0.482488] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1070client # [ 0.383736] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1071lb # [ 0.386171] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001072worker2 # [ 0.482510] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1073lb # [ 0.386181] pci_bus 0000:00: root bus resource [bus 00-ff]1074worker2 # [ 0.482534] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1075client # [ 0.384200] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1076lb # [ 0.386187] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1077worker2 # [ 0.482555] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1078client # [ 0.384389] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1079worker1 # [ 0.444220] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1080lb # [ 0.386192] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1081worker2 # [ 0.482577] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1082client # [ 0.384406] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1083worker1 # [ 0.444468] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1084worker2 # [ 0.482599] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1085client # [ 0.384436] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086lb # [ 0.386198] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1087worker1 # [ 0.444500] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1088worker2 # [ 0.482621] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1089lb # [ 0.386291] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1090client # [ 0.384927] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1091worker2 # [ 0.482644] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1092worker1 # [ 0.444914] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1093client # [ 0.385112] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1094worker2 # [ 0.482667] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1095worker1 # [ 0.445103] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1096client # [ 0.385128] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1097worker2 # [ 0.482699] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1098worker1 # [ 0.445357] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1099client # [ 0.385160] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100worker2 # [ 0.482710] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1101worker1 # [ 0.445669] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1102client # [ 0.385631] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1103worker2 # [ 0.482714] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1104worker1 # [ 0.445690] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1105client # [ 0.385831] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1106worker2 # [ 0.483926] pci 0000:00:0a.0: enabling device (0000 -> 0002)1107worker1 # [ 0.445720] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1108client # [ 0.385848] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1109lb # [ 0.398735] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1110worker1 # [ 0.446213] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1111client # [ 0.385879] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1112lb # [ 0.398938] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1113worker1 # [ 0.446404] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1114lb # [ 0.398956] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1115client # [ 0.386343] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1116worker1 # [ 0.446420] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1117client # [ 0.386528] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1118lb # [ 0.398991] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1119worker1 # [ 0.446452] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1120client # [ 0.386544] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1121lb # [ 0.399008] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1122worker1 # [ 0.447071] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1123client # [ 0.386573] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124lb # [ 0.399490] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1125worker1 # [ 0.447082] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1126client # [ 0.387035] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1127lb # [ 0.399688] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1128client # [ 0.387223] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1129worker1 # [ 0.447088] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1130lb # [ 0.399707] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1131client # [ 0.387239] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1132lb # [ 0.399743] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133worker1 # [ 0.447138] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1134client # [ 0.387270] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135lb # [ 0.400225] pci 0000:00:03.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1136worker1 # [ 0.447185] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1137client # [ 0.387286] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1138lb # [ 0.400424] pci 0000:00:03.0: BAR 0 [io 0x0000-0x003f]1139worker1 # [ 0.447238] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1140client # [ 0.387786] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1141lb # [ 0.400444] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1142worker1 # [ 0.447286] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1143client # [ 0.387980] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1144lb # [ 0.400477] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145client # [ 0.388010] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146worker1 # [ 0.447337] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1147lb # [ 0.400970] pci 0000:00:04.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1148lb # [ 0.401161] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1149client # [ 0.388469] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1150worker1 # [ 0.447385] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1151lb # [ 0.401180] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1152client # [ 0.388659] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1153worker1 # [ 0.447439] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1154lb # [ 0.401215] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1155client # [ 0.388689] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1156worker1 # [ 0.447487] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1157lb # [ 0.401711] pci 0000:00:05.0: [1af4:1009] type 00 class 0x000200 conventional PCI endpoint1158lb # [ 0.401904] pci 0000:00:05.0: BAR 0 [io 0x0000-0x001f]1159worker1 # [ 0.447536] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1160lb # [ 0.401920] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1161worker1 # [ 0.447617] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1162lb # [ 0.401951] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1163worker1 # [ 0.447687] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1164worker2 # [ 0.520444] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1165lb # [ 0.402421] pci 0000:00:06.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1166worker1 # [ 0.447709] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1167worker1 # [ 0.447733] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1168worker1 # [ 0.447758] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1169worker1 # [ 0.447785] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1170worker1 # [ 0.447807] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1171worker1 # [ 0.447829] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1172worker1 # [ 0.447853] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1173worker1 # [ 0.447875] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1174client # [ 0.437230] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1175worker1 # [ 0.447899] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1176client # [ 0.437451] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1177worker1 # [ 0.447923] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1178client # [ 0.437724] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1179worker1 # [ 0.447957] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1180client # [ 0.438029] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1181worker1 # [ 0.447987] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1182client # [ 0.438047] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1183worker1 # [ 0.448018] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1184client # [ 0.438077] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1185worker1 # [ 0.448049] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1186worker1 # [ 0.448073] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1187client # [ 0.438564] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1188client # [ 0.438754] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1189worker1 # [ 0.448096] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1190client # [ 0.438770] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1191worker1 # [ 0.448118] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1192worker1 # [ 0.448143] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1193client # [ 0.438801] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1194worker1 # [ 0.448167] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1195lb # [ 0.426727] pci 0000:00:06.0: BAR 0 [io 0x0000-0x007f]1196client # [ 0.439406] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1197worker1 # [ 0.448197] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1198lb # [ 0.426767] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1199client # [ 0.439419] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1200worker1 # [ 0.448225] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1201lb # [ 0.426798] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1202client # [ 0.439424] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1203worker1 # [ 0.448234] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1204lb # [ 0.427356] pci 0000:00:07.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1205worker1 # [ 0.448240] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1206client # [ 0.439473] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1207lb # [ 0.427559] pci 0000:00:07.0: BAR 0 [io 0x0000-0x001f]1208worker1 # [ 0.449455] pci 0000:00:0a.0: enabling device (0000 -> 0002)1209lb # [ 0.427577] pci 0000:00:07.0: BAR 1 [mem 0x00000000-0x00000fff]1210client # [ 0.439523] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1211lb # [ 0.427607] pci 0000:00:07.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1212worker2 # [ 0.538849] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1213client # [ 0.439571] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1214lb # [ 0.427626] pci 0000:00:07.0: ROM [mem 0x00000000-0x0003ffff pref]1215worker2 # [ 0.541040] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1216client # [ 0.439620] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1217lb # [ 0.428144] pci 0000:00:08.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1218client # [ 0.439669] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1219lb # [ 0.428338] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1220client # [ 0.439718] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1221lb # [ 0.428372] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1222worker2 # [ 0.545033] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1223client # [ 0.439772] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1224lb # [ 0.428890] pci 0000:00:09.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1225lb # [ 0.429098] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1226client # [ 0.439824] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1227lb # [ 0.429131] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1228client # [ 0.439872] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1229lb # [ 0.429564] pci 0000:00:0a.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1230client # [ 0.439954] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1231lb # [ 0.429751] pci 0000:00:0a.0: BAR 0 [mem 0x00000000-0x00000fff]1232client # [ 0.440013] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1233worker2 # [ 0.548211] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1234lb # [ 0.430022] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1235client # [ 0.440035] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1236lb # [ 0.430328] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1237client # [ 0.440057] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1238lb # [ 0.430352] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1239client # [ 0.440079] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1240lb # [ 0.430383] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1241client # [ 0.440103] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1242lb # [ 0.430878] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1243client # [ 0.440126] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1244lb # [ 0.431072] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1245client # [ 0.440150] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1246lb # [ 0.431091] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1247client # [ 0.440172] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1248lb # [ 0.431123] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1249client # [ 0.440194] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1250lb # [ 0.431772] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1251client # [ 0.440217] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1252lb # [ 0.431783] pci 0000:00:07.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1253client # [ 0.440241] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1254lb # [ 0.431789] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1255client # [ 0.440264] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1256lb # [ 0.431838] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1257client # [ 0.440287] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1258client # [ 0.440309] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1259lb # [ 0.431888] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1260client # [ 0.440330] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1261lb # [ 0.431939] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1262client # [ 0.440351] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1263worker2 # [ 0.559678] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1264lb # [ 0.431993] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1265client # [ 0.440372] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1266worker2 # [ 0.561982] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1267lb # [ 0.432041] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1268client # [ 0.440394] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1269client # [ 0.440415] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1270lb # [ 0.432092] pci 0000:00:07.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1271client # [ 0.440437] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1272lb # [ 0.432140] pci 0000:00:08.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1273client # [ 0.440459] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1274worker1 # [ 0.518560] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1275lb # [ 0.432188] pci 0000:00:09.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1276client # [ 0.440485] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1277client # [ 0.440495] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1278lb # [ 0.432237] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1279client # [ 0.440500] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1280lb # [ 0.432305] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1281lb # [ 0.432352] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1282lb # [ 0.432376] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1283lb # [ 0.432398] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1284client # [ 0.482802] pci 0000:00:0a.0: enabling device (0000 -> 0002)1285lb # [ 0.432421] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1286lb # [ 0.432448] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1287worker2 # [ 0.573517] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1288lb # [ 0.432476] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1289lb # [ 0.432501] pci 0000:00:07.0: BAR 1 [mem 0x10086000-0x10086fff]: assigned1290lb # [ 0.432524] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1291worker1 # [ 0.525177] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1292lb # [ 0.432549] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1293worker1 # [ 0.527389] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1294lb # [ 0.432585] pci 0000:00:0a.0: BAR 0 [mem 0x10089000-0x10089fff]: assigned1295lb # [ 0.432609] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1296lb # [ 0.432637] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1297lb # [ 0.432662] pci 0000:00:06.0: BAR 0 [io 0x1000-0x107f]: assigned1298lb # [ 0.432685] pci 0000:00:03.0: BAR 0 [io 0x1080-0x10bf]: assigned1299lb # [ 0.432708] pci 0000:00:0b.0: BAR 0 [io 0x10c0-0x10ff]: assigned1300lb # [ 0.432733] pci 0000:00:01.0: BAR 0 [io 0x1100-0x111f]: assigned1301lb # [ 0.432757] pci 0000:00:02.0: BAR 0 [io 0x1120-0x113f]: assigned1302lb # [ 0.432782] pci 0000:00:04.0: BAR 0 [io 0x1140-0x115f]: assigned1303lb # [ 0.432807] pci 0000:00:05.0: BAR 0 [io 0x1160-0x117f]: assigned1304lb # [ 0.432833] pci 0000:00:07.0: BAR 0 [io 0x1180-0x119f]: assigned1305lb # [ 0.432857] pci 0000:00:0c.0: BAR 0 [io 0x11a0-0x11bf]: assigned1306worker1 # [ 0.537247] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1307lb # [ 0.432889] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1308worker2 # [ 0.583975] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1309worker1 # [ 0.539629] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1310lb # [ 0.432899] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1311lb # [ 0.432904] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1312lb # [ 0.434110] pci 0000:00:0a.0: enabling device (0000 -> 0002)1313worker2 # [ 0.588536] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1314worker2 # [ 0.590913] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1315lb # [ 0.494735] pci 0000:00:0a.0: quirk_usb_early_handoff+0x0/0xa60 took 59207 usecs1316worker1 # [ 0.551561] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1317client # [ 0.511755] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1318worker2 # [ 0.604191] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1319client # [ 0.514973] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1320worker1 # [ 0.561114] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1321worker2 # [ 0.611447] msm_serial: driver initialized1322worker2 # [ 0.611646] SuperH (H)SCI(F) driver initialized1323worker2 # [ 0.611706] STM32 USART driver initialized1324client # [ 0.524962] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1325client # [ 0.527071] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1326client # [ 0.531059] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1327lb # [ 0.516074] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1328lb # [ 0.518549] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1329worker1 # [ 0.572385] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1330worker1 # [ 0.574937] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1331client # [ 0.537230] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1332worker1 # [ 0.576956] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1333client # [ 0.540646] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1334worker1 # [ 0.579535] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1335client # [ 0.551935] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1336lb # [ 0.536137] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1337lb # [ 0.538332] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1338lb # [ 0.542410] virtio-pci 0000:00:05.0: enabling device (0000 -> 0003)1339worker2 # [ 0.646794] loop: module loaded1340worker1 # [ 0.600435] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1341worker2 # [ 0.647024] virtio_blk virtio5: 1/0/0 default/read/poll queues1342worker2 # [ 0.647909] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1343worker1 # [ 0.603173] msm_serial: driver initialized1344lb # [ 0.545562] virtio-pci 0000:00:06.0: enabling device (0000 -> 0003)1345client # [ 0.558461] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1346worker1 # [ 0.603314] SuperH (H)SCI(F) driver initialized1347client # [ 0.560630] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1348worker1 # [ 0.603370] STM32 USART driver initialized1349worker2 # [ 0.654792] megasas: 07.734.00.00-rc11350worker2 # [ 0.655637] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1351client # [ 0.570390] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1352lb # [ 0.556296] virtio-pci 0000:00:07.0: enabling device (0000 -> 0003)1353lb # [ 0.562822] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1354lb # [ 0.565558] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1355worker2 # [ 0.666192] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001356worker2 # [ 0.666247] Intel/Sharp Extended Query Table at 0x00311357worker2 # [ 0.668908] Using buffer write method1358worker2 # [ 0.669019] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1359client # [ 0.587918] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1360worker2 # [ 0.670979] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001361worker2 # [ 0.671011] Intel/Sharp Extended Query Table at 0x00311362worker2 # [ 0.672844] Using buffer write method1363worker2 # [ 0.672874] Concatenating MTD devices:1364worker2 # [ 0.672879] (0): "0.flash"1365worker2 # [ 0.672883] (1): "0.flash"1366worker2 # [ 0.672886] into device "0.flash"1367client # [ 0.590624] msm_serial: driver initialized1368client # [ 0.590773] SuperH (H)SCI(F) driver initialized1369client # [ 0.590828] STM32 USART driver initialized1370lb # [ 0.575749] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1371lb # [ 0.578176] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1372worker1 # [ 0.643982] loop: module loaded1373worker1 # [ 0.644221] virtio_blk virtio5: 1/0/0 default/read/poll queues1374worker1 # [ 0.645205] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1375lb # [ 0.596164] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1376lb # [ 0.598833] msm_serial: driver initialized1377lb # [ 0.599003] SuperH (H)SCI(F) driver initialized1378lb # [ 0.599062] STM32 USART driver initialized1379worker1 # [ 0.656454] megasas: 07.734.00.00-rc11380worker1 # [ 0.657273] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1381client # [ 0.623742] loop: module loaded1382client # [ 0.623961] virtio_blk virtio5: 1/0/0 default/read/poll queues1383worker1 # [ 0.659740] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001384worker1 # [ 0.659765] Intel/Sharp Extended Query Table at 0x00311385worker1 # [ 0.661567] Using buffer write method1386worker1 # [ 0.661647] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1387worker1 # [ 0.663541] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001388worker1 # [ 0.663582] Intel/Sharp Extended Query Table at 0x00311389client # [ 0.632877] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1390worker1 # [ 0.681531] Using buffer write method1391client # [ 0.636109] megasas: 07.734.00.00-rc11392worker1 # [ 0.681606] Concatenating MTD devices:1393worker1 # [ 0.681611] (0): "0.flash"1394client # [ 0.636858] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1395worker1 # [ 0.681615] (1): "0.flash"1396worker1 # [ 0.681618] into device "0.flash"1397client # [ 0.642857] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001398client # [ 0.642886] Intel/Sharp Extended Query Table at 0x00311399client # [ 0.644632] Using buffer write method1400lb # [ 0.635942] loop: module loaded1401lb # [ 0.636134] virtio_blk virtio5: 1/0/0 default/read/poll queues1402lb # [ 0.637016] virtio_blk virtio5: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1403client # [ 0.652760] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1404client # [ 0.655819] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001405client # [ 0.655853] Intel/Sharp Extended Query Table at 0x00311406client # [ 0.657581] Using buffer write method1407client # [ 0.657609] Concatenating MTD devices:1408client # [ 0.657613] (0): "0.flash"1409client # [ 0.657617] (1): "0.flash"1410client # [ 0.657620] into device "0.flash"1411lb # [ 0.647359] megasas: 07.734.00.00-rc11412lb # [ 0.648175] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1413lb # [ 0.650340] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001414lb # [ 0.650405] Intel/Sharp Extended Query Table at 0x00311415lb # [ 0.656013] Using buffer write method1416lb # [ 0.656097] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1417lb # [ 0.658039] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001418lb # [ 0.658073] Intel/Sharp Extended Query Table at 0x00311419lb # [ 0.676031] Using buffer write method1420lb # [ 0.676079] Concatenating MTD devices:1421lb # [ 0.676084] (0): "0.flash"1422lb # [ 0.676088] (1): "0.flash"1423lb # [ 0.676091] into device "0.flash"1424worker2 # [ 1.009763] Freeing initrd memory: 26148K1425worker1 # [ 0.967185] Freeing initrd memory: 26148K1426worker2 # [ 1.017411] tun: Universal TUN/TAP device driver, 1.61427worker1 # [ 0.973621] tun: Universal TUN/TAP device driver, 1.61428worker2 # [ 1.021730] thunder_xcv, ver 1.01429worker2 # [ 1.021873] thunder_bgx, ver 1.01430worker2 # [ 1.021895] nicpf, ver 1.01431worker1 # [ 0.977632] thunder_xcv, ver 1.01432worker1 # [ 0.977684] thunder_bgx, ver 1.01433worker2 # [ 1.023910] e1000: Intel(R) PRO/1000 Network Driver1434worker1 # [ 0.977708] nicpf, ver 1.01435worker2 # [ 1.023926] e1000: Copyright (c) 1999-2006 Intel Corporation.1436worker1 # [ 0.978316] e1000: Intel(R) PRO/1000 Network Driver1437worker2 # [ 1.023953] e1000e: Intel(R) PRO/1000 Network Driver1438worker1 # [ 0.978329] e1000: Copyright (c) 1999-2006 Intel Corporation.1439worker2 # [ 1.023960] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1440worker1 # [ 0.978354] e1000e: Intel(R) PRO/1000 Network Driver1441worker2 # [ 1.023990] igb: Intel(R) Gigabit Ethernet Network Driver1442worker1 # [ 0.978363] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1443worker1 # [ 0.978393] igb: Intel(R) Gigabit Ethernet Network Driver1444worker2 # [ 1.023999] igb: Copyright (c) 2007-2014 Intel Corporation.1445worker1 # [ 0.978399] igb: Copyright (c) 2007-2014 Intel Corporation.1446worker2 # [ 1.024022] igbvf: Intel(R) Gigabit Virtual Function Network Driver1447worker1 # [ 0.978424] igbvf: Intel(R) Gigabit Virtual Function Network Driver1448worker2 # [ 1.024029] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1449worker2 # [ 1.024187] sky2: driver version 1.301450worker1 # [ 0.978431] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1451worker1 # [ 0.978567] sky2: driver version 1.301452worker2 # [ 1.026856] ehci-pci 0000:00:0a.0: EHCI Host Controller1453worker2 # [ 1.026894] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11454worker2 # [ 1.027076] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001455worker1 # [ 0.987590] usbcore: registered new interface driver usb-storage1456worker1 # [ 0.988544] ehci-pci 0000:00:0a.0: EHCI Host Controller1457worker1 # [ 0.988574] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11458worker2 # [ 1.035908] usbcore: registered new interface driver usb-storage1459worker1 # [ 0.988825] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001460worker2 # [ 1.036038] usbcore: registered new interface driver usbserial_generic1461client # [ 0.949402] Freeing initrd memory: 26160K1462worker2 # [ 1.036061] usbserial: USB Serial support registered for generic1463worker2 # [ 1.036859] hv_vmbus: registering driver hyperv_keyboard1464worker1 # [ 0.991995] usbcore: registered new interface driver usbserial_generic1465worker1 # [ 0.992014] usbserial: USB Serial support registered for generic1466worker2 # [ 1.040781] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001467worker1 # [ 0.992725] hv_vmbus: registering driver hyperv_keyboard1468worker2 # [ 1.041217] hub 1-0:1.0: USB hub found1469worker2 # [ 1.041250] hub 1-0:1.0: 6 ports detected1470client # [ 0.955974] tun: Universal TUN/TAP device driver, 1.61471worker1 # [ 0.995983] rtc-pl031 9010000.pl031: registered as rtc01472worker2 # [ 1.043930] rtc-pl031 9010000.pl031: registered as rtc01473worker1 # [ 0.996014] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:56:33 UTC (1790074593)1474worker1 # [ 0.996352] i2c_dev: i2c /dev entries driver1475worker2 # [ 1.043978] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:56:33 UTC (1790074593)1476worker2 # [ 1.044360] i2c_dev: i2c /dev entries driver1477client # [ 0.959929] thunder_xcv, ver 1.01478client # [ 0.959974] thunder_bgx, ver 1.01479client # [ 0.959996] nicpf, ver 1.01480client # [ 0.960564] e1000: Intel(R) PRO/1000 Network Driver1481worker2 # [ 1.049679] sdhci: Secure Digital Host Controller Interface driver1482client # [ 0.960571] e1000: Copyright (c) 1999-2006 Intel Corporation.1483worker2 # [ 1.049695] sdhci: Copyright(c) Pierre Ossman1484worker1 # [ 1.001487] sdhci: Secure Digital Host Controller Interface driver1485worker1 # [ 1.001497] sdhci: Copyright(c) Pierre Ossman1486worker2 # [ 1.049966] Synopsys Designware Multimedia Card Interface Driver1487client # [ 0.960595] e1000e: Intel(R) PRO/1000 Network Driver1488client # [ 0.960603] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1489worker1 # [ 1.001765] Synopsys Designware Multimedia Card Interface Driver1490worker1 # [ 1.002171] sdhci-pltfm: SDHCI platform and OF driver helper1491client # [ 0.960634] igb: Intel(R) Gigabit Ethernet Network Driver1492client # [ 0.960640] igb: Copyright (c) 2007-2014 Intel Corporation.1493worker1 # [ 1.006396] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001494worker2 # [ 1.052553] sdhci-pltfm: SDHCI platform and OF driver helper1495client # [ 0.960663] igbvf: Intel(R) Gigabit Virtual Function Network Driver1496worker1 # [ 1.007845] hub 1-0:1.0: USB hub found1497client # [ 0.960669] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1498worker1 # [ 1.008381] hub 1-0:1.0: 6 ports detected1499client # [ 0.967818] sky2: driver version 1.301500worker2 # [ 1.055410] hid: raw HID events driver (C) Jiri Kosina1501worker2 # [ 1.055732] usbcore: registered new interface driver usbhid1502worker1 # [ 1.009592] hid: raw HID events driver (C) Jiri Kosina1503worker2 # [ 1.055751] usbhid: USB HID core driver1504worker1 # [ 1.009838] usbcore: registered new interface driver usbhid1505worker1 # [ 1.009845] usbhid: USB HID core driver1506client # [ 0.970066] usbcore: registered new interface driver usb-storage1507worker2 # [ 1.058894] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1508client # [ 0.970125] usbcore: registered new interface driver usbserial_generic1509worker1 # [ 1.012831] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1510worker2 # [ 1.060698] drop_monitor: Initializing network drop monitor service1511client # [ 0.970140] usbserial: USB Serial support registered for generic1512worker2 # [ 1.060868] NET: Registered PF_INET6 protocol family1513worker1 # [ 1.014402] drop_monitor: Initializing network drop monitor service1514client # [ 0.970746] hv_vmbus: registering driver hyperv_keyboard1515worker1 # [ 1.014542] NET: Registered PF_INET6 protocol family1516client # [ 0.972194] ehci-pci 0000:00:0a.0: EHCI Host Controller1517worker2 # [ 1.064398] Segment Routing with IPv61518worker1 # [ 1.017698] Segment Routing with IPv61519worker2 # [ 1.064425] In-situ OAM (IOAM) with IPv61520client # [ 0.972223] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11521worker1 # [ 1.017721] In-situ OAM (IOAM) with IPv61522worker2 # [ 1.064461] NET: Registered PF_PACKET protocol family1523client # [ 0.972658] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001524worker1 # [ 1.017759] NET: Registered PF_PACKET protocol family1525worker1 # [ 1.019407] 9pnet: Installing 9P2000 support1526client # [ 0.978412] rtc-pl031 9010000.pl031: registered as rtc01527lb # [ 0.965765] Freeing initrd memory: 26140K1528client # [ 0.978447] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:56:33 UTC (1790074593)1529client # [ 0.978826] i2c_dev: i2c /dev entries driver1530worker2 # [ 1.066228] 9pnet: Installing 9P2000 support1531worker2 # [ 1.068691] Key type dns_resolver registered1532worker1 # [ 1.022368] Key type dns_resolver registered1533client # [ 0.984156] sdhci: Secure Digital Host Controller Interface driver1534client # [ 0.984172] sdhci: Copyright(c) Pierre Ossman1535client # [ 0.984448] Synopsys Designware Multimedia Card Interface Driver1536client # [ 0.986826] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001537client # [ 0.987931] hub 1-0:1.0: USB hub found1538lb # [ 0.973130] tun: Universal TUN/TAP device driver, 1.61539client # [ 0.988447] hub 1-0:1.0: 6 ports detected1540worker2 # [ 1.077133] registered taskstats version 11541worker2 # [ 1.077412] Loading compiled-in X.509 certificates1542worker1 # [ 1.028951] registered taskstats version 11543worker1 # [ 1.029145] Loading compiled-in X.509 certificates1544client # [ 0.989425] sdhci-pltfm: SDHCI platform and OF driver helper1545lb # [ 0.977285] thunder_xcv, ver 1.01546client # [ 0.991492] hid: raw HID events driver (C) Jiri Kosina1547lb # [ 0.977334] thunder_bgx, ver 1.01548lb # [ 0.977360] nicpf, ver 1.01549client # [ 0.991787] usbcore: registered new interface driver usbhid1550client # [ 0.991797] usbhid: USB HID core driver1551lb # [ 0.978003] e1000: Intel(R) PRO/1000 Network Driver1552lb # [ 0.978014] e1000: Copyright (c) 1999-2006 Intel Corporation.1553lb # [ 0.978038] e1000e: Intel(R) PRO/1000 Network Driver1554lb # [ 0.978053] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1555client # [ 0.995853] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1556lb # [ 0.978096] igb: Intel(R) Gigabit Ethernet Network Driver1557lb # [ 0.978106] igb: Copyright (c) 2007-2014 Intel Corporation.1558worker1 # [ 1.038938] Demotion targets for Node 0: null1559worker2 # [ 1.086515] Demotion targets for Node 0: null1560worker1 # [ 1.039066] Key type .fscrypt registered1561lb # [ 0.978131] igbvf: Intel(R) Gigabit Virtual Function Network Driver1562client # [ 0.998632] drop_monitor: Initializing network drop monitor service1563worker2 # [ 1.086647] Key type .fscrypt registered1564worker1 # [ 1.039075] Key type fscrypt-provisioning registered1565lb # [ 0.978137] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1566client # [ 0.998836] NET: Registered PF_INET6 protocol family1567worker2 # [ 1.086653] Key type fscrypt-provisioning registered1568lb # [ 0.978278] sky2: driver version 1.301569worker1 # [ 1.039178] ima: No TPM chip found, activating TPM-bypass!1570worker2 # [ 1.086761] ima: No TPM chip found, activating TPM-bypass!1571client # [ 1.000797] Segment Routing with IPv61572worker1 # [ 1.039200] ima: Allocated hash algorithm: sha11573worker2 # [ 1.086788] ima: Allocated hash algorithm: sha11574client # [ 1.000817] In-situ OAM (IOAM) with IPv61575worker1 # [ 1.039224] ima: No architecture policies found1576worker2 # [ 1.086813] ima: No architecture policies found1577client # [ 1.000847] NET: Registered PF_PACKET protocol family1578worker1 # [ 1.043498] input: gpio-keys as /devices/platform/gpio-keys/input/input01579worker2 # [ 1.091391] input: gpio-keys as /devices/platform/gpio-keys/input/input01580client # [ 1.002526] 9pnet: Installing 9P2000 support1581lb # [ 0.987530] usbcore: registered new interface driver usb-storage1582lb # [ 0.987591] usbcore: registered new interface driver usbserial_generic1583lb # [ 0.987606] usbserial: USB Serial support registered for generic1584lb # [ 0.988272] hv_vmbus: registering driver hyperv_keyboard1585lb # [ 0.989801] ehci-pci 0000:00:0a.0: EHCI Host Controller1586client # [ 1.005382] Key type dns_resolver registered1587lb # [ 0.989829] ehci-pci 0000:00:0a.0: new USB bus registered, assigned bus number 11588lb # [ 0.990118] ehci-pci 0000:00:0a.0: irq 15, io mem 0x100890001589lb # [ 0.995925] rtc-pl031 9010000.pl031: registered as rtc01590client # [ 1.011872] registered taskstats version 11591lb # [ 0.995961] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T10:56:33 UTC (1790074593)1592client # [ 1.012041] Loading compiled-in X.509 certificates1593lb # [ 0.996367] i2c_dev: i2c /dev entries driver1594lb # [ 0.999602] ehci-pci 0000:00:0a.0: USB 2.0 started, EHCI 1.001595lb # [ 1.000004] hub 1-0:1.0: USB hub found1596lb # [ 1.000034] hub 1-0:1.0: 6 ports detected1597client # [ 1.021099] Demotion targets for Node 0: null1598lb # [ 1.004458] sdhci: Secure Digital Host Controller Interface driver1599client # [ 1.021215] Key type .fscrypt registered1600lb # [ 1.004475] sdhci: Copyright(c) Pierre Ossman1601worker1 # [ 1.063259] clk: Disabling unused clocks1602client # [ 1.021221] Key type fscrypt-provisioning registered1603lb # [ 1.004786] Synopsys Designware Multimedia Card Interface Driver1604worker1 # [ 1.063296] PM: genpd: Disabling unused power domains1605client # [ 1.021323] ima: No TPM chip found, activating TPM-bypass!1606lb # [ 1.005180] sdhci-pltfm: SDHCI platform and OF driver helper1607client # [ 1.021343] ima: Allocated hash algorithm: sha11608client # [ 1.021366] ima: No architecture policies found1609lb # [ 1.009745] hid: raw HID events driver (C) Jiri Kosina1610client # [ 1.025595] input: gpio-keys as /devices/platform/gpio-keys/input/input01611worker2 # [ 1.112718] clk: Disabling unused clocks1612lb # [ 1.010048] usbcore: registered new interface driver usbhid1613worker2 # [ 1.112773] PM: genpd: Disabling unused power domains1614lb # [ 1.010058] usbhid: USB HID core driver1615worker1 # [ 1.067729] Freeing unused kernel memory: 4736K1616worker1 # [ 1.067942] Run /init as init process1617lb # [ 1.013080] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1618worker2 # [ 1.118519] Freeing unused kernel memory: 4736K1619lb # [ 1.015809] drop_monitor: Initializing network drop monitor service1620worker2 # [ 1.118856] Run /init as init process1621lb # [ 1.015977] NET: Registered PF_INET6 protocol family1622lb # [ 1.018092] Segment Routing with IPv61623lb # [ 1.018115] In-situ OAM (IOAM) with IPv61624lb # [ 1.018143] NET: Registered PF_PACKET protocol family1625lb # [ 1.019956] 9pnet: Installing 9P2000 support1626lb # [ 1.022330] Key type dns_resolver registered1627client # [ 1.044474] clk: Disabling unused clocks1628client # [ 1.044513] PM: genpd: Disabling unused power domains1629lb # [ 1.029973] registered taskstats version 11630worker1 # [ 1.086272] systemd[1]: Successfully made /usr/ read-only.1631lb # [ 1.030134] Loading compiled-in X.509 certificates1632worker2 # [ 1.136736] systemd[1]: Successfully made /usr/ read-only.1633client # [ 1.048837] Freeing unused kernel memory: 4736K1634client # [ 1.049064] Run /init as init process1635lb # [ 1.039044] Demotion targets for Node 0: null1636lb # [ 1.039169] Key type .fscrypt registered1637lb # [ 1.039177] Key type fscrypt-provisioning registered1638lb # [ 1.039283] ima: No TPM chip found, activating TPM-bypass!1639lb # [ 1.039303] ima: Allocated hash algorithm: sha11640lb # [ 1.039327] ima: No architecture policies found1641lb # [ 1.043955] input: gpio-keys as /devices/platform/gpio-keys/input/input01642client # [ 1.067363] systemd[1]: Successfully made /usr/ read-only.1643lb # [ 1.063079] clk: Disabling unused clocks1644lb # [ 1.063116] PM: genpd: Disabling unused power domains1645lb # [ 1.067540] Freeing unused kernel memory: 4736K1646lb # [ 1.067766] Run /init as init process1647lb # [ 1.085049] systemd[1]: Successfully made /usr/ read-only.1648worker2 # [ 1.286208] usb 1-1: new high-speed USB device number 2 using ehci-pci1649worker1 # [ 1.255749] usb 1-1: new high-speed USB device number 2 using ehci-pci1650client # [ 1.240803] usb 1-1: new high-speed USB device number 2 using ehci-pci1651lb # [ 1.246763] usb 1-1: new high-speed USB device number 2 using ehci-pci1652worker2 # [ 1.445188] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11653worker1 # [ 1.408507] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11654worker2 # [ 1.471630] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1655worker2 # [ 1.475900] systemd[1]: Detected virtualization qemu.1656worker2 # [ 1.476646] systemd[1]: Detected architecture arm64.1657worker2 # [ 1.477330] systemd[1]: Running in initrd.1658worker2 # [ 1.478425] systemd[1]: Initializing machine ID from random generator.1659worker2 # [ 1.479489] systemd[1]: Hostname set to <worker2>.1660worker1 # [ 1.422125] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1661worker1 # [ 1.435240] systemd[1]: Detected virtualization qemu.1662worker1 # [ 1.437535] systemd[1]: Detected architecture arm64.1663worker1 # [ 1.439796] systemd[1]: Running in initrd.1664client # [ 1.395175] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11665worker1 # [ 1.443178] systemd[1]: Initializing machine ID from random generator.1666worker1 # [ 1.446482] systemd[1]: Hostname set to <worker1>.1667client # [ 1.402439] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1668client # [ 1.415862] systemd[1]: Detected virtualization qemu.1669client # [ 1.418204] systemd[1]: Detected architecture arm64.1670client # [ 1.420275] systemd[1]: Running in initrd.1671lb # [ 1.401713] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11672client # [ 1.423186] systemd[1]: Initializing machine ID from random generator.1673client # [ 1.426297] systemd[1]: Hostname set to <client>.1674lb # [ 1.420494] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1675lb # [ 1.433272] systemd[1]: Detected virtualization qemu.1676lb # [ 1.435478] systemd[1]: Detected architecture arm64.1677worker2 # [ 1.538449] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01678lb # [ 1.437532] systemd[1]: Running in initrd.1679lb # [ 1.440418] systemd[1]: Initializing machine ID from random generator.1680lb # [ 1.443658] systemd[1]: Hostname set to <lb>.1681worker1 # [ 1.499985] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01682client # [ 1.485017] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01683lb # [ 1.495005] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:0a.0-1/input01684worker2 # [ 1.662160] usb 1-2: new high-speed USB device number 3 using ehci-pci1685worker1 # [ 1.623717] usb 1-2: new high-speed USB device number 3 using ehci-pci1686client # [ 1.608782] usb 1-2: new high-speed USB device number 3 using ehci-pci1687lb # [ 1.618781] usb 1-2: new high-speed USB device number 3 using ehci-pci1688worker2 # [ 1.812991] systemd[1]: bpf-restrict-fs: LSM BPF program attached1689worker1 # [ 1.779028] systemd[1]: bpf-restrict-fs: LSM BPF program attached1690worker2 # [ 1.832077] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21691worker1 # [ 1.787246] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21692worker2 # [ 1.837199] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01693worker1 # [ 1.787560] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01694client # [ 1.759394] systemd[1]: bpf-restrict-fs: LSM BPF program attached1695client # [ 1.772415] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21696client # [ 1.778179] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01697lb # [ 1.783338] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:0a.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21698lb # [ 1.789014] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:0a.0-2/input01699lb # [ 1.795605] systemd[1]: bpf-restrict-fs: LSM BPF program attached1700worker2 # [ 1.929395] systemd[1]: Queued start job for default target Initrd Default Target.1701worker1 # [ 1.886581] systemd[1]: Queued start job for default target Initrd Default Target.1702worker2 # [ 1.940675] systemd[1]: Created slice Slice /system/modprobe.1703worker1 # [ 1.893854] systemd[1]: Created slice Slice /system/modprobe.1704worker2 # [ 1.941919] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1705worker1 # [ 1.894918] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1706worker1 # [ 1.896108] systemd[1]: Expecting device /dev/disk/by-label/nixos...1707worker2 # [ 1.943314] systemd[1]: Expecting device /dev/disk/by-label/nixos...1708worker1 # [ 1.897058] systemd[1]: Reached target Path Units.1709worker2 # [ 1.944349] systemd[1]: Reached target Path Units.1710worker1 # [ 1.897750] systemd[1]: Reached target Slice Units.1711worker2 # [ 1.945122] systemd[1]: Reached target Slice Units.1712worker1 # [ 1.898450] systemd[1]: Reached target Swaps.1713worker2 # [ 1.945914] systemd[1]: Reached target Swaps.1714worker1 # [ 1.899104] systemd[1]: Reached target Timer Units.1715worker2 # [ 1.946686] systemd[1]: Reached target Timer Units.1716worker1 # [ 1.900041] systemd[1]: Listening on D-Bus System Message Bus Socket.1717worker2 # [ 1.947708] systemd[1]: Listening on D-Bus System Message Bus Socket.1718worker1 # [ 1.901114] systemd[1]: Listening on Journal Socket (/dev/log).1719worker2 # [ 1.948901] systemd[1]: Listening on Journal Socket (/dev/log).1720worker1 # [ 1.902103] systemd[1]: Listening on Journal Sockets.1721worker2 # [ 1.949967] systemd[1]: Listening on Journal Sockets.1722worker1 # [ 1.902956] systemd[1]: Listening on udev Control Socket.1723worker1 # [ 1.903889] systemd[1]: Listening on udev Kernel Socket.1724worker2 # [ 1.951078] systemd[1]: Listening on udev Control Socket.1725worker1 # [ 1.903915] systemd[1]: Reached target Socket Units.1726worker2 # [ 1.951203] systemd[1]: Listening on udev Kernel Socket.1727worker2 # [ 1.951226] systemd[1]: Reached target Socket Units.1728worker1 # [ 1.906965] systemd[1]: Starting Create List of Static Device Nodes...1729worker2 # [ 1.955567] systemd[1]: Starting Create List of Static Device Nodes...1730worker1 # [ 1.915849] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1731client # [ 1.875265] systemd[1]: Queued start job for default target Initrd Default Target.1732worker1 # [ 1.916889] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1733worker2 # [ 1.966354] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1734worker2 # [ 1.967454] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1735worker1 # [ 1.923942] systemd[1]: Mounting Kernel Configuration File System...1736client # [ 1.887332] systemd[1]: Created slice Slice /system/modprobe.1737client # [ 1.888555] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1738client # [ 1.889906] systemd[1]: Expecting device /dev/disk/by-label/nixos...1739client # [ 1.890919] systemd[1]: Reached target Path Units.1740client # [ 1.891811] systemd[1]: Reached target Slice Units.1741client # [ 1.892617] systemd[1]: Reached target Swaps.1742worker2 # [ 1.979502] systemd[1]: Mounting Kernel Configuration File System...1743client # [ 1.893408] systemd[1]: Reached target Timer Units.1744client # [ 1.894422] systemd[1]: Listening on D-Bus System Message Bus Socket.1745client # [ 1.895629] systemd[1]: Listening on Journal Socket (/dev/log).1746client # [ 1.896755] systemd[1]: Listening on Journal Sockets.1747client # [ 1.897684] systemd[1]: Listening on udev Control Socket.1748client # [ 1.898656] systemd[1]: Listening on udev Kernel Socket.1749client # [ 1.899548] systemd[1]: Reached target Socket Units.1750lb # [ 1.886143] systemd[1]: Queued start job for default target Initrd Default Target.1751client # [ 1.902360] systemd[1]: Starting Create List of Static Device Nodes...1752client # [ 1.909651] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1753worker2 # [ 1.998413] systemd[1]: Starting Journal Service...1754client # [ 1.910813] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1755lb # [ 1.897108] systemd[1]: Created slice Slice /system/modprobe.1756lb # [ 1.898270] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1757lb # [ 1.899557] systemd[1]: Expecting device /dev/disk/by-label/nixos...1758worker1 # [ 1.955995] systemd[1]: Starting Journal Service...1759lb # [ 1.900569] systemd[1]: Reached target Path Units.1760lb # [ 1.901332] systemd[1]: Reached target Slice Units.1761worker2 # [ 2.004621] systemd[1]: Starting Load Kernel Modules...1762lb # [ 1.902108] systemd[1]: Reached target Swaps.1763lb # [ 1.902846] systemd[1]: Reached target Timer Units.1764worker2 # [ 2.005550] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1765lb # [ 1.903884] systemd[1]: Listening on D-Bus System Message Bus Socket.1766lb # [ 1.905073] systemd[1]: Listening on Journal Socket (/dev/log).1767lb # [ 1.906146] systemd[1]: Listening on Journal Sockets.1768worker1 # [ 1.962203] systemd[1]: Starting Load Kernel Modules...1769client # [ 1.922144] systemd[1]: Mounting Kernel Configuration File System...1770lb # [ 1.907154] systemd[1]: Listening on udev Control Socket.1771worker1 # [ 1.962359] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1772lb # [ 1.907287] systemd[1]: Listening on udev Kernel Socket.1773lb # [ 1.907311] systemd[1]: Reached target Socket Units.1774lb # [ 1.911432] systemd[1]: Starting Create List of Static Device Nodes...1775worker1 # [ 1.967791] systemd[1]: Starting Coldplug All udev Devices...1776lb # [ 1.922127] systemd[1]: Starting Load Kernel Module 9pnet_virtio...1777lb # [ 1.923465] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1778worker2 # [ 2.026690] systemd[1]: Starting Coldplug All udev Devices...1779worker1 # [ 1.991936] systemd[1]: Finished Create List of Static Device Nodes.1780worker1 # [ 1.992999] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1781client # [ 1.953071] systemd[1]: Starting Journal Service...1782worker1 # [ 1.993308] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1783worker1 # [ 1.993552] systemd[1]: Mounted Kernel Configuration File System.1784lb # [ 1.939187] systemd[1]: Mounting Kernel Configuration File System...1785client # [ 1.964980] systemd[1]: Starting Load Kernel Modules...1786client # [ 1.965927] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1787client # [ 1.973106] systemd[1]: Starting Coldplug All udev Devices...1788lb # [ 1.958937] systemd[1]: Starting Journal Service...1789worker2 # [ 2.060908] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1790worker2 # [ 2.063046] systemd[1]: Finished Create List of Static Device Nodes.1791worker2 # [ 2.064510] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1792worker1 # [ 2.017674] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1793worker2 # [ 2.064824] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1794lb # [ 1.966007] systemd[1]: Starting Load Kernel Modules...1795lb # [ 1.967098] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1796worker2 # [ 2.070569] systemd-journald[73]: Collecting audit messages is disabled.1797worker1 # [ 2.023869] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1798worker1 # [ 2.027016] systemd-journald[73]: Collecting audit messages is disabled.1799worker2 # [ 2.078564] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1800worker2 # [ 2.078872] systemd[1]: Mounted Kernel Configuration File System.1801worker2 # [ 2.082331] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1802client # [ 1.993432] systemd[1]: Finished Create List of Static Device Nodes.1803client # [ 1.994476] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1804client # [ 1.994816] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1805client # [ 1.995088] systemd[1]: Mounted Kernel Configuration File System.1806worker1 # [ 2.040224] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1807worker1 # [ 2.045075] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01808worker1 # [ 2.045322] [drm] features: -virgl +edid -resource_blob -host_visible1809worker2 # [ 2.094614] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01810worker1 # [ 2.045332] [drm] features: -context_init1811worker1 # [ 2.046104] [drm] number of scanouts: 11812worker2 # [ 2.094866] [drm] features: -virgl +edid -resource_blob -host_visible1813worker1 # [ 2.046122] [drm] number of cap sets: 01814worker2 # [ 2.094877] [drm] features: -context_init1815worker2 # [ 2.095676] [drm] number of scanouts: 11816client # [ 2.009360] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1817worker2 # [ 2.095695] [drm] number of cap sets: 01818lb # [ 1.999647] systemd[1]: Starting Coldplug All udev Devices...1819lb # [ 2.002289] systemd[1]: Finished Create List of Static Device Nodes.1820lb # [ 2.004071] systemd[1]: modprobe@9pnet_virtio.service: Deactivated successfully.1821worker2 # [ 2.110573] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1822worker2 # [ 2.110598] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01823client # [ 2.028322] systemd-journald[73]: Collecting audit messages is disabled.1824worker1 # [ 2.072442] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1825lb # [ 2.015126] systemd[1]: Finished Load Kernel Module 9pnet_virtio.1826worker1 # [ 2.075169] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1827worker1 # [ 2.075197] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01828worker1 # [ 2.081913] systemd[1]: Starting Create Static Device Nodes in /dev...1829lb # [ 2.031052] systemd-journald[73]: Collecting audit messages is disabled.1830client # [ 2.053033] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1831lb # [ 2.042897] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1832lb # [ 2.044614] systemd[1]: Mounted Kernel Configuration File System.1833client # [ 2.060791] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1834worker2 # [ 2.146458] Console: switching to colour frame buffer device 160x501835worker2 # [ 2.153490] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1836client # [ 2.069298] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01837client # [ 2.069545] [drm] features: -virgl +edid -resource_blob -host_visible1838client # [ 2.069555] [drm] features: -context_init1839client # [ 2.070321] [drm] number of scanouts: 11840client # [ 2.070339] [drm] number of cap sets: 01841worker1 # [ 2.108066] Console: switching to colour frame buffer device 160x501842lb # [ 2.059454] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1843worker1 # [ 2.114809] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1844client # [ 2.074526] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1845worker2 # [ 2.166979] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1846worker2 # [ 2.174518] systemd[1]: Starting Create Static Device Nodes in /dev...1847client # [ 2.085263] systemd[1]: Starting Create Static Device Nodes in /dev...1848lb # [ 2.078720] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1849client # [ 2.097905] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1850worker1 # [ 2.140344] systemd[1]: Finished Create Static Device Nodes in /dev.1851client # [ 2.097936] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01852worker1 # [ 2.140765] systemd[1]: Reached target Preparation for Local File Systems.1853lb # [ 2.086804] [drm] pci: virtio-gpu-pci detected at 0000:00:08.01854worker1 # [ 2.140806] systemd[1]: Reached target Local File Systems.1855lb # [ 2.087052] [drm] features: -virgl +edid -resource_blob -host_visible1856lb # [ 2.087063] [drm] features: -context_init1857lb # [ 2.087793] [drm] number of scanouts: 11858lb # [ 2.087812] [drm] number of cap sets: 01859worker1 # [ 2.145057] systemd[1]: Starting Rule-based Manager for Device Events and Files...1860worker2 # [ 2.198393] systemd[1]: Finished Load Kernel Modules.1861worker2 # [ 2.201994] systemd[1]: Starting Apply Kernel Variables...1862worker1 # [ 2.159890] systemd[1]: Finished Load Kernel Modules.1863lb # [ 2.104638] virtio-pci 0000:00:08.0: [drm] Registered 1 planes with drm panic1864lb # [ 2.104672] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:08.0 on minor 01865lb # [ 2.111411] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1866client # [ 2.125116] Console: switching to colour frame buffer device 160x501867lb # [ 2.119011] systemd[1]: Starting Create Static Device Nodes in /dev...1868worker1 # [ 2.172114] systemd[1]: Starting Apply Kernel Variables...1869worker2 # [ 2.234308] systemd[1]: Finished Create Static Device Nodes in /dev.1870worker2 # [ 2.234590] systemd[1]: Reached target Preparation for Local File Systems.1871worker2 # [ 2.234631] systemd[1]: Reached target Local File Systems.1872worker2 # [ 2.240121] systemd[1]: Starting Rule-based Manager for Device Events and Files...1873client # [ 2.153437] systemd[1]: Finished Create Static Device Nodes in /dev.1874client # [ 2.153834] systemd[1]: Reached target Preparation for Local File Systems.1875client # [ 2.153858] systemd[1]: Reached target Local File Systems.1876client # [ 2.158240] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1877lb # [ 2.138937] Console: switching to colour frame buffer device 160x501878client # [ 2.160675] systemd[1]: Starting Rule-based Manager for Device Events and Files...1879worker1 # [ 2.185481] systemd-modules-load[74]: Inserted module 'dm_mod'1880worker1 # [ 2.207949] systemd[1]: Started Journal Service.1881worker1 # [ 2.196233] systemd-modules-load[74]: Module 'virtio_balloon' is built in1882worker2 # [ 2.268202] systemd[1]: Finished Apply Kernel Variables.1883client # [ 2.180967] systemd[1]: Finished Load Kernel Modules.1884worker1 # [ 2.205630] systemd-modules-load[74]: Module 'virtio_console' is built in1885lb # [ 2.171389] virtio-pci 0000:00:08.0: [drm] fb0: virtio_gpudrmfb frame buffer device1886worker2 # [ 2.252512] systemd-modules-load[74]: Inserted module 'dm_mod'1887worker1 # [ 2.208314] systemd-modules-load[74]: Inserted module 'virtio_gpu'1888lb # [ 2.173244] systemd[1]: Finished Create Static Device Nodes in /dev.1889worker2 # [ 2.276104] systemd[1]: Started Journal Service.1890lb # [ 2.176479] systemd[1]: Reached target Preparation for Local File Systems.1891worker2 # [ 2.257499] systemd-modules-load[74]: Module 'virtio_balloon' is built in1892lb # [ 2.177439] systemd[1]: Reached target Local File Systems.1893worker2 # [ 2.259645] systemd-modules-load[74]: Module 'virtio_console' is built in1894client # [ 2.189121] systemd[1]: Starting Apply Kernel Variables...1895worker1 # [ 2.216394] systemd-modules-load[74]: Module 'virtio_rng' is built in1896worker2 # [ 2.265997] systemd-modules-load[74]: Inserted module 'virtio_gpu'1897worker2 # [ 2.267041] systemd-modules-load[74]: Module 'virtio_rng' is built in1898worker1 # [ 2.224325] systemd[1]: Starting Create System Files and Directories...1899lb # [ 2.187607] systemd[1]: Starting Rule-based Manager for Device Events and Files...1900worker1 # [ 2.227879] systemd[1]: Finished Apply Kernel Variables.1901worker2 # [ 2.276249] systemd[1]: Starting Create System Files and Directories...1902lb # [ 2.203289] systemd[1]: Finished Load Kernel Modules.1903client # [ 2.207156] systemd-modules-load[74]: Inserted module 'dm_mod'1904client # [ 2.229021] systemd[1]: Started Journal Service.1905lb # [ 2.215200] systemd[1]: Starting Apply Kernel Variables...1906lb # [ 2.204295] systemd-modules-load[74]: Inserted module 'dm_mod'1907lb # [ 2.205403] systemd-modules-load[74]: Module 'virtio_balloon' is built in1908worker1 # [ 2.261668] systemd[1]: Finished Create System Files and Directories.1909lb # [ 2.225100] systemd[1]: Started Journal Service.1910client # [ 2.225525] systemd-modules-load[74]: Module 'virtio_balloon' is built in1911client # [ 2.226720] systemd-modules-load[74]: Module 'virtio_console' is built in1912client # [ 2.227790] systemd-modules-load[74]: Inserted module 'virtio_gpu'1913lb # [ 2.212386] systemd-modules-load[74]: Module 'virtio_console' is built in1914worker1 # [ 2.269281] systemd-udevd[79]: Using default interface naming scheme 'v261'.1915lb # [ 2.216362] systemd-modules-load[74]: Inserted module 'virtio_gpu'1916lb # [ 2.220329] systemd-modules-load[74]: Module 'virtio_rng' is built in1917client # [ 2.237721] systemd-modules-load[74]: Module 'virtio_rng' is built in1918worker2 # [ 2.321551] systemd-udevd[81]: Using default interface naming scheme 'v261'.1919lb # [ 2.222228] systemd[1]: Starting Create System Files and Directories...1920client # [ 2.244353] systemd[1]: Starting Create System Files and Directories...1921client # [ 2.250745] systemd[1]: Finished Apply Kernel Variables.1922worker2 # [ 2.334988] systemd[1]: Finished Create System Files and Directories.1923worker1 # [ 2.300747] systemd[1]: Started Rule-based Manager for Device Events and Files.1924worker2 # [ 2.361973] systemd[1]: Started Rule-based Manager for Device Events and Files.1925client # [ 2.281681] systemd[1]: Finished Create System Files and Directories.1926client # [ 2.288462] systemd-udevd[79]: Using default interface naming scheme 'v261'.1927lb # [ 2.281321] systemd[1]: Finished Apply Kernel Variables.1928lb # [ 2.299941] systemd-udevd[80]: Using default interface naming scheme 'v261'.1929lb # [ 2.304342] systemd[1]: Finished Create System Files and Directories.1930client # [ 2.322258] systemd[1]: Started Rule-based Manager for Device Events and Files.1931worker1 # [ 2.368138] systemd[1]: Starting Virtual Console Setup...1932worker2 # [ 2.424127] systemd[1]: Starting Virtual Console Setup...1933lb # [ 2.338822] systemd[1]: Started Rule-based Manager for Device Events and Files.1934client # [ 2.384684] systemd[1]: Starting Virtual Console Setup...1935worker1 # [ 2.424643] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1936worker1 # [ 2.428991] systemd[1]: Finished Virtual Console Setup.1937worker2 # [ 2.488735] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1938worker2 # [ 2.493069] systemd[1]: Finished Virtual Console Setup.1939lb # [ 2.404728] systemd[1]: Starting Virtual Console Setup...1940client # [ 2.440685] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1941client # [ 2.444754] systemd[1]: Finished Virtual Console Setup.1942lb # [ 2.466356] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1943lb # [ 2.468449] systemd[1]: Finished Virtual Console Setup.1944worker1 # [ 3.096374] systemd[1]: Finished Coldplug All udev Devices.1945worker1 # [ 3.097429] systemd[1]: Reached target System Initialization.1946worker1 # [ 3.098578] systemd[1]: Reached target Basic System.1947worker2 # [ 3.217302] systemd[1]: Finished Coldplug All udev Devices.1948worker2 # [ 3.218372] systemd[1]: Reached target System Initialization.1949worker2 # [ 3.219189] systemd[1]: Reached target Basic System.1950client # [ 3.136637] systemd[1]: Finished Coldplug All udev Devices.1951client # [ 3.138158] systemd[1]: Reached target System Initialization.1952client # [ 3.140152] systemd[1]: Reached target Basic System.1953lb # [ 3.130223] systemd[1]: Finished Coldplug All udev Devices.1954lb # [ 3.131246] systemd[1]: Reached target System Initialization.1955lb # [ 3.132138] systemd[1]: Reached target Basic System.1956worker1 # [ 3.295244] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1957worker1 # [ 3.314685] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1958worker1 # [ 3.324538] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1959worker2 # [ 3.410344] (udev-worker)[106]: Network interface NamePolicy= disabled on kernel command line.1960client # [ 3.334040] (udev-worker)[101]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1961client # [ 3.337571] (udev-worker)[101]: Network interface NamePolicy= disabled on kernel command line.1962lb # [ 3.323809] (udev-worker)[102]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1963client # [ 3.345031] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1964lb # [ 3.328572] (udev-worker)[102]: Network interface NamePolicy= disabled on kernel command line.1965lb # [ 3.345616] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1966worker2 # [ 3.455684] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1967worker2 # [ 3.460907] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1968worker1 # [ 3.424146] systemd[1]: Found device /dev/disk/by-label/nixos.1969worker1 # [ 3.428974] systemd[1]: Reached target Initrd Root Device.1970worker1 # [ 3.433513] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1971lb # [ 3.417976] systemd[1]: Found device /dev/disk/by-label/nixos.1972lb # [ 3.422606] systemd[1]: Reached target Initrd Root Device.1973lb # [ 3.425838] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1974worker2 # [ 3.528610] systemd[1]: Found device /dev/disk/by-label/nixos.1975client # [ 3.446828] systemd[1]: Found device /dev/disk/by-label/nixos.1976worker2 # [ 3.533053] systemd[1]: Reached target Initrd Root Device.1977client # [ 3.450315] systemd[1]: Reached target Initrd Root Device.1978worker1 # [ 3.489885] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1979client # [ 3.456604] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1980worker1 # [ 3.499255] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1981worker2 # [ 3.544501] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982worker1 # [ 3.505990] systemd[1]: Mounting /sysroot...1983client # [ 3.504305] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1984lb # [ 3.494609] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1985client # [ 3.514340] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1986worker1 # [ 3.570624] EXT4-fs (vda): mounted filesystem f4aed186-e9ce-4b8b-aecf-d37ccfbd1eaa r/w with ordered data mode. Quota mode: none.1987worker2 # [ 3.599587] systemd-fsck[112]: nixos: clean, 12/65536 files, 13019/262144 blocks1988client # [ 3.518318] systemd[1]: Mounting /sysroot...1989worker1 # [ 3.559386] systemd[1]: Mounted /sysroot.1990worker1 # [ 3.561405] systemd[1]: Reached target Initrd Root File System.1991lb # [ 3.508093] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1992worker2 # [ 3.608273] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993worker1 # [ 3.566134] systemd[1]: Starting Mountpoints Configured in the Real Root...1994lb # [ 3.512797] systemd[1]: Mounting /sysroot...1995worker2 # [ 3.621073] systemd[1]: Mounting /sysroot...1996worker1 # [ 3.600321] systemd-sysroot-fstab-check[120]: /sysroot should be mounted in the initrd, will request daemon-reload.1997client # [ 3.580950] EXT4-fs (vda): mounted filesystem 0e706931-f17b-4b37-b6a0-448fe9e8d98d r/w with ordered data mode. Quota mode: none.1998worker1 # [ 3.606200] systemd[1]: Reload requested from client PID 120 ('systemd-sysroot') (unit initrd-parse-etc.service)...1999client # [ 3.573488] systemd[1]: Mounted /sysroot.2000worker1 # [ 3.612586] systemd[1]: Reloading...
loading 2,020 hidden lines…
4021worker2 # [ 40.317798] nix-grpc-daemon[776]: ts=2026-09-22T10:57:12Z level=info event=scheduler_connected addr=lb:500514022worker2 # [ 40.321648] nix-grpc-daemon[776]: ts=2026-09-22T10:57:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4023lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.15 seconds)4024worker1 # [ 40.455586] nix-grpc-daemon[774]: ts=2026-09-22T10:57:12Z level=info event=scheduler_connected addr=lb:500514025worker1 # [ 40.461704] nix-grpc-daemon[774]: ts=2026-09-22T10:57:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4026worker2 # [ 40.523405] nix-grpc-daemon[776]: ts=2026-09-22T10:57:13Z level=info event=scheduler_connected addr=lb:500514027worker2 # [ 40.529529] nix-grpc-daemon[776]: ts=2026-09-22T10:57:13Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4028worker1 # [ 40.664671] nix-grpc-daemon[774]: ts=2026-09-22T10:57:13Z level=info event=scheduler_connected addr=lb:500514029worker2 # [ 40.715070] nix-grpc-daemon[776]: ts=2026-09-22T10:57:13Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:509104030worker2 # [ 40.721210] nix-grpc-daemon[776]: ts=2026-09-22T10:57:13Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=04031worker2 # [ 40.730525] nix-grpc-daemon[776]: ts=2026-09-22T10:57:13Z level=info event=scheduler_connected addr=lb:500514032worker2 # [ 40.751338] nix-grpc-daemon[776]: ts=2026-09-22T10:57:13Z level=info event=oidc_jwks provider=mock keys=14033worker2 # [ 40.756284] nix-grpc-daemon[776]: ts=2026-09-22T10:57:13Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:509104034worker2 # [ 40.761691] nix-grpc-daemon[776]: ts=2026-09-22T10:57:13Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=04035lb: must succeed: curl -sf localhost:9901/clusters4036lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4037worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4038worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4039lb: must succeed: curl -sf localhost:9901/clusters4040lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4041lb: must succeed: curl -sf localhost:9901/clusters4042lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4043lb: must succeed: curl -sf localhost:9901/clusters4044lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4045lb: must succeed: curl -sf localhost:9901/clusters4046lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4047worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4048worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.07 seconds)4049worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4050worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4051lb: must succeed: curl -sf localhost:9901/clusters4052lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4053lb: must succeed: curl -sf localhost:9901/clusters4054lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4055worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4056worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)4057worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4058worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4059worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4060worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4061worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4062worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4063(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.85 seconds)4064subtest: balancer auth: client cert, bearer token, nothing, unknown CN4065worker1 # [ 42.771982] nix-grpc-daemon[774]: ts=2026-09-22T10:57:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:358764066worker1 # [ 42.790989] systemd[1]: Started Nix Daemon.4067worker1 # [ 43.105774] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4068worker1 # [ 43.133690] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)4069worker1 # [ 43.154733] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 2/5)4070worker1 # [ 43.222678] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 3/5)4071worker1 # [ 43.352729] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 570 ms (attempt 4/5)4072worker1 # [ 43.933999] nix-grpc-daemon[774]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4073worker1 # [ 43.941253] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4074worker1 # [ 43.963673] nix-grpc-daemon[774]: ts=2026-09-22T10:57:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:358764075worker1 # [ 43.981882] nix-grpc-daemon[774]: ts=2026-09-22T10:57:16Z level=debug event=path_info_miss path=fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test error="error: path '/nix/store/fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test' is required, but there is no substituter that can build it"4076worker1 # [ 43.995208] nix-grpc-daemon[774]: ts=2026-09-22T10:57:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:358764077worker1 # [ 44.008620] nix-grpc-daemon[774]: ts=2026-09-22T10:57:16Z level=debug event=path_info_miss path=fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test error="error: path '/nix/store/fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test' is required, but there is no substituter that can build it"4078client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4079client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.11 seconds)4080client: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4081client: (finished: must succeed: install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.05 seconds)4082worker1 # [ 44.492199] nix-grpc-daemon[774]: ts=2026-09-22T10:57:17Z level=info event=oidc_jwks provider=mock keys=14083worker1 # [ 44.499127] nix-grpc-daemon[774]: ts=2026-09-22T10:57:17Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:358764084worker2 # [ 44.575619] nix-grpc-daemon[776]: ts=2026-09-22T10:57:17Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:364564085worker2 # [ 44.595434] systemd[1]: Started Nix Daemon.4086worker2 # [ 44.935785] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4087worker2 # [ 44.962506] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)4088worker2 # [ 45.004075] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)4089worker2 # [ 45.119487] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 3/5)4090worker2 # [ 45.333730] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 4/5)4091worker2 # [ 45.351740] nix-grpc-daemon[776]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4092worker2 # [ 45.358058] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4093worker2 # [ 45.371393] nix-grpc-daemon[776]: ts=2026-09-22T10:57:17Z level=debug event=path_info_miss path=fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test error="error: path '/nix/store/fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test' is required, but there is no substituter that can build it"4094worker1 # [ 45.352155] nix-grpc-daemon[774]: ts=2026-09-22T10:57:17Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:358764095worker1 # [ 45.365329] nix-grpc-daemon[774]: ts=2026-09-22T10:57:17Z level=debug event=path_info_miss path=fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test error="error: path '/nix/store/fgq6s5j0c5a9yvpdbw3wwxx580bjrg8c-nixos-system-client-test' is required, but there is no substituter that can build it"4096client: must succeed: rm -r /var/lib/nix-grpc-store4097client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.05 seconds)4098worker2 # [ 45.781384] nix-grpc-daemon[776]: ts=2026-09-22T10:57:18Z level=info event=denied method=QueryMissing cn=- role=none4099worker2 # [ 46.072329] nix-grpc-daemon[776]: ts=2026-09-22T10:57:18Z level=info event=denied method=QueryMissing cn=stranger role=none4100(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 3.69 seconds)4101subtest: DAG build is scheduled across workers and lands in the cache4102client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24103worker1 # [ 46.314555] nix-grpc-daemon[774]: ts=2026-09-22T10:57:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:358764104client # don't know how to build these paths:4105client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4106worker1 # [ 46.329222] nix-grpc-daemon[774]: ts=2026-09-22T10:57:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764107worker2 # [ 46.421434] nix-grpc-daemon[776]: ts=2026-09-22T10:57:18Z level=debug event=want client=1 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv cached=04108worker2 # [ 46.427643] nix-grpc-daemon[776]: ts=2026-09-22T10:57:18Z level=debug event=assigned drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv worker=192.168.1.3:50051 clients=14109worker1 # [ 46.426787] nix-grpc-daemon[774]: ts=2026-09-22T10:57:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764110worker1 # [ 46.450228] nix-grpc-daemon[774]: ts=2026-09-22T10:57:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764111client # copying 1 paths to 'lb:50051'...4112worker1 # [ 46.469696] nix-grpc-daemon[774]: ts=2026-09-22T10:57:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358764113client # copying path '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' to 'lb:50051'...4114worker1 # [ 46.498111] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4115worker1 # [ 46.537043] nix-grpc-daemon[774]: ts=2026-09-22T10:57:19Z level=info event=niks3_push_started pid=8534116worker1 # [ 46.872812] nix-grpc-daemon[853]: time=2026-09-22T10:57:19.394Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""4117worker1 # [ 47.091681] nix-daemon[840]: accepted connection from pid 866, user nix-grpc-daemon (trusted)4118worker1 # [ 47.112662] nix-daemon[840]: reaped child process 870, status = succeeded4119worker1 # [ 47.305241] nix-daemon[840]: accepted connection from pid 872, user nix-grpc-daemon (trusted)4120worker1 # [ 47.325926] nix-daemon[840]: reaped child process 875, status = succeeded4121lb # [ 47.283839] niks3-server[1013]: 2026/09/22 10:57:19 INFO Received uploads request method=POST path=/api/pending_closures4122worker1 # [ 47.369063] nix-grpc-daemon[853]: time=2026-09-22T10:57:19.892Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4123worker1 # [ 47.372980] nix-grpc-daemon[853]: time=2026-09-22T10:57:19.894Z level=INFO msg="Uploading 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv (504B)"4124lb # [ 47.370399] niks3-server[1013]: 2026/09/22 10:57:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4125lb # [ 47.382164] niks3-server[1013]: 2026/09/22 10:57:19 INFO Registered completed upload object_key=nar/1il50j4vra11xfnqgjhjgrjn1753frlhxnss6mh1dmp5ww3is34w.nar.zst4126lb # [ 47.427911] niks3-server[1013]: 2026/09/22 10:57:19 INFO Signed narinfos id=1 count=14127worker1 # [ 47.484423] nix-grpc-daemon[853]: time=2026-09-22T10:57:20.006Z level=INFO msg="Uploading 1 narinfos"4128lb # [ 47.454037] niks3-server[1013]: 2026/09/22 10:57:19 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.ls4129lb # [ 47.463138] niks3-server[1013]: 2026/09/22 10:57:19 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4130lb # [ 47.473721] niks3-server[1013]: 2026/09/22 10:57:19 INFO Registered completed upload object_key=619ccx60pfv52kv4hhgqpj1ivdmfcr5r.narinfo4131lb # [ 47.483745] niks3-server[1013]: 2026/09/22 10:57:20 INFO Completed upload id=14132worker1 # [ 47.539007] nix-grpc-daemon[853]: time=2026-09-22T10:57:20.062Z level=INFO msg="Upload complete. (657ms)"4133worker1 # [ 47.551564] nix-grpc-daemon[774]: ts=2026-09-22T10:57:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=1 paths=1 nar_bytes_in=5044134worker1 # [ 47.572648] nix-grpc-daemon[774]: ts=2026-09-22T10:57:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764135worker1 # [ 47.586634] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4136worker1 # [ 47.590646] nix-daemon[840]: reaped child process 851, status = succeeded4137client # building '/nix/store/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv' on lb:50051...4138client # farm-a> node-a: building 619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4139worker1 # [ 47.609086] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4140client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4141client # farm-a> LOG-farm-a-t14142worker1 # [ 49.248376] nix-daemon[840]: reaped child process 879, status = succeeded4143worker1 # [ 49.443659] nix-daemon[840]: accepted connection from pid 895, user nix-grpc-daemon (trusted)4144worker1 # [ 49.466383] nix-daemon[840]: reaped child process 898, status = succeeded4145lb # [ 49.425417] niks3-server[1013]: 2026/09/22 10:57:21 INFO Received uploads request method=POST path=/api/pending_closures4146worker1 # [ 49.498629] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.020Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4147worker1 # [ 49.501062] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.020Z level=INFO msg="Uploading dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a (128B)"4148lb # [ 49.489335] niks3-server[1013]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4149lb # [ 49.500445] niks3-server[1013]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=log/619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv4150lb # [ 49.510156] niks3-server[1013]: 2026/09/22 10:57:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4151lb # [ 49.518008] niks3-server[1013]: 2026/09/22 10:57:22 INFO Signed narinfos id=2 count=14152worker1 # [ 49.573632] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.095Z level=INFO msg="Uploading 1 narinfos"4153lb # [ 49.520732] niks3-server[1013]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.ls4154lb # [ 49.533406] niks3-server[1013]: 2026/09/22 10:57:22 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4155lb # [ 49.549152] niks3-server[1013]: 2026/09/22 10:57:22 INFO Completed upload id=24156lb # [ 49.552176] niks3-server[1013]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=dg404zlvxfrspnxl6cc1qkc6i83m5a3a.narinfo4157worker1 # [ 49.606902] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.130Z level=INFO msg="Upload complete. (382ms)"4158worker1 # [ 49.619564] nix-grpc-daemon[774]: ts=2026-09-22T10:57:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=2 drv=619ccx60pfv52kv4hhgqpj1ivdmfcr5r-farm-a.drv assign_id=1 outputs=14159worker2 # [ 49.677433] nix-grpc-daemon[776]: ts=2026-09-22T10:57:22Z level=debug event=want client=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv cached=04160worker2 # [ 49.683160] nix-grpc-daemon[776]: ts=2026-09-22T10:57:22Z level=debug event=assigned drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv worker=192.168.1.3:50051 clients=14161worker1 # [ 49.645211] nix-daemon[840]: reaped child process 877, status = succeeded4162worker1 # [ 49.680332] nix-grpc-daemon[774]: ts=2026-09-22T10:57:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764163worker1 # [ 49.699623] nix-grpc-daemon[774]: ts=2026-09-22T10:57:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764164worker1 # [ 49.717620] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.241Z level=INFO msg="All 1 paths already cached"4165client # copying 1 paths to 'lb:50051'...4166worker1 # [ 49.730017] nix-grpc-daemon[774]: ts=2026-09-22T10:57:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358764167client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4168worker1 # [ 49.754752] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4169worker1 # [ 49.979963] nix-daemon[840]: accepted connection from pid 902, user nix-grpc-daemon (trusted)4170worker1 # [ 50.003447] nix-daemon[840]: reaped child process 905, status = succeeded4171worker1 # [ 50.191041] nix-daemon[840]: accepted connection from pid 907, user nix-grpc-daemon (trusted)4172worker1 # [ 50.211591] nix-daemon[840]: reaped child process 910, status = succeeded4173lb # [ 50.169666] niks3-server[1013]: 2026/09/22 10:57:22 INFO Received uploads request method=POST path=/api/pending_closures4174worker1 # [ 50.232159] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.755Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4175worker1 # [ 50.233796] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.755Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4176lb # [ 50.208133] niks3-server[1013]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4177worker1 # [ 50.278765] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.801Z level=INFO msg="Uploading 1 narinfos"4178lb # [ 50.223685] niks3-server[1013]: 2026/09/22 10:57:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4179lb # [ 50.230424] niks3-server[1013]: 2026/09/22 10:57:22 INFO Signed narinfos id=3 count=14180lb # [ 50.236717] niks3-server[1013]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4181lb # [ 50.248281] niks3-server[1013]: 2026/09/22 10:57:22 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4182lb # [ 50.257284] niks3-server[1013]: 2026/09/22 10:57:22 INFO Completed upload id=34183worker1 # [ 50.312304] nix-grpc-daemon[853]: time=2026-09-22T10:57:22.835Z level=INFO msg="Upload complete. (537ms)"4184lb # [ 50.260140] niks3-server[1013]: 2026/09/22 10:57:22 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4185worker1 # [ 50.320190] nix-grpc-daemon[774]: ts=2026-09-22T10:57:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_in=6244186worker1 # [ 50.335271] nix-grpc-daemon[774]: ts=2026-09-22T10:57:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764187worker1 # [ 50.346684] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4188worker1 # [ 50.349841] nix-daemon[840]: reaped child process 900, status = succeeded4189client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4190client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4191worker1 # [ 50.367094] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4192client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4193client # farm-b> LOG-farm-b-t14194worker1 # [ 51.412313] nix-daemon[840]: reaped child process 914, status = succeeded4195worker1 # [ 51.596419] nix-daemon[840]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4196worker1 # [ 51.620107] nix-daemon[840]: reaped child process 933, status = succeeded4197lb # [ 51.578911] niks3-server[1013]: 2026/09/22 10:57:24 INFO Received uploads request method=POST path=/api/pending_closures4198worker1 # [ 51.648895] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.172Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4199worker1 # [ 51.651237] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.174Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4200lb # [ 51.638802] niks3-server[1013]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4201worker1 # [ 51.708566] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.231Z level=INFO msg="Uploading 1 narinfos"4202lb # [ 51.654882] niks3-server[1013]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4203lb # [ 51.662960] niks3-server[1013]: 2026/09/22 10:57:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4204lb # [ 51.671518] niks3-server[1013]: 2026/09/22 10:57:24 INFO Signed narinfos id=4 count=14205lb # [ 51.686102] niks3-server[1013]: 2026/09/22 10:57:24 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4206lb # [ 51.692653] niks3-server[1013]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4207lb # [ 51.698901] niks3-server[1013]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4208lb # [ 51.702412] niks3-server[1013]: 2026/09/22 10:57:24 INFO Completed upload id=44209worker1 # [ 51.757369] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.280Z level=INFO msg="Upload complete. (378ms)"4210worker1 # [ 51.769563] nix-grpc-daemon[774]: ts=2026-09-22T10:57:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14211worker2 # [ 51.821161] nix-grpc-daemon[776]: ts=2026-09-22T10:57:24Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04212worker2 # [ 51.821587] nix-grpc-daemon[776]: ts=2026-09-22T10:57:24Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14213worker1 # [ 51.792233] nix-daemon[840]: reaped child process 912, status = succeeded4214worker1 # [ 51.823881] nix-grpc-daemon[774]: ts=2026-09-22T10:57:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764215worker1 # [ 51.842136] nix-grpc-daemon[774]: ts=2026-09-22T10:57:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764216worker1 # [ 51.859116] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.382Z level=INFO msg="All 2 paths already cached"4217client # copying 1 paths to 'lb:50051'...4218client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4219worker1 # [ 51.871502] nix-grpc-daemon[774]: ts=2026-09-22T10:57:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358764220worker1 # [ 51.891177] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4221lb # [ 51.852902] envoy[923]: conn peer=[2001:db8:1::1]:33820 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=715 tx=365 ms=12974222lb # [ 51.860389] envoy[923]: conn peer=[2001:db8:1::1]:33830 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2613 tx=406 ms=9054223lb # [ 51.865107] envoy[923]: conn peer=[2001:db8:1::1]:33842 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=214224lb # [ 51.869582] envoy[923]: conn peer=[2001:db8:1::1]:33852 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=204225worker1 # [ 52.107053] nix-daemon[840]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4226worker1 # [ 52.127669] nix-daemon[840]: reaped child process 940, status = succeeded4227worker1 # [ 52.308565] nix-daemon[840]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4228worker1 # [ 52.327429] nix-daemon[840]: reaped child process 945, status = succeeded4229lb # [ 52.289967] niks3-server[1013]: 2026/09/22 10:57:24 INFO Received uploads request method=POST path=/api/pending_closures4230worker1 # [ 52.357484] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.880Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4231worker1 # [ 52.359122] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.880Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4232worker1 # [ 52.393082] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.916Z level=INFO msg="Uploading 1 narinfos"4233lb # [ 52.338141] niks3-server[1013]: 2026/09/22 10:57:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4234lb # [ 52.342912] niks3-server[1013]: 2026/09/22 10:57:24 INFO Signed narinfos id=5 count=14235lb # [ 52.345607] niks3-server[1013]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4236lb # [ 52.357353] niks3-server[1013]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4237lb # [ 52.372129] niks3-server[1013]: 2026/09/22 10:57:24 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4238lb # [ 52.380484] niks3-server[1013]: 2026/09/22 10:57:24 INFO Completed upload id=54239worker1 # [ 52.434789] nix-grpc-daemon[853]: time=2026-09-22T10:57:24.958Z level=INFO msg="Upload complete. (524ms)"4240lb # [ 52.383368] niks3-server[1013]: 2026/09/22 10:57:24 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4241worker1 # [ 52.444483] nix-grpc-daemon[774]: ts=2026-09-22T10:57:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_in=7524242worker1 # [ 52.460861] nix-grpc-daemon[774]: ts=2026-09-22T10:57:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764243worker1 # [ 52.469326] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4244worker1 # [ 52.480539] nix-daemon[840]: reaped child process 935, status = succeeded4245client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4246client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4247worker1 # [ 52.492259] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4248client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4249client # farm-top> LOG-farm-top-t14250worker1 # [ 53.527960] nix-daemon[840]: reaped child process 949, status = succeeded4251worker1 # [ 53.695899] nix-daemon[840]: accepted connection from pid 965, user nix-grpc-daemon (trusted)4252worker1 # [ 53.719159] nix-daemon[840]: reaped child process 968, status = succeeded4253lb # [ 53.677399] niks3-server[1013]: 2026/09/22 10:57:26 INFO Received uploads request method=POST path=/api/pending_closures4254worker1 # [ 53.744491] nix-grpc-daemon[853]: time=2026-09-22T10:57:26.267Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4255worker1 # [ 53.746015] nix-grpc-daemon[853]: time=2026-09-22T10:57:26.267Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4256lb # [ 53.723581] niks3-server[1013]: 2026/09/22 10:57:26 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4257lb # [ 53.742819] niks3-server[1013]: 2026/09/22 10:57:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4258worker1 # [ 53.803768] nix-grpc-daemon[853]: time=2026-09-22T10:57:26.326Z level=INFO msg="Uploading 1 narinfos"4259lb # [ 53.754647] niks3-server[1013]: 2026/09/22 10:57:26 INFO Signed narinfos id=6 count=14260lb # [ 53.759342] niks3-server[1013]: 2026/09/22 10:57:26 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4261lb # [ 53.767694] niks3-server[1013]: 2026/09/22 10:57:26 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4262lb # [ 53.775699] niks3-server[1013]: 2026/09/22 10:57:26 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4263worker1 # [ 53.830068] nix-grpc-daemon[853]: time=2026-09-22T10:57:26.353Z level=INFO msg="Upload complete. (326ms)"4264lb # [ 53.778816] niks3-server[1013]: 2026/09/22 10:57:26 INFO Completed upload id=64265lb # [ 53.782097] niks3-server[1013]: 2026/09/22 10:57:26 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4266worker1 # [ 53.841304] nix-grpc-daemon[774]: ts=2026-09-22T10:57:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14267worker1 # [ 53.859631] nix-daemon[840]: reaped child process 947, status = succeeded4268client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 7.85 seconds)4269client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4270client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.26 seconds)4271client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4272client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4273client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4274client # copying 3 paths...4275client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4276client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4277client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4278client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.32 seconds)4279(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 8.57 seconds)4280subtest: repeat is answered Cached by the scheduler without building4281worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4282worker1 # finding garbage collector roots...4283worker1 # deleting specified paths...4284worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4285worker1 # deleting unused links...4286worker1 # note: hard linking is currently saving 0.0 KiB4287worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.29 seconds)4288worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4289worker2 # finding garbage collector roots...4290worker2 # deleting specified paths...4291worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4292worker2 # deleting unused links...4293worker2 # note: hard linking is currently saving 0.0 KiB4294worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.32 seconds)4295client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&24296worker1 # [ 55.521029] nix-grpc-daemon[774]: ts=2026-09-22T10:57:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:358764297client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4298client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4299worker1 # [ 55.542808] nix-grpc-daemon[774]: ts=2026-09-22T10:57:28Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764300worker1 # [ 55.555704] nix-grpc-daemon[853]: time=2026-09-22T10:57:28.078Z level=INFO msg="All 2 paths already cached"4301worker2 # [ 55.621546] nix-grpc-daemon[776]: ts=2026-09-22T10:57:28Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14302client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 >&2, in 0.38 seconds)4303client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath4304client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 outPath, in 0.23 seconds)4305lb: must succeed: curl -sf localhost:9901/clusters4306lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4307lb: must succeed: curl -sf localhost:9901/clusters4308lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4309worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4310worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4311(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.57 seconds)4312subtest: inputs already in the cache are not sent to the scheduler4313lb: must succeed: curl -sf localhost:9901/clusters4314lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4315lb: must succeed: curl -sf localhost:9901/clusters4316lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4317worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4318worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4319client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&24320worker2 # [ 56.745898] nix-grpc-daemon[776]: ts=2026-09-22T10:57:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364564321client # don't know how to build these paths:4322client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4323worker1 # [ 56.715578] nix-grpc-daemon[774]: ts=2026-09-22T10:57:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764324worker1 # [ 56.730612] nix-grpc-daemon[853]: time=2026-09-22T10:57:29.251Z level=INFO msg="All 2 paths already cached"4325worker2 # [ 56.788116] nix-grpc-daemon[776]: ts=2026-09-22T10:57:29Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04326worker2 # [ 56.794450] nix-grpc-daemon[776]: ts=2026-09-22T10:57:29Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14327worker1 # [ 56.792999] nix-grpc-daemon[774]: ts=2026-09-22T10:57:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764328worker1 # [ 56.812742] nix-grpc-daemon[774]: ts=2026-09-22T10:57:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764329worker1 # [ 56.829617] nix-grpc-daemon[853]: time=2026-09-22T10:57:29.353Z level=INFO msg="All 2 paths already cached"4330client # copying 1 paths to 'lb:50051'...4331client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4332worker1 # [ 56.842240] nix-grpc-daemon[774]: ts=2026-09-22T10:57:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358764333worker1 # [ 56.863539] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4334worker1 # [ 57.086063] nix-daemon[840]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4335worker1 # [ 57.109628] nix-daemon[840]: reaped child process 988, status = succeeded4336worker1 # [ 57.298159] nix-daemon[840]: accepted connection from pid 990, user nix-grpc-daemon (trusted)4337worker1 # [ 57.320541] nix-daemon[840]: reaped child process 993, status = succeeded4338lb # [ 57.279007] niks3-server[1013]: 2026/09/22 10:57:29 INFO Received uploads request method=POST path=/api/pending_closures4339worker1 # [ 57.348493] nix-grpc-daemon[853]: time=2026-09-22T10:57:29.871Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4340worker1 # [ 57.350045] nix-grpc-daemon[853]: time=2026-09-22T10:57:29.871Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4341lb # [ 57.316509] niks3-server[1013]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4342lb # [ 57.335290] niks3-server[1013]: 2026/09/22 10:57:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4343worker1 # [ 57.395666] nix-grpc-daemon[853]: time=2026-09-22T10:57:29.918Z level=INFO msg="Uploading 1 narinfos"4344lb # [ 57.347885] niks3-server[1013]: 2026/09/22 10:57:29 INFO Signed narinfos id=7 count=14345lb # [ 57.355053] niks3-server[1013]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4346lb # [ 57.364815] niks3-server[1013]: 2026/09/22 10:57:29 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4347lb # [ 57.374770] niks3-server[1013]: 2026/09/22 10:57:29 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4348lb # [ 57.379988] niks3-server[1013]: 2026/09/22 10:57:29 INFO Completed upload id=74349worker1 # [ 57.435098] nix-grpc-daemon[853]: time=2026-09-22T10:57:29.958Z level=INFO msg="Upload complete. (550ms)"4350worker1 # [ 57.444488] nix-grpc-daemon[774]: ts=2026-09-22T10:57:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_in=7524351worker1 # [ 57.464483] nix-grpc-daemon[774]: ts=2026-09-22T10:57:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764352worker1 # [ 57.476999] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4353worker1 # [ 57.480512] nix-daemon[840]: reaped child process 983, status = succeeded4354client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4355worker1 # [ 57.494619] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4356client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4357client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4358client # farm-top> LOG-farm-top-t1b4359worker1 # [ 59.155803] nix-daemon[840]: reaped child process 997, status = succeeded4360worker1 # [ 59.344763] nix-daemon[840]: accepted connection from pid 1013, user nix-grpc-daemon (trusted)4361worker1 # [ 59.366208] nix-daemon[840]: reaped child process 1016, status = succeeded4362lb # [ 59.323792] niks3-server[1013]: 2026/09/22 10:57:31 INFO Received uploads request method=POST path=/api/pending_closures4363worker1 # [ 59.392495] nix-grpc-daemon[853]: time=2026-09-22T10:57:31.914Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4364worker1 # [ 59.394040] nix-grpc-daemon[853]: time=2026-09-22T10:57:31.914Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4365lb # [ 59.372480] niks3-server[1013]: 2026/09/22 10:57:31 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4366lb # [ 59.385004] niks3-server[1013]: 2026/09/22 10:57:31 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4367lb # [ 59.395359] niks3-server[1013]: 2026/09/22 10:57:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4368worker1 # [ 59.457582] nix-grpc-daemon[853]: time=2026-09-22T10:57:31.980Z level=INFO msg="Uploading 1 narinfos"4369lb # [ 59.405575] niks3-server[1013]: 2026/09/22 10:57:31 INFO Signed narinfos id=8 count=14370lb # [ 59.411562] niks3-server[1013]: 2026/09/22 10:57:31 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4371lb # [ 59.424493] niks3-server[1013]: 2026/09/22 10:57:31 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4372lb # [ 59.431973] niks3-server[1013]: 2026/09/22 10:57:31 INFO Completed upload id=84373worker1 # [ 59.487036] nix-grpc-daemon[853]: time=2026-09-22T10:57:32.010Z level=INFO msg="Upload complete. (373ms)"4374lb # [ 59.435006] niks3-server[1013]: 2026/09/22 10:57:31 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4375worker1 # [ 59.501598] nix-grpc-daemon[774]: ts=2026-09-22T10:57:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=2 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14376worker1 # [ 59.520152] nix-daemon[840]: reaped child process 995, status = succeeded4377client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b >&2, in 3.15 seconds)4378client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath4379client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.25 seconds)4380lb: must succeed: curl -sf localhost:9901/clusters4381lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)4382lb: must succeed: curl -sf localhost:9901/clusters4383lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4384worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4385worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4386(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.83 seconds)4387subtest: two clients wanting the same drv build it once4388client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4389client # [ 60.069425] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4390client # Running as unit: dedup1.service; invocation ID: e6fdd5f2e35b4fbcb5e2cbbf44437cb34391client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 0.10 seconds)4392client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup4393client # [ 60.355912] nix[880]: don't know how to build these paths:4394client # [ 60.358082] nix[880]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4395worker2 # [ 60.438659] nix-grpc-daemon[776]: ts=2026-09-22T10:57:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364564396worker2 # [ 60.454016] nix-grpc-daemon[776]: ts=2026-09-22T10:57:32Z level=debug event=want client=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04397worker2 # [ 60.462494] nix-grpc-daemon[776]: ts=2026-09-22T10:57:32Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14398worker2 # [ 60.471494] nix-grpc-daemon[776]: ts=2026-09-22T10:57:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564399worker2 # [ 60.499129] nix-grpc-daemon[776]: ts=2026-09-22T10:57:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564400client # [ 60.432523] nix[880]: copying 1 paths to 'lb:50051'...4401client # [ 60.434145] nix[880]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4402worker2 # [ 60.516150] nix-grpc-daemon[776]: ts=2026-09-22T10:57:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364564403worker2 # [ 60.530937] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4404worker2 # [ 60.551065] nix-grpc-daemon[776]: ts=2026-09-22T10:57:33Z level=info event=niks3_push_started pid=9144405worker2 # [ 60.824101] nix-grpc-daemon[914]: time=2026-09-22T10:57:33.300Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""4406worker2 # [ 61.026024] nix-daemon[863]: accepted connection from pid 927, user nix-grpc-daemon (trusted)4407worker2 # [ 61.049881] nix-daemon[863]: reaped child process 930, status = succeeded4408worker2 # [ 61.212981] nix-daemon[863]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4409worker2 # [ 61.234077] nix-daemon[863]: reaped child process 935, status = succeeded4410lb # [ 61.146523] niks3-server[1013]: 2026/09/22 10:57:33 INFO Received uploads request method=POST path=/api/pending_closures4411worker2 # [ 61.258280] nix-grpc-daemon[914]: time=2026-09-22T10:57:33.736Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4412worker2 # [ 61.262084] nix-grpc-daemon[914]: time=2026-09-22T10:57:33.738Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4413lb # [ 61.176854] niks3-server[1013]: 2026/09/22 10:57:33 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4414lb # [ 61.182676] niks3-server[1013]: 2026/09/22 10:57:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4415client # [ 61.189673] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup.4416lb # [ 61.185021] niks3-server[1013]: 2026/09/22 10:57:33 INFO Signed narinfos id=9 count=14417worker2 # [ 61.285121] nix-grpc-daemon[914]: time=2026-09-22T10:57:33.762Z level=INFO msg="Uploading 1 narinfos"4418client # Running as unit: dedup2.service; invocation ID: 4b61fc10f853449ea0cec7f5358ed4a04419lb # [ 61.190563] niks3-server[1013]: 2026/09/22 10:57:33 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4420lb # [ 61.197255] niks3-server[1013]: 2026/09/22 10:57:33 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4421lb # [ 61.201962] niks3-server[1013]: 2026/09/22 10:57:33 INFO Completed upload id=94422worker2 # [ 61.301446] nix-grpc-daemon[914]: time=2026-09-22T10:57:33.779Z level=INFO msg="Upload complete. (470ms)"4423lb # [ 61.204076] niks3-server[1013]: 2026/09/22 10:57:33 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4424worker2 # [ 61.308153] nix-grpc-daemon[776]: ts=2026-09-22T10:57:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=0 paths=1 nar_bytes_in=4324425client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag dedup, in 1.14 seconds)4426worker2 # [ 61.313801] nix-grpc-daemon[776]: ts=2026-09-22T10:57:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564427worker2 # [ 61.324370] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4428worker2 # [ 61.326375] nix-daemon[863]: reaped child process 912, status = succeeded4429client # [ 61.251442] nix[880]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4430worker2 # [ 61.336784] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4431worker1 # [ 61.803041] nix-grpc-daemon[774]: ts=2026-09-22T10:57:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:358764432client # [ 61.775213] nix[896]: don't know how to build these paths:4433client # [ 61.777631] nix[896]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4434worker2 # [ 61.873067] nix-grpc-daemon[776]: ts=2026-09-22T10:57:34Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04435worker2 # [ 61.881840] nix-grpc-daemon[776]: ts=2026-09-22T10:57:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564436client # [ 61.806642] nix[896]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4437lb # [ 61.857296] envoy[923]: conn peer=[2001:db8:1::1]:33868 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9575 tx=4099 ms=75584438lb # [ 61.862822] envoy[923]: conn peer=[2001:db8:1::1]:57440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=834439lb # [ 61.868222] envoy[923]: conn peer=[2001:db8:1::1]:57442 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=1822 ms=28144440client: must succeed: sleep 24441client: (finished: must succeed: sleep 2, in 2.07 seconds)4442client: waiting for success: ! systemctl is-active dedup1 dedup24443worker2 # [ 64.960259] nix-daemon[863]: reaped child process 939, status = succeeded4444worker2 # [ 65.127390] nix-daemon[863]: accepted connection from pid 971, user nix-grpc-daemon (trusted)4445worker2 # [ 65.145777] nix-daemon[863]: reaped child process 974, status = succeeded4446lb # [ 65.060781] niks3-server[1013]: 2026/09/22 10:57:37 INFO Received uploads request method=POST path=/api/pending_closures4447worker2 # [ 65.172608] nix-grpc-daemon[914]: time=2026-09-22T10:57:37.650Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4448worker2 # [ 65.175007] nix-grpc-daemon[914]: time=2026-09-22T10:57:37.652Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4449lb # [ 65.113851] niks3-server[1013]: 2026/09/22 10:57:37 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4450lb # [ 65.125959] niks3-server[1013]: 2026/09/22 10:57:37 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4451worker2 # [ 65.235676] nix-grpc-daemon[914]: time=2026-09-22T10:57:37.712Z level=INFO msg="Uploading 1 narinfos"4452lb # [ 65.137600] niks3-server[1013]: 2026/09/22 10:57:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4453lb # [ 65.143776] niks3-server[1013]: 2026/09/22 10:57:37 INFO Signed narinfos id=10 count=14454lb # [ 65.149113] niks3-server[1013]: 2026/09/22 10:57:37 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4455lb # [ 65.157713] niks3-server[1013]: 2026/09/22 10:57:37 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4456lb # [ 65.164368] niks3-server[1013]: 2026/09/22 10:57:37 INFO Completed upload id=104457worker2 # [ 65.262258] nix-grpc-daemon[914]: time=2026-09-22T10:57:37.740Z level=INFO msg="Upload complete. (315ms)"4458lb # [ 65.166491] niks3-server[1013]: 2026/09/22 10:57:37 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4459worker2 # [ 65.275288] nix-grpc-daemon[776]: ts=2026-09-22T10:57:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14460worker2 # [ 65.292342] nix-grpc-daemon[776]: ts=2026-09-22T10:57:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=5 outputs=14461worker2 # [ 65.298489] nix-daemon[863]: reaped child process 937, status = succeeded4462client # [ 65.242375] systemd[1]: dedup2.service: Deactivated successfully.4463client # [ 65.250460] systemd[1]: dedup2.service: Consumed 291ms CPU time over 4.060s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4464client # [ 65.291651] systemd[1]: dedup1.service: Deactivated successfully.4465client # [ 65.301118] systemd[1]: dedup1.service: Consumed 191ms CPU time over 5.230s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4466client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4467client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4468client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4469worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4470worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4471worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4472worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4473worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4474worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4475worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4476worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)4477(finished: subtest: two clients wanting the same drv build it once, in 6.33 seconds)4478subtest: --timeout and --max-silent-time reach the builder4479worker1 # [ 66.649437] nix-grpc-daemon[774]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:358764480worker2 # [ 66.707442] nix-grpc-daemon[776]: ts=2026-09-22T10:57:39Z level=debug event=want client=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04481worker2 # [ 66.707907] nix-grpc-daemon[776]: ts=2026-09-22T10:57:39Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14482worker1 # [ 66.707726] nix-grpc-daemon[774]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764483worker1 # [ 66.725505] nix-grpc-daemon[774]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764484worker1 # [ 66.741719] nix-grpc-daemon[774]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358764485worker1 # [ 66.757572] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4486worker1 # [ 66.973427] nix-daemon[840]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4487worker1 # [ 66.995093] nix-daemon[840]: reaped child process 1044, status = succeeded4488worker1 # [ 67.173228] nix-daemon[840]: accepted connection from pid 1046, user nix-grpc-daemon (trusted)4489worker1 # [ 67.192621] nix-daemon[840]: reaped child process 1049, status = succeeded4490lb # [ 67.150369] niks3-server[1013]: 2026/09/22 10:57:39 INFO Received uploads request method=POST path=/api/pending_closures4491worker1 # [ 67.215667] nix-grpc-daemon[853]: time=2026-09-22T10:57:39.738Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4492worker1 # [ 67.217417] nix-grpc-daemon[853]: time=2026-09-22T10:57:39.738Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4493lb # [ 67.180229] niks3-server[1013]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4494lb # [ 67.195449] niks3-server[1013]: 2026/09/22 10:57:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4495worker1 # [ 67.256174] nix-grpc-daemon[853]: time=2026-09-22T10:57:39.779Z level=INFO msg="Uploading 1 narinfos"4496lb # [ 67.207916] niks3-server[1013]: 2026/09/22 10:57:39 INFO Signed narinfos id=11 count=14497lb # [ 67.214184] niks3-server[1013]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4498lb # [ 67.223808] niks3-server[1013]: 2026/09/22 10:57:39 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4499lb # [ 67.229009] niks3-server[1013]: 2026/09/22 10:57:39 INFO Completed upload id=114500worker1 # [ 67.283946] nix-grpc-daemon[853]: time=2026-09-22T10:57:39.807Z level=INFO msg="Upload complete. (507ms)"4501lb # [ 67.231827] niks3-server[1013]: 2026/09/22 10:57:39 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4502worker1 # [ 67.292380] nix-grpc-daemon[774]: ts=2026-09-22T10:57:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_in=4244503worker1 # [ 67.307210] nix-grpc-daemon[774]: ts=2026-09-22T10:57:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764504worker1 # [ 67.313778] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4505worker1 # [ 67.319167] nix-daemon[840]: reaped child process 1039, status = succeeded4506worker1 # [ 67.330635] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4507lb # [ 71.855881] envoy[923]: conn peer=[2001:db8:1::1]:56706 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1086 ms=34464508lb # [ 71.861128] envoy[923]: conn peer=[2001:db8:1::1]:57454 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1453 ms=49094509worker1 # [ 73.752368] nix-grpc-daemon[774]: ts=2026-09-22T10:57:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=6 outputs=04510worker1 # [ 73.792624] nix-daemon[840]: reaped child process 1051, status = succeeded4511worker1 # [ 73.795403] nix-daemon[840]: reaped child process 1053, status = succeeded4512worker1 # [ 74.049769] nix-grpc-daemon[774]: ts=2026-09-22T10:57:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:358764513worker2 # [ 74.108800] nix-grpc-daemon[776]: ts=2026-09-22T10:57:46Z level=debug event=want client=7 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04514worker2 # [ 74.116648] nix-grpc-daemon[776]: ts=2026-09-22T10:57:46Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14515worker2 # [ 74.124656] nix-grpc-daemon[776]: ts=2026-09-22T10:57:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564516worker2 # [ 74.136270] nix-grpc-daemon[776]: ts=2026-09-22T10:57:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564517worker2 # [ 74.151239] nix-grpc-daemon[776]: ts=2026-09-22T10:57:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364564518worker2 # [ 74.167655] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4519worker2 # [ 74.370757] nix-daemon[863]: accepted connection from pid 991, user nix-grpc-daemon (trusted)4520worker2 # [ 74.391162] nix-daemon[863]: reaped child process 994, status = succeeded4521worker2 # [ 74.563544] nix-daemon[863]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4522worker2 # [ 74.582833] nix-daemon[863]: reaped child process 999, status = succeeded4523lb # [ 74.495616] niks3-server[1013]: 2026/09/22 10:57:47 INFO Received uploads request method=POST path=/api/pending_closures4524worker2 # [ 74.605573] nix-grpc-daemon[914]: time=2026-09-22T10:57:47.082Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4525worker2 # [ 74.607058] nix-grpc-daemon[914]: time=2026-09-22T10:57:47.082Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4526lb # [ 74.525391] niks3-server[1013]: 2026/09/22 10:57:47 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4527lb # [ 74.553666] niks3-server[1013]: 2026/09/22 10:57:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4528worker2 # [ 74.658196] nix-grpc-daemon[914]: time=2026-09-22T10:57:47.135Z level=INFO msg="Uploading 1 narinfos"4529lb # [ 74.559452] niks3-server[1013]: 2026/09/22 10:57:47 INFO Signed narinfos id=12 count=14530lb # [ 74.567579] niks3-server[1013]: 2026/09/22 10:57:47 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4531lb # [ 74.577008] niks3-server[1013]: 2026/09/22 10:57:47 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4532lb # [ 74.583244] niks3-server[1013]: 2026/09/22 10:57:47 INFO Completed upload id=124533lb # [ 74.586677] niks3-server[1013]: 2026/09/22 10:57:47 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4534worker2 # [ 74.682545] nix-grpc-daemon[914]: time=2026-09-22T10:57:47.160Z level=INFO msg="Upload complete. (497ms)"4535worker2 # [ 74.692586] nix-grpc-daemon[776]: ts=2026-09-22T10:57:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=0 paths=1 nar_bytes_in=4244536worker2 # [ 74.707791] nix-grpc-daemon[776]: ts=2026-09-22T10:57:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564537worker2 # [ 74.715662] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4538worker2 # [ 74.722068] nix-daemon[863]: reaped child process 989, status = succeeded4539worker2 # [ 74.736993] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4540worker2 # [ 81.155622] nix-grpc-daemon[776]: ts=2026-09-22T10:57:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=7 outputs=04541worker2 # [ 81.189755] nix-daemon[863]: reaped child process 1001, status = succeeded4542worker2 # [ 81.193238] nix-daemon[863]: reaped child process 1003, status = succeeded4543(finished: subtest: --timeout and --max-silent-time reach the builder, in 14.97 seconds)4544subtest: the first client leaving does not fail the second4545client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share4546client # [ 81.361665] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share.4547client # Running as unit: share1.service; invocation ID: 601c10c94aa24aa493287eea6b5cdd604548client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share, in 0.09 seconds)4549client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share4550worker2 # [ 81.708084] nix-grpc-daemon[776]: ts=2026-09-22T10:57:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364564551client # [ 81.632413] nix[970]: don't know how to build these paths:4552client # [ 81.633501] nix[970]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4553worker2 # [ 81.726888] nix-grpc-daemon[776]: ts=2026-09-22T10:57:54Z level=debug event=want client=8 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04554worker1 # [ 81.686417] nix-grpc-daemon[774]: ts=2026-09-22T10:57:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764555worker2 # [ 81.733101] nix-grpc-daemon[776]: ts=2026-09-22T10:57:54Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14556worker1 # [ 81.704786] nix-grpc-daemon[774]: ts=2026-09-22T10:57:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764557client # [ 81.680083] nix[970]: copying 1 paths to 'lb:50051'...4558client # [ 81.682267] nix[970]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4559worker1 # [ 81.719587] nix-grpc-daemon[774]: ts=2026-09-22T10:57:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358764560worker1 # [ 81.735431] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4561lb # [ 81.856534] envoy[923]: conn peer=[2001:db8:1::1]:56718 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=976 ms=71264562lb # [ 81.857062] envoy[923]: conn peer=[2001:db8:1::1]:56842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=954 ms=70844563worker1 # [ 81.945179] nix-daemon[840]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)4564worker1 # [ 81.964525] nix-daemon[840]: reaped child process 1079, status = succeeded4565worker1 # [ 82.143201] nix-daemon[840]: accepted connection from pid 1081, user nix-grpc-daemon (trusted)4566worker1 # [ 82.160938] nix-daemon[840]: reaped child process 1084, status = succeeded4567lb # [ 82.119312] niks3-server[1013]: 2026/09/22 10:57:54 INFO Received uploads request method=POST path=/api/pending_closures4568worker1 # [ 82.213917] nix-grpc-daemon[853]: time=2026-09-22T10:57:54.736Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4569worker1 # [ 82.215962] nix-grpc-daemon[853]: time=2026-09-22T10:57:54.736Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4570lb # [ 82.182985] niks3-server[1013]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4571worker1 # [ 82.259821] nix-grpc-daemon[853]: time=2026-09-22T10:57:54.783Z level=INFO msg="Uploading 1 narinfos"4572lb # [ 82.196784] niks3-server[1013]: 2026/09/22 10:57:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4573lb # [ 82.209572] niks3-server[1013]: 2026/09/22 10:57:54 INFO Signed narinfos id=13 count=14574lb # [ 82.216291] niks3-server[1013]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4575worker1 # [ 82.284303] nix-grpc-daemon[853]: time=2026-09-22T10:57:54.807Z level=INFO msg="Upload complete. (530ms)"4576lb # [ 82.228001] niks3-server[1013]: 2026/09/22 10:57:54 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4577lb # [ 82.235909] niks3-server[1013]: 2026/09/22 10:57:54 INFO Completed upload id=134578lb # [ 82.240531] niks3-server[1013]: 2026/09/22 10:57:54 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4579worker1 # [ 82.294851] nix-grpc-daemon[774]: ts=2026-09-22T10:57:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_in=4324580worker1 # [ 82.315348] nix-grpc-daemon[774]: ts=2026-09-22T10:57:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764581worker1 # [ 82.322660] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4582worker1 # [ 82.328958] nix-daemon[840]: reaped child process 1074, status = succeeded4583client # [ 82.302936] nix[970]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4584worker1 # [ 82.345639] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4585client # [ 82.477469] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share.4586client # Running as unit: share2.service; invocation ID: 88d8080a68d64d52ab854235222a25ba4587client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag share, in 1.12 seconds)4588worker2 # [ 82.827468] nix-grpc-daemon[776]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:364564589client # [ 82.752531] nix[986]: don't know how to build these paths:4590client # [ 82.753517] nix[986]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4591worker2 # [ 82.846482] nix-grpc-daemon[776]: ts=2026-09-22T10:57:55Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04592worker1 # [ 82.806761] nix-grpc-daemon[774]: ts=2026-09-22T10:57:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764593client # [ 82.776075] nix[986]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4594client: must succeed: sleep 2; systemctl kill -s INT share14595client # [ 85.942907] systemd[1]: share1.service: Sent signal SIGINT to main process 970 (nix) on client request.4596client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4597client: waiting for success: ! systemctl is-active share14598client # [ 85.947756] nix[970]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4599client # [ 85.949568] nix[970]: error: interrupted by the user4600client # [ 86.000912] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4601client # [ 86.008331] systemd[1]: share1.service: Failed with result 'exit-code'.4602client # [ 86.010347] systemd[1]: share1.service: Consumed 166ms CPU time over 4.639s wall clock time, 41.3M memory peak, 6.2K incoming IP traffic, 8.3K outgoing IP traffic.4603client: (finished: waiting for success: ! systemctl is-active share1, in 0.08 seconds)4604worker1 # [ 86.373258] nix-daemon[840]: reaped child process 1088, status = succeeded4605client: waiting for success: ! systemctl is-active share24606worker1 # [ 86.540004] nix-daemon[840]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4607worker1 # [ 86.561298] nix-daemon[840]: reaped child process 1125, status = succeeded4608lb # [ 86.519604] niks3-server[1013]: 2026/09/22 10:57:59 INFO Received uploads request method=POST path=/api/pending_closures4609worker1 # [ 86.585821] nix-grpc-daemon[853]: time=2026-09-22T10:57:59.108Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4610worker1 # [ 86.587336] nix-grpc-daemon[853]: time=2026-09-22T10:57:59.109Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4611lb # [ 86.565871] niks3-server[1013]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4612worker1 # [ 86.633279] nix-grpc-daemon[853]: time=2026-09-22T10:57:59.156Z level=INFO msg="Uploading 1 narinfos"4613lb # [ 86.581170] niks3-server[1013]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4614lb # [ 86.590445] niks3-server[1013]: 2026/09/22 10:57:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4615lb # [ 86.595040] niks3-server[1013]: 2026/09/22 10:57:59 INFO Signed narinfos id=14 count=14616worker1 # [ 86.657660] nix-grpc-daemon[853]: time=2026-09-22T10:57:59.181Z level=INFO msg="Upload complete. (297ms)"4617lb # [ 86.603453] niks3-server[1013]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4618lb # [ 86.606438] niks3-server[1013]: 2026/09/22 10:57:59 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4619lb # [ 86.610605] niks3-server[1013]: 2026/09/22 10:57:59 INFO Completed upload id=144620lb # [ 86.614419] niks3-server[1013]: 2026/09/22 10:57:59 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4621worker1 # [ 86.669104] nix-grpc-daemon[774]: ts=2026-09-22T10:57:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=8 outputs=14622worker1 # [ 86.683575] nix-grpc-daemon[774]: ts=2026-09-22T10:57:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=3 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=8 outputs=14623worker1 # [ 86.689650] nix-daemon[840]: reaped child process 1086, status = succeeded4624client # [ 86.671538] systemd[1]: share2.service: Deactivated successfully.4625client # [ 86.674636] systemd[1]: share2.service: Consumed 164ms CPU time over 4.197s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4626client: (finished: waiting for success: ! systemctl is-active share2, in 1.14 seconds)4627client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4628client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4629(finished: subtest: the first client leaving does not fail the second, in 6.24 seconds)4630subtest: build hook: nix-daemon with builders = grpc://lb4631client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log4632client # [ 87.731711] systemd[1]: Started Nix Daemon.4633client # [ 87.922696] nix-daemon[1033]: accepted connection from pid 1030, user root (trusted)4634worker2 # [ 88.227914] nix-grpc-daemon[776]: ts=2026-09-22T10:58:00Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364564635worker2 # [ 88.235095] nix-grpc-daemon[776]: ts=2026-09-22T10:58:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564636worker2 # [ 88.249479] nix-grpc-daemon[776]: ts=2026-09-22T10:58:00Z level=debug event=want client=10 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04637worker2 # [ 88.256896] nix-grpc-daemon[776]: ts=2026-09-22T10:58:00Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14638worker2 # [ 88.266010] nix-grpc-daemon[776]: ts=2026-09-22T10:58:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564639worker2 # [ 88.277740] nix-grpc-daemon[776]: ts=2026-09-22T10:58:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564640worker2 # [ 88.293140] nix-grpc-daemon[776]: ts=2026-09-22T10:58:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364564641worker2 # [ 88.309112] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4642worker2 # [ 88.513893] nix-daemon[863]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)4643worker2 # [ 88.534970] nix-daemon[863]: reaped child process 1051, status = succeeded4644worker2 # [ 88.710639] nix-daemon[863]: accepted connection from pid 1053, user nix-grpc-daemon (trusted)4645worker2 # [ 88.730653] nix-daemon[863]: reaped child process 1056, status = succeeded4646lb # [ 88.645304] niks3-server[1013]: 2026/09/22 10:58:01 INFO Received uploads request method=POST path=/api/pending_closures4647worker2 # [ 88.755642] nix-grpc-daemon[914]: time=2026-09-22T10:58:01.232Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4648worker2 # [ 88.757276] nix-grpc-daemon[914]: time=2026-09-22T10:58:01.232Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4649lb # [ 88.676798] niks3-server[1013]: 2026/09/22 10:58:01 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4650worker2 # [ 88.801902] nix-grpc-daemon[914]: time=2026-09-22T10:58:01.279Z level=INFO msg="Uploading 1 narinfos"4651lb # [ 88.696740] niks3-server[1013]: 2026/09/22 10:58:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4652lb # [ 88.708503] niks3-server[1013]: 2026/09/22 10:58:01 INFO Signed narinfos id=15 count=14653lb # [ 88.715005] niks3-server[1013]: 2026/09/22 10:58:01 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4654worker2 # [ 88.823992] nix-grpc-daemon[914]: time=2026-09-22T10:58:01.302Z level=INFO msg="Upload complete. (499ms)"4655lb # [ 88.725866] niks3-server[1013]: 2026/09/22 10:58:01 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4656lb # [ 88.732094] niks3-server[1013]: 2026/09/22 10:58:01 INFO Completed upload id=154657worker2 # [ 88.832196] nix-grpc-daemon[776]: ts=2026-09-22T10:58:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=0 paths=1 nar_bytes_in=5124658lb # [ 88.735326] niks3-server[1013]: 2026/09/22 10:58:01 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4659worker2 # [ 88.847270] nix-grpc-daemon[776]: ts=2026-09-22T10:58:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564660worker2 # [ 88.854561] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4661worker2 # [ 88.859673] nix-daemon[863]: reaped child process 1046, status = succeeded4662worker2 # [ 88.872224] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4663worker2 # [ 90.310405] nix-daemon[863]: reaped child process 1060, status = succeeded4664worker2 # [ 90.488948] nix-daemon[863]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)4665worker2 # [ 90.509126] nix-daemon[863]: reaped child process 1079, status = succeeded4666lb # [ 90.418447] niks3-server[1013]: 2026/09/22 10:58:02 INFO Received uploads request method=POST path=/api/pending_closures4667worker2 # [ 90.529636] nix-grpc-daemon[914]: time=2026-09-22T10:58:03.006Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4668worker2 # [ 90.531188] nix-grpc-daemon[914]: time=2026-09-22T10:58:03.006Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4669lb # [ 90.460442] niks3-server[1013]: 2026/09/22 10:58:02 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4670worker2 # [ 90.573815] nix-grpc-daemon[914]: time=2026-09-22T10:58:03.051Z level=INFO msg="Uploading 1 narinfos"4671lb # [ 90.475425] niks3-server[1013]: 2026/09/22 10:58:02 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4672lb # [ 90.487319] niks3-server[1013]: 2026/09/22 10:58:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4673lb # [ 90.494642] niks3-server[1013]: 2026/09/22 10:58:02 INFO Signed narinfos id=16 count=14674lb # [ 90.502981] niks3-server[1013]: 2026/09/22 10:58:03 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4675worker2 # [ 90.608322] nix-grpc-daemon[914]: time=2026-09-22T10:58:03.086Z level=INFO msg="Upload complete. (324ms)"4676lb # [ 90.509764] niks3-server[1013]: 2026/09/22 10:58:03 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4677lb # [ 90.516887] niks3-server[1013]: 2026/09/22 10:58:03 INFO Completed upload id=164678lb # [ 90.520367] niks3-server[1013]: 2026/09/22 10:58:03 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4679worker2 # [ 90.620792] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=9 outputs=14680worker2 # [ 90.634689] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364564681worker2 # [ 90.638867] nix-daemon[863]: reaped child process 1058, status = succeeded4682worker1 # [ 90.606970] nix-grpc-daemon[774]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:358764683worker1 # [ 90.625052] nix-grpc-daemon[774]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4684worker1 # [ 90.648153] nix-grpc-daemon[774]: ts=2026-09-22T10:58:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_out=1284685worker2 # [ 91.027195] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364564686worker2 # [ 91.035448] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564687worker2 # [ 91.042865] nix-grpc-daemon[914]: time=2026-09-22T10:58:03.519Z level=INFO msg="All 1 paths already cached"4688worker2 # [ 91.058053] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=debug event=want client=11 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04689worker2 # [ 91.065919] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14690worker2 # [ 91.075782] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564691worker2 # [ 91.089584] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564692worker2 # [ 91.105704] nix-grpc-daemon[914]: time=2026-09-22T10:58:03.583Z level=INFO msg="All 1 paths already cached"4693worker2 # [ 91.116140] nix-grpc-daemon[776]: ts=2026-09-22T10:58:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364564694worker2 # [ 91.131487] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4695worker2 # [ 91.331449] nix-daemon[863]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)4696worker2 # [ 91.349647] nix-daemon[863]: reaped child process 1088, status = succeeded4697worker2 # [ 91.527877] nix-daemon[863]: accepted connection from pid 1090, user nix-grpc-daemon (trusted)4698worker2 # [ 91.545704] nix-daemon[863]: reaped child process 1093, status = succeeded4699lb # [ 91.456719] niks3-server[1013]: 2026/09/22 10:58:03 INFO Received uploads request method=POST path=/api/pending_closures4700worker2 # [ 91.566419] nix-grpc-daemon[914]: time=2026-09-22T10:58:04.044Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4701worker2 # [ 91.568289] nix-grpc-daemon[914]: time=2026-09-22T10:58:04.044Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4702lb # [ 91.488701] niks3-server[1013]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4703worker2 # [ 91.609087] nix-grpc-daemon[914]: time=2026-09-22T10:58:04.086Z level=INFO msg="Uploading 1 narinfos"4704lb # [ 91.501956] niks3-server[1013]: 2026/09/22 10:58:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4705lb # [ 91.513889] niks3-server[1013]: 2026/09/22 10:58:04 INFO Signed narinfos id=17 count=14706lb # [ 91.520466] niks3-server[1013]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4707lb # [ 91.527142] niks3-server[1013]: 2026/09/22 10:58:04 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4708lb # [ 91.535192] niks3-server[1013]: 2026/09/22 10:58:04 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4709lb # [ 91.538935] niks3-server[1013]: 2026/09/22 10:58:04 INFO Completed upload id=174710worker2 # [ 91.638413] nix-grpc-daemon[914]: time=2026-09-22T10:58:04.116Z level=INFO msg="Upload complete. (489ms)"4711worker2 # [ 91.648445] nix-grpc-daemon[776]: ts=2026-09-22T10:58:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=0 paths=1 nar_bytes_in=6244712worker2 # [ 91.662743] nix-grpc-daemon[776]: ts=2026-09-22T10:58:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564713worker2 # [ 91.669044] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4714worker2 # [ 91.674186] nix-daemon[863]: reaped child process 1083, status = succeeded4715worker2 # [ 91.688154] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4716lb # [ 91.861469] envoy[923]: conn peer=[2001:db8:1::1]:47752 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3003 tx=788 ms=43414717lb # [ 91.866539] envoy[923]: conn peer=[2001:db8:1::1]:47768 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1082 ms=39064718lb # [ 91.871903] envoy[923]: conn peer=[2001:db8:1::1]:47774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3444 tx=1760 ms=25264719lb # [ 91.877727] envoy[923]: conn peer=[2001:db8:1::1]:47776 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1074720worker2 # [ 92.513346] nix-daemon[863]: reaped child process 1097, status = succeeded4721worker2 # [ 92.675637] nix-daemon[863]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4722worker2 # [ 92.696123] nix-daemon[863]: reaped child process 1116, status = succeeded4723lb # [ 92.609458] niks3-server[1013]: 2026/09/22 10:58:05 INFO Received uploads request method=POST path=/api/pending_closures4724worker2 # [ 92.719464] nix-grpc-daemon[914]: time=2026-09-22T10:58:05.197Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4725worker2 # [ 92.722616] nix-grpc-daemon[914]: time=2026-09-22T10:58:05.197Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4726lb # [ 92.646445] niks3-server[1013]: 2026/09/22 10:58:05 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4727lb # [ 92.664211] niks3-server[1013]: 2026/09/22 10:58:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4728worker2 # [ 92.769281] nix-grpc-daemon[914]: time=2026-09-22T10:58:05.247Z level=INFO msg="Uploading 1 narinfos"4729lb # [ 92.674244] niks3-server[1013]: 2026/09/22 10:58:05 INFO Signed narinfos id=18 count=14730lb # [ 92.682070] niks3-server[1013]: 2026/09/22 10:58:05 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4731lb # [ 92.688362] niks3-server[1013]: 2026/09/22 10:58:05 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4732lb # [ 92.695484] niks3-server[1013]: 2026/09/22 10:58:05 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4733lb # [ 92.701079] niks3-server[1013]: 2026/09/22 10:58:05 INFO Completed upload id=184734worker2 # [ 92.800277] nix-grpc-daemon[914]: time=2026-09-22T10:58:05.278Z level=INFO msg="Upload complete. (300ms)"4735lb # [ 92.703628] niks3-server[1013]: 2026/09/22 10:58:05 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4736worker2 # [ 92.812926] nix-grpc-daemon[776]: ts=2026-09-22T10:58:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=10 outputs=14737worker2 # [ 92.829358] nix-grpc-daemon[776]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364564738worker2 # [ 92.834160] nix-daemon[863]: reaped child process 1095, status = succeeded4739worker1 # [ 92.812876] nix-grpc-daemon[774]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:358764740worker1 # [ 92.825592] nix-grpc-daemon[774]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4741worker1 # [ 92.843923] nix-grpc-daemon[774]: ts=2026-09-22T10:58:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_out=1764742worker2 # [ 93.217076] nix-grpc-daemon[776]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364564743worker1 # [ 93.176805] nix-grpc-daemon[774]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764744worker1 # [ 93.187238] nix-grpc-daemon[853]: time=2026-09-22T10:58:05.709Z level=INFO msg="All 2 paths already cached"4745worker2 # [ 93.252248] nix-grpc-daemon[776]: ts=2026-09-22T10:58:05Z level=debug event=want client=12 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04746worker2 # [ 93.260284] nix-grpc-daemon[776]: ts=2026-09-22T10:58:05Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14747worker2 # [ 93.270692] nix-grpc-daemon[776]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564748worker2 # [ 93.283061] nix-grpc-daemon[776]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564749worker2 # [ 93.299037] nix-grpc-daemon[914]: time=2026-09-22T10:58:05.776Z level=INFO msg="All 2 paths already cached"4750worker2 # [ 93.309118] nix-grpc-daemon[776]: ts=2026-09-22T10:58:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364564751worker2 # [ 93.324375] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4752worker2 # [ 93.529945] nix-daemon[863]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4753worker2 # [ 93.549819] nix-daemon[863]: reaped child process 1125, status = succeeded4754worker2 # [ 93.726544] nix-daemon[863]: accepted connection from pid 1127, user nix-grpc-daemon (trusted)4755worker2 # [ 93.746378] nix-daemon[863]: reaped child process 1130, status = succeeded4756lb # [ 93.654943] niks3-server[1013]: 2026/09/22 10:58:06 INFO Received uploads request method=POST path=/api/pending_closures4757worker2 # [ 93.764391] nix-grpc-daemon[914]: time=2026-09-22T10:58:06.242Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4758worker2 # [ 93.766320] nix-grpc-daemon[914]: time=2026-09-22T10:58:06.242Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4759lb # [ 93.685977] niks3-server[1013]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4760worker2 # [ 93.798662] nix-grpc-daemon[914]: time=2026-09-22T10:58:06.276Z level=INFO msg="Uploading 1 narinfos"4761lb # [ 93.699882] niks3-server[1013]: 2026/09/22 10:58:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4762lb # [ 93.711990] niks3-server[1013]: 2026/09/22 10:58:06 INFO Signed narinfos id=19 count=14763lb # [ 93.718536] niks3-server[1013]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4764lb # [ 93.728589] niks3-server[1013]: 2026/09/22 10:58:06 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4765worker2 # [ 93.833335] nix-grpc-daemon[914]: time=2026-09-22T10:58:06.311Z level=INFO msg="Upload complete. (489ms)"4766lb # [ 93.734214] niks3-server[1013]: 2026/09/22 10:58:06 INFO Completed upload id=194767worker2 # [ 93.838518] nix-grpc-daemon[776]: ts=2026-09-22T10:58:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=0 paths=1 nar_bytes_in=7524768lb # [ 93.738771] niks3-server[1013]: 2026/09/22 10:58:06 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4769worker2 # [ 93.844450] nix-grpc-daemon[776]: ts=2026-09-22T10:58:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564770worker2 # [ 93.853598] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4771worker2 # [ 93.855363] nix-daemon[863]: reaped child process 1120, status = succeeded4772worker2 # [ 93.865020] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4773worker2 # [ 95.087546] nix-daemon[863]: reaped child process 1134, status = succeeded4774worker2 # [ 95.242438] nix-daemon[863]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)4775worker2 # [ 95.262524] nix-daemon[863]: reaped child process 1153, status = succeeded4776lb # [ 95.176690] niks3-server[1013]: 2026/09/22 10:58:07 INFO Received uploads request method=POST path=/api/pending_closures4777worker2 # [ 95.288485] nix-grpc-daemon[914]: time=2026-09-22T10:58:07.765Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4778worker2 # [ 95.290078] nix-grpc-daemon[914]: time=2026-09-22T10:58:07.765Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4779lb # [ 95.227258] niks3-server[1013]: 2026/09/22 10:58:07 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4780worker2 # [ 95.343982] nix-grpc-daemon[914]: time=2026-09-22T10:58:07.821Z level=INFO msg="Uploading 1 narinfos"4781lb # [ 95.244280] niks3-server[1013]: 2026/09/22 10:58:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4782lb # [ 95.252319] niks3-server[1013]: 2026/09/22 10:58:07 INFO Signed narinfos id=20 count=14783lb # [ 95.258658] niks3-server[1013]: 2026/09/22 10:58:07 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4784lb # [ 95.266460] niks3-server[1013]: 2026/09/22 10:58:07 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4785lb # [ 95.272209] niks3-server[1013]: 2026/09/22 10:58:07 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4786lb # [ 95.281079] niks3-server[1013]: 2026/09/22 10:58:07 INFO Completed upload id=204787worker2 # [ 95.380291] nix-grpc-daemon[914]: time=2026-09-22T10:58:07.858Z level=INFO msg="Upload complete. (307ms)"4788lb # [ 95.284965] niks3-server[1013]: 2026/09/22 10:58:07 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4789worker2 # [ 95.397316] nix-grpc-daemon[776]: ts=2026-09-22T10:58:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=11 outputs=14790worker2 # [ 95.413577] nix-daemon[863]: reaped child process 1132, status = succeeded4791worker1 # [ 95.399362] nix-grpc-daemon[774]: ts=2026-09-22T10:58:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:358764792worker1 # [ 95.414470] nix-grpc-daemon[774]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4793worker1 # [ 95.460164] nix-grpc-daemon[774]: ts=2026-09-22T10:58:07Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:358764794worker1 # [ 95.467539] nix-grpc-daemon[774]: ts=2026-09-22T10:58:07Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_out=2324795client # [ 95.569414] nix-daemon[1033]: reaped child process 1035, status = succeeded4796client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag hook 2>/tmp/hook.log, in 8.06 seconds)4797client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4798client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4799client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4800client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.06 seconds)4801client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4802client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.04 seconds)4803(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 8.21 seconds)4804subtest: an input only one worker has still reaches the builder4805client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only4806client # this derivation will be built:4807client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4808client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4809client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag w1only, in 0.47 seconds)4810client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4811client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.22 seconds)4812worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4813worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.27 seconds)4814worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4815worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4816client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&24817client # [ 96.963540] nix-daemon[1033]: accepted connection from pid 1115, user root (trusted)4818client # this derivation will be built:4819client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4820client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4821worker1 # [ 97.228776] nix-grpc-daemon[774]: ts=2026-09-22T10:58:09Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:358764822worker1 # [ 97.237686] nix-grpc-daemon[774]: ts=2026-09-22T10:58:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764823worker1 # [ 97.451707] nix-daemon[840]: accepted connection from pid 1135, user nix-grpc-daemon (trusted)4824worker1 # [ 97.471856] nix-daemon[840]: reaped child process 1138, status = succeeded4825lb # [ 97.431922] niks3-server[1013]: 2026/09/22 10:58:09 INFO Received uploads request method=POST path=/api/pending_closures4826worker1 # [ 97.498638] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.021Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4827worker1 # [ 97.500293] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.021Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4828lb # [ 97.453549] niks3-server[1013]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4829lb # [ 97.460252] niks3-server[1013]: 2026/09/22 10:58:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4830worker1 # [ 97.518152] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.041Z level=INFO msg="Uploading 1 narinfos"4831lb # [ 97.463478] niks3-server[1013]: 2026/09/22 10:58:09 INFO Signed narinfos id=21 count=14832lb # [ 97.466503] niks3-server[1013]: 2026/09/22 10:58:09 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4833lb # [ 97.474319] niks3-server[1013]: 2026/09/22 10:58:09 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4834worker1 # [ 97.532460] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.055Z level=INFO msg="Upload complete. (296ms)"4835lb # [ 97.479674] niks3-server[1013]: 2026/09/22 10:58:09 INFO Completed upload id=214836lb # [ 97.482200] niks3-server[1013]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4837worker2 # [ 97.603912] nix-grpc-daemon[776]: ts=2026-09-22T10:58:10Z level=debug event=want client=13 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04838worker2 # [ 97.610600] nix-grpc-daemon[776]: ts=2026-09-22T10:58:10Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14839worker1 # [ 97.606564] nix-grpc-daemon[774]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764840worker1 # [ 97.629547] nix-grpc-daemon[774]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764841client # copying 2 paths to 'lb:50051'...4842client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4843worker1 # [ 97.659762] nix-grpc-daemon[774]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358764844client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4845worker1 # [ 97.680781] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4846worker1 # [ 97.920198] nix-daemon[840]: accepted connection from pid 1142, user nix-grpc-daemon (trusted)4847worker1 # [ 97.945775] nix-daemon[840]: reaped child process 1145, status = succeeded4848worker1 # [ 98.153811] nix-daemon[840]: accepted connection from pid 1147, user nix-grpc-daemon (trusted)4849worker1 # [ 98.174985] nix-daemon[840]: reaped child process 1150, status = succeeded4850lb # [ 98.133197] niks3-server[1013]: 2026/09/22 10:58:10 INFO Received uploads request method=POST path=/api/pending_closures4851lb # [ 98.146940] niks3-server[1013]: 2026/09/22 10:58:10 INFO Received uploads request method=POST path=/api/pending_closures4852worker1 # [ 98.213516] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.736Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4853worker1 # [ 98.215237] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.736Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4854worker1 # [ 98.217302] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.736Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4855lb # [ 98.214957] niks3-server[1013]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4856lb # [ 98.224765] niks3-server[1013]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4857lb # [ 98.231303] niks3-server[1013]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4858lb # [ 98.240196] niks3-server[1013]: 2026/09/22 10:58:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4859worker1 # [ 98.295179] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.816Z level=INFO msg="Uploading 3 narinfos"4860lb # [ 98.243042] niks3-server[1013]: 2026/09/22 10:58:10 INFO Signed narinfos id=23 count=24861lb # [ 98.250822] niks3-server[1013]: 2026/09/22 10:58:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4862lb # [ 98.257381] niks3-server[1013]: 2026/09/22 10:58:10 INFO Signed narinfos id=22 count=14863lb # [ 98.259231] niks3-server[1013]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4864lb # [ 98.275441] niks3-server[1013]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4865lb # [ 98.288292] niks3-server[1013]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4866lb # [ 98.296090] niks3-server[1013]: 2026/09/22 10:58:10 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4867lb # [ 98.298717] niks3-server[1013]: 2026/09/22 10:58:10 INFO Completed upload id=224868lb # [ 98.300481] niks3-server[1013]: 2026/09/22 10:58:10 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4869lb # [ 98.303425] niks3-server[1013]: 2026/09/22 10:58:10 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4870lb # [ 98.305866] niks3-server[1013]: 2026/09/22 10:58:10 INFO Completed upload id=234871worker1 # [ 98.358620] nix-grpc-daemon[853]: time=2026-09-22T10:58:10.881Z level=INFO msg="Upload complete. (653ms)"4872worker1 # [ 98.375044] nix-grpc-daemon[774]: ts=2026-09-22T10:58:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=2 nar_bytes_in=9844873worker1 # [ 98.389328] nix-grpc-daemon[774]: ts=2026-09-22T10:58:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764874worker1 # [ 98.403752] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4875worker1 # [ 98.409824] nix-daemon[840]: reaped child process 1140, status = succeeded4876client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4877worker1 # [ 98.428545] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4878client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4879client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4880worker1 # [ 99.888669] nix-daemon[840]: reaped child process 1154, status = succeeded4881worker1 # [ 100.099679] nix-daemon[840]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)4882worker1 # [ 100.121280] nix-daemon[840]: reaped child process 1173, status = succeeded4883lb # [ 100.086876] niks3-server[1013]: 2026/09/22 10:58:12 INFO Received uploads request method=POST path=/api/pending_closures4884worker1 # [ 100.159405] nix-grpc-daemon[853]: time=2026-09-22T10:58:12.679Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4885lb # [ 100.130447] niks3-server[1013]: 2026/09/22 10:58:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4886worker1 # [ 100.196957] nix-grpc-daemon[853]: time=2026-09-22T10:58:12.719Z level=INFO msg="Uploading 1 narinfos"4887lb # [ 100.142930] niks3-server[1013]: 2026/09/22 10:58:12 INFO Signed narinfos id=24 count=14888lb # [ 100.158943] niks3-server[1013]: 2026/09/22 10:58:12 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4889lb # [ 100.166017] niks3-server[1013]: 2026/09/22 10:58:12 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4890lb # [ 100.175236] niks3-server[1013]: 2026/09/22 10:58:12 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4891lb # [ 100.182282] niks3-server[1013]: 2026/09/22 10:58:12 INFO Completed upload id=244892worker1 # [ 100.237282] nix-grpc-daemon[853]: time=2026-09-22T10:58:12.760Z level=INFO msg="Upload complete. (394ms)"4893lb # [ 100.184425] niks3-server[1013]: 2026/09/22 10:58:12 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4894client # copying 1 paths...4895worker1 # [ 100.251970] nix-grpc-daemon[774]: ts=2026-09-22T10:58:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=12 outputs=14896worker2 # [ 100.308552] nix-grpc-daemon[776]: ts=2026-09-22T10:58:12Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364564897worker1 # [ 100.271939] nix-daemon[840]: reaped child process 1152, status = succeeded4898worker2 # [ 100.330771] nix-grpc-daemon[776]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4899client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4900worker2 # [ 100.391608] nix-grpc-daemon[776]: ts=2026-09-22T10:58:12Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364564901worker2 # [ 100.398841] nix-grpc-daemon[776]: ts=2026-09-22T10:58:12Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=0 paths=1 nar_bytes_out=1204902client # [ 100.469321] nix-daemon[1033]: reaped child process 1118, status = succeeded4903client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.76 seconds)4904client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&24905client # [ 100.699234] nix-daemon[1033]: accepted connection from pid 1137, user root (trusted)4906client # this derivation will be built:4907client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4908worker1 # [ 100.980356] nix-grpc-daemon[774]: ts=2026-09-22T10:58:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:358764909client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4910worker2 # [ 101.038241] nix-grpc-daemon[776]: ts=2026-09-22T10:58:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564911worker2 # [ 101.078340] nix-grpc-daemon[776]: ts=2026-09-22T10:58:13Z level=debug event=want client=14 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04912worker2 # [ 101.086967] nix-grpc-daemon[776]: ts=2026-09-22T10:58:13Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14913worker2 # [ 101.130044] nix-grpc-daemon[776]: ts=2026-09-22T10:58:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564914worker2 # [ 101.153443] nix-grpc-daemon[776]: ts=2026-09-22T10:58:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564915client # copying 1 paths to 'lb:50051'...4916worker2 # [ 101.183819] nix-grpc-daemon[776]: ts=2026-09-22T10:58:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:364564917client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4918worker2 # [ 101.197850] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4919worker2 # [ 101.218031] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 1/5)4920worker2 # [ 101.275107] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 2/5)4921worker2 # [ 101.378495] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 295 ms (attempt 3/5)4922worker2 # [ 101.681467] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 440 ms (attempt 4/5)4923lb # [ 101.859968] envoy[923]: conn peer=[2001:db8:1::1]:55694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3900 tx=1958 ms=19234924lb # [ 101.865853] envoy[923]: conn peer=[2001:db8:1::1]:55700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=1014925lb # [ 101.870888] envoy[923]: conn peer=[2001:db8:1::1]:55704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4339 tx=2304 ms=23584926lb # [ 101.878794] envoy[923]: conn peer=[2001:db8:1::1]:55710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=754927lb # [ 101.883864] envoy[923]: conn peer=[2001:db8:1::1]:55714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3892 tx=1973 ms=31854928lb # [ 101.889180] envoy[923]: conn peer=[2001:db8:1::1]:55730 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=774929worker2 # [ 102.126480] nix-grpc-daemon[776]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4930worker2 # [ 102.134769] nix-grpc-daemon[776]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4931worker2 # [ 102.142825] nix-grpc-daemon[776]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4932worker2 # [ 102.402104] nix-daemon[863]: accepted connection from pid 1170, user nix-grpc-daemon (trusted)4933worker2 # [ 102.424355] nix-daemon[863]: reaped child process 1173, status = succeeded4934worker2 # [ 102.609738] nix-daemon[863]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)4935worker2 # [ 102.630798] nix-daemon[863]: reaped child process 1178, status = succeeded4936lb # [ 102.546554] niks3-server[1013]: 2026/09/22 10:58:15 INFO Received uploads request method=POST path=/api/pending_closures4937worker2 # [ 102.660388] nix-grpc-daemon[914]: time=2026-09-22T10:58:15.137Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4938worker2 # [ 102.662136] nix-grpc-daemon[914]: time=2026-09-22T10:58:15.137Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4939lb # [ 102.593580] niks3-server[1013]: 2026/09/22 10:58:15 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4940lb # [ 102.607909] niks3-server[1013]: 2026/09/22 10:58:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4941worker2 # [ 102.713487] nix-grpc-daemon[914]: time=2026-09-22T10:58:15.191Z level=INFO msg="Uploading 1 narinfos"4942lb # [ 102.619117] niks3-server[1013]: 2026/09/22 10:58:15 INFO Signed narinfos id=25 count=14943lb # [ 102.624331] niks3-server[1013]: 2026/09/22 10:58:15 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4944lb # [ 102.637782] niks3-server[1013]: 2026/09/22 10:58:15 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4945lb # [ 102.644458] niks3-server[1013]: 2026/09/22 10:58:15 INFO Completed upload id=254946worker2 # [ 102.743949] nix-grpc-daemon[914]: time=2026-09-22T10:58:15.221Z level=INFO msg="Upload complete. (570ms)"4947lb # [ 102.648095] niks3-server[1013]: 2026/09/22 10:58:15 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4948worker2 # [ 102.753206] nix-grpc-daemon[776]: ts=2026-09-22T10:58:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=1 paths=1 nar_bytes_in=5684949worker2 # [ 102.764771] nix-grpc-daemon[776]: ts=2026-09-22T10:58:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:364564950worker2 # [ 102.772983] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4951worker2 # [ 102.783613] nix-daemon[863]: reaped child process 1163, status = succeeded4952worker2 # [ 102.786768] nix-grpc-daemon[776]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4953client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4954worker2 # [ 102.807338] nix-daemon[863]: accepted connection from pid 776, user nix-grpc-daemon (trusted)4955client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4956client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4957worker2 # [ 103.831566] nix-daemon[863]: reaped child process 1183, status = succeeded4958worker2 # [ 104.011140] nix-daemon[863]: accepted connection from pid 1199, user nix-grpc-daemon (trusted)4959worker2 # [ 104.030605] nix-daemon[863]: reaped child process 1202, status = succeeded4960lb # [ 103.946292] niks3-server[1013]: 2026/09/22 10:58:16 INFO Received uploads request method=POST path=/api/pending_closures4961worker2 # [ 104.060538] nix-grpc-daemon[914]: time=2026-09-22T10:58:16.535Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4962lb # [ 103.974892] niks3-server[1013]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4963lb # [ 103.979743] niks3-server[1013]: 2026/09/22 10:58:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4964worker2 # [ 104.083047] nix-grpc-daemon[914]: time=2026-09-22T10:58:16.560Z level=INFO msg="Uploading 1 narinfos"4965lb # [ 103.987088] niks3-server[1013]: 2026/09/22 10:58:16 INFO Signed narinfos id=26 count=14966lb # [ 103.990520] niks3-server[1013]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4967lb # [ 103.999975] niks3-server[1013]: 2026/09/22 10:58:16 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4968lb # [ 104.004617] niks3-server[1013]: 2026/09/22 10:58:16 INFO Completed upload id=264969worker2 # [ 104.104354] nix-grpc-daemon[914]: time=2026-09-22T10:58:16.582Z level=INFO msg="Upload complete. (287ms)"4970lb # [ 104.006576] niks3-server[1013]: 2026/09/22 10:58:16 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4971worker2 # [ 104.113414] nix-grpc-daemon[776]: ts=2026-09-22T10:58:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=13 outputs=14972client # copying 1 paths...4973worker2 # [ 104.125724] nix-grpc-daemon[776]: ts=2026-09-22T10:58:16Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:364564974client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4975worker2 # [ 104.128489] nix-daemon[863]: reaped child process 1180, status = succeeded4976worker1 # [ 104.114798] nix-grpc-daemon[774]: ts=2026-09-22T10:58:16Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:358764977worker1 # [ 104.128558] nix-grpc-daemon[774]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4978worker1 # [ 104.148157] nix-grpc-daemon[774]: ts=2026-09-22T10:58:16Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_out=1204979client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.79 seconds)4980client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&24981client # [ 104.297820] nix-daemon[1033]: reaped child process 1140, status = succeeded4982client # [ 104.518105] nix-daemon[1033]: accepted connection from pid 1159, user root (trusted)4983client # this derivation will be built:4984client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4985worker2 # [ 104.846212] nix-grpc-daemon[776]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:364564986client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux'...4987worker2 # [ 104.859627] nix-grpc-daemon[776]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:364564988worker2 # [ 104.871607] nix-grpc-daemon[914]: time=2026-09-22T10:58:17.349Z level=INFO msg="All 1 paths already cached"4989worker2 # [ 104.891566] nix-grpc-daemon[776]: ts=2026-09-22T10:58:17Z level=debug event=want client=15 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04990worker2 # [ 104.898610] nix-grpc-daemon[776]: ts=2026-09-22T10:58:17Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14991worker1 # [ 104.896180] nix-grpc-daemon[774]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358764992worker1 # [ 104.916778] nix-grpc-daemon[774]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:358764993client # copying 1 paths to 'lb:50051'...4994worker1 # [ 104.937645] nix-grpc-daemon[853]: time=2026-09-22T10:58:17.458Z level=INFO msg="All 1 paths already cached"4995client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4996worker1 # [ 104.944213] nix-grpc-daemon[774]: ts=2026-09-22T10:58:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:358764997worker1 # [ 104.965220] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4998worker1 # [ 105.204240] nix-daemon[840]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)4999worker1 # [ 105.227885] nix-daemon[840]: reaped child process 1181, status = succeeded5000worker1 # [ 105.412552] nix-daemon[840]: accepted connection from pid 1183, user nix-grpc-daemon (trusted)5001worker1 # [ 105.433630] nix-daemon[840]: reaped child process 1186, status = succeeded5002lb # [ 105.391608] niks3-server[1013]: 2026/09/22 10:58:17 INFO Received uploads request method=POST path=/api/pending_closures5003worker1 # [ 105.457789] nix-grpc-daemon[853]: time=2026-09-22T10:58:17.980Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5004worker1 # [ 105.459333] nix-grpc-daemon[853]: time=2026-09-22T10:58:17.980Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5005lb # [ 105.427014] niks3-server[1013]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5006worker1 # [ 105.491900] nix-grpc-daemon[853]: time=2026-09-22T10:58:18.015Z level=INFO msg="Uploading 1 narinfos"5007lb # [ 105.438900] niks3-server[1013]: 2026/09/22 10:58:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5008lb # [ 105.445162] niks3-server[1013]: 2026/09/22 10:58:17 INFO Signed narinfos id=27 count=15009lb # [ 105.446848] niks3-server[1013]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5010lb # [ 105.454150] niks3-server[1013]: 2026/09/22 10:58:17 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5011lb # [ 105.460058] niks3-server[1013]: 2026/09/22 10:58:17 INFO Completed upload id=275012worker1 # [ 105.514101] nix-grpc-daemon[853]: time=2026-09-22T10:58:18.037Z level=INFO msg="Upload complete. (525ms)"5013lb # [ 105.461985] niks3-server[1013]: 2026/09/22 10:58:17 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5014worker1 # [ 105.521015] nix-grpc-daemon[774]: ts=2026-09-22T10:58:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=0 paths=1 nar_bytes_in=5685015worker1 # [ 105.537526] nix-grpc-daemon[774]: ts=2026-09-22T10:58:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:358765016worker1 # [ 105.543866] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5017worker1 # [ 105.549201] nix-daemon[840]: reaped child process 1176, status = succeeded5018client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5019client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5020worker1 # [ 105.563111] nix-daemon[840]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5021client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5022worker1 # [ 106.395887] nix-daemon[840]: reaped child process 1190, status = succeeded5023worker1 # [ 106.576117] nix-daemon[840]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5024worker1 # [ 106.598640] nix-daemon[840]: reaped child process 1209, status = succeeded5025lb # [ 106.558519] niks3-server[1013]: 2026/09/22 10:58:19 INFO Received uploads request method=POST path=/api/pending_closures5026worker1 # [ 106.624478] nix-grpc-daemon[853]: time=2026-09-22T10:58:19.146Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5027lb # [ 106.603987] niks3-server[1013]: 2026/09/22 10:58:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5028worker1 # [ 106.666733] nix-grpc-daemon[853]: time=2026-09-22T10:58:19.189Z level=INFO msg="Uploading 1 narinfos"5029lb # [ 106.615891] niks3-server[1013]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5030lb # [ 106.623941] niks3-server[1013]: 2026/09/22 10:58:19 INFO Signed narinfos id=28 count=15031lb # [ 106.626902] niks3-server[1013]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5032lb # [ 106.632848] niks3-server[1013]: 2026/09/22 10:58:19 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5033lb # [ 106.640635] niks3-server[1013]: 2026/09/22 10:58:19 INFO Completed upload id=285034worker1 # [ 106.695664] nix-grpc-daemon[853]: time=2026-09-22T10:58:19.219Z level=INFO msg="Upload complete. (322ms)"5035lb # [ 106.643119] niks3-server[1013]: 2026/09/22 10:58:19 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5036worker1 # [ 106.705154] nix-grpc-daemon[774]: ts=2026-09-22T10:58:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35876 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=14 outputs=15037worker1 # [ 106.725074] nix-daemon[840]: reaped child process 1188, status = succeeded5038client # copying 1 paths...5039worker1 # [ 106.754485] nix-grpc-daemon[774]: ts=2026-09-22T10:58:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:358765040client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5041worker2 # [ 106.834605] nix-grpc-daemon[776]: ts=2026-09-22T10:58:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:364565042worker2 # [ 106.854356] nix-grpc-daemon[776]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5043worker2 # [ 106.876932] nix-grpc-daemon[776]: ts=2026-09-22T10:58:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36456 duration_s=0 paths=1 nar_bytes_out=1205044client # [ 106.925621] nix-daemon[1033]: reaped child process 1162, status = succeeded5045client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.68 seconds)5046(finished: subtest: an input only one worker has still reaches the builder, in 11.20 seconds)5047subtest: upload whose reference the worker lacks is completed from the cache5048client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache5049client # this derivation will be built:5050client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5051client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5052client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A input --argstr tag viacache, in 0.62 seconds)5053client: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache5054client # this derivation will be built:5055client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5056client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5057client: (finished: must succeed: nix-build --no-out-link /nix/store/d2v6gfscfnwmpkifqgfkvs9vxxfraw1i-dep.nix -A referrer --argstr tag viacache, in 0.46 seconds)5058client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5059client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.18 seconds)5060worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5061worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.26 seconds)5062worker2 # [ 108.615488] systemd[1]: Stopping Nix worker-protocol over gRPC...5063worker2 # [ 109.511981] nix-grpc-daemon[776]: ts=2026-09-22T10:58:21Z level=info event=signal_exit5064lb # [ 109.662632] niks3-server[1013]: 2026/09/22 10:58:22 INFO lead: released remote=192.168.1.4:601105065lb # [ 110.156630] niks3-server[1013]: 2026/09/22 10:58:22 INFO lead: acquired remote=192.168.1.3:453985066worker1 # [ 110.217059] nix-grpc-daemon[774]: ts=2026-09-22T10:58:22Z level=info event=scheduler_take_over5067worker1 # [ 110.720672] nix-grpc-daemon[774]: ts=2026-09-22T10:58:23Z level=info event=scheduler_restarting addr=lb:500515068worker2 # [ 110.770644] nix-grpc-daemon[776]: ts=2026-09-22T10:58:23Z level=info event=scheduler_restarting5069worker2 # [ 110.771228] nix-grpc-daemon[776]: ts=2026-09-22T10:58:23Z level=info event=scheduler_restarting addr=lb:500515070worker2 # [ 110.771535] nix-grpc-daemon[776]: ts=2026-09-22T10:58:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:509105071worker2 # [ 110.771700] nix-grpc-daemon[776]: ts=2026-09-22T10:58:23Z level=info event=worker_gone addr=192.168.1.4:500515072worker2 # [ 110.771885] nix-grpc-daemon[776]: ts=2026-09-22T10:58:23Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:509105073worker2 # [ 110.793850] nix-grpc-daemon[776]: ts=2026-09-22T10:58:23Z level=info event=worker_gone addr=192.168.1.3:500515074worker2 # [ 110.832239] nix-daemon[863]: reaped child process 865, status = succeeded5075worker1 # [ 110.927503] nix-grpc-daemon[774]: ts=2026-09-22T10:58:23Z level=info event=scheduler_connected addr=lb:500515076worker1 # [ 110.933512] nix-grpc-daemon[774]: ts=2026-09-22T10:58:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5077worker2 # [ 110.994461] nix-grpc-daemon[776]: ts=2026-09-22T10:58:23Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5078worker1 # [ 111.134404] nix-grpc-daemon[774]: ts=2026-09-22T10:58:23Z level=info event=scheduler_connected addr=lb:500515079worker1 # [ 111.139920] nix-grpc-daemon[774]: ts=2026-09-22T10:58:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5080worker1 # [ 111.342717] nix-grpc-daemon[774]: ts=2026-09-22T10:58:23Z level=info event=scheduler_connected addr=lb:500515081worker1 # [ 111.346983] nix-grpc-daemon[774]: ts=2026-09-22T10:58:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5082worker1 # [ 111.543568] nix-grpc-daemon[774]: ts=2026-09-22T10:58:24Z level=info event=scheduler_connected addr=lb:500515083worker1 # [ 111.565487] nix-grpc-daemon[774]: ts=2026-09-22T10:58:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:356065084worker1 # [ 111.570963] nix-grpc-daemon[774]: ts=2026-09-22T10:58:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05085worker2 # [ 111.636706] nix-grpc-daemon[776]: E0000 00:00:1790074704.114406 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5086worker2 # [ 111.706024] nix-grpc-daemon[776]: E0000 00:00:1790074704.183828 821 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5087lb # [ 111.864189] envoy[923]: conn peer=[2001:db8:1::1]:59458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1045088lb # [ 111.864521] envoy[923]: conn peer=[2001:db8:1::1]:59450 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=1902 ms=32365089lb # [ 111.864629] envoy[923]: conn peer=[2001:db8:1::1]:59472 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1105090lb # [ 111.864813] envoy[923]: conn peer=[2001:db8:1::1]:59470 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3728 tx=1873 ms=20925091lb # [ 111.864939] envoy[923]: conn peer=[2001:db8:1::4]:60876 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=68380 tx=3682 ms=884335092worker2 # [ 112.489295] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5093worker2 # [ 112.492593] systemd[1]: Stopped Nix worker-protocol over gRPC.5094worker2 # [ 112.495110] systemd[1]: nix-grpc-daemon.service: Consumed 3.078s CPU time over 1min 30.364s wall clock time, 59.4M memory peak, 80.5K incoming IP traffic, 215.4K outgoing IP traffic.5095worker2 # [ 112.511925] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5096worker2 # [ 112.516599] systemd[1]: Closed Nix worker-protocol over gRPC.5097worker2 # [ 112.518383] systemd[1]: nix-grpc-daemon.socket: Consumed 864us CPU time over 1min 40.336s wall clock time, 256K memory peak, 103K incoming IP traffic, 84.4K outgoing IP traffic.5098lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5099lb: must succeed: curl -sf localhost:9901/clusters5100lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5101lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.10 seconds)5102client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25103worker1 # [ 112.853531] nix-grpc-daemon[774]: ts=2026-09-22T10:58:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:358765104worker1 # [ 112.864227] nix-grpc-daemon[774]: ts=2026-09-22T10:58:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:358765105client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5106client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.32 seconds)5107worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5108worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.04 seconds)5109worker2 # [ 113.053768] systemd[1]: Listening on Nix worker-protocol over gRPC.5110worker1 # [ 113.092832] systemd[1]: Stopping Nix worker-protocol over gRPC...5111worker1 # [ 113.318530] nix-grpc-daemon[774]: ts=2026-09-22T10:58:25Z level=info event=signal_exit5112worker2 # [ 113.476188] systemd[1]: Starting scheduler-token.service...5113worker2 # [ 113.582059] systemd[1]: scheduler-token.service: Deactivated successfully.5114worker2 # [ 113.588862] systemd[1]: Finished scheduler-token.service.5115worker2 # [ 113.590466] systemd[1]: scheduler-token.service: Consumed 53ms CPU time over 108ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5116worker2 # [ 113.598759] systemd[1]: Starting Nix worker-protocol over gRPC...5117lb # [ 113.531625] niks3-server[1013]: 2026/09/22 10:58:26 INFO lead: released remote=192.168.1.3:453985118worker2 # [ 113.785780] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5119worker2 # [ 113.788482] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5120worker2 # [ 113.792570] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15121worker2 # [ 113.800100] nix-grpc-daemon[1229]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5122worker2 # [ 113.801657] nix-grpc-daemon[1229]: E0000 00:00:1790074706.276745 1229 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5123worker2 # [ 113.814262] nix-grpc-daemon[1229]: E0000 00:00:1790074706.283797 1229 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5124worker2 # [ 113.819227] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5125worker2 # [ 113.823323] systemd[1]: Started Nix worker-protocol over gRPC.5126lb # [ 113.725227] niks3-server[1013]: 2026/09/22 10:58:26 INFO lead: acquired remote=192.168.1.4:370805127worker2 # [ 113.828068] nix-grpc-daemon[1229]: E0000 00:00:1790074706.305602 1268 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5128worker2 # [ 113.860209] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_take_over5129worker2 # [ 113.861449] nix-grpc-daemon[1229]: E0000 00:00:1790074706.324553 1268 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5130worker2 # [ 113.866181] nix-grpc-daemon[1229]: E0000 00:00:1790074706.324707 1268 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5131worker2 # [ 113.869143] nix-grpc-daemon[1229]: E0000 00:00:1790074706.343419 1268 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5132worker2 # [ 113.879540] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_connected addr=lb:500515133worker2 # [ 113.881348] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5134worker2 # [ 114.083150] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_connected addr=lb:500515135worker2 # [ 114.088285] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5136worker2 # [ 114.289701] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_connected addr=lb:500515137worker2 # [ 114.295312] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5138worker2 # [ 114.496554] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_connected addr=lb:500515139worker2 # [ 114.501808] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5140worker1 # [ 114.593676] nix-grpc-daemon[774]: ts=2026-09-22T10:58:27Z level=info event=scheduler_restarting5141worker1 # [ 114.597878] nix-grpc-daemon[774]: ts=2026-09-22T10:58:27Z level=info event=scheduler_restarting addr=lb:500515142worker1 # [ 114.598057] nix-grpc-daemon[774]: ts=2026-09-22T10:58:27Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:356065143worker1 # [ 114.598200] nix-grpc-daemon[774]: ts=2026-09-22T10:58:27Z level=info event=worker_gone addr=192.168.1.3:500515144worker1 # [ 114.625606] nix-daemon[840]: reaped child process 842, status = succeeded5145worker2 # [ 114.703164] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:27Z level=info event=scheduler_connected addr=lb:500515146worker2 # [ 114.708764] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5147worker1 # [ 114.813071] nix-grpc-daemon[774]: ts=2026-09-22T10:58:27Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5148worker2 # [ 114.911582] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:27Z level=info event=scheduler_connected addr=lb:500515149worker2 # [ 114.928157] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:27Z level=info event=oidc_jwks provider=mock keys=15150worker2 # [ 114.933805] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:27Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:485405151worker2 # [ 114.939184] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05152worker1 # [ 115.468862] nix-grpc-daemon[774]: E0000 00:00:1790074707.991997 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5153worker1 # [ 115.513817] nix-grpc-daemon[774]: E0000 00:00:1790074708.037143 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5154worker1 # [ 116.368855] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5155worker1 # [ 116.372787] systemd[1]: Stopped Nix worker-protocol over gRPC.5156worker1 # [ 116.381352] systemd[1]: nix-grpc-daemon.service: Consumed 4.117s CPU time over 1min 34.457s wall clock time, 59.8M memory peak, 97.1K incoming IP traffic, 173.1K outgoing IP traffic.5157worker1 # [ 116.398144] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5158worker1 # [ 116.403531] systemd[1]: Closed Nix worker-protocol over gRPC.5159worker1 # [ 116.405933] systemd[1]: nix-grpc-daemon.socket: Consumed 872us CPU time over 1min 44.072s wall clock time, 256K memory peak, 99.2K incoming IP traffic, 81.7K outgoing IP traffic.5160lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5161lb: must succeed: curl -sf localhost:9901/clusters5162lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5163lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)5164client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25165worker2 # [ 116.819693] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485565166worker2 # [ 116.837656] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5167worker2 # [ 116.869378] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 1/5)5168worker2 # [ 116.914843] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5)5169worker2 # [ 116.940111] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 241 ms (attempt 3/5)5170worker2 # [ 117.186646] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 4/5)5171worker2 # [ 117.218518] nix-grpc-daemon[1229]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5172worker2 # [ 117.223037] nix-grpc-daemon[1229]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5173client # copying 2 paths...5174worker2 # [ 117.244323] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485565175client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5176client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5177worker2 # [ 117.259813] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5178worker2 # [ 117.275964] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:29Z level=info event=niks3_push_started pid=12895179worker2 # [ 117.470395] nix-grpc-daemon[1289]: time=2026-09-22T10:58:29.947Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5180worker2 # [ 117.651753] nix-daemon[863]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)5181worker2 # [ 117.671278] nix-daemon[863]: reaped child process 1305, status = succeeded5182lb # [ 117.586545] niks3-server[1013]: 2026/09/22 10:58:30 INFO Received uploads request method=POST path=/api/pending_closures5183lb # [ 117.597592] niks3-server[1013]: 2026/09/22 10:58:30 INFO Received uploads request method=POST path=/api/pending_closures5184worker2 # [ 117.708098] nix-grpc-daemon[1289]: time=2026-09-22T10:58:30.184Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5185worker2 # [ 117.709664] nix-grpc-daemon[1289]: time=2026-09-22T10:58:30.184Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5186worker2 # [ 117.711371] nix-grpc-daemon[1289]: time=2026-09-22T10:58:30.184Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5187lb # [ 117.648631] niks3-server[1013]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5188lb # [ 117.655071] niks3-server[1013]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5189lb # [ 117.663125] niks3-server[1013]: 2026/09/22 10:58:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5190lb # [ 117.669919] niks3-server[1013]: 2026/09/22 10:58:30 INFO Signed narinfos id=29 count=15191lb # [ 117.671013] niks3-server[1013]: 2026/09/22 10:58:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5192lb # [ 117.675724] niks3-server[1013]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5193lb # [ 117.680226] niks3-server[1013]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5194lb # [ 117.683956] niks3-server[1013]: 2026/09/22 10:58:30 INFO Signed narinfos id=30 count=25195worker2 # [ 117.785840] nix-grpc-daemon[1289]: time=2026-09-22T10:58:30.261Z level=INFO msg="Uploading 3 narinfos"5196lb # [ 117.705680] niks3-server[1013]: 2026/09/22 10:58:30 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5197lb # [ 117.712366] niks3-server[1013]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5198lb # [ 117.714084] niks3-server[1013]: 2026/09/22 10:58:30 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5199lb # [ 117.721069] niks3-server[1013]: 2026/09/22 10:58:30 INFO Completed upload id=305200lb # [ 117.722094] niks3-server[1013]: 2026/09/22 10:58:30 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5201lb # [ 117.726930] niks3-server[1013]: 2026/09/22 10:58:30 INFO Completed upload id=295202worker2 # [ 117.825082] nix-grpc-daemon[1289]: time=2026-09-22T10:58:30.302Z level=INFO msg="Upload complete. (350ms)"5203worker2 # [ 117.837550] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48556 duration_s=0 paths=2 nar_bytes_in=3125204worker2 # [ 117.858753] nix-daemon[863]: reaped child process 1287, status = succeeded5205client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.34 seconds)5206worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5207worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.04 seconds)5208worker1 # [ 117.958562] systemd[1]: Listening on Nix worker-protocol over gRPC.5209worker1 # [ 117.967331] systemd[1]: Starting Nix worker-protocol over gRPC...5210worker1 # [ 118.163707] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5211worker1 # [ 118.166332] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5212worker1 # [ 118.170588] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15213worker1 # [ 118.179861] nix-grpc-daemon[1232]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5214worker1 # [ 118.187571] nix-grpc-daemon[1232]: E0000 00:00:1790074710.701047 1232 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5215worker1 # [ 118.192084] nix-grpc-daemon[1232]: E0000 00:00:1790074710.703210 1232 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5216worker1 # [ 118.196005] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5217worker1 # [ 118.199332] systemd[1]: Started Nix worker-protocol over gRPC.5218lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5219lb: must succeed: curl -sf localhost:9901/clusters5220worker1 # [ 118.204109] nix-grpc-daemon[1232]: E0000 00:00:1790074710.726110 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5221worker1 # [ 118.232115] nix-grpc-daemon[1232]: E0000 00:00:1790074710.743968 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5222worker1 # [ 118.234486] nix-grpc-daemon[1232]: E0000 00:00:1790074710.744181 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5223worker1 # [ 118.241303] nix-grpc-daemon[1232]: E0000 00:00:1790074710.763730 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5224worker1 # [ 118.255877] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:30Z level=info event=scheduler_connected addr=lb:500515225worker2 # [ 118.302727] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:485405226worker2 # [ 118.308619] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05227lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)5228lb: must succeed: curl -sf localhost:9901/clusters5229lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5230lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.16 seconds)5231(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 12.38 seconds)5232subtest: a niks3 restart does not cost the leader its role5233lb: must succeed: curl -sf localhost:9901/clusters5234lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5235lb: must succeed: curl -sf localhost:9901/clusters5236lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5237lb # [ 119.452307] systemd[1]: Stopping niks3 server...5238lb # [ 119.452948] niks3-server[1013]: 2026/09/22 10:58:31 INFO Shutdown signal received, draining in-flight requests timeout=10s5239lb # [ 119.457383] niks3-server[1013]: 2026/09/22 10:58:31 INFO lead: released remote=192.168.1.4:370805240worker1 # [ 119.512244] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:32Z level=info event=scheduler_lock_lost status=2005241worker2 # [ 119.557872] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:32Z level=info event=scheduler_lock_lost status=2005242lb # [ 119.475625] systemd[1]: niks3.service: Deactivated successfully.5243lb # [ 119.478271] systemd[1]: Stopped niks3 server.5244lb # [ 119.478978] systemd[1]: niks3.service: Consumed 671ms CPU time over 1min 29.859s wall clock time, 22.7M memory peak, 80.1K incoming IP traffic, 115.8K outgoing IP traffic.5245lb # [ 119.489445] systemd[1]: Starting niks3 server...5246lb # [ 119.568590] niks3-server[1160]: 2026/09/22 10:58:32 goose: no migrations to run. current version: 202609200000005247lb # [ 119.577235] niks3-server[1160]: 2026/09/22 10:58:32 OK 1_commit_pending_closure.sql (6.66ms)5248lb # [ 119.581138] niks3-server[1160]: 2026/09/22 10:58:32 OK 2_object_stats_trigger.sql (2.11ms)5249lb # [ 119.582576] niks3-server[1160]: 2026/09/22 10:58:32 goose: up to current file version: 25250lb # [ 119.584589] niks3-server[1160]: 2026/09/22 10:58:32 INFO Loaded signing key name=farm-test-1 path=/nix/store/8lzz3wcf17v05vfdp1zcfaa9hipw3f70-key5251lb # [ 119.596993] niks3-server[1160]: 2026/09/22 10:58:32 INFO Read proxy enabled — serving cache objects from S35252lb # [ 119.598363] niks3-server[1160]: 2026/09/22 10:58:32 INFO Using socket-activated listener address=0.0.0.0:57515253lb # [ 119.602871] niks3-server[1160]: 2026/09/22 10:58:32 INFO systemd watchdog enabled interval=15s5254lb # [ 119.604312] niks3-server[1160]: 2026/09/22 10:58:32 INFO Starting HTTP server address=0.0.0.0:57515255lb # [ 119.605501] systemd[1]: Started niks3 server.5256lb: waiting for unit niks3.service5257lb: (finished: waiting for unit niks3.service, in 0.04 seconds)5258lb: must succeed: curl -sf localhost:9901/clusters5259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5260worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5261worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5262lb: must succeed: curl -sf localhost:9901/clusters5263lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5264lb: must succeed: curl -sf localhost:9901/clusters5265lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5266worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5267worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5268client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&25269worker1 # [ 120.299865] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:356385270worker1 # [ 120.315531] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5271client # don't know how to build these paths:5272client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5273worker1 # [ 120.338647] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356385274worker1 # [ 120.362728] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)5275worker1 # [ 120.458661] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 2/5)5276worker1 # [ 120.503348] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 254 ms (attempt 3/5)5277lb # [ 120.463595] niks3-server[1160]: 2026/09/22 10:58:32 INFO lead: acquired remote=192.168.1.4:371425278worker1 # [ 120.766474] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 387 ms (attempt 4/5)5279worker1 # [ 121.161944] nix-grpc-daemon[1232]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5280worker1 # [ 121.168613] nix-grpc-daemon[1232]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5281worker2 # [ 121.260405] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:33Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05282worker2 # [ 121.260663] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:33Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15283worker2 # [ 121.305147] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485565284worker2 # [ 121.325100] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485565285client # copying 1 paths to 'lb:50051'...5286client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5287worker2 # [ 121.341207] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485565288worker2 # [ 121.357251] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5289worker2 # [ 121.570972] nix-daemon[863]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5290worker2 # [ 121.593398] nix-daemon[863]: reaped child process 1327, status = succeeded5291worker2 # [ 121.763959] nix-daemon[863]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5292worker2 # [ 121.785269] nix-daemon[863]: reaped child process 1332, status = succeeded5293lb # [ 121.697770] niks3-server[1160]: 2026/09/22 10:58:34 INFO Received uploads request method=POST path=/api/pending_closures5294worker2 # [ 121.808591] nix-grpc-daemon[1289]: time=2026-09-22T10:58:34.285Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5295worker2 # [ 121.810147] nix-grpc-daemon[1289]: time=2026-09-22T10:58:34.285Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5296lb # [ 121.729978] niks3-server[1160]: 2026/09/22 10:58:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5297lb # [ 121.738742] niks3-server[1160]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5298lb # [ 121.761092] niks3-server[1160]: 2026/09/22 10:58:34 INFO Signed narinfos id=31 count=15299worker2 # [ 121.863388] nix-grpc-daemon[1289]: time=2026-09-22T10:58:34.340Z level=INFO msg="Uploading 1 narinfos"5300lb # [ 121.779058] niks3-server[1160]: 2026/09/22 10:58:34 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5301lb # [ 121.785996] niks3-server[1160]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5302lb # [ 121.788677] niks3-server[1160]: 2026/09/22 10:58:34 INFO Completed upload id=315303worker2 # [ 121.888583] nix-grpc-daemon[1289]: time=2026-09-22T10:58:34.366Z level=INFO msg="Upload complete. (513ms)"5304lb # [ 121.794833] niks3-server[1160]: 2026/09/22 10:58:34 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5305worker2 # [ 121.898186] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48556 duration_s=0 paths=1 nar_bytes_in=5445306worker2 # [ 121.912204] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485565307worker2 # [ 121.918492] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5308worker2 # [ 121.923232] nix-daemon[863]: reaped child process 1322, status = succeeded5309client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5310worker2 # [ 121.934602] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5311client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5312lb # [ 121.862649] envoy[923]: conn peer=[2001:db8:1::1]:36104 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=523 ms=355313lb # [ 121.867702] envoy[923]: conn peer=[2001:db8:1::3]:49776 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=14638 tx=4013 ms=924435314lb # [ 121.872960] envoy[923]: conn peer=[2001:db8:1::1]:36106 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1007 tx=305 ms=10335315client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5316client # farm-a> LOG-farm-a-after-niks3-restart5317worker2 # [ 122.768597] nix-daemon[863]: reaped child process 1336, status = succeeded5318worker2 # [ 122.947870] nix-daemon[863]: accepted connection from pid 1352, user nix-grpc-daemon (trusted)5319worker2 # [ 122.966624] nix-daemon[863]: reaped child process 1355, status = succeeded5320lb # [ 122.881555] niks3-server[1160]: 2026/09/22 10:58:35 INFO Received uploads request method=POST path=/api/pending_closures5321worker2 # [ 122.996454] nix-grpc-daemon[1289]: time=2026-09-22T10:58:35.473Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5322worker2 # [ 122.998033] nix-grpc-daemon[1289]: time=2026-09-22T10:58:35.473Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5323lb # [ 122.917669] niks3-server[1160]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5324lb # [ 122.924162] niks3-server[1160]: 2026/09/22 10:58:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5325worker2 # [ 123.030865] nix-grpc-daemon[1289]: time=2026-09-22T10:58:35.508Z level=INFO msg="Uploading 1 narinfos"5326lb # [ 122.931236] niks3-server[1160]: 2026/09/22 10:58:35 INFO Signed narinfos id=32 count=15327lb # [ 122.937365] niks3-server[1160]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5328lb # [ 122.941877] niks3-server[1160]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5329lb # [ 122.947006] niks3-server[1160]: 2026/09/22 10:58:35 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5330lb # [ 122.953651] niks3-server[1160]: 2026/09/22 10:58:35 INFO Completed upload id=325331worker2 # [ 123.053263] nix-grpc-daemon[1289]: time=2026-09-22T10:58:35.531Z level=INFO msg="Upload complete. (306ms)"5332lb # [ 122.955691] niks3-server[1160]: 2026/09/22 10:58:35 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5333worker2 # [ 123.062402] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48556 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15334worker2 # [ 123.073155] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:35Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05335worker2 # [ 123.075024] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:35Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15336worker2 # [ 123.077405] nix-daemon[863]: reaped child process 1334, status = succeeded5337worker2 # [ 123.116874] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485565338worker2 # [ 123.131147] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485565339worker2 # [ 123.147562] nix-grpc-daemon[1289]: time=2026-09-22T10:58:35.625Z level=INFO msg="All 1 paths already cached"5340client # copying 1 paths to 'lb:50051'...5341client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5342worker2 # [ 123.154716] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485565343worker2 # [ 123.167852] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5344worker2 # [ 123.367979] nix-daemon[863]: accepted connection from pid 1361, user nix-grpc-daemon (trusted)5345worker2 # [ 123.386640] nix-daemon[863]: reaped child process 1364, status = succeeded5346worker2 # [ 123.568974] nix-daemon[863]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5347worker2 # [ 123.587637] nix-daemon[863]: reaped child process 1369, status = succeeded5348lb # [ 123.500528] niks3-server[1160]: 2026/09/22 10:58:36 INFO Received uploads request method=POST path=/api/pending_closures5349worker2 # [ 123.611915] nix-grpc-daemon[1289]: time=2026-09-22T10:58:36.089Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5350worker2 # [ 123.613595] nix-grpc-daemon[1289]: time=2026-09-22T10:58:36.089Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5351lb # [ 123.531517] niks3-server[1160]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5352lb # [ 123.544960] niks3-server[1160]: 2026/09/22 10:58:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5353worker2 # [ 123.650175] nix-grpc-daemon[1289]: time=2026-09-22T10:58:36.127Z level=INFO msg="Uploading 1 narinfos"5354lb # [ 123.557438] niks3-server[1160]: 2026/09/22 10:58:36 INFO Signed narinfos id=33 count=15355lb # [ 123.564411] niks3-server[1160]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5356lb # [ 123.577929] niks3-server[1160]: 2026/09/22 10:58:36 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5357lb # [ 123.582803] niks3-server[1160]: 2026/09/22 10:58:36 INFO Completed upload id=335358worker2 # [ 123.682518] nix-grpc-daemon[1289]: time=2026-09-22T10:58:36.160Z level=INFO msg="Upload complete. (499ms)"5359lb # [ 123.586707] niks3-server[1160]: 2026/09/22 10:58:36 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5360worker2 # [ 123.692340] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48556 duration_s=0 paths=1 nar_bytes_in=6565361worker2 # [ 123.707244] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485565362worker2 # [ 123.713842] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5363worker2 # [ 123.718848] nix-daemon[863]: reaped child process 1359, status = succeeded5364client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5365client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5366worker2 # [ 123.732148] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5367client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5368worker2 # [ 124.555814] nix-daemon[863]: reaped child process 1373, status = succeeded5369client # farm-b> LOG-farm-b-after-niks3-restart5370worker2 # [ 124.718822] nix-daemon[863]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)5371worker2 # [ 124.739843] nix-daemon[863]: reaped child process 1392, status = succeeded5372lb # [ 124.654488] niks3-server[1160]: 2026/09/22 10:58:37 INFO Received uploads request method=POST path=/api/pending_closures5373worker2 # [ 124.768884] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.245Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5374worker2 # [ 124.770409] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.245Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5375lb # [ 124.696183] niks3-server[1160]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5376lb # [ 124.709289] niks3-server[1160]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5377lb # [ 124.717598] niks3-server[1160]: 2026/09/22 10:58:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5378worker2 # [ 124.822428] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.300Z level=INFO msg="Uploading 1 narinfos"5379lb # [ 124.727207] niks3-server[1160]: 2026/09/22 10:58:37 INFO Signed narinfos id=34 count=15380lb # [ 124.732920] niks3-server[1160]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5381lb # [ 124.740619] niks3-server[1160]: 2026/09/22 10:58:37 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5382lb # [ 124.748774] niks3-server[1160]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5383lb # [ 124.753303] niks3-server[1160]: 2026/09/22 10:58:37 INFO Completed upload id=345384worker2 # [ 124.853120] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.331Z level=INFO msg="Upload complete. (308ms)"5385worker2 # [ 124.870539] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48556 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15386worker2 # [ 124.887479] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:37Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05387worker2 # [ 124.891330] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:37Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15388worker2 # [ 124.895028] nix-daemon[863]: reaped child process 1371, status = succeeded5389worker2 # [ 124.915249] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485565390worker2 # [ 124.928141] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:485565391worker2 # [ 124.939540] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.417Z level=INFO msg="All 2 paths already cached"5392client # copying 1 paths to 'lb:50051'...5393client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5394worker2 # [ 124.945934] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:485565395worker2 # [ 124.960116] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5396worker2 # [ 125.139899] nix-daemon[863]: accepted connection from pid 1398, user nix-grpc-daemon (trusted)5397worker2 # [ 125.162153] nix-daemon[863]: reaped child process 1401, status = succeeded5398worker2 # [ 125.341772] nix-daemon[863]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)5399worker2 # [ 125.359649] nix-daemon[863]: reaped child process 1406, status = succeeded5400lb # [ 125.274339] niks3-server[1160]: 2026/09/22 10:58:37 INFO Received uploads request method=POST path=/api/pending_closures5401worker2 # [ 125.383659] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.861Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5402worker2 # [ 125.385346] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.861Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5403lb # [ 125.303363] niks3-server[1160]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5404worker2 # [ 125.416201] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.894Z level=INFO msg="Uploading 1 narinfos"5405lb # [ 125.319804] niks3-server[1160]: 2026/09/22 10:58:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5406lb # [ 125.330704] niks3-server[1160]: 2026/09/22 10:58:37 INFO Signed narinfos id=35 count=15407lb # [ 125.336282] niks3-server[1160]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5408lb # [ 125.345470] niks3-server[1160]: 2026/09/22 10:58:37 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5409worker2 # [ 125.449560] nix-grpc-daemon[1289]: time=2026-09-22T10:58:37.927Z level=INFO msg="Upload complete. (475ms)"5410lb # [ 125.351276] niks3-server[1160]: 2026/09/22 10:58:37 INFO Completed upload id=355411lb # [ 125.353484] niks3-server[1160]: 2026/09/22 10:58:37 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5412worker2 # [ 125.460229] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48556 duration_s=0 paths=1 nar_bytes_in=7845413worker2 # [ 125.468332] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:485565414worker2 # [ 125.480170] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5415worker2 # [ 125.482784] nix-daemon[863]: reaped child process 1396, status = succeeded5416client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5417client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5418worker2 # [ 125.495933] nix-daemon[863]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5419client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5420client # farm-top> LOG-farm-top-after-niks3-restart5421worker2 # [ 126.341576] nix-daemon[863]: reaped child process 1410, status = succeeded5422worker2 # [ 126.532105] nix-daemon[863]: accepted connection from pid 1426, user nix-grpc-daemon (trusted)5423worker2 # [ 126.550214] nix-daemon[863]: reaped child process 1429, status = succeeded5424lb # [ 126.464668] niks3-server[1160]: 2026/09/22 10:58:38 INFO Received uploads request method=POST path=/api/pending_closures5425worker2 # [ 126.576488] nix-grpc-daemon[1289]: time=2026-09-22T10:58:39.053Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5426worker2 # [ 126.578160] nix-grpc-daemon[1289]: time=2026-09-22T10:58:39.053Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5427lb # [ 126.494499] niks3-server[1160]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5428lb # [ 126.510985] niks3-server[1160]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5429lb # [ 126.522546] niks3-server[1160]: 2026/09/22 10:58:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5430worker2 # [ 126.628594] nix-grpc-daemon[1289]: time=2026-09-22T10:58:39.106Z level=INFO msg="Uploading 1 narinfos"5431lb # [ 126.533338] niks3-server[1160]: 2026/09/22 10:58:39 INFO Signed narinfos id=36 count=15432lb # [ 126.535806] niks3-server[1160]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5433lb # [ 126.545623] niks3-server[1160]: 2026/09/22 10:58:39 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5434lb # [ 126.552748] niks3-server[1160]: 2026/09/22 10:58:39 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5435worker2 # [ 126.656141] nix-grpc-daemon[1289]: time=2026-09-22T10:58:39.134Z level=INFO msg="Upload complete. (350ms)"5436lb # [ 126.556864] niks3-server[1160]: 2026/09/22 10:58:39 INFO Completed upload id=365437worker2 # [ 126.664623] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48556 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15438worker2 # [ 126.674897] nix-daemon[863]: reaped child process 1408, status = succeeded5439client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart >&2, in 6.65 seconds)5440client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath5441client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag after-niks3-restart outPath, in 0.24 seconds)5442(finished: subtest: a niks3 restart does not cost the leader its role, in 7.55 seconds)5443subtest: scheduler down: the other node takes the lock and keeps it5444lb: must succeed: curl -sf localhost:9901/clusters5445lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5446lb: must succeed: curl -sf localhost:9901/clusters5447lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5448lb: must succeed: curl -sf localhost:9901/clusters5449lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5450lb: must succeed: curl -sf localhost:9901/clusters5451lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5452worker2 # [ 127.262322] systemd[1]: Stopping Nix worker-protocol over gRPC...5453worker2 # [ 127.817365] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:40Z level=info event=signal_exit5454lb # [ 128.681441] niks3-server[1160]: 2026/09/22 10:58:41 INFO lead: released remote=192.168.1.4:371425455worker1 # [ 129.732284] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:42Z level=info event=scheduler_restarting addr=lb:500515456worker2 # [ 129.778683] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:42Z level=info event=scheduler_restarting5457worker2 # [ 129.779050] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:485405458worker2 # [ 129.779177] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:42Z level=info event=worker_gone addr=192.168.1.3:500515459worker2 # [ 129.779344] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:42Z level=info event=scheduler_restarting addr=lb:500515460worker2 # [ 129.797968] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:485405461worker2 # [ 129.803429] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:42Z level=info event=worker_gone addr=192.168.1.4:500515462worker2 # [ 129.831485] nix-daemon[863]: reaped child process 1281, status = succeeded5463worker2 # [ 129.888599] nix-grpc-daemon[1229]: E0000 00:00:1790074722.366382 1274 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true5464worker2 # [ 129.895045] nix-grpc-daemon[1229]: E0000 00:00:1790074722.366596 1274 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5465worker1 # [ 129.931886] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:42Z level=info event=scheduler_connected addr=lb:500515466worker1 # [ 129.937437] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5467worker2 # [ 129.998788] nix-grpc-daemon[1229]: ts=2026-09-22T10:58:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5468worker1 # [ 130.138417] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:42Z level=info event=scheduler_connected addr=lb:500515469worker1 # [ 130.143925] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5470worker1 # [ 130.344853] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:42Z level=info event=scheduler_connected addr=lb:500515471worker1 # [ 130.350367] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5472worker1 # [ 130.525433] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_take_over5473lb # [ 130.471486] niks3-server[1160]: 2026/09/22 10:58:42 INFO lead: acquired remote=192.168.1.3:329445474worker1 # [ 130.551143] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_connected addr=lb:500515475worker1 # [ 130.556584] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5476worker1 # [ 130.759283] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_connected addr=lb:500515477worker1 # [ 130.763566] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5478worker1 # [ 130.959986] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_connected addr=lb:500515479worker1 # [ 130.965118] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5480worker1 # [ 131.165736] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_connected addr=lb:500515481worker1 # [ 131.170846] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5482worker1 # [ 131.371740] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_connected addr=lb:500515483worker1 # [ 131.377118] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker1 # [ 131.578166] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=info event=scheduler_connected addr=lb:500515485worker1 # [ 131.587921] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:526665486worker1 # [ 131.593322] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05487worker2 # [ 131.939291] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5488worker2 # [ 131.944973] systemd[1]: Stopped Nix worker-protocol over gRPC.5489worker2 # [ 131.947436] systemd[1]: nix-grpc-daemon.service: Consumed 1.537s CPU time over 18.336s wall clock time, 36.1M memory peak, 46.5K incoming IP traffic, 81.3K outgoing IP traffic.5490lb # [ 131.859817] envoy[923]: conn peer=[2001:db8:1::1]:36122 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9710 tx=4144 ms=63385491worker2 # [ 131.964294] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5492worker2 # [ 131.966422] systemd[1]: Closed Nix worker-protocol over gRPC.5493lb # [ 131.864985] envoy[923]: conn peer=[2001:db8:1::4]:38452 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5496 tx=648 ms=161095494worker2 # [ 131.969239] systemd[1]: nix-grpc-daemon.socket: Consumed 768us CPU time over 18.914s wall clock time, 256K memory peak, 38.3K incoming IP traffic, 28.6K outgoing IP traffic.5495worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5496worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.09 seconds)5497worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5498worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5499client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&25500client # don't know how to build these paths:5501client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5502worker1 # [ 132.338552] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:356385503worker1 # [ 132.346432] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356385504worker1 # [ 132.372135] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05505worker1 # [ 132.378584] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15506worker1 # [ 132.423578] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356385507worker1 # [ 132.439929] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356385508client # copying 1 paths to 'lb:50051'...5509client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5510worker1 # [ 132.456121] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:356385511worker1 # [ 132.472461] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5512worker1 # [ 132.501000] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:45Z level=info event=niks3_push_started pid=13055513worker1 # [ 132.713708] nix-grpc-daemon[1305]: time=2026-09-22T10:58:45.236Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5514worker1 # [ 132.889516] nix-daemon[840]: accepted connection from pid 1318, user nix-grpc-daemon (trusted)5515worker1 # [ 132.906843] nix-daemon[840]: reaped child process 1321, status = succeeded5516worker1 # [ 133.081322] nix-daemon[840]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5517worker1 # [ 133.099596] nix-daemon[840]: reaped child process 1326, status = succeeded5518lb # [ 133.057489] niks3-server[1160]: 2026/09/22 10:58:45 INFO Received uploads request method=POST path=/api/pending_closures5519worker1 # [ 133.123078] nix-grpc-daemon[1305]: time=2026-09-22T10:58:45.645Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5520worker1 # [ 133.124845] nix-grpc-daemon[1305]: time=2026-09-22T10:58:45.645Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5521lb # [ 133.090622] niks3-server[1160]: 2026/09/22 10:58:45 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5522lb # [ 133.103647] niks3-server[1160]: 2026/09/22 10:58:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5523worker1 # [ 133.166644] nix-grpc-daemon[1305]: time=2026-09-22T10:58:45.689Z level=INFO msg="Uploading 1 narinfos"5524lb # [ 133.118545] niks3-server[1160]: 2026/09/22 10:58:45 INFO Signed narinfos id=37 count=15525lb # [ 133.126049] niks3-server[1160]: 2026/09/22 10:58:45 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5526lb # [ 133.138523] niks3-server[1160]: 2026/09/22 10:58:45 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5527lb # [ 133.145620] niks3-server[1160]: 2026/09/22 10:58:45 INFO Completed upload id=375528worker1 # [ 133.199745] nix-grpc-daemon[1305]: time=2026-09-22T10:58:45.723Z level=INFO msg="Upload complete. (481ms)"5529lb # [ 133.149468] niks3-server[1160]: 2026/09/22 10:58:45 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5530worker1 # [ 133.208453] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35638 duration_s=0 paths=1 nar_bytes_in=5205531worker1 # [ 133.223658] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356385532worker1 # [ 133.230124] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5533worker1 # [ 133.234853] nix-daemon[840]: reaped child process 1303, status = succeeded5534client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5535client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5536worker1 # [ 133.249006] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5537client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5538client # farm-a> LOG-farm-a-on-standby5539worker1 # [ 134.895649] nix-daemon[840]: reaped child process 1330, status = succeeded5540worker1 # [ 135.081496] nix-daemon[840]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5541worker1 # [ 135.099842] nix-daemon[840]: reaped child process 1349, status = succeeded5542lb # [ 135.060274] niks3-server[1160]: 2026/09/22 10:58:47 INFO Received uploads request method=POST path=/api/pending_closures5543worker1 # [ 135.128447] nix-grpc-daemon[1305]: time=2026-09-22T10:58:47.648Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5544worker1 # [ 135.130098] nix-grpc-daemon[1305]: time=2026-09-22T10:58:47.648Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5545lb # [ 135.108242] niks3-server[1160]: 2026/09/22 10:58:47 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5546lb # [ 135.119649] niks3-server[1160]: 2026/09/22 10:58:47 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5547lb # [ 135.127922] niks3-server[1160]: 2026/09/22 10:58:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5548lb # [ 135.134051] niks3-server[1160]: 2026/09/22 10:58:47 INFO Signed narinfos id=38 count=15549worker1 # [ 135.189770] nix-grpc-daemon[1305]: time=2026-09-22T10:58:47.712Z level=INFO msg="Uploading 1 narinfos"5550lb # [ 135.136930] niks3-server[1160]: 2026/09/22 10:58:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5551lb # [ 135.148197] niks3-server[1160]: 2026/09/22 10:58:47 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5552lb # [ 135.157390] niks3-server[1160]: 2026/09/22 10:58:47 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5553lb # [ 135.161898] niks3-server[1160]: 2026/09/22 10:58:47 INFO Completed upload id=385554worker1 # [ 135.216748] nix-grpc-daemon[1305]: time=2026-09-22T10:58:47.740Z level=INFO msg="Upload complete. (356ms)"5555worker1 # [ 135.244411] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35638 duration_s=2 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15556worker1 # [ 135.255822] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:47Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05557worker1 # [ 135.259634] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:47Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15558worker1 # [ 135.263784] nix-daemon[840]: reaped child process 1328, status = succeeded5559worker1 # [ 135.283997] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356385560worker1 # [ 135.298790] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356385561worker1 # [ 135.314163] nix-grpc-daemon[1305]: time=2026-09-22T10:58:47.837Z level=INFO msg="All 1 paths already cached"5562client # copying 1 paths to 'lb:50051'...5563client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5564worker1 # [ 135.324342] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:356385565worker1 # [ 135.340232] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5566worker1 # [ 135.544974] nix-daemon[840]: accepted connection from pid 1355, user nix-grpc-daemon (trusted)5567worker1 # [ 135.564457] nix-daemon[840]: reaped child process 1358, status = succeeded5568worker1 # [ 135.735891] nix-daemon[840]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5569worker1 # [ 135.754650] nix-daemon[840]: reaped child process 1363, status = succeeded5570lb # [ 135.711109] niks3-server[1160]: 2026/09/22 10:58:48 INFO Received uploads request method=POST path=/api/pending_closures5571worker1 # [ 135.776176] nix-grpc-daemon[1305]: time=2026-09-22T10:58:48.299Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5572worker1 # [ 135.777690] nix-grpc-daemon[1305]: time=2026-09-22T10:58:48.299Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5573worker1 # [ 135.804517] nix-grpc-daemon[1305]: time=2026-09-22T10:58:48.327Z level=INFO msg="Uploading 1 narinfos"5574lb # [ 135.741841] niks3-server[1160]: 2026/09/22 10:58:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5575lb # [ 135.759135] niks3-server[1160]: 2026/09/22 10:58:48 INFO Signed narinfos id=39 count=15576lb # [ 135.765655] niks3-server[1160]: 2026/09/22 10:58:48 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5577lb # [ 135.772847] niks3-server[1160]: 2026/09/22 10:58:48 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5578lb # [ 135.781831] niks3-server[1160]: 2026/09/22 10:58:48 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5579worker1 # [ 135.845145] nix-grpc-daemon[1305]: time=2026-09-22T10:58:48.368Z level=INFO msg="Upload complete. (489ms)"5580lb # [ 135.790114] niks3-server[1160]: 2026/09/22 10:58:48 INFO Completed upload id=395581lb # [ 135.794050] niks3-server[1160]: 2026/09/22 10:58:48 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5582worker1 # [ 135.852136] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35638 duration_s=0 paths=1 nar_bytes_in=6405583worker1 # [ 135.867287] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356385584worker1 # [ 135.874580] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5585worker1 # [ 135.879675] nix-daemon[840]: reaped child process 1353, status = succeeded5586client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5587worker1 # [ 135.891359] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5588client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5589client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5590client # farm-b> LOG-farm-b-on-standby5591worker1 # [ 137.536895] nix-daemon[840]: reaped child process 1367, status = succeeded5592worker1 # [ 137.723003] nix-daemon[840]: accepted connection from pid 1383, user nix-grpc-daemon (trusted)5593worker1 # [ 137.740711] nix-daemon[840]: reaped child process 1386, status = succeeded5594lb # [ 137.698027] niks3-server[1160]: 2026/09/22 10:58:50 INFO Received uploads request method=POST path=/api/pending_closures5595worker1 # [ 137.762867] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.285Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5596worker1 # [ 137.764969] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.285Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5597lb # [ 137.726840] niks3-server[1160]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5598lb # [ 137.743809] niks3-server[1160]: 2026/09/22 10:58:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5599lb # [ 137.758537] niks3-server[1160]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5600lb # [ 137.768090] niks3-server[1160]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5601lb # [ 137.774250] niks3-server[1160]: 2026/09/22 10:58:50 INFO Signed narinfos id=40 count=15602worker1 # [ 137.830092] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.352Z level=INFO msg="Uploading 1 narinfos"5603lb # [ 137.782783] niks3-server[1160]: 2026/09/22 10:58:50 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5604lb # [ 137.792145] niks3-server[1160]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5605lb # [ 137.799420] niks3-server[1160]: 2026/09/22 10:58:50 INFO Completed upload id=405606worker1 # [ 137.854520] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.377Z level=INFO msg="Upload complete. (349ms)"5607worker1 # [ 137.876256] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35638 duration_s=2 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15608worker1 # [ 137.888414] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:50Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05609worker1 # [ 137.895571] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:50Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15610worker1 # [ 137.900189] nix-daemon[840]: reaped child process 1365, status = succeeded5611worker1 # [ 137.915400] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356385612worker1 # [ 137.930990] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356385613worker1 # [ 137.946485] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.469Z level=INFO msg="All 2 paths already cached"5614client # copying 1 paths to 'lb:50051'...5615client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5616worker1 # [ 137.956979] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:356385617worker1 # [ 137.970888] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5618worker1 # [ 138.172695] nix-daemon[840]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)5619worker1 # [ 138.193310] nix-daemon[840]: reaped child process 1395, status = succeeded5620worker1 # [ 138.369172] nix-daemon[840]: accepted connection from pid 1397, user nix-grpc-daemon (trusted)5621worker1 # [ 138.388885] nix-daemon[840]: reaped child process 1400, status = succeeded5622lb # [ 138.346494] niks3-server[1160]: 2026/09/22 10:58:50 INFO Received uploads request method=POST path=/api/pending_closures5623worker1 # [ 138.413832] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.937Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5624worker1 # [ 138.415361] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.937Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5625lb # [ 138.382719] niks3-server[1160]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5626lb # [ 138.394972] niks3-server[1160]: 2026/09/22 10:58:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5627worker1 # [ 138.456469] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.979Z level=INFO msg="Uploading 1 narinfos"5628lb # [ 138.406630] niks3-server[1160]: 2026/09/22 10:58:50 INFO Signed narinfos id=41 count=15629lb # [ 138.412150] niks3-server[1160]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5630worker1 # [ 138.474058] nix-grpc-daemon[1305]: time=2026-09-22T10:58:50.997Z level=INFO msg="Upload complete. (485ms)"5631lb # [ 138.420652] niks3-server[1160]: 2026/09/22 10:58:50 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5632lb # [ 138.425551] niks3-server[1160]: 2026/09/22 10:58:50 INFO Completed upload id=415633lb # [ 138.428204] niks3-server[1160]: 2026/09/22 10:58:50 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5634worker1 # [ 138.484200] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35638 duration_s=0 paths=1 nar_bytes_in=7685635worker1 # [ 138.494220] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356385636worker1 # [ 138.501421] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5637worker1 # [ 138.506267] nix-daemon[840]: reaped child process 1390, status = succeeded5638client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5639worker1 # [ 138.519270] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5640client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5641client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5642client # farm-top> LOG-farm-top-on-standby5643worker1 # [ 139.944946] nix-daemon[840]: reaped child process 1404, status = succeeded5644worker1 # [ 140.113541] nix-daemon[840]: accepted connection from pid 1420, user nix-grpc-daemon (trusted)5645worker1 # [ 140.133775] nix-daemon[840]: reaped child process 1423, status = succeeded5646lb # [ 140.093729] niks3-server[1160]: 2026/09/22 10:58:52 INFO Received uploads request method=POST path=/api/pending_closures5647worker1 # [ 140.160543] nix-grpc-daemon[1305]: time=2026-09-22T10:58:52.683Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5648worker1 # [ 140.163889] nix-grpc-daemon[1305]: time=2026-09-22T10:58:52.683Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5649lb # [ 140.135555] niks3-server[1160]: 2026/09/22 10:58:52 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5650lb # [ 140.151467] niks3-server[1160]: 2026/09/22 10:58:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5651worker1 # [ 140.212395] nix-grpc-daemon[1305]: time=2026-09-22T10:58:52.735Z level=INFO msg="Uploading 1 narinfos"5652lb # [ 140.162569] niks3-server[1160]: 2026/09/22 10:58:52 INFO Signed narinfos id=42 count=15653lb # [ 140.169197] niks3-server[1160]: 2026/09/22 10:58:52 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5654lb # [ 140.174305] niks3-server[1160]: 2026/09/22 10:58:52 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5655lb # [ 140.181459] niks3-server[1160]: 2026/09/22 10:58:52 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5656lb # [ 140.185446] niks3-server[1160]: 2026/09/22 10:58:52 INFO Completed upload id=425657worker1 # [ 140.240517] nix-grpc-daemon[1305]: time=2026-09-22T10:58:52.764Z level=INFO msg="Upload complete. (311ms)"5658lb # [ 140.187860] niks3-server[1160]: 2026/09/22 10:58:52 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5659worker1 # [ 140.252995] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35638 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15660worker1 # [ 140.269514] nix-daemon[840]: reaped child process 1402, status = succeeded5661client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby >&2, in 8.24 seconds)5662client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath5663client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag on-standby outPath, in 0.25 seconds)5664worker2 # [ 140.690402] systemd[1]: Listening on Nix worker-protocol over gRPC.5665worker2 # [ 140.699460] systemd[1]: Starting scheduler-token.service...5666worker2 # [ 140.789095] systemd[1]: scheduler-token.service: Deactivated successfully.5667worker2 # [ 140.794583] systemd[1]: Finished scheduler-token.service.5668worker2 # [ 140.795944] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 92ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5669worker2 # [ 140.800487] systemd[1]: Starting Nix worker-protocol over gRPC...5670worker2 # [ 140.976479] nix-grpc-daemon[1450]: ts=2026-09-22T10:58:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5671worker2 # [ 140.979162] nix-grpc-daemon[1450]: ts=2026-09-22T10:58:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5672worker2 # [ 140.983276] nix-grpc-daemon[1450]: ts=2026-09-22T10:58:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15673worker2 # [ 140.991931] nix-grpc-daemon[1450]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5674worker2 # [ 141.006479] nix-grpc-daemon[1450]: E0000 00:00:1790074733.467812 1450 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5675worker2 # [ 141.010879] nix-grpc-daemon[1450]: E0000 00:00:1790074733.469837 1450 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5676worker2 # [ 141.016323] nix-grpc-daemon[1450]: ts=2026-09-22T10:58:53Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5677lb: must succeed: curl -sf localhost:9901/clusters5678worker2 # [ 141.019159] systemd[1]: Started Nix worker-protocol over gRPC.5679worker2 # [ 141.024124] nix-grpc-daemon[1450]: E0000 00:00:1790074733.498757 1492 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5680worker2 # [ 141.048059] nix-grpc-daemon[1450]: E0000 00:00:1790074733.516369 1492 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5681worker2 # [ 141.050383] nix-grpc-daemon[1450]: E0000 00:00:1790074733.516540 1492 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5682worker2 # [ 141.058316] nix-grpc-daemon[1450]: E0000 00:00:1790074733.535501 1492 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5683worker2 # [ 141.074393] nix-grpc-daemon[1450]: ts=2026-09-22T10:58:53Z level=info event=scheduler_connected addr=lb:500515684worker1 # [ 141.039345] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:53Z level=info event=oidc_jwks provider=mock keys=15685worker1 # [ 141.045112] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:526665686worker1 # [ 141.050626] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05687lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5688worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5689worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5690lb: must succeed: curl -sf localhost:9901/clusters5691lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5692lb: must succeed: curl -sf localhost:9901/clusters5693lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5694worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5695worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5696(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 14.45 seconds)5697subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5698client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs5699client # [ 141.393674] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs.5700client # Running as unit: rs.service; invocation ID: 5a4f211273944788b88b6fa580e32fb55701client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag rs, in 0.10 seconds)5702worker2 # [ 141.741801] nix-grpc-daemon[1450]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:417085703worker2 # [ 141.757694] nix-daemon[863]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5704client # [ 141.693986] nix[1285]: don't know how to build these paths:5705client # [ 141.695241] nix[1285]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5706worker1 # [ 141.742847] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:54Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05707worker1 # [ 141.749399] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:54Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15708worker1 # [ 141.797494] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356385709worker1 # [ 141.814821] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356385710client # [ 141.792168] nix[1285]: copying 1 paths to 'lb:50051'...5711worker1 # [ 141.830594] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:356385712client # [ 141.795763] nix[1285]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5713worker1 # [ 141.846492] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5714lb # [ 141.864724] envoy[923]: conn peer=[2001:db8:1::1]:52818 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9679 tx=4111 ms=79275715worker1 # [ 142.063064] nix-daemon[840]: accepted connection from pid 1436, user nix-grpc-daemon (trusted)5716worker1 # [ 142.083963] nix-daemon[840]: reaped child process 1439, status = succeeded5717worker1 # [ 142.261255] nix-daemon[840]: accepted connection from pid 1441, user nix-grpc-daemon (trusted)5718worker1 # [ 142.280703] nix-daemon[840]: reaped child process 1444, status = succeeded5719lb # [ 142.238127] niks3-server[1160]: 2026/09/22 10:58:54 INFO Received uploads request method=POST path=/api/pending_closures5720worker1 # [ 142.304081] nix-grpc-daemon[1305]: time=2026-09-22T10:58:54.826Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5721worker1 # [ 142.305759] nix-grpc-daemon[1305]: time=2026-09-22T10:58:54.826Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5722lb # [ 142.277308] niks3-server[1160]: 2026/09/22 10:58:54 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5723worker1 # [ 142.343590] nix-grpc-daemon[1305]: time=2026-09-22T10:58:54.866Z level=INFO msg="Uploading 1 narinfos"5724lb # [ 142.294141] niks3-server[1160]: 2026/09/22 10:58:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5725lb # [ 142.302008] niks3-server[1160]: 2026/09/22 10:58:54 INFO Signed narinfos id=43 count=15726lb # [ 142.310949] niks3-server[1160]: 2026/09/22 10:58:54 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5727lb # [ 142.314337] niks3-server[1160]: 2026/09/22 10:58:54 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5728worker1 # [ 142.372788] nix-grpc-daemon[1305]: time=2026-09-22T10:58:54.896Z level=INFO msg="Upload complete. (506ms)"5729lb # [ 142.320839] niks3-server[1160]: 2026/09/22 10:58:54 INFO Completed upload id=435730lb # [ 142.323334] niks3-server[1160]: 2026/09/22 10:58:54 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5731worker1 # [ 142.380355] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35638 duration_s=0 paths=1 nar_bytes_in=4165732worker1 # [ 142.396978] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356385733worker1 # [ 142.403518] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5734worker1 # [ 142.409144] nix-daemon[840]: reaped child process 1434, status = succeeded5735client # [ 142.381257] nix[1285]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5736worker1 # [ 142.423536] nix-daemon[840]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5737lb: must succeed: curl -sf localhost:9901/clusters5738lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5739lb: must succeed: curl -sf localhost:9901/clusters5740lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5741lb: must succeed: curl -sf localhost:9901/clusters5742lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5743lb: must succeed: curl -sf localhost:9901/clusters5744lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5745worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5746worker2 # [ 144.405955] systemd-journald[273]: Received client request to rotate journal, rotating.5747worker2 # [ 144.426979] systemd-journald[273]: Vacuuming done, freed 0B of archived journals from /var/log/journal/3713ca001c85475b931045bea46d0f35.5748worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5749worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5750worker1 # [ 144.434596] systemd[1]: Stopping Nix worker-protocol over gRPC...5751worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5752client: must succeed: sleep 5.05753worker1 # [ 145.198357] nix-grpc-daemon[1232]: ts=2026-09-22T10:58:57Z level=info event=draining5754client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5755worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5756worker1 # [ 149.860920] nix-daemon[840]: reaped child process 1448, status = succeeded5757worker1 # [ 150.036348] nix-daemon[840]: accepted connection from pid 1489, user nix-grpc-daemon (trusted)5758worker1 # [ 150.054120] nix-daemon[840]: reaped child process 1492, status = succeeded5759lb # [ 150.017352] niks3-server[1160]: 2026/09/22 10:59:02 INFO Received uploads request method=POST path=/api/pending_closures5760worker1 # [ 150.084412] nix-grpc-daemon[1305]: time=2026-09-22T10:59:02.605Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5761worker1 # [ 150.085913] nix-grpc-daemon[1305]: time=2026-09-22T10:59:02.605Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5762lb # [ 150.051150] niks3-server[1160]: 2026/09/22 10:59:02 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5763lb # [ 150.072369] niks3-server[1160]: 2026/09/22 10:59:02 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5764worker1 # [ 150.140637] nix-grpc-daemon[1305]: time=2026-09-22T10:59:02.663Z level=INFO msg="Uploading 1 narinfos"5765lb # [ 150.085667] niks3-server[1160]: 2026/09/22 10:59:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5766lb # [ 150.096933] niks3-server[1160]: 2026/09/22 10:59:02 INFO Signed narinfos id=44 count=15767lb # [ 150.103194] niks3-server[1160]: 2026/09/22 10:59:02 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5768lb # [ 150.110925] niks3-server[1160]: 2026/09/22 10:59:02 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5769lb # [ 150.117868] niks3-server[1160]: 2026/09/22 10:59:02 INFO Completed upload id=445770worker1 # [ 150.171855] nix-grpc-daemon[1305]: time=2026-09-22T10:59:02.695Z level=INFO msg="Upload complete. (332ms)"5771lb # [ 150.121073] niks3-server[1160]: 2026/09/22 10:59:02 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5772worker1 # [ 150.189126] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35638 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15773worker1 # [ 150.204407] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:02Z level=info event=signal_exit5774worker1 # [ 150.208915] nix-daemon[840]: reaped child process 1446, status = succeeded5775client # [ 150.189202] systemd[1]: rs.service: Deactivated successfully.5776client # [ 150.194551] systemd[1]: rs.service: Consumed 185ms CPU time over 8.799s wall clock time, 41.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5777lb # [ 150.472223] niks3-server[1160]: 2026/09/22 10:59:02 INFO lead: released remote=192.168.1.3:329445778worker2 # [ 151.013065] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:03Z level=info event=scheduler_take_over5779lb # [ 150.912395] niks3-server[1160]: 2026/09/22 10:59:03 INFO lead: acquired remote=192.168.1.4:528045780worker2 # [ 151.573947] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=scheduler_restarting addr=lb:500515781worker1 # [ 151.534008] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:04Z level=info event=scheduler_restarting5782worker1 # [ 151.539210] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:04Z level=info event=scheduler_restarting addr=lb:500515783worker1 # [ 151.544357] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:526665784worker1 # [ 151.549418] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:04Z level=info event=worker_gone addr=192.168.1.3:500515785worker1 # [ 151.554938] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:526665786worker1 # [ 151.559838] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:04Z level=info event=worker_gone addr=192.168.1.4:500515787worker1 # [ 151.584189] nix-daemon[840]: reaped child process 1284, status = succeeded5788worker2 # [ 151.775796] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=scheduler_connected addr=lb:500515789worker2 # [ 151.781259] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5790worker1 # [ 151.752569] nix-grpc-daemon[1232]: ts=2026-09-22T10:59:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5791lb # [ 151.866109] envoy[923]: conn peer=[2001:db8:1::1]:40152 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1405 ms=85085792lb # [ 151.871189] envoy[923]: conn peer=[2001:db8:1::3]:55822 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2535 tx=864 ms=334865793worker2 # [ 151.982258] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=scheduler_connected addr=lb:500515794worker2 # [ 151.987542] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5795worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.20 seconds)5796client: waiting for success: ! systemctl is-active rs5797client: (finished: waiting for success: ! systemctl is-active rs, in 0.07 seconds)5798client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5799worker2 # [ 152.190171] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=scheduler_connected addr=lb:500515800client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.06 seconds)5801client: must fail: journalctl -u rs | grep -q 'reconnecting'5802worker2 # [ 152.206914] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=oidc_jwks provider=mock keys=15803worker2 # [ 152.211560] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:576465804worker2 # [ 152.216933] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05805client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.04 seconds)5806lb: must succeed: curl -sf localhost:9901/clusters5807lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5808worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5809worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5810worker1 # [ 152.450198] nix-grpc-daemon[1232]: E0000 00:00:1790074744.973311 1277 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5811worker1 # [ 152.450636] nix-grpc-daemon[1232]: E0000 00:00:1790074744.973611 1277 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5812worker1 # [ 153.192791] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5813worker1 # [ 153.196845] systemd[1]: Stopped Nix worker-protocol over gRPC.5814worker1 # [ 153.199343] systemd[1]: nix-grpc-daemon.service: Consumed 1.897s CPU time over 35.223s wall clock time, 37M memory peak, 46.9K incoming IP traffic, 89K outgoing IP traffic.5815worker1 # [ 153.218172] systemd[1]: Starting Nix worker-protocol over gRPC...5816lb: must succeed: curl -sf localhost:9901/clusters5817worker1 # [ 153.415099] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5818worker1 # [ 153.417676] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5819worker1 # [ 153.421761] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15820lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5821worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5822worker1 # [ 153.430724] nix-grpc-daemon[1499]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5823worker1 # [ 153.435558] nix-grpc-daemon[1499]: E0000 00:00:1790074745.951957 1499 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5824worker1 # [ 153.439462] nix-grpc-daemon[1499]: E0000 00:00:1790074745.954075 1499 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5825worker1 # [ 153.442987] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5826worker1 # [ 153.446116] systemd[1]: Started Nix worker-protocol over gRPC.5827worker1 # [ 153.452069] nix-grpc-daemon[1499]: E0000 00:00:1790074745.972126 1538 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5828worker1 # [ 153.476061] nix-grpc-daemon[1499]: E0000 00:00:1790074745.989741 1538 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5829worker1 # [ 153.478376] nix-grpc-daemon[1499]: E0000 00:00:1790074745.989912 1538 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5830worker1 # [ 153.486891] nix-grpc-daemon[1499]: E0000 00:00:1790074746.009236 1538 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5831worker1 # [ 153.500258] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:06Z level=info event=scheduler_connected addr=lb:500515832worker2 # [ 153.547760] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:576465833worker2 # [ 153.552773] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05834worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)5835(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.16 seconds)5836subtest: scheduler restart mid-build: build finishes, no double build5837client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid5838client # [ 153.554492] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid.5839client # Running as unit: mid.service; invocation ID: c12de9a226b746be9b272a5597d928785840client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag mid, in 0.09 seconds)5841worker2 # [ 153.877124] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:417085842client # [ 153.802330] nix[1321]: don't know how to build these paths:5843client # [ 153.803446] nix[1321]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5844worker2 # [ 153.897478] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:06Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05845worker2 # [ 153.904729] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:06Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15846worker2 # [ 153.943595] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:417085847worker2 # [ 153.959913] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 1/5)5848worker2 # [ 154.003283] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 2/5)5849worker2 # [ 154.181636] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 323 ms (attempt 3/5)5850worker2 # [ 154.504959] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 472 ms (attempt 4/5)5851worker2 # [ 154.979303] nix-grpc-daemon[1450]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5852worker2 # [ 154.986535] nix-grpc-daemon[1450]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5853worker2 # [ 155.000222] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:417085854client # [ 154.929496] nix[1321]: copying 1 paths to 'lb:50051'...5855worker2 # [ 155.012106] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:417085856client # [ 154.934189] nix[1321]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5857worker2 # [ 155.023787] nix-daemon[863]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5858worker2 # [ 155.047647] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:07Z level=info event=niks3_push_started pid=15765859worker2 # [ 155.249813] nix-grpc-daemon[1576]: time=2026-09-22T10:59:07.727Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""5860worker2 # [ 155.424510] nix-daemon[863]: accepted connection from pid 1589, user nix-grpc-daemon (trusted)5861worker2 # [ 155.440905] nix-daemon[863]: reaped child process 1592, status = succeeded5862worker2 # [ 155.616386] nix-daemon[863]: accepted connection from pid 1594, user nix-grpc-daemon (trusted)5863worker2 # [ 155.634759] nix-daemon[863]: reaped child process 1597, status = succeeded5864lb # [ 155.547395] niks3-server[1160]: 2026/09/22 10:59:08 INFO Received uploads request method=POST path=/api/pending_closures5865worker2 # [ 155.659145] nix-grpc-daemon[1576]: time=2026-09-22T10:59:08.136Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5866worker2 # [ 155.660855] nix-grpc-daemon[1576]: time=2026-09-22T10:59:08.136Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5867lb # [ 155.586455] niks3-server[1160]: 2026/09/22 10:59:08 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5868worker2 # [ 155.696397] nix-grpc-daemon[1576]: time=2026-09-22T10:59:08.173Z level=INFO msg="Uploading 1 narinfos"5869lb # [ 155.596739] niks3-server[1160]: 2026/09/22 10:59:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5870lb # [ 155.604503] niks3-server[1160]: 2026/09/22 10:59:08 INFO Signed narinfos id=45 count=15871lb # [ 155.614825] niks3-server[1160]: 2026/09/22 10:59:08 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5872lb # [ 155.625651] niks3-server[1160]: 2026/09/22 10:59:08 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5873worker2 # [ 155.730958] nix-grpc-daemon[1576]: time=2026-09-22T10:59:08.208Z level=INFO msg="Upload complete. (477ms)"5874lb # [ 155.634288] niks3-server[1160]: 2026/09/22 10:59:08 INFO Completed upload id=455875lb # [ 155.638052] niks3-server[1160]: 2026/09/22 10:59:08 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5876worker2 # [ 155.737429] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41708 duration_s=0 paths=1 nar_bytes_in=4245877worker2 # [ 155.750274] nix-grpc-daemon[1450]: ts=2026-09-22T10:59:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:417085878worker2 # [ 155.757449] nix-daemon[863]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5879worker2 # [ 155.762428] nix-daemon[863]: reaped child process 1574, status = succeeded5880client # [ 155.688847] nix[1321]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5881worker2 # [ 155.775010] nix-daemon[863]: accepted connection from pid 1450, user nix-grpc-daemon (trusted)5882lb: must succeed: curl -sf localhost:9901/clusters5883lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5884lb: must succeed: curl -sf localhost:9901/clusters5885lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5886worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5887worker2 # [ 157.647283] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1450 (nix-grpc-daemon) on client request.5888worker2 # [ 157.649290] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5889lb # [ 157.568504] niks3-server[1160]: 2026/09/22 10:59:10 INFO lead: released remote=192.168.1.4:528045890worker2 # [ 157.681587] systemd[1]: Starting scheduler-token.service...5891client # [ 157.593846] nix[1321]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5892worker1 # [ 157.631671] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:10Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5893client # [ 157.600140] nix[1321]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5894worker2 # [ 157.692122] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5895worker2 # [ 157.693616] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5896worker2 # [ 157.701488] systemd[1]: nix-grpc-daemon.service: Consumed 548ms CPU time over 16.891s wall clock time, 34.5M memory peak, 15.7K incoming IP traffic, 26.1K outgoing IP traffic.5897worker2 # [ 157.718543] nix-daemon[863]: reaped child process 1509, status = succeeded5898worker2 # [ 157.721220] nix-daemon[863]: reaped child process 1599, status = succeeded5899worker2 # [ 157.730085] nix-daemon[1601]: unexpected Nix daemon error: error: interrupted by the user5900worker2 # [ 157.737176] nix-daemon[863]: reaped child process 1601, status = failed with exit code 15901worker2 # [ 157.783134] systemd[1]: scheduler-token.service: Deactivated successfully.5902worker2 # [ 157.787367] systemd[1]: Finished scheduler-token.service.5903worker2 # [ 157.788799] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 101ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5904worker2 # [ 157.791962] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5905worker2 # [ 157.793914] systemd[1]: Starting scheduler-token.service...5906worker2 # [ 157.861437] systemd[1]: scheduler-token.service: Deactivated successfully.5907worker2 # [ 157.864247] systemd[1]: Finished scheduler-token.service.5908worker2 # [ 157.865035] systemd[1]: scheduler-token.service: Consumed 22ms CPU time over 69ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5909worker2 # [ 157.869729] systemd[1]: Starting Nix worker-protocol over gRPC...5910worker1 # [ 157.831758] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:10Z level=info event=scheduler_connected addr=lb:500515911worker2 # [ 158.035738] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5912worker2 # [ 158.038328] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5913worker2 # [ 158.042291] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15914worker2 # [ 158.050691] nix-grpc-daemon[1644]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5915worker2 # [ 158.063060] nix-grpc-daemon[1644]: E0000 00:00:1790074750.526680 1644 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5916worker2 # [ 158.069263] nix-grpc-daemon[1644]: E0000 00:00:1790074750.528612 1644 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5917worker2 # [ 158.075305] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5918worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.48 seconds)5919lb # [ 157.977363] niks3-server[1160]: 2026/09/22 10:59:10 INFO lead: acquired remote=192.168.1.4:582145920client: must succeed: sleep 5.05921worker2 # [ 158.078835] systemd[1]: Started Nix worker-protocol over gRPC.5922worker1 # [ 158.037927] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:10Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5923worker2 # [ 158.080254] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=scheduler_take_over5924worker2 # [ 158.093254] nix-grpc-daemon[1644]: E0000 00:00:1790074750.558643 1685 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5925worker2 # [ 158.114351] nix-grpc-daemon[1644]: E0000 00:00:1790074750.577506 1685 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5926worker2 # [ 158.117163] nix-grpc-daemon[1644]: E0000 00:00:1790074750.577652 1685 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5927worker2 # [ 158.119609] nix-grpc-daemon[1644]: E0000 00:00:1790074750.591827 1685 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5928worker2 # [ 158.127096] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=scheduler_connected addr=lb:500515929worker2 # [ 158.128780] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5930client # [ 158.096889] nix[1321]: scheduler at lb:50051: no healthy upstream, reconnecting5931worker1 # [ 158.238815] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:10Z level=info event=scheduler_connected addr=lb:500515932worker1 # [ 158.244236] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5933worker2 # [ 158.330354] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=scheduler_connected addr=lb:500515934worker2 # [ 158.335148] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5935worker1 # [ 158.445494] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:10Z level=info event=scheduler_connected addr=lb:500515936worker1 # [ 158.450602] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5937worker2 # [ 158.536375] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515938worker2 # [ 158.541425] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5939worker1 # [ 158.652968] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515940worker1 # [ 158.657750] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5941worker2 # [ 158.742563] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515942worker2 # [ 158.747817] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5943worker1 # [ 158.858515] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515944worker1 # [ 158.863800] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5945worker2 # [ 158.949287] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515946worker2 # [ 158.954662] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5947worker1 # [ 159.064595] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515948worker1 # [ 159.069676] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5949worker2 # [ 159.157805] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515950worker2 # [ 159.162037] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5951client # [ 159.100720] nix[1321]: scheduler at lb:50051: no healthy upstream, reconnecting5952worker1 # [ 159.270535] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515953worker1 # [ 159.275899] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5954worker2 # [ 159.358695] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_connected addr=lb:500515955worker2 # [ 159.364082] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5956worker1 # [ 159.476645] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:12Z level=info event=scheduler_connected addr=lb:500515957worker1 # [ 159.481845] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5958worker2 # [ 159.565156] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=scheduler_connected addr=lb:500515959worker2 # [ 159.570298] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5960worker1 # [ 159.684421] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:12Z level=info event=scheduler_connected addr=lb:500515961worker1 # [ 159.688509] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5962worker2 # [ 159.771403] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=scheduler_connected addr=lb:500515963worker2 # [ 159.776597] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5964worker1 # [ 159.884946] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:12Z level=info event=scheduler_connected addr=lb:500515965worker1 # [ 159.890034] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5966worker2 # [ 159.977728] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=scheduler_connected addr=lb:500515967worker2 # [ 159.982820] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5968worker1 # [ 160.091148] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:12Z level=info event=scheduler_connected addr=lb:500515969worker2 # [ 160.141399] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:576765970worker2 # [ 160.146664] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05971worker2 # [ 160.186802] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=scheduler_connected addr=lb:500515972worker2 # [ 160.196154] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=oidc_jwks provider=mock keys=15973worker2 # [ 160.201731] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:576765974worker2 # [ 160.207038] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05975worker2 # [ 161.197372] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:13Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05976worker2 # [ 161.203521] nix-grpc-daemon[1644]: ts=2026-09-22T10:59:13Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15977worker1 # [ 161.219697] nix-grpc-daemon[1499]: ts=2026-09-22T10:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351465978worker1 # [ 161.234597] nix-daemon[840]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5979worker1 # [ 161.260300] nix-grpc-daemon[1499]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 44 ms (attempt 1/5)5980worker1 # [ 161.311851] nix-grpc-daemon[1499]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 2/5)5981worker1 # [ 161.409527] nix-grpc-daemon[1499]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 201 ms (attempt 3/5)5982worker1 # [ 161.618183] nix-grpc-daemon[1499]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 624 ms (attempt 4/5)5983lb # [ 161.868913] envoy[923]: conn peer=[2001:db8:1::4]:47710 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3440 tx=427 ms=166125984worker1 # [ 162.251818] nix-grpc-daemon[1499]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5985worker1 # [ 162.257420] nix-grpc-daemon[1499]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5986worker1 # [ 162.276840] nix-grpc-daemon[1499]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5987worker1 # [ 162.304788] nix-daemon[840]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5988worker1 # [ 162.324237] nix-daemon[840]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5989client # [ 162.285724] nix[1321]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5990client: (finished: must succeed: sleep 5.0, in 5.06 seconds)5991client: waiting for success: ! systemctl is-active mid5992worker1 # [ 253.818520] nix-grpc-daemon[1499]: ts=2026-09-22T11:00:46Z level=info event=niks3_push_started pid=16055993worker1 # [ 253.862107] nix-daemon[840]: reaped child process 1580, status = succeeded5994worker1 # [ 254.053899] nix-grpc-daemon[1605]: time=2026-09-22T11:00:46.576Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""5995worker1 # [ 254.226350] nix-daemon[840]: accepted connection from pid 1618, user nix-grpc-daemon (trusted)5996worker1 # [ 254.243436] nix-daemon[840]: reaped child process 1621, status = succeeded5997lb # [ 254.202810] niks3-server[1160]: 2026/09/22 11:00:46 INFO Received uploads request method=POST path=/api/pending_closures5998worker1 # [ 254.274363] nix-grpc-daemon[1605]: time=2026-09-22T11:00:46.796Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5999worker1 # [ 254.277444] nix-grpc-daemon[1605]: time=2026-09-22T11:00:46.796Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6000lb # [ 254.269296] niks3-server[1160]: 2026/09/22 11:00:46 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6001worker1 # [ 254.333429] nix-grpc-daemon[1605]: time=2026-09-22T11:00:46.855Z level=INFO msg="Uploading 1 narinfos"6002lb # [ 254.281794] niks3-server[1160]: 2026/09/22 11:00:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6003lb # [ 254.291428] niks3-server[1160]: 2026/09/22 11:00:46 INFO Signed narinfos id=46 count=16004lb # [ 254.297613] niks3-server[1160]: 2026/09/22 11:00:46 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6005lb # [ 254.314653] niks3-server[1160]: 2026/09/22 11:00:46 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6006lb # [ 254.321497] niks3-server[1160]: 2026/09/22 11:00:46 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6007lb # [ 254.327945] niks3-server[1160]: 2026/09/22 11:00:46 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6008lb # [ 254.344096] niks3-server[1160]: 2026/09/22 11:00:46 INFO Completed upload id=466009worker1 # [ 254.399299] nix-grpc-daemon[1605]: time=2026-09-22T11:00:46.922Z level=INFO msg="Upload complete. (341ms)"6010worker1 # [ 254.412455] nix-grpc-daemon[1499]: ts=2026-09-22T11:00:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35146 duration_s=93 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16011worker1 # [ 254.430276] nix-daemon[840]: reaped child process 1578, status = succeeded6012client # [ 254.416250] systemd[1]: mid.service: Deactivated successfully.6013client # [ 254.420825] systemd[1]: mid.service: Consumed 243ms CPU time over 1min 40.865s wall clock time, 41.3M memory peak, 4K written to disk, 9.1K incoming IP traffic, 11.1K outgoing IP traffic.6014client: (finished: waiting for success: ! systemctl is-active mid, in 91.85 seconds)6015client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6016client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)6017(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 101.80 seconds)6018subtest: interrupting the client stops the build on the worker6019client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr6020client # [ 255.327553] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr.6021client # Running as unit: intr.service; invocation ID: ecd9a41b63004439b932a61bf109f02a6022client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag intr, in 0.06 seconds)6023worker2 # [ 255.662360] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:501786024worker2 # [ 255.675828] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6025client # [ 255.610402] nix[1782]: don't know how to build these paths:6026client # [ 255.612118] nix[1782]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6027worker2 # [ 255.708321] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:48Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06028worker2 # [ 255.715046] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:48Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16029worker2 # [ 255.759822] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786030worker2 # [ 255.777277] nix-grpc-daemon[1644]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 81 ms (attempt 1/5)6031worker2 # [ 255.865417] nix-grpc-daemon[1644]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 2/5)6032worker2 # [ 256.008985] nix-grpc-daemon[1644]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 314 ms (attempt 3/5)6033worker2 # [ 256.330727] nix-grpc-daemon[1644]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 4/5)6034worker2 # [ 256.363874] nix-grpc-daemon[1644]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6035worker2 # [ 256.370655] nix-grpc-daemon[1644]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6036worker2 # [ 256.389265] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786037client # [ 256.320328] nix[1782]: copying 1 paths to 'lb:50051'...6038client # [ 256.325871] nix[1782]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6039worker2 # [ 256.408118] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501786040worker2 # [ 256.423466] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6041worker2 # [ 256.451190] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:48Z level=info event=niks3_push_started pid=17206042worker2 # [ 256.672567] nix-grpc-daemon[1720]: time=2026-09-22T11:00:49.150Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker2.key ca=""6043worker2 # [ 256.855381] nix-daemon[863]: accepted connection from pid 1737, user nix-grpc-daemon (trusted)6044worker2 # [ 256.874960] nix-daemon[863]: reaped child process 1740, status = succeeded6045worker2 # [ 257.061393] nix-daemon[863]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6046worker2 # [ 257.083303] nix-daemon[863]: reaped child process 1748, status = succeeded6047lb # [ 257.001592] niks3-server[1160]: 2026/09/22 11:00:49 INFO Received uploads request method=POST path=/api/pending_closures6048worker2 # [ 257.114658] nix-grpc-daemon[1720]: time=2026-09-22T11:00:49.591Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6049worker2 # [ 257.116437] nix-grpc-daemon[1720]: time=2026-09-22T11:00:49.591Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6050lb # [ 257.041690] niks3-server[1160]: 2026/09/22 11:00:49 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6051worker2 # [ 257.159328] nix-grpc-daemon[1720]: time=2026-09-22T11:00:49.636Z level=INFO msg="Uploading 1 narinfos"6052lb # [ 257.053278] niks3-server[1160]: 2026/09/22 11:00:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6053lb # [ 257.063858] niks3-server[1160]: 2026/09/22 11:00:49 INFO Signed narinfos id=47 count=16054lb # [ 257.074995] niks3-server[1160]: 2026/09/22 11:00:49 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6055lb # [ 257.085656] niks3-server[1160]: 2026/09/22 11:00:49 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6056worker2 # [ 257.190132] nix-grpc-daemon[1720]: time=2026-09-22T11:00:49.668Z level=INFO msg="Upload complete. (512ms)"6057lb # [ 257.091218] niks3-server[1160]: 2026/09/22 11:00:49 INFO Completed upload id=476058lb # [ 257.095087] niks3-server[1160]: 2026/09/22 11:00:49 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6059worker2 # [ 257.201245] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=0 paths=1 nar_bytes_in=4246060worker2 # [ 257.211774] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786061worker2 # [ 257.219035] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6062worker2 # [ 257.223880] nix-daemon[863]: reaped child process 1718, status = succeeded6063client # [ 257.151790] nix[1782]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6064worker2 # [ 257.237049] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6065client: must succeed: systemctl kill -s INT intr6066worker2 # [ 259.094305] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:51Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6067client # [ 259.010601] nix[1782]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6068client # [ 259.018378] systemd[1]: intr.service: Sent signal SIGINT to main process 1782 (nix) on client request.6069client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6070client # [ 259.027367] nix[1782]: error: interrupted by the user6071client # [ 259.059643] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6072client # [ 259.064611] systemd[1]: intr.service: Failed with result 'exit-code'.6073client # [ 259.066765] systemd[1]: intr.service: Consumed 159ms CPU time over 3.735s wall clock time, 41.2M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6074worker2 # [ 259.271391] nix-daemon[863]: reaped child process 1752, status = succeeded6075worker2 # [ 259.273900] nix-daemon[1754]: unexpected Nix daemon error: error: interrupted by the user6076worker2 # [ 259.282304] nix-daemon[863]: reaped child process 1754, status = failed with exit code 16077(finished: subtest: interrupting the client stops the build on the worker, in 5.04 seconds)6078subtest: a deploy mid-build drains: the build finishes, the new generation takes over6079client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw6080client # [ 260.388700] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key" --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw.6081client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/mpvagzcaddzb5j0jgpk1hnp5dkckqbn5-slow.nix --argstr tag sw, in 0.09 seconds)6082worker2 # [ 260.737609] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:501786083client # Running as unit: sw.service; invocation ID: 67f911e498b645d78c19ab7a06fb8f386084client # [ 260.663446] nix[1803]: don't know how to build these paths:6085client # [ 260.665844] nix[1803]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6086worker2 # [ 260.759144] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:53Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06087worker2 # [ 260.765274] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:53Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16088worker1 # [ 260.761309] nix-grpc-daemon[1499]: ts=2026-09-22T11:00:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351466089worker1 # [ 260.783366] nix-grpc-daemon[1499]: ts=2026-09-22T11:00:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:351466090client # [ 260.760146] nix[1803]: copying 1 paths to 'lb:50051'...6091worker1 # [ 260.799577] nix-grpc-daemon[1499]: ts=2026-09-22T11:00:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:351466092client # [ 260.764148] nix[1803]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6093worker1 # [ 260.815082] nix-daemon[840]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)6094worker1 # [ 261.026818] nix-daemon[840]: accepted connection from pid 1655, user nix-grpc-daemon (trusted)6095worker1 # [ 261.046100] nix-daemon[840]: reaped child process 1658, status = succeeded6096worker1 # [ 261.228649] nix-daemon[840]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6097worker1 # [ 261.245856] nix-daemon[840]: reaped child process 1663, status = succeeded6098lb # [ 261.208891] niks3-server[1160]: 2026/09/22 11:00:53 INFO Received uploads request method=POST path=/api/pending_closures6099worker1 # [ 261.278716] nix-grpc-daemon[1605]: time=2026-09-22T11:00:53.801Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6100worker1 # [ 261.280383] nix-grpc-daemon[1605]: time=2026-09-22T11:00:53.801Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6101lb # [ 261.246132] niks3-server[1160]: 2026/09/22 11:00:53 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6102lb # [ 261.260983] niks3-server[1160]: 2026/09/22 11:00:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6103worker1 # [ 261.320622] nix-grpc-daemon[1605]: time=2026-09-22T11:00:53.843Z level=INFO msg="Uploading 1 narinfos"6104lb # [ 261.272105] niks3-server[1160]: 2026/09/22 11:00:53 INFO Signed narinfos id=48 count=16105lb # [ 261.277467] niks3-server[1160]: 2026/09/22 11:00:53 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6106lb # [ 261.290050] niks3-server[1160]: 2026/09/22 11:00:53 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6107lb # [ 261.301334] niks3-server[1160]: 2026/09/22 11:00:53 INFO Completed upload id=486108worker1 # [ 261.355861] nix-grpc-daemon[1605]: time=2026-09-22T11:00:53.879Z level=INFO msg="Upload complete. (523ms)"6109lb # [ 261.304570] niks3-server[1160]: 2026/09/22 11:00:53 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6110worker1 # [ 261.365832] nix-grpc-daemon[1499]: ts=2026-09-22T11:00:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35146 duration_s=0 paths=1 nar_bytes_in=4166111worker1 # [ 261.384587] nix-grpc-daemon[1499]: ts=2026-09-22T11:00:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:351466112worker1 # [ 261.394494] nix-daemon[840]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)6113worker1 # [ 261.398549] nix-daemon[840]: reaped child process 1653, status = succeeded6114worker1 # [ 261.408891] nix-daemon[840]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)6115client # [ 261.369355] nix[1803]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6116lb # [ 261.872703] envoy[923]: conn peer=[2001:db8:1::1]:33244 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4288 tx=1971 ms=1005926117lb # [ 261.877855] envoy[923]: conn peer=[2001:db8:1::1]:48376 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=853 ms=34736118worker1: must succeed: systemctl reload nix-grpc-daemon.service6119worker1 # [ 263.139245] systemd[1]: Reloading Nix worker-protocol over gRPC...6120worker1 # [ 263.189631] systemd[1]: Reloaded Nix worker-protocol over gRPC.6121worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.11 seconds)6122worker1: must succeed: pgrep -f 'read -t [9]0 x'6123worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.06 seconds)6124client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&26125worker1 # [ 263.471041] nix-grpc-daemon[1499]: ts=2026-09-22T11:00:55Z level=info event=draining6126client # don't know how to build these paths:6127client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6128worker2 # [ 263.564810] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:501786129worker2 # [ 263.571238] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786130worker2 # [ 263.596261] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06131worker2 # [ 263.602999] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16132worker2 # [ 263.652834] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786133worker2 # [ 263.667615] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786134client # copying 1 paths to 'lb:50051'...6135worker2 # [ 263.683180] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501786136client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6137worker2 # [ 263.707643] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6138worker2 # [ 263.911142] nix-daemon[863]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6139worker2 # [ 263.930728] nix-daemon[863]: reaped child process 1812, status = succeeded6140worker2 # [ 264.106249] nix-daemon[863]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)6141worker2 # [ 264.123754] nix-daemon[863]: reaped child process 1817, status = succeeded6142lb # [ 264.036986] niks3-server[1160]: 2026/09/22 11:00:56 INFO Received uploads request method=POST path=/api/pending_closures6143worker2 # [ 264.147355] nix-grpc-daemon[1720]: time=2026-09-22T11:00:56.624Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6144worker2 # [ 264.149002] nix-grpc-daemon[1720]: time=2026-09-22T11:00:56.624Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6145lb # [ 264.067830] niks3-server[1160]: 2026/09/22 11:00:56 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6146worker2 # [ 264.180232] nix-grpc-daemon[1720]: time=2026-09-22T11:00:56.658Z level=INFO msg="Uploading 1 narinfos"6147lb # [ 264.083817] niks3-server[1160]: 2026/09/22 11:00:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6148lb # [ 264.094555] niks3-server[1160]: 2026/09/22 11:00:56 INFO Signed narinfos id=49 count=16149lb # [ 264.098931] niks3-server[1160]: 2026/09/22 11:00:56 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6150lb # [ 264.107886] niks3-server[1160]: 2026/09/22 11:00:56 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6151lb # [ 264.113394] niks3-server[1160]: 2026/09/22 11:00:56 INFO Completed upload id=496152worker2 # [ 264.213078] nix-grpc-daemon[1720]: time=2026-09-22T11:00:56.691Z level=INFO msg="Upload complete. (488ms)"6153lb # [ 264.115854] niks3-server[1160]: 2026/09/22 11:00:56 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6154worker2 # [ 264.222145] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=0 paths=1 nar_bytes_in=5286155worker2 # [ 264.234050] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786156worker2 # [ 264.239630] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6157worker2 # [ 264.244725] nix-daemon[863]: reaped child process 1807, status = succeeded6158client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6159client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6160worker2 # [ 264.259009] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6161client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6162client # farm-a> LOG-farm-a-during-drain6163worker2 # [ 265.095802] nix-daemon[863]: reaped child process 1821, status = succeeded6164worker2 # [ 265.276096] nix-daemon[863]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6165worker2 # [ 265.295066] nix-daemon[863]: reaped child process 1840, status = succeeded6166lb # [ 265.210056] niks3-server[1160]: 2026/09/22 11:00:57 INFO Received uploads request method=POST path=/api/pending_closures6167worker2 # [ 265.320400] nix-grpc-daemon[1720]: time=2026-09-22T11:00:57.795Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6168worker2 # [ 265.321878] nix-grpc-daemon[1720]: time=2026-09-22T11:00:57.795Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6169lb # [ 265.250421] niks3-server[1160]: 2026/09/22 11:00:57 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6170lb # [ 265.260689] niks3-server[1160]: 2026/09/22 11:00:57 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6171worker2 # [ 265.371758] nix-grpc-daemon[1720]: time=2026-09-22T11:00:57.848Z level=INFO msg="Uploading 1 narinfos"6172lb # [ 265.274319] niks3-server[1160]: 2026/09/22 11:00:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6173lb # [ 265.282207] niks3-server[1160]: 2026/09/22 11:00:57 INFO Signed narinfos id=50 count=16174lb # [ 265.285345] niks3-server[1160]: 2026/09/22 11:00:57 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6175lb # [ 265.292110] niks3-server[1160]: 2026/09/22 11:00:57 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6176lb # [ 265.295784] niks3-server[1160]: 2026/09/22 11:00:57 INFO Completed upload id=506177worker2 # [ 265.395256] nix-grpc-daemon[1720]: time=2026-09-22T11:00:57.873Z level=INFO msg="Upload complete. (326ms)"6178lb # [ 265.298330] niks3-server[1160]: 2026/09/22 11:00:57 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6179worker2 # [ 265.407529] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16180worker2 # [ 265.426115] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:57Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06181worker2 # [ 265.430152] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:57Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16182worker2 # [ 265.434370] nix-daemon[863]: reaped child process 1819, status = succeeded6183worker2 # [ 265.463819] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786184worker2 # [ 265.478103] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786185client # copying 1 paths to 'lb:50051'...6186worker2 # [ 265.492808] nix-grpc-daemon[1720]: time=2026-09-22T11:00:57.970Z level=INFO msg="All 1 paths already cached"6187worker2 # [ 265.499302] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501786188client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6189worker2 # [ 265.524236] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6190worker2 # [ 265.731534] nix-daemon[863]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6191worker2 # [ 265.748731] nix-daemon[863]: reaped child process 1849, status = succeeded6192worker2 # [ 265.923628] nix-daemon[863]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6193worker2 # [ 265.940719] nix-daemon[863]: reaped child process 1854, status = succeeded6194lb # [ 265.855149] niks3-server[1160]: 2026/09/22 11:00:58 INFO Received uploads request method=POST path=/api/pending_closures6195worker2 # [ 265.964068] nix-grpc-daemon[1720]: time=2026-09-22T11:00:58.441Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6196worker2 # [ 265.966045] nix-grpc-daemon[1720]: time=2026-09-22T11:00:58.441Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6197lb # [ 265.882512] niks3-server[1160]: 2026/09/22 11:00:58 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6198worker2 # [ 265.996559] nix-grpc-daemon[1720]: time=2026-09-22T11:00:58.474Z level=INFO msg="Uploading 1 narinfos"6199lb # [ 265.897573] niks3-server[1160]: 2026/09/22 11:00:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6200lb # [ 265.909003] niks3-server[1160]: 2026/09/22 11:00:58 INFO Signed narinfos id=51 count=16201lb # [ 265.915556] niks3-server[1160]: 2026/09/22 11:00:58 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6202lb # [ 265.924603] niks3-server[1160]: 2026/09/22 11:00:58 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6203lb # [ 265.929205] niks3-server[1160]: 2026/09/22 11:00:58 INFO Completed upload id=516204worker2 # [ 266.028815] nix-grpc-daemon[1720]: time=2026-09-22T11:00:58.506Z level=INFO msg="Upload complete. (487ms)"6205lb # [ 265.931786] niks3-server[1160]: 2026/09/22 11:00:58 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6206worker2 # [ 266.036130] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=0 paths=1 nar_bytes_in=6406207worker2 # [ 266.047557] nix-grpc-daemon[1644]: ts=2026-09-22T11:00:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786208worker2 # [ 266.053712] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6209worker2 # [ 266.058757] nix-daemon[863]: reaped child process 1844, status = succeeded6210client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6211client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6212worker2 # [ 266.071878] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6213client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6214client # farm-b> LOG-farm-b-during-drain6215worker2 # [ 267.296668] nix-daemon[863]: reaped child process 1858, status = succeeded6216worker2 # [ 267.457069] nix-daemon[863]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6217worker2 # [ 267.475661] nix-daemon[863]: reaped child process 1877, status = succeeded6218lb # [ 267.389005] niks3-server[1160]: 2026/09/22 11:00:59 INFO Received uploads request method=POST path=/api/pending_closures6219worker2 # [ 267.498944] nix-grpc-daemon[1720]: time=2026-09-22T11:00:59.976Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6220worker2 # [ 267.500501] nix-grpc-daemon[1720]: time=2026-09-22T11:00:59.976Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6221lb # [ 267.419939] niks3-server[1160]: 2026/09/22 11:00:59 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6222lb # [ 267.431611] niks3-server[1160]: 2026/09/22 11:00:59 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6223lb # [ 267.445902] niks3-server[1160]: 2026/09/22 11:00:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6224worker2 # [ 267.550252] nix-grpc-daemon[1720]: time=2026-09-22T11:01:00.028Z level=INFO msg="Uploading 1 narinfos"6225lb # [ 267.455751] niks3-server[1160]: 2026/09/22 11:00:59 INFO Signed narinfos id=52 count=16226lb # [ 267.461819] niks3-server[1160]: 2026/09/22 11:00:59 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6227lb # [ 267.469079] niks3-server[1160]: 2026/09/22 11:00:59 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6228worker2 # [ 267.573696] nix-grpc-daemon[1720]: time=2026-09-22T11:01:00.051Z level=INFO msg="Upload complete. (292ms)"6229lb # [ 267.476616] niks3-server[1160]: 2026/09/22 11:00:59 INFO Completed upload id=526230lb # [ 267.480986] niks3-server[1160]: 2026/09/22 11:01:00 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6231worker2 # [ 267.587370] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16232worker2 # [ 267.602844] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:00Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06233worker2 # [ 267.606295] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:00Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16234worker2 # [ 267.610048] nix-daemon[863]: reaped child process 1856, status = succeeded6235worker2 # [ 267.644233] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786236worker2 # [ 267.659436] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786237worker2 # [ 267.673651] nix-grpc-daemon[1720]: time=2026-09-22T11:01:00.151Z level=INFO msg="All 2 paths already cached"6238client # copying 1 paths to 'lb:50051'...6239worker2 # [ 267.683854] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501786240client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6241worker2 # [ 267.701506] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6242worker2 # [ 267.911992] nix-daemon[863]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6243worker2 # [ 267.931888] nix-daemon[863]: reaped child process 1886, status = succeeded6244worker2 # [ 268.110785] nix-daemon[863]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)6245worker2 # [ 268.127824] nix-daemon[863]: reaped child process 1891, status = succeeded6246lb # [ 268.040409] niks3-server[1160]: 2026/09/22 11:01:00 INFO Received uploads request method=POST path=/api/pending_closures6247worker2 # [ 268.149295] nix-grpc-daemon[1720]: time=2026-09-22T11:01:00.627Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6248worker2 # [ 268.150819] nix-grpc-daemon[1720]: time=2026-09-22T11:01:00.627Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6249lb # [ 268.067188] niks3-server[1160]: 2026/09/22 11:01:00 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6250lb # [ 268.080605] niks3-server[1160]: 2026/09/22 11:01:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6251worker2 # [ 268.189116] nix-grpc-daemon[1720]: time=2026-09-22T11:01:00.666Z level=INFO msg="Uploading 1 narinfos"6252lb # [ 268.091286] niks3-server[1160]: 2026/09/22 11:01:00 INFO Signed narinfos id=53 count=16253lb # [ 268.098190] niks3-server[1160]: 2026/09/22 11:01:00 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6254lb # [ 268.110775] niks3-server[1160]: 2026/09/22 11:01:00 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6255lb # [ 268.116857] niks3-server[1160]: 2026/09/22 11:01:00 INFO Completed upload id=536256worker2 # [ 268.216533] nix-grpc-daemon[1720]: time=2026-09-22T11:01:00.694Z level=INFO msg="Upload complete. (497ms)"6257lb # [ 268.120484] niks3-server[1160]: 2026/09/22 11:01:00 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6258worker2 # [ 268.231473] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=0 paths=1 nar_bytes_in=7686259worker2 # [ 268.235409] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786260worker2 # [ 268.239489] nix-daemon[863]: reaped child process 1881, status = succeeded6261worker2 # [ 268.246226] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6262client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6263worker2 # [ 268.258752] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6264client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6265client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6266client # farm-top> LOG-farm-top-during-drain6267worker2 # [ 269.506820] nix-daemon[863]: reaped child process 1895, status = succeeded6268worker2 # [ 269.686412] nix-daemon[863]: accepted connection from pid 1911, user nix-grpc-daemon (trusted)6269worker2 # [ 269.706140] nix-daemon[863]: reaped child process 1914, status = succeeded6270lb # [ 269.619120] niks3-server[1160]: 2026/09/22 11:01:02 INFO Received uploads request method=POST path=/api/pending_closures6271worker2 # [ 269.730055] nix-grpc-daemon[1720]: time=2026-09-22T11:01:02.206Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6272worker2 # [ 269.731562] nix-grpc-daemon[1720]: time=2026-09-22T11:01:02.206Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6273lb # [ 269.655027] niks3-server[1160]: 2026/09/22 11:01:02 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6274lb # [ 269.667235] niks3-server[1160]: 2026/09/22 11:01:02 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6275lb # [ 269.675928] niks3-server[1160]: 2026/09/22 11:01:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6276worker2 # [ 269.781849] nix-grpc-daemon[1720]: time=2026-09-22T11:01:02.259Z level=INFO msg="Uploading 1 narinfos"6277lb # [ 269.682053] niks3-server[1160]: 2026/09/22 11:01:02 INFO Signed narinfos id=54 count=16278lb # [ 269.687310] niks3-server[1160]: 2026/09/22 11:01:02 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6279lb # [ 269.697274] niks3-server[1160]: 2026/09/22 11:01:02 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6280worker2 # [ 269.802122] nix-grpc-daemon[1720]: time=2026-09-22T11:01:02.280Z level=INFO msg="Upload complete. (328ms)"6281lb # [ 269.704589] niks3-server[1160]: 2026/09/22 11:01:02 INFO Completed upload id=546282lb # [ 269.708751] niks3-server[1160]: 2026/09/22 11:01:02 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6283worker2 # [ 269.813951] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16284worker2 # [ 269.829023] nix-daemon[863]: reaped child process 1893, status = succeeded6285client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain >&2, in 6.57 seconds)6286client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath6287client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag during-drain outPath, in 0.23 seconds)6288worker1: must succeed: pkill -f 'read -t [9]0 x'6289worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6290client: waiting for success: ! systemctl is-active sw6291worker1 # [ 270.255301] nix-daemon[840]: reaped child process 1667, status = succeeded6292worker1 # [ 270.431774] nix-daemon[840]: accepted connection from pid 1708, user nix-grpc-daemon (trusted)6293worker1 # [ 270.450947] nix-daemon[840]: reaped child process 1711, status = succeeded6294lb # [ 270.408102] niks3-server[1160]: 2026/09/22 11:01:02 INFO Received uploads request method=POST path=/api/pending_closures6295worker1 # [ 270.472383] nix-grpc-daemon[1605]: time=2026-09-22T11:01:02.995Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6296worker1 # [ 270.474779] nix-grpc-daemon[1605]: time=2026-09-22T11:01:02.995Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6297lb # [ 270.443983] niks3-server[1160]: 2026/09/22 11:01:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6298lb # [ 270.458031] niks3-server[1160]: 2026/09/22 11:01:02 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6299lb # [ 270.466779] niks3-server[1160]: 2026/09/22 11:01:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6300worker1 # [ 270.529127] nix-grpc-daemon[1605]: time=2026-09-22T11:01:03.051Z level=INFO msg="Uploading 1 narinfos"6301lb # [ 270.473087] niks3-server[1160]: 2026/09/22 11:01:02 INFO Signed narinfos id=55 count=16302lb # [ 270.479155] niks3-server[1160]: 2026/09/22 11:01:02 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6303lb # [ 270.489411] niks3-server[1160]: 2026/09/22 11:01:03 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6304lb # [ 270.497823] niks3-server[1160]: 2026/09/22 11:01:03 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6305lb # [ 270.502696] niks3-server[1160]: 2026/09/22 11:01:03 INFO Completed upload id=556306worker1 # [ 270.557409] nix-grpc-daemon[1605]: time=2026-09-22T11:01:03.080Z level=INFO msg="Upload complete. (330ms)"6307worker1 # [ 270.569364] nix-grpc-daemon[1499]: ts=2026-09-22T11:01:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35146 duration_s=9 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16308worker1 # [ 270.586373] nix-daemon[840]: reaped child process 1665, status = succeeded6309client # [ 270.573612] systemd[1]: sw.service: Deactivated successfully.6310client # [ 270.578139] systemd[1]: sw.service: Consumed 189ms CPU time over 10.189s wall clock time, 41.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6311client: (finished: waiting for success: ! systemctl is-active sw, in 1.12 seconds)6312client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6313client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6314worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6315worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.07 seconds)6316lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6317lb: must succeed: curl -sf localhost:9901/clusters6318lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6319worker1 # [ 271.475645] nix-grpc-daemon[1499]: ts=2026-09-22T11:01:03Z level=info event=signal_exit6320lb # [ 271.872619] envoy[923]: conn peer=[2001:db8:1::1]:34440 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=4130 ms=62686321lb: must succeed: curl -sf localhost:9901/clusters6322lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6323lb # [ 271.878478] envoy[923]: conn peer=[2001:db8:1::1]:48386 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=1430 ms=98906324worker2 # [ 272.804009] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:576766325worker1 # [ 272.761044] nix-grpc-daemon[1499]: ts=2026-09-22T11:01:05Z level=info event=scheduler_restarting6326worker2 # [ 272.804436] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:05Z level=info event=worker_gone addr=192.168.1.3:500516327worker1 # [ 272.784898] nix-grpc-daemon[1499]: ts=2026-09-22T11:01:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6328worker1 # [ 272.808974] nix-daemon[840]: reaped child process 1571, status = succeeded6329worker1 # [ 273.392523] nix-grpc-daemon[1499]: E0000 00:00:1790074865.915755 1544 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 15, started true, shutdown true6330worker1 # [ 273.392922] nix-grpc-daemon[1499]: E0000 00:00:1790074865.915966 1544 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6331lb: must succeed: curl -sf localhost:9901/clusters6332worker1 # [ 273.575712] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6333worker1 # [ 273.582845] systemd[1]: nix-grpc-daemon.service: Consumed 1.052s CPU time over 2min 355ms wall clock time, 33.1M memory peak, 24.4K incoming IP traffic, 32.6K outgoing IP traffic.6334lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6335worker1 # [ 274.564977] systemd[1]: Starting Nix worker-protocol over gRPC...6336lb: must succeed: curl -sf localhost:9901/clusters6337lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6338worker1 # [ 274.773732] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6339worker1 # [ 274.776328] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6340worker1 # [ 274.780353] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16341worker1 # [ 274.788076] nix-grpc-daemon[1723]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6342worker1 # [ 274.789530] nix-grpc-daemon[1723]: E0000 00:00:1790074867.310117 1723 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6343worker1 # [ 274.802041] nix-grpc-daemon[1723]: E0000 00:00:1790074867.316365 1723 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6344worker1 # [ 274.806655] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6345worker1 # [ 274.811039] systemd[1]: Started Nix worker-protocol over gRPC.6346worker1 # [ 274.816108] nix-grpc-daemon[1723]: E0000 00:00:1790074867.336963 1763 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6347worker1 # [ 274.844047] nix-grpc-daemon[1723]: E0000 00:00:1790074867.354486 1763 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6348worker1 # [ 274.848634] nix-grpc-daemon[1723]: E0000 00:00:1790074867.354630 1763 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6349worker1 # [ 274.851422] nix-grpc-daemon[1723]: E0000 00:00:1790074867.371434 1763 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6350worker1 # [ 274.860760] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:07Z level=info event=scheduler_connected addr=lb:500516351worker2 # [ 274.907307] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:07Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:576766352worker2 # [ 274.914214] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:07Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06353lb: must succeed: curl -sf localhost:9901/clusters6354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6355lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 4.42 seconds)6356lb: must succeed: curl -sf localhost:9901/clusters6357lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6358worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6359worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6360(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.56 seconds)6361subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6362client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16363worker2 # [ 276.241408] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:08Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:501786364worker1 # [ 276.204966] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526365worker1 # [ 276.219713] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6366worker1 # [ 276.243999] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 86 ms (attempt 1/5)6367worker1 # [ 276.337647] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)6368worker1 # [ 276.543281] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 3/5)6369worker1 # [ 276.600068] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 4/5)6370worker1 # [ 276.659872] nix-grpc-daemon[1723]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6371worker1 # [ 276.666665] nix-grpc-daemon[1723]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6372worker2 # [ 276.759225] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:09Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06373worker2 # [ 276.765378] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:09Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16374worker2 # [ 276.771424] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:09Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06375worker2 # [ 276.776694] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:09Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06376worker1 # [ 276.759883] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526377worker1 # [ 276.777591] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526378worker1 # [ 276.792107] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526379worker1 # [ 276.807522] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6380worker1 # [ 276.837483] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:09Z level=info event=niks3_push_started pid=17836381worker1 # [ 277.054570] nix-grpc-daemon[1783]: time=2026-09-22T11:01:09.577Z level=INFO msg="Configuring client TLS" cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.pem key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/worker1.key ca=""6382worker1 # [ 277.234884] nix-daemon[840]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6383worker1 # [ 277.252077] nix-daemon[840]: reaped child process 1799, status = succeeded6384worker1 # [ 277.429422] nix-daemon[840]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6385worker1 # [ 277.447371] nix-daemon[840]: reaped child process 1804, status = succeeded6386lb # [ 277.405214] niks3-server[1160]: 2026/09/22 11:01:09 INFO Received uploads request method=POST path=/api/pending_closures6387worker1 # [ 277.470797] nix-grpc-daemon[1783]: time=2026-09-22T11:01:09.993Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6388worker1 # [ 277.472710] nix-grpc-daemon[1783]: time=2026-09-22T11:01:09.993Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6389lb # [ 277.442010] niks3-server[1160]: 2026/09/22 11:01:09 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6390worker1 # [ 277.517784] nix-grpc-daemon[1783]: time=2026-09-22T11:01:10.040Z level=INFO msg="Uploading 1 narinfos"6391lb # [ 277.456927] niks3-server[1160]: 2026/09/22 11:01:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6392lb # [ 277.466400] niks3-server[1160]: 2026/09/22 11:01:09 INFO Signed narinfos id=56 count=16393lb # [ 277.474753] niks3-server[1160]: 2026/09/22 11:01:09 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6394lb # [ 277.487453] niks3-server[1160]: 2026/09/22 11:01:10 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6395worker1 # [ 277.548625] nix-grpc-daemon[1783]: time=2026-09-22T11:01:10.072Z level=INFO msg="Upload complete. (489ms)"6396lb # [ 277.493926] niks3-server[1160]: 2026/09/22 11:01:10 INFO Completed upload id=566397lb # [ 277.496826] niks3-server[1160]: 2026/09/22 11:01:10 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6398worker1 # [ 277.556941] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=5126399worker1 # [ 277.568473] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526400worker1 # [ 277.581906] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6401worker1 # [ 277.587559] nix-daemon[840]: reaped child process 1781, status = succeeded6402worker1 # [ 277.603797] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6403worker1 # [ 278.853196] nix-daemon[840]: reaped child process 1808, status = succeeded6404worker1 # [ 279.032524] nix-daemon[840]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6405worker1 # [ 279.051592] nix-daemon[840]: reaped child process 1827, status = succeeded6406lb # [ 279.009455] niks3-server[1160]: 2026/09/22 11:01:11 INFO Received uploads request method=POST path=/api/pending_closures6407worker1 # [ 279.074517] nix-grpc-daemon[1783]: time=2026-09-22T11:01:11.597Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6408worker1 # [ 279.080157] nix-grpc-daemon[1783]: time=2026-09-22T11:01:11.599Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6409lb # [ 279.066118] niks3-server[1160]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6410lb # [ 279.077327] niks3-server[1160]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6411lb # [ 279.088307] niks3-server[1160]: 2026/09/22 11:01:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6412worker1 # [ 279.144232] nix-grpc-daemon[1783]: time=2026-09-22T11:01:11.666Z level=INFO msg="Uploading 1 narinfos"6413lb # [ 279.093675] niks3-server[1160]: 2026/09/22 11:01:11 INFO Signed narinfos id=57 count=16414lb # [ 279.099596] niks3-server[1160]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6415lb # [ 279.110550] niks3-server[1160]: 2026/09/22 11:01:11 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6416lb # [ 279.119310] niks3-server[1160]: 2026/09/22 11:01:11 INFO Completed upload id=576417worker1 # [ 279.174485] nix-grpc-daemon[1783]: time=2026-09-22T11:01:11.697Z level=INFO msg="Upload complete. (357ms)"6418lb # [ 279.122174] niks3-server[1160]: 2026/09/22 11:01:11 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6419worker2 # [ 279.230059] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:11Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16420worker1 # [ 279.187512] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16421worker2 # [ 279.240568] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:11Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06422worker1 # [ 279.212538] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526423worker1 # [ 279.218074] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526424worker1 # [ 279.225390] nix-daemon[840]: reaped child process 1806, status = succeeded6425worker1 # [ 279.230604] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526426worker1 # [ 279.241102] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6427worker1 # [ 279.434657] nix-daemon[840]: accepted connection from pid 1831, user nix-grpc-daemon (trusted)6428worker1 # [ 279.451607] nix-daemon[840]: reaped child process 1834, status = succeeded6429worker1 # [ 279.628827] nix-daemon[840]: accepted connection from pid 1836, user nix-grpc-daemon (trusted)6430worker1 # [ 279.645183] nix-daemon[840]: reaped child process 1839, status = succeeded6431lb # [ 279.603673] niks3-server[1160]: 2026/09/22 11:01:12 INFO Received uploads request method=POST path=/api/pending_closures6432worker1 # [ 279.669025] nix-grpc-daemon[1783]: time=2026-09-22T11:01:12.192Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6433worker1 # [ 279.670502] nix-grpc-daemon[1783]: time=2026-09-22T11:01:12.192Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6434lb # [ 279.631432] niks3-server[1160]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6435lb # [ 279.645656] niks3-server[1160]: 2026/09/22 11:01:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6436worker1 # [ 279.706330] nix-grpc-daemon[1783]: time=2026-09-22T11:01:12.229Z level=INFO msg="Uploading 1 narinfos"6437lb # [ 279.656351] niks3-server[1160]: 2026/09/22 11:01:12 INFO Signed narinfos id=58 count=16438lb # [ 279.662851] niks3-server[1160]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6439lb # [ 279.670885] niks3-server[1160]: 2026/09/22 11:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6440worker1 # [ 279.731782] nix-grpc-daemon[1783]: time=2026-09-22T11:01:12.255Z level=INFO msg="Upload complete. (476ms)"6441lb # [ 279.677768] niks3-server[1160]: 2026/09/22 11:01:12 INFO Completed upload id=586442lb # [ 279.681548] niks3-server[1160]: 2026/09/22 11:01:12 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6443worker1 # [ 279.740279] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=5126444worker1 # [ 279.751657] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526445worker1 # [ 279.757999] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6446worker1 # [ 279.762719] nix-daemon[840]: reaped child process 1829, status = succeeded6447worker1 # [ 279.776129] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6448worker1 # [ 281.021456] nix-daemon[840]: reaped child process 1843, status = succeeded6449worker1 # [ 281.210062] nix-daemon[840]: accepted connection from pid 1859, user nix-grpc-daemon (trusted)6450worker1 # [ 281.228145] nix-daemon[840]: reaped child process 1862, status = succeeded6451lb # [ 281.189117] niks3-server[1160]: 2026/09/22 11:01:13 INFO Received uploads request method=POST path=/api/pending_closures6452worker1 # [ 281.254825] nix-grpc-daemon[1783]: time=2026-09-22T11:01:13.777Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6453worker1 # [ 281.256555] nix-grpc-daemon[1783]: time=2026-09-22T11:01:13.777Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6454lb # [ 281.230698] niks3-server[1160]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6455worker1 # [ 281.299468] nix-grpc-daemon[1783]: time=2026-09-22T11:01:13.822Z level=INFO msg="Uploading 1 narinfos"6456lb # [ 281.246155] niks3-server[1160]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6457lb # [ 281.255918] niks3-server[1160]: 2026/09/22 11:01:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6458lb # [ 281.260938] niks3-server[1160]: 2026/09/22 11:01:13 INFO Signed narinfos id=59 count=16459lb # [ 281.268146] niks3-server[1160]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6460worker1 # [ 281.322935] nix-grpc-daemon[1783]: time=2026-09-22T11:01:13.846Z level=INFO msg="Upload complete. (333ms)"6461lb # [ 281.271433] niks3-server[1160]: 2026/09/22 11:01:13 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6462lb # [ 281.275574] niks3-server[1160]: 2026/09/22 11:01:13 INFO Completed upload id=596463lb # [ 281.278273] niks3-server[1160]: 2026/09/22 11:01:13 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6464worker2 # [ 281.377421] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:13Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16465worker1 # [ 281.336278] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16466worker2 # [ 281.390249] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:13Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06467worker1 # [ 281.350878] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526468worker1 # [ 281.356835] nix-daemon[840]: reaped child process 1841, status = succeeded6469worker1 # [ 281.364252] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526470worker1 # [ 281.376980] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526471worker1 # [ 281.388370] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6472worker1 # [ 281.583526] nix-daemon[840]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)6473worker1 # [ 281.600132] nix-daemon[840]: reaped child process 1869, status = succeeded6474worker1 # [ 281.776941] nix-daemon[840]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6475worker1 # [ 281.793217] nix-daemon[840]: reaped child process 1874, status = succeeded6476lb # [ 281.752230] niks3-server[1160]: 2026/09/22 11:01:14 INFO Received uploads request method=POST path=/api/pending_closures6477worker1 # [ 281.815892] nix-grpc-daemon[1783]: time=2026-09-22T11:01:14.339Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6478worker1 # [ 281.817539] nix-grpc-daemon[1783]: time=2026-09-22T11:01:14.339Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6479lb # [ 281.781354] niks3-server[1160]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6480worker1 # [ 281.850471] nix-grpc-daemon[1783]: time=2026-09-22T11:01:14.373Z level=INFO msg="Uploading 1 narinfos"6481lb # [ 281.797599] niks3-server[1160]: 2026/09/22 11:01:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6482lb # [ 281.807948] niks3-server[1160]: 2026/09/22 11:01:14 INFO Signed narinfos id=60 count=16483lb # [ 281.813591] niks3-server[1160]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6484lb # [ 281.823417] niks3-server[1160]: 2026/09/22 11:01:14 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6485worker1 # [ 281.877814] nix-grpc-daemon[1783]: time=2026-09-22T11:01:14.401Z level=INFO msg="Upload complete. (477ms)"6486lb # [ 281.828425] niks3-server[1160]: 2026/09/22 11:01:14 INFO Completed upload id=606487worker1 # [ 281.884352] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=5126488lb # [ 281.831701] niks3-server[1160]: 2026/09/22 11:01:14 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6489worker1 # [ 281.895949] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526490worker1 # [ 281.899338] nix-daemon[840]: reaped child process 1864, status = succeeded6491worker1 # [ 281.906022] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6492worker1 # [ 281.918646] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6493lb # [ 281.872851] envoy[923]: conn peer=[2001:db8:1::3]:34084 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2387 tx=893 ms=1192676494worker1 # [ 282.940656] nix-daemon[840]: reaped child process 1878, status = succeeded6495worker1 # [ 283.103484] nix-daemon[840]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6496worker1 # [ 283.120501] nix-daemon[840]: reaped child process 1897, status = succeeded6497lb # [ 283.081498] niks3-server[1160]: 2026/09/22 11:01:15 INFO Received uploads request method=POST path=/api/pending_closures6498worker1 # [ 283.148420] nix-grpc-daemon[1783]: time=2026-09-22T11:01:15.670Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6499worker1 # [ 283.149921] nix-grpc-daemon[1783]: time=2026-09-22T11:01:15.670Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6500lb # [ 283.123622] niks3-server[1160]: 2026/09/22 11:01:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6501worker1 # [ 283.187657] nix-grpc-daemon[1783]: time=2026-09-22T11:01:15.710Z level=INFO msg="Uploading 1 narinfos"6502lb # [ 283.138644] niks3-server[1160]: 2026/09/22 11:01:15 INFO Signed narinfos id=61 count=16503lb # [ 283.149068] niks3-server[1160]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6504lb # [ 283.159311] niks3-server[1160]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6505lb # [ 283.167888] niks3-server[1160]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6506worker1 # [ 283.227568] nix-grpc-daemon[1783]: time=2026-09-22T11:01:15.750Z level=INFO msg="Upload complete. (300ms)"6507lb # [ 283.175841] niks3-server[1160]: 2026/09/22 11:01:15 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6508lb # [ 283.182162] niks3-server[1160]: 2026/09/22 11:01:15 INFO Completed upload id=616509worker1 # [ 283.236866] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16510lb # [ 283.185755] niks3-server[1160]: 2026/09/22 11:01:15 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6511worker2 # [ 283.284200] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:15Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16512worker1 # [ 283.251544] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526513worker1 # [ 283.256428] nix-daemon[840]: reaped child process 1876, status = succeeded6514worker2 # [ 283.298581] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:15Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06515worker1 # [ 283.267778] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526516worker1 # [ 283.281906] nix-grpc-daemon[1783]: time=2026-09-22T11:01:15.805Z level=INFO msg="All 1 paths already cached"6517worker1 # [ 283.290083] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526518worker1 # [ 283.301926] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6519worker1 # [ 283.492110] nix-daemon[840]: accepted connection from pid 1901, user nix-grpc-daemon (trusted)6520worker1 # [ 283.512205] nix-daemon[840]: reaped child process 1904, status = succeeded6521worker1 # [ 283.685581] nix-daemon[840]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6522worker1 # [ 283.704546] nix-daemon[840]: reaped child process 1909, status = succeeded6523lb # [ 283.665213] niks3-server[1160]: 2026/09/22 11:01:16 INFO Received uploads request method=POST path=/api/pending_closures6524worker1 # [ 283.729959] nix-grpc-daemon[1783]: time=2026-09-22T11:01:16.253Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6525worker1 # [ 283.731484] nix-grpc-daemon[1783]: time=2026-09-22T11:01:16.253Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6526lb # [ 283.697050] niks3-server[1160]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6527worker1 # [ 283.762252] nix-grpc-daemon[1783]: time=2026-09-22T11:01:16.285Z level=INFO msg="Uploading 1 narinfos"6528lb # [ 283.710351] niks3-server[1160]: 2026/09/22 11:01:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6529lb # [ 283.720709] niks3-server[1160]: 2026/09/22 11:01:16 INFO Signed narinfos id=62 count=16530lb # [ 283.726669] niks3-server[1160]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6531worker1 # [ 283.790585] nix-grpc-daemon[1783]: time=2026-09-22T11:01:16.314Z level=INFO msg="Upload complete. (475ms)"6532lb # [ 283.736123] niks3-server[1160]: 2026/09/22 11:01:16 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6533lb # [ 283.741247] niks3-server[1160]: 2026/09/22 11:01:16 INFO Completed upload id=626534lb # [ 283.745074] niks3-server[1160]: 2026/09/22 11:01:16 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6535worker1 # [ 283.800708] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=6246536worker1 # [ 283.808837] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526537worker1 # [ 283.818278] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6538worker1 # [ 283.823029] nix-daemon[840]: reaped child process 1899, status = succeeded6539worker1 # [ 283.836166] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6540worker1 # [ 285.082224] nix-daemon[840]: reaped child process 1913, status = succeeded6541worker1 # [ 285.272674] nix-daemon[840]: accepted connection from pid 1929, user nix-grpc-daemon (trusted)6542worker1 # [ 285.292505] nix-daemon[840]: reaped child process 1932, status = succeeded6543lb # [ 285.250780] niks3-server[1160]: 2026/09/22 11:01:17 INFO Received uploads request method=POST path=/api/pending_closures6544worker1 # [ 285.320147] nix-grpc-daemon[1783]: time=2026-09-22T11:01:17.840Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6545worker1 # [ 285.321699] nix-grpc-daemon[1783]: time=2026-09-22T11:01:17.840Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6546lb # [ 285.283670] niks3-server[1160]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6547lb # [ 285.304388] niks3-server[1160]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6548worker1 # [ 285.367609] nix-grpc-daemon[1783]: time=2026-09-22T11:01:17.890Z level=INFO msg="Uploading 1 narinfos"6549lb # [ 285.315946] niks3-server[1160]: 2026/09/22 11:01:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6550lb # [ 285.319410] niks3-server[1160]: 2026/09/22 11:01:17 INFO Signed narinfos id=63 count=16551worker1 # [ 285.384994] nix-grpc-daemon[1783]: time=2026-09-22T11:01:17.908Z level=INFO msg="Upload complete. (335ms)"6552lb # [ 285.330989] niks3-server[1160]: 2026/09/22 11:01:17 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6553lb # [ 285.336199] niks3-server[1160]: 2026/09/22 11:01:17 INFO Completed upload id=636554lb # [ 285.338539] niks3-server[1160]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6555worker2 # [ 285.436934] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:17Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16556lb # [ 285.341857] niks3-server[1160]: 2026/09/22 11:01:17 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6557worker1 # [ 285.394743] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16558worker2 # [ 285.451845] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:17Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06559worker1 # [ 285.411200] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526560worker1 # [ 285.420843] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526561worker1 # [ 285.425834] nix-daemon[840]: reaped child process 1911, status = succeeded6562worker1 # [ 285.432268] nix-grpc-daemon[1783]: time=2026-09-22T11:01:17.955Z level=INFO msg="All 1 paths already cached"6563worker1 # [ 285.438973] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526564worker1 # [ 285.448519] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6565worker1 # [ 285.633772] nix-daemon[840]: accepted connection from pid 1936, user nix-grpc-daemon (trusted)6566worker1 # [ 285.651833] nix-daemon[840]: reaped child process 1939, status = succeeded6567worker1 # [ 285.829563] nix-daemon[840]: accepted connection from pid 1941, user nix-grpc-daemon (trusted)6568worker1 # [ 285.847140] nix-daemon[840]: reaped child process 1944, status = succeeded6569lb # [ 285.803806] niks3-server[1160]: 2026/09/22 11:01:18 INFO Received uploads request method=POST path=/api/pending_closures6570worker1 # [ 285.868924] nix-grpc-daemon[1783]: time=2026-09-22T11:01:18.392Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6571worker1 # [ 285.870822] nix-grpc-daemon[1783]: time=2026-09-22T11:01:18.394Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6572lb # [ 285.832112] niks3-server[1160]: 2026/09/22 11:01:18 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6573worker1 # [ 285.900346] nix-grpc-daemon[1783]: time=2026-09-22T11:01:18.423Z level=INFO msg="Uploading 1 narinfos"6574lb # [ 285.847010] niks3-server[1160]: 2026/09/22 11:01:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6575lb # [ 285.858251] niks3-server[1160]: 2026/09/22 11:01:18 INFO Signed narinfos id=64 count=16576lb # [ 285.864342] niks3-server[1160]: 2026/09/22 11:01:18 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6577lb # [ 285.870858] niks3-server[1160]: 2026/09/22 11:01:18 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6578lb # [ 285.878350] niks3-server[1160]: 2026/09/22 11:01:18 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6579lb # [ 285.882723] niks3-server[1160]: 2026/09/22 11:01:18 INFO Completed upload id=646580worker1 # [ 285.937283] nix-grpc-daemon[1783]: time=2026-09-22T11:01:18.460Z level=INFO msg="Upload complete. (476ms)"6581worker1 # [ 285.944677] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=6246582worker1 # [ 285.957692] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526583worker1 # [ 285.963700] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6584worker1 # [ 285.968663] nix-daemon[840]: reaped child process 1934, status = succeeded6585worker1 # [ 285.980217] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6586worker1 # [ 286.830211] nix-daemon[840]: reaped child process 1948, status = succeeded6587worker1 # [ 287.005141] nix-daemon[840]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6588worker1 # [ 287.022296] nix-daemon[840]: reaped child process 1967, status = succeeded6589lb # [ 286.985046] niks3-server[1160]: 2026/09/22 11:01:19 INFO Received uploads request method=POST path=/api/pending_closures6590worker1 # [ 287.052591] nix-grpc-daemon[1783]: time=2026-09-22T11:01:19.574Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6591worker1 # [ 287.054869] nix-grpc-daemon[1783]: time=2026-09-22T11:01:19.574Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6592lb # [ 287.029048] niks3-server[1160]: 2026/09/22 11:01:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6593worker1 # [ 287.092641] nix-grpc-daemon[1783]: time=2026-09-22T11:01:19.615Z level=INFO msg="Uploading 1 narinfos"6594lb # [ 287.042405] niks3-server[1160]: 2026/09/22 11:01:19 INFO Signed narinfos id=65 count=16595lb # [ 287.050544] niks3-server[1160]: 2026/09/22 11:01:19 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6596lb # [ 287.064407] niks3-server[1160]: 2026/09/22 11:01:19 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6597lb # [ 287.070795] niks3-server[1160]: 2026/09/22 11:01:19 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6598lb # [ 287.075659] niks3-server[1160]: 2026/09/22 11:01:19 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6599worker1 # [ 287.125386] nix-grpc-daemon[1783]: time=2026-09-22T11:01:19.648Z level=INFO msg="Upload complete. (334ms)"6600lb # [ 287.081372] niks3-server[1160]: 2026/09/22 11:01:19 INFO Completed upload id=656601lb # [ 287.084839] niks3-server[1160]: 2026/09/22 11:01:19 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6602worker2 # [ 287.183174] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:19Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16603worker1 # [ 287.137630] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16604worker2 # [ 287.197329] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:19Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06605worker1 # [ 287.158866] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526606worker1 # [ 287.165193] nix-daemon[840]: reaped child process 1946, status = succeeded6607worker1 # [ 287.172643] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526608worker1 # [ 287.181403] nix-grpc-daemon[1783]: time=2026-09-22T11:01:19.704Z level=INFO msg="All 1 paths already cached"6609worker1 # [ 287.188288] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526610worker1 # [ 287.195271] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6611worker1 # [ 287.370119] nix-daemon[840]: accepted connection from pid 1971, user nix-grpc-daemon (trusted)6612worker1 # [ 287.396522] nix-daemon[840]: reaped child process 1974, status = succeeded6613worker1 # [ 287.574505] nix-daemon[840]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6614worker1 # [ 287.596966] nix-daemon[840]: reaped child process 1979, status = succeeded6615lb # [ 287.554247] niks3-server[1160]: 2026/09/22 11:01:20 INFO Received uploads request method=POST path=/api/pending_closures6616worker1 # [ 287.619396] nix-grpc-daemon[1783]: time=2026-09-22T11:01:20.142Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6617worker1 # [ 287.621877] nix-grpc-daemon[1783]: time=2026-09-22T11:01:20.144Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6618lb # [ 287.578498] niks3-server[1160]: 2026/09/22 11:01:20 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6619lb # [ 287.584063] niks3-server[1160]: 2026/09/22 11:01:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6620worker1 # [ 287.642602] nix-grpc-daemon[1783]: time=2026-09-22T11:01:20.165Z level=INFO msg="Uploading 1 narinfos"6621lb # [ 287.587429] niks3-server[1160]: 2026/09/22 11:01:20 INFO Signed narinfos id=66 count=16622lb # [ 287.590270] niks3-server[1160]: 2026/09/22 11:01:20 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6623lb # [ 287.597723] niks3-server[1160]: 2026/09/22 11:01:20 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6624lb # [ 287.601549] niks3-server[1160]: 2026/09/22 11:01:20 INFO Completed upload id=666625worker1 # [ 287.655936] nix-grpc-daemon[1783]: time=2026-09-22T11:01:20.179Z level=INFO msg="Upload complete. (452ms)"6626lb # [ 287.603617] niks3-server[1160]: 2026/09/22 11:01:20 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6627worker1 # [ 287.664200] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=6246628worker1 # [ 287.667888] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526629worker1 # [ 287.677778] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6630worker1 # [ 287.681759] nix-daemon[840]: reaped child process 1969, status = succeeded6631worker1 # [ 287.693463] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6632worker1 # [ 288.720275] nix-daemon[840]: reaped child process 1983, status = succeeded6633worker1 # [ 288.890911] nix-daemon[840]: accepted connection from pid 1999, user nix-grpc-daemon (trusted)6634worker1 # [ 288.910724] nix-daemon[840]: reaped child process 2002, status = succeeded6635lb # [ 288.870029] niks3-server[1160]: 2026/09/22 11:01:21 INFO Received uploads request method=POST path=/api/pending_closures6636worker1 # [ 288.936182] nix-grpc-daemon[1783]: time=2026-09-22T11:01:21.458Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6637worker1 # [ 288.937698] nix-grpc-daemon[1783]: time=2026-09-22T11:01:21.458Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6638lb # [ 288.903793] niks3-server[1160]: 2026/09/22 11:01:21 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6639lb # [ 288.917806] niks3-server[1160]: 2026/09/22 11:01:21 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6640lb # [ 288.932770] niks3-server[1160]: 2026/09/22 11:01:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6641worker1 # [ 288.994898] nix-grpc-daemon[1783]: time=2026-09-22T11:01:21.518Z level=INFO msg="Uploading 1 narinfos"6642lb # [ 288.943320] niks3-server[1160]: 2026/09/22 11:01:21 INFO Signed narinfos id=67 count=16643lb # [ 288.952335] niks3-server[1160]: 2026/09/22 11:01:21 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6644lb # [ 288.959968] niks3-server[1160]: 2026/09/22 11:01:21 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6645lb # [ 288.966794] niks3-server[1160]: 2026/09/22 11:01:21 INFO Completed upload id=676646lb # [ 288.969682] niks3-server[1160]: 2026/09/22 11:01:21 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6647worker1 # [ 289.021812] nix-grpc-daemon[1783]: time=2026-09-22T11:01:21.545Z level=INFO msg="Upload complete. (320ms)"6648worker2 # [ 289.076935] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:21Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16649worker1 # [ 289.035671] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16650worker2 # [ 289.091338] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:21Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06651worker1 # [ 289.053075] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526652worker1 # [ 289.061748] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526653worker1 # [ 289.067323] nix-daemon[840]: reaped child process 1981, status = succeeded6654worker1 # [ 289.070309] nix-grpc-daemon[1783]: time=2026-09-22T11:01:21.593Z level=INFO msg="All 2 paths already cached"6655worker1 # [ 289.078294] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526656worker1 # [ 289.089190] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6657worker1 # [ 289.289901] nix-daemon[840]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)6658worker1 # [ 289.309404] nix-daemon[840]: reaped child process 2009, status = succeeded6659worker1 # [ 289.494481] nix-daemon[840]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)6660worker1 # [ 289.513162] nix-daemon[840]: reaped child process 2014, status = succeeded6661lb # [ 289.469755] niks3-server[1160]: 2026/09/22 11:01:21 INFO Received uploads request method=POST path=/api/pending_closures6662worker1 # [ 289.533938] nix-grpc-daemon[1783]: time=2026-09-22T11:01:22.057Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6663worker1 # [ 289.535506] nix-grpc-daemon[1783]: time=2026-09-22T11:01:22.057Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6664lb # [ 289.501145] niks3-server[1160]: 2026/09/22 11:01:22 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6665lb # [ 289.512134] niks3-server[1160]: 2026/09/22 11:01:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6666worker1 # [ 289.574232] nix-grpc-daemon[1783]: time=2026-09-22T11:01:22.097Z level=INFO msg="Uploading 1 narinfos"6667lb # [ 289.521730] niks3-server[1160]: 2026/09/22 11:01:22 INFO Signed narinfos id=68 count=16668lb # [ 289.527470] niks3-server[1160]: 2026/09/22 11:01:22 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6669lb # [ 289.538272] niks3-server[1160]: 2026/09/22 11:01:22 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6670worker1 # [ 289.598372] nix-grpc-daemon[1783]: time=2026-09-22T11:01:22.121Z level=INFO msg="Upload complete. (496ms)"6671lb # [ 289.543530] niks3-server[1160]: 2026/09/22 11:01:22 INFO Completed upload id=686672lb # [ 289.546424] niks3-server[1160]: 2026/09/22 11:01:22 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6673worker1 # [ 289.604319] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=7526674worker1 # [ 289.618457] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526675worker1 # [ 289.625042] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6676worker1 # [ 289.630077] nix-daemon[840]: reaped child process 2004, status = succeeded6677worker1 # [ 289.644214] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6678worker1 # [ 290.667208] nix-daemon[840]: reaped child process 2018, status = succeeded6679worker1 # [ 290.832909] nix-daemon[840]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6680worker1 # [ 290.852449] nix-daemon[840]: reaped child process 2037, status = succeeded6681lb # [ 290.811285] niks3-server[1160]: 2026/09/22 11:01:23 INFO Received uploads request method=POST path=/api/pending_closures6682worker1 # [ 290.876450] nix-grpc-daemon[1783]: time=2026-09-22T11:01:23.399Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6683worker1 # [ 290.877926] nix-grpc-daemon[1783]: time=2026-09-22T11:01:23.399Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6684lb # [ 290.846673] niks3-server[1160]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6685worker1 # [ 290.907421] nix-grpc-daemon[1783]: time=2026-09-22T11:01:23.430Z level=INFO msg="Uploading 1 narinfos"6686lb # [ 290.854290] niks3-server[1160]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6687lb # [ 290.860483] niks3-server[1160]: 2026/09/22 11:01:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6688lb # [ 290.864757] niks3-server[1160]: 2026/09/22 11:01:23 INFO Signed narinfos id=69 count=16689lb # [ 290.868898] niks3-server[1160]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6690worker1 # [ 290.927538] nix-grpc-daemon[1783]: time=2026-09-22T11:01:23.450Z level=INFO msg="Upload complete. (273ms)"6691lb # [ 290.872841] niks3-server[1160]: 2026/09/22 11:01:23 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6692lb # [ 290.875374] niks3-server[1160]: 2026/09/22 11:01:23 INFO Completed upload id=696693lb # [ 290.878910] niks3-server[1160]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6694worker1 # [ 290.936873] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16695worker2 # [ 290.980369] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:23Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16696worker1 # [ 290.946871] nix-daemon[840]: reaped child process 2016, status = succeeded6697worker1 # [ 290.982933] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526698worker1 # [ 290.997126] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526699worker1 # [ 291.010751] nix-grpc-daemon[1783]: time=2026-09-22T11:01:23.534Z level=INFO msg="All 2 paths already cached"6700worker1 # [ 291.019835] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526701worker1 # [ 291.033306] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6702worker1 # [ 291.231619] nix-daemon[840]: accepted connection from pid 2041, user nix-grpc-daemon (trusted)6703worker1 # [ 291.254997] nix-daemon[840]: reaped child process 2044, status = succeeded6704worker1 # [ 291.435888] nix-daemon[840]: accepted connection from pid 2046, user nix-grpc-daemon (trusted)6705worker1 # [ 291.458903] nix-daemon[840]: reaped child process 2049, status = succeeded6706lb # [ 291.417357] niks3-server[1160]: 2026/09/22 11:01:23 INFO Received uploads request method=POST path=/api/pending_closures6707worker1 # [ 291.477699] nix-grpc-daemon[1783]: time=2026-09-22T11:01:24.001Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6708worker1 # [ 291.479951] nix-grpc-daemon[1783]: time=2026-09-22T11:01:24.003Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6709lb # [ 291.436750] niks3-server[1160]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6710lb # [ 291.442237] niks3-server[1160]: 2026/09/22 11:01:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6711worker1 # [ 291.500300] nix-grpc-daemon[1783]: time=2026-09-22T11:01:24.023Z level=INFO msg="Uploading 1 narinfos"6712lb # [ 291.446377] niks3-server[1160]: 2026/09/22 11:01:23 INFO Signed narinfos id=70 count=16713lb # [ 291.451389] niks3-server[1160]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6714lb # [ 291.459950] niks3-server[1160]: 2026/09/22 11:01:23 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6715lb # [ 291.463931] niks3-server[1160]: 2026/09/22 11:01:23 INFO Completed upload id=706716worker1 # [ 291.518591] nix-grpc-daemon[1783]: time=2026-09-22T11:01:24.042Z level=INFO msg="Upload complete. (470ms)"6717lb # [ 291.465674] niks3-server[1160]: 2026/09/22 11:01:23 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6718worker1 # [ 291.524292] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=7526719worker1 # [ 291.529915] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526720worker1 # [ 291.533749] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6721worker1 # [ 291.540191] nix-daemon[840]: reaped child process 2039, status = succeeded6722worker1 # [ 291.549354] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6723worker1 # [ 292.773779] nix-daemon[840]: reaped child process 2053, status = succeeded6724worker1 # [ 292.996525] nix-daemon[840]: accepted connection from pid 2069, user nix-grpc-daemon (trusted)6725worker1 # [ 293.020160] nix-daemon[840]: reaped child process 2072, status = succeeded6726lb # [ 292.980964] niks3-server[1160]: 2026/09/22 11:01:25 INFO Received uploads request method=POST path=/api/pending_closures6727worker1 # [ 293.044201] nix-grpc-daemon[1783]: time=2026-09-22T11:01:25.566Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6728worker1 # [ 293.045847] nix-grpc-daemon[1783]: time=2026-09-22T11:01:25.566Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6729lb # [ 293.004203] niks3-server[1160]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6730lb # [ 293.010172] niks3-server[1160]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6731lb # [ 293.017255] niks3-server[1160]: 2026/09/22 11:01:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6732lb # [ 293.021333] niks3-server[1160]: 2026/09/22 11:01:25 INFO Signed narinfos id=71 count=16733worker1 # [ 293.076333] nix-grpc-daemon[1783]: time=2026-09-22T11:01:25.599Z level=INFO msg="Uploading 1 narinfos"6734lb # [ 293.024871] niks3-server[1160]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6735lb # [ 293.033561] niks3-server[1160]: 2026/09/22 11:01:25 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6736lb # [ 293.039184] niks3-server[1160]: 2026/09/22 11:01:25 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6737lb # [ 293.043318] niks3-server[1160]: 2026/09/22 11:01:25 INFO Completed upload id=716738worker1 # [ 293.098417] nix-grpc-daemon[1783]: time=2026-09-22T11:01:25.621Z level=INFO msg="Upload complete. (341ms)"6739worker2 # [ 293.148523] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:25Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16740worker1 # [ 293.106643] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16741worker1 # [ 293.119435] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526742worker1 # [ 293.122049] nix-daemon[840]: reaped child process 2051, status = succeeded6743worker1 # [ 293.129809] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526744worker1 # [ 293.138978] nix-grpc-daemon[1783]: time=2026-09-22T11:01:25.662Z level=INFO msg="All 2 paths already cached"6745worker1 # [ 293.148462] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:422526746worker1 # [ 293.155406] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6747worker1 # [ 293.433373] nix-daemon[840]: accepted connection from pid 2076, user nix-grpc-daemon (trusted)6748worker1 # [ 293.456200] nix-daemon[840]: reaped child process 2079, status = succeeded6749worker1 # [ 293.629215] nix-daemon[840]: accepted connection from pid 2081, user nix-grpc-daemon (trusted)6750worker1 # [ 293.651892] nix-daemon[840]: reaped child process 2084, status = succeeded6751lb # [ 293.608254] niks3-server[1160]: 2026/09/22 11:01:26 INFO Received uploads request method=POST path=/api/pending_closures6752worker1 # [ 293.671468] nix-grpc-daemon[1783]: time=2026-09-22T11:01:26.194Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6753worker1 # [ 293.673190] nix-grpc-daemon[1783]: time=2026-09-22T11:01:26.194Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6754lb # [ 293.635355] niks3-server[1160]: 2026/09/22 11:01:26 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6755lb # [ 293.647361] niks3-server[1160]: 2026/09/22 11:01:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6756worker1 # [ 293.705968] nix-grpc-daemon[1783]: time=2026-09-22T11:01:26.229Z level=INFO msg="Uploading 1 narinfos"6757lb # [ 293.654987] niks3-server[1160]: 2026/09/22 11:01:26 INFO Signed narinfos id=72 count=16758lb # [ 293.658527] niks3-server[1160]: 2026/09/22 11:01:26 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6759lb # [ 293.668001] niks3-server[1160]: 2026/09/22 11:01:26 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6760lb # [ 293.673862] niks3-server[1160]: 2026/09/22 11:01:26 INFO Completed upload id=726761worker1 # [ 293.729136] nix-grpc-daemon[1783]: time=2026-09-22T11:01:26.252Z level=INFO msg="Upload complete. (563ms)"6762lb # [ 293.675645] niks3-server[1160]: 2026/09/22 11:01:26 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6763worker1 # [ 293.735637] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=0 paths=1 nar_bytes_in=7526764worker1 # [ 293.740444] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:422526765worker1 # [ 293.749596] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6766worker1 # [ 293.753931] nix-daemon[840]: reaped child process 2074, status = succeeded6767worker1 # [ 293.767159] nix-daemon[840]: accepted connection from pid 1723, user nix-grpc-daemon (trusted)6768worker1 # [ 294.605006] nix-daemon[840]: reaped child process 2088, status = succeeded6769worker1 # [ 294.790953] nix-daemon[840]: accepted connection from pid 2104, user nix-grpc-daemon (trusted)6770worker1 # [ 294.809935] nix-daemon[840]: reaped child process 2107, status = succeeded6771lb # [ 294.773896] niks3-server[1160]: 2026/09/22 11:01:27 INFO Received uploads request method=POST path=/api/pending_closures6772worker1 # [ 294.840463] nix-grpc-daemon[1783]: time=2026-09-22T11:01:27.360Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6773worker1 # [ 294.842019] nix-grpc-daemon[1783]: time=2026-09-22T11:01:27.361Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6774lb # [ 294.815273] niks3-server[1160]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6775lb # [ 294.829811] niks3-server[1160]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6776worker1 # [ 294.891954] nix-grpc-daemon[1783]: time=2026-09-22T11:01:27.415Z level=INFO msg="Uploading 1 narinfos"6777lb # [ 294.839174] niks3-server[1160]: 2026/09/22 11:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6778lb # [ 294.845742] niks3-server[1160]: 2026/09/22 11:01:27 INFO Signed narinfos id=73 count=16779lb # [ 294.847618] niks3-server[1160]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6780lb # [ 294.855803] niks3-server[1160]: 2026/09/22 11:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6781lb # [ 294.859625] niks3-server[1160]: 2026/09/22 11:01:27 INFO Completed upload id=736782worker1 # [ 294.914683] nix-grpc-daemon[1783]: time=2026-09-22T11:01:27.438Z level=INFO msg="Upload complete. (338ms)"6783lb # [ 294.862133] niks3-server[1160]: 2026/09/22 11:01:27 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6784worker1 # [ 294.925074] nix-grpc-daemon[1723]: ts=2026-09-22T11:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42252 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16785worker1 # [ 294.940709] nix-daemon[840]: reaped child process 2086, status = succeeded6786client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 19.08 seconds)6787worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6788worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6789client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]'6790client # [ 295.115984] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6791client # Running as unit: gpu.service; invocation ID: 0b1733e96f19427294b1eda535eecb436792client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6793lb: must succeed: curl -sf localhost:9901/clusters6794lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6795lb: must succeed: curl -sf localhost:9901/clusters6796lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6797worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6798worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6799client # [ 295.391022] nix[1883]: don't know how to build these paths:6800client # [ 295.392316] nix[1883]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6801worker2 # [ 295.473034] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:501786802worker2 # [ 295.478915] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786803client # [ 295.422651] nix[1883]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, waiting up to 30s for a worker6804worker2 # [ 295.504266] nix-grpc-daemon[1644]: ts=2026-09-22T11:01:27Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06805lb: must succeed: curl -sf localhost:9901/clusters6806lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6807lb: must succeed: curl -sf localhost:9901/clusters6808lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6809worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6810worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6811client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6812lb # [ 301.870891] envoy[923]: conn peer=[2001:db8:1::1]:36506 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27118 tx=11352 ms=187466813lb # [ 318.981076] postgres[826]: [826] LOG: checkpoint starting: time6814client # [ 325.426786] nix[1883]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}6815client # [ 325.441663] nix[1883]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6816client # [ 325.488266] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6817client # [ 325.494811] systemd[1]: gpu.service: Failed with result 'exit-code'.6818client # [ 325.497767] systemd[1]: gpu.service: Consumed 201ms CPU time over 30.377s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.6819client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.17 seconds)6820client: must succeed: journalctl -u gpu -o cat6821client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6822lb: must succeed: curl -sf localhost:9901/clusters6823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)6824lb: must succeed: curl -sf localhost:9901/clusters6825lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6826worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6827worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6828(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 50.10 seconds)6829subtest: low disk drains a worker and builds go to the other6830worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6831worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.16 seconds)6832worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6833worker1 # [ 327.830376] nix-grpc-daemon[1723]: ts=2026-09-22T11:02:00Z level=info event=unhealthy reason=min_free6834worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.13 seconds)6835client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&26836client # don't know how to build these paths:6837client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6838worker2 # [ 328.635059] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:501786839worker1 # [ 328.594953] nix-grpc-daemon[1723]: ts=2026-09-22T11:02:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:422526840worker2 # [ 328.667580] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:01Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06841worker2 # [ 328.675688] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:01Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16842worker2 # [ 328.716633] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786843worker2 # [ 328.732812] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786844client # copying 1 paths to 'lb:50051'...6845client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6846worker2 # [ 328.749697] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501786847worker2 # [ 328.765927] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6848worker2 # [ 328.980830] nix-daemon[863]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6849worker2 # [ 329.000949] nix-daemon[863]: reaped child process 1952, status = succeeded6850worker2 # [ 329.179640] nix-daemon[863]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)6851worker2 # [ 329.197179] nix-daemon[863]: reaped child process 1957, status = succeeded6852lb # [ 329.110183] niks3-server[1160]: 2026/09/22 11:02:01 INFO Received uploads request method=POST path=/api/pending_closures6853worker2 # [ 329.221629] nix-grpc-daemon[1720]: time=2026-09-22T11:02:01.699Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6854worker2 # [ 329.223237] nix-grpc-daemon[1720]: time=2026-09-22T11:02:01.699Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6855lb # [ 329.144436] niks3-server[1160]: 2026/09/22 11:02:01 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6856lb # [ 329.160593] niks3-server[1160]: 2026/09/22 11:02:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6857worker2 # [ 329.266170] nix-grpc-daemon[1720]: time=2026-09-22T11:02:01.743Z level=INFO msg="Uploading 1 narinfos"6858lb # [ 329.169392] niks3-server[1160]: 2026/09/22 11:02:01 INFO Signed narinfos id=74 count=16859lb # [ 329.176169] niks3-server[1160]: 2026/09/22 11:02:01 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6860lb # [ 329.185558] niks3-server[1160]: 2026/09/22 11:02:01 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6861lb # [ 329.193611] niks3-server[1160]: 2026/09/22 11:02:01 INFO Completed upload id=746862lb # [ 329.196693] niks3-server[1160]: 2026/09/22 11:02:01 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6863worker2 # [ 329.293278] nix-grpc-daemon[1720]: time=2026-09-22T11:02:01.771Z level=INFO msg="Upload complete. (510ms)"6864worker2 # [ 329.304660] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=0 paths=1 nar_bytes_in=5126865worker2 # [ 329.315868] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786866worker2 # [ 329.323524] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6867worker2 # [ 329.328733] nix-daemon[863]: reaped child process 1947, status = succeeded6868client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6869client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6870worker2 # [ 329.343979] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6871client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6872client # farm-a> LOG-farm-a-drain6873worker2 # [ 330.190815] nix-daemon[863]: reaped child process 1961, status = succeeded6874worker2 # [ 330.372954] nix-daemon[863]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)6875worker2 # [ 330.393674] nix-daemon[863]: reaped child process 1980, status = succeeded6876lb # [ 330.308902] niks3-server[1160]: 2026/09/22 11:02:02 INFO Received uploads request method=POST path=/api/pending_closures6877worker2 # [ 330.417953] nix-grpc-daemon[1720]: time=2026-09-22T11:02:02.894Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6878worker2 # [ 330.419765] nix-grpc-daemon[1720]: time=2026-09-22T11:02:02.894Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6879lb # [ 330.352715] niks3-server[1160]: 2026/09/22 11:02:02 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6880lb # [ 330.366192] niks3-server[1160]: 2026/09/22 11:02:02 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6881lb # [ 330.375227] niks3-server[1160]: 2026/09/22 11:02:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6882lb # [ 330.381131] niks3-server[1160]: 2026/09/22 11:02:02 INFO Signed narinfos id=75 count=16883worker2 # [ 330.481117] nix-grpc-daemon[1720]: time=2026-09-22T11:02:02.958Z level=INFO msg="Uploading 1 narinfos"6884lb # [ 330.386927] niks3-server[1160]: 2026/09/22 11:02:02 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6885lb # [ 330.399338] niks3-server[1160]: 2026/09/22 11:02:02 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6886lb # [ 330.408590] niks3-server[1160]: 2026/09/22 11:02:02 INFO Completed upload id=756887worker2 # [ 330.508402] nix-grpc-daemon[1720]: time=2026-09-22T11:02:02.986Z level=INFO msg="Upload complete. (353ms)"6888lb # [ 330.412580] niks3-server[1160]: 2026/09/22 11:02:02 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6889worker2 # [ 330.527402] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16890worker2 # [ 330.544403] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:03Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06891worker2 # [ 330.550268] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:03Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16892worker2 # [ 330.554857] nix-daemon[863]: reaped child process 1959, status = succeeded6893worker2 # [ 330.573861] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786894worker2 # [ 330.590535] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786895worker2 # [ 330.607928] nix-grpc-daemon[1720]: time=2026-09-22T11:02:03.085Z level=INFO msg="All 1 paths already cached"6896client # copying 1 paths to 'lb:50051'...6897client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6898worker2 # [ 330.619950] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501786899worker2 # [ 330.634942] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6900worker2 # [ 330.847394] nix-daemon[863]: accepted connection from pid 1986, user nix-grpc-daemon (trusted)6901worker2 # [ 330.867534] nix-daemon[863]: reaped child process 1989, status = succeeded6902worker2 # [ 331.041470] nix-daemon[863]: accepted connection from pid 1991, user nix-grpc-daemon (trusted)6903worker2 # [ 331.059400] nix-daemon[863]: reaped child process 1994, status = succeeded6904lb # [ 330.972344] niks3-server[1160]: 2026/09/22 11:02:03 INFO Received uploads request method=POST path=/api/pending_closures6905worker2 # [ 331.084683] nix-grpc-daemon[1720]: time=2026-09-22T11:02:03.562Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6906worker2 # [ 331.086221] nix-grpc-daemon[1720]: time=2026-09-22T11:02:03.562Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6907lb # [ 331.015742] niks3-server[1160]: 2026/09/22 11:02:03 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6908worker2 # [ 331.127159] nix-grpc-daemon[1720]: time=2026-09-22T11:02:03.604Z level=INFO msg="Uploading 1 narinfos"6909lb # [ 331.030319] niks3-server[1160]: 2026/09/22 11:02:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6910lb # [ 331.036748] niks3-server[1160]: 2026/09/22 11:02:03 INFO Signed narinfos id=76 count=16911lb # [ 331.038658] niks3-server[1160]: 2026/09/22 11:02:03 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6912lb # [ 331.046960] niks3-server[1160]: 2026/09/22 11:02:03 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6913lb # [ 331.052713] niks3-server[1160]: 2026/09/22 11:02:03 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6914worker2 # [ 331.156212] nix-grpc-daemon[1720]: time=2026-09-22T11:02:03.634Z level=INFO msg="Upload complete. (503ms)"6915lb # [ 331.059231] niks3-server[1160]: 2026/09/22 11:02:03 INFO Completed upload id=766916worker2 # [ 331.165163] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=0 paths=1 nar_bytes_in=6326917worker2 # [ 331.179476] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786918worker2 # [ 331.186220] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6919worker2 # [ 331.191432] nix-daemon[863]: reaped child process 1984, status = succeeded6920client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6921client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6922worker2 # [ 331.205422] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6923lb # [ 331.871118] envoy[923]: conn peer=[2001:db8:1::1]:52652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=499 ms=300616924client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6925client # farm-b> LOG-farm-b-drain6926worker2 # [ 332.849505] nix-daemon[863]: reaped child process 1998, status = succeeded6927worker2 # [ 333.014186] nix-daemon[863]: accepted connection from pid 2014, user nix-grpc-daemon (trusted)6928worker2 # [ 333.033104] nix-daemon[863]: reaped child process 2017, status = succeeded6929lb # [ 332.951403] niks3-server[1160]: 2026/09/22 11:02:05 INFO Received uploads request method=POST path=/api/pending_closures6930worker2 # [ 333.063025] nix-grpc-daemon[1720]: time=2026-09-22T11:02:05.540Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6931worker2 # [ 333.066190] nix-grpc-daemon[1720]: time=2026-09-22T11:02:05.542Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6932lb # [ 332.995857] niks3-server[1160]: 2026/09/22 11:02:05 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6933lb # [ 333.006513] niks3-server[1160]: 2026/09/22 11:02:05 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6934worker2 # [ 333.118489] nix-grpc-daemon[1720]: time=2026-09-22T11:02:05.596Z level=INFO msg="Uploading 1 narinfos"6935lb # [ 333.018815] niks3-server[1160]: 2026/09/22 11:02:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6936lb # [ 333.025381] niks3-server[1160]: 2026/09/22 11:02:05 INFO Signed narinfos id=77 count=16937lb # [ 333.027825] niks3-server[1160]: 2026/09/22 11:02:05 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6938lb # [ 333.040515] niks3-server[1160]: 2026/09/22 11:02:05 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6939worker2 # [ 333.145019] nix-grpc-daemon[1720]: time=2026-09-22T11:02:05.623Z level=INFO msg="Upload complete. (327ms)"6940lb # [ 333.045788] niks3-server[1160]: 2026/09/22 11:02:05 INFO Completed upload id=776941lb # [ 333.048923] niks3-server[1160]: 2026/09/22 11:02:05 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6942worker2 # [ 333.154141] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16943worker2 # [ 333.170786] nix-daemon[863]: reaped child process 1996, status = succeeded6944worker2 # [ 333.175307] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:05Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06945worker2 # [ 333.179528] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:05Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16946worker2 # [ 333.220649] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786947worker2 # [ 333.235458] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:501786948worker2 # [ 333.244962] nix-grpc-daemon[1720]: time=2026-09-22T11:02:05.722Z level=INFO msg="All 2 paths already cached"6949client # copying 1 paths to 'lb:50051'...6950client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6951worker2 # [ 333.255776] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:501786952worker2 # [ 333.270297] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6953worker2 # [ 333.474526] nix-daemon[863]: accepted connection from pid 2023, user nix-grpc-daemon (trusted)6954worker2 # [ 333.492931] nix-daemon[863]: reaped child process 2026, status = succeeded6955worker2 # [ 333.663797] nix-daemon[863]: accepted connection from pid 2028, user nix-grpc-daemon (trusted)6956worker2 # [ 333.681257] nix-daemon[863]: reaped child process 2031, status = succeeded6957lb # [ 333.594778] niks3-server[1160]: 2026/09/22 11:02:06 INFO Received uploads request method=POST path=/api/pending_closures6958worker2 # [ 333.706255] nix-grpc-daemon[1720]: time=2026-09-22T11:02:06.183Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6959worker2 # [ 333.707788] nix-grpc-daemon[1720]: time=2026-09-22T11:02:06.184Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6960lb # [ 333.626638] niks3-server[1160]: 2026/09/22 11:02:06 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6961worker2 # [ 333.741089] nix-grpc-daemon[1720]: time=2026-09-22T11:02:06.218Z level=INFO msg="Uploading 1 narinfos"6962lb # [ 333.644668] niks3-server[1160]: 2026/09/22 11:02:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6963lb # [ 333.654874] niks3-server[1160]: 2026/09/22 11:02:06 INFO Signed narinfos id=78 count=16964worker2 # [ 333.764217] nix-grpc-daemon[1720]: time=2026-09-22T11:02:06.242Z level=INFO msg="Upload complete. (476ms)"6965lb # [ 333.664894] niks3-server[1160]: 2026/09/22 11:02:06 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6966lb # [ 333.669274] niks3-server[1160]: 2026/09/22 11:02:06 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6967lb # [ 333.674005] niks3-server[1160]: 2026/09/22 11:02:06 INFO Completed upload id=786968worker2 # [ 333.772141] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=0 paths=1 nar_bytes_in=7606969lb # [ 333.676147] niks3-server[1160]: 2026/09/22 11:02:06 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo6970worker2 # [ 333.784061] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:501786971worker2 # [ 333.790292] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6972worker2 # [ 333.795741] nix-daemon[863]: reaped child process 2021, status = succeeded6973client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...6974worker2 # [ 333.809600] nix-daemon[863]: accepted connection from pid 1644, user nix-grpc-daemon (trusted)6975client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6976client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6977client # farm-top> LOG-farm-top-drain6978worker2 # [ 334.629039] nix-daemon[863]: reaped child process 2035, status = succeeded6979worker2 # [ 334.787988] nix-daemon[863]: accepted connection from pid 2051, user nix-grpc-daemon (trusted)6980worker2 # [ 334.808310] nix-daemon[863]: reaped child process 2054, status = succeeded6981lb # [ 334.721028] niks3-server[1160]: 2026/09/22 11:02:07 INFO Received uploads request method=POST path=/api/pending_closures6982worker2 # [ 334.831440] nix-grpc-daemon[1720]: time=2026-09-22T11:02:07.308Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6983worker2 # [ 334.833073] nix-grpc-daemon[1720]: time=2026-09-22T11:02:07.308Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"6984lb # [ 334.751736] niks3-server[1160]: 2026/09/22 11:02:07 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6985lb # [ 334.770039] niks3-server[1160]: 2026/09/22 11:02:07 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst6986lb # [ 334.785596] niks3-server[1160]: 2026/09/22 11:02:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6987worker2 # [ 334.891873] nix-grpc-daemon[1720]: time=2026-09-22T11:02:07.369Z level=INFO msg="Uploading 1 narinfos"6988lb # [ 334.797735] niks3-server[1160]: 2026/09/22 11:02:07 INFO Signed narinfos id=79 count=16989lb # [ 334.803895] niks3-server[1160]: 2026/09/22 11:02:07 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls6990lb # [ 334.812360] niks3-server[1160]: 2026/09/22 11:02:07 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6991lb # [ 334.818149] niks3-server[1160]: 2026/09/22 11:02:07 INFO Completed upload id=796992lb # [ 334.821208] niks3-server[1160]: 2026/09/22 11:02:07 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo6993worker2 # [ 334.917418] nix-grpc-daemon[1720]: time=2026-09-22T11:02:07.395Z level=INFO msg="Upload complete. (299ms)"6994worker2 # [ 334.929268] nix-grpc-daemon[1644]: ts=2026-09-22T11:02:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50178 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=16995worker2 # [ 334.944981] nix-daemon[863]: reaped child process 2033, status = succeeded6996client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.pem&client-key=/nix/store/r1p0v55v6igq97giy4hh7w7h1khnwhk6-farm-certs/ci.key' --eval-store auto -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain >&2, in 6.67 seconds)6997client: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath6998client: (finished: must succeed: nix eval --raw -f /nix/store/r498lgb40xdi1s9di8nh5bgv2h8vrrp1-job.nix --argstr tag drain outPath, in 0.24 seconds)6999worker1: must succeed: rm /nix/.rw-store/fill7000worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7001worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7002worker1 # [ 335.840823] nix-grpc-daemon[1723]: ts=2026-09-22T11:02:08Z level=info event=healthy reason=min_free7003worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.13 seconds)7004(finished: subtest: low disk drains a worker and builds go to the other, in 10.60 seconds)7005(finished: run the VM test script, in 337.58 seconds)7006test script finished in 337.90s7007cleanup7008kill QemuMachine (pid 50)7009client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)7010kill QemuMachine (pid 49)7011lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)7012kill QemuMachine (pid 52)7013worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)7014kill QemuMachine (pid 51)7015worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/lfh101cfjj8p8npcari9hk0lbqi33yvh-python3-3.14.7/bin/python3.14)7016(finished: cleanup, in 1.16 seconds)7017additionally exposed symbols:7018 client, lb, worker1, worker2,7019 vlan1,7020 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh