vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #535
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.u2TIzZtfXI', fmt=raw size=107374182414worker2 # Disk image does not exist, creating the virtualisation disk image...15worker2: QEMU running (pid 51)16worker1 # Disk image does not exist, creating the virtualisation disk image...17lb # Disk image does not exist, creating the virtualisation disk image...18lb: QEMU running (pid 50)19client # mke2fs 1.47.4 (6-Mar-2025)20worker1 # Formatting '/build/vm-state-worker1/tmp.KKEGgw2m3y', fmt=raw size=107374182421lb # Formatting '/build/vm-state-lb/tmp.qT35F1tNkD', fmt=raw size=107374182422client: QEMU running (pid 49)23worker1: QEMU running (pid 52)24worker2 # Formatting '/build/vm-state-worker2/tmp.QFHS1GhFKd', fmt=raw size=107374182425client # Discarding device blocks: 0/262144 done26lb # mke2fs 1.47.4 (6-Mar-2025)27worker1 # mke2fs 1.47.4 (6-Mar-2025)28worker2 # mke2fs 1.47.4 (6-Mar-2025)29client # Creating filesystem with 262144 4k blocks and 65536 inodes30worker1 # Discarding device blocks: 0/262144 done31worker2 # Discarding device blocks: 0/262144 done32client # Filesystem UUID: ae37c38b-e469-4e07-87a2-8424a782917533worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes34worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes35client # Superblock backups stored on blocks:36worker1 # Filesystem UUID: 933f64f8-b512-4fd9-b502-2906bfb81b7837worker2 # Filesystem UUID: db3a8e93-7bbf-4dc9-b691-45970539d6f138client # 32768, 98304, 163840, 22937639worker1 # Superblock backups stored on blocks:40worker2 # Superblock backups stored on blocks:41client # 42worker1 # 32768, 98304, 163840, 22937643worker2 # 32768, 98304, 163840, 22937644client # Allocating group tables: 0/8 done45worker1 # 46worker2 # 47client # Writing inode tables: 0/8 done48worker1 # Allocating group tables: 0/8 done49worker2 # Allocating group tables: 0/8 done50client # Creating journal (8192 blocks): done51worker1 # Writing inode tables: 0/8 done52worker2 # Writing inode tables: 0/8 done53client # Writing superblocks and filesystem accounting information: 0/8 done54worker1 # Creating journal (8192 blocks): done55worker2 # Creating journal (8192 blocks): done56client # 57worker1 # Writing superblocks and filesystem accounting information: 0/8 done58worker2 # Writing superblocks and filesystem accounting information: 0/8 done59client # Virtualisation disk image created.60worker1 # 61worker2 # 62client # Starting virtiofs daemons...63worker1 # Virtualisation disk image created.64worker2 # Virtualisation disk image created.65client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)66worker1 # Starting virtiofs daemons...67worker2 # Starting virtiofs daemons...68client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether69worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)71client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...72worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether74client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...76worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...77client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...81worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...85worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...86(finished: start all VMs, in 0.50 seconds)87lb # Discarding device blocks: 0/262144 done88client # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91client: waiting for unit network-addresses-eth1.service92worker1 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93worker2 # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether94client: waiting for the VM to finish booting95worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...96worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...97lb # Creating filesystem with 262144 4k blocks and 65536 inodes98client # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...99worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests100worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests101lb # Filesystem UUID: 4e796412-306a-4bfb-ae60-e56a025ef008102client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests103worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests104worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests105lb # Superblock backups stored on blocks:106client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests107worker1 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests108worker2 # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests109lb # 32768, 98304, 163840, 229376110client # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests111lb # 112lb # Allocating group tables: 0/8 done113lb # Writing inode tables: 0/8 done114lb # Creating journal (8192 blocks): done115lb # Writing superblocks and filesystem accounting information: 0/8 done116lb # 117lb # Virtualisation disk image created.118lb # Starting virtiofs daemons...119lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...122lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-23T06:58:58Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-23T06:58:58Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-23T06:58:58Z INFO virtiofsd] Client connected, servicing requests131lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133lb # [ 0.000000] KASLR enabled134lb # [ 0.000000] random: crng init done135lb # [ 0.000000] Machine model: linux,dummy-virt136lb # [ 0.000000] efi: UEFI not found.137lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140lb # [ 0.000000] Zone ranges:141lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142lb # [ 0.000000] DMA32 empty143lb # [ 0.000000] Normal empty144lb # [ 0.000000] Device empty145lb # [ 0.000000] Movable zone start for each node146lb # [ 0.000000] Early memory node ranges147lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150lb # [ 0.000000] psci: probing for conduit method from DT.151lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.152lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153lb # [ 0.000000] psci: Trusted OS migration not required154lb # [ 0.000000] psci: SMC Calling Convention v1.1155lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157lb # [ 0.000000] Detected PIPT I-cache on CPU0158lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159lb # [ 0.000000] CPU features: detected: GICv3 CPU interface160lb # [ 0.000000] CPU features: detected: Spectre-v4161lb # [ 0.000000] CPU features: detected: Spectre-BHB162lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164lb # [ 0.000000] alternatives: applying boot alternatives165lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/3in2qynk4v72bmzzvgahb746q83gs855-nixos-system-lb-test/init regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration console=ttyAMA0,115200n8 console=tty0166lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x7007ikm6bw7126pklv8b26h3gxx7bya-closure-info/registration", will be passed to user space.167lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)169lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)170lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB171lb # [ 0.000000] software IO TLB: area num 1.172lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB173lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)174lb # [ 0.000000] Fallback order for Node 0: 0175lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216176lb # [ 0.000000] Policy zone: DMA177lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off178lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1179lb # [ 0.000000] allocated 3145728 bytes of page_ext180lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages181lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups182lb # [ 0.000000] rcu: Hierarchical RCU implementation.183lb # [ 0.000000] rcu: RCU event tracing is enabled.184lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.185lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.186lb # [ 0.000000] Rude variant of Tasks RCU enabled.187lb # [ 0.000000] Tracing variant of Tasks RCU enabled.188lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.189lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1190lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.191lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.192lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.193lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0194lb # [ 0.000000] GICv3: 256 SPIs implemented195lb # [ 0.000000] GICv3: 0 Extended SPIs implemented196lb # [ 0.000000] Root IRQ handler: gic_handle_irq197lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI198lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0199lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000200lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]201lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)202lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)203lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000204lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000205lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.206lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns207lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).208lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns209lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns210lb # [ 0.000036] arm-pv: using stolen time PV211lb # [ 0.000661] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)212worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]213lb # [ 0.000805] Console: colour dummy device 80x25214lb # [ 0.000813] printk: legacy console [tty0] enabled215worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026216worker1 # [ 0.000000] KASLR enabled217lb # [ 0.001002] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)218worker1 # [ 0.000000] random: crng init done219lb # [ 0.001009] pid_max: default: 32768 minimum: 301220worker1 # [ 0.000000] Machine model: linux,dummy-virt221worker1 # [ 0.000000] efi: UEFI not found.222lb # [ 0.001094] LSM: initializing lsm=capability,landlock,yama,bpf,ima223lb # [ 0.001279] landlock: Up and running.224worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT225lb # [ 0.001282] Yama: becoming mindful.226lb # [ 0.001951] LSM support for eBPF active227worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]228lb # [ 0.002124] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)229worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]230worker1 # [ 0.000000] Zone ranges:231lb # [ 0.002156] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)232worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]233worker1 # [ 0.000000] DMA32 empty234lb # [ 0.004027] rcu: Hierarchical SRCU implementation.235worker1 # [ 0.000000] Normal empty236lb # [ 0.004032] rcu: Max phase no-delay instances is 1000.237worker1 # [ 0.000000] Device empty238lb # [ 0.005258] fsl-mc MSI: its@8080000 domain created239worker1 # [ 0.000000] Movable zone start for each node240worker1 # [ 0.000000] Early memory node ranges241lb # [ 0.005356] EFI services will not be available.242lb # [ 0.005480] smp: Bringing up secondary CPUs ...243worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]244lb # [ 0.005489] smp: Brought up 1 node, 1 CPU245worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]246worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]247lb # [ 0.005492] SMP: Total of 1 processors activated.248lb # [ 0.005497] CPU: All CPU(s) started at EL1249worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000250worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026251worker1 # [ 0.000000] psci: probing for conduit method from DT.252lb # [ 0.005507] CPU features: detected: Branch Target Identification253worker2 # [ 0.000000] KASLR enabled254worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.255worker2 # [ 0.000000] random: crng init done256lb # [ 0.005511] CPU features: detected: ARMv8.4 Translation Table Level257worker2 # [ 0.000000] Machine model: linux,dummy-virt258worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs259worker2 # [ 0.000000] efi: UEFI not found.260lb # [ 0.005515] CPU features: detected: Instruction cache invalidation not required for I/D coherence261worker1 # [ 0.000000] psci: Trusted OS migration not required262client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]263worker1 # [ 0.000000] psci: SMC Calling Convention v1.1264worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT265lb # [ 0.005518] CPU features: detected: Data cache clean to the PoU not required for I/D coherence266worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]267worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)268lb # [ 0.005522] CPU features: detected: Common not Private translations269client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026270client # [ 0.000000] KASLR enabled271worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]272lb # [ 0.005525] CPU features: detected: CRC32 instructions273worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296274worker2 # [ 0.000000] Zone ranges:275client # [ 0.000000] random: crng init done276worker1 # [ 0.000000] Detected PIPT I-cache on CPU0277client # [ 0.000000] Machine model: linux,dummy-virt278worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]279lb # [ 0.005528] CPU features: detected: Data cache clean to Point of Deep Persistence280client # [ 0.000000] efi: UEFI not found.281worker2 # [ 0.000000] DMA32 empty282worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)283worker2 # [ 0.000000] Normal empty284lb # [ 0.005531] CPU features: detected: Data cache clean to Point of Persistence285worker2 # [ 0.000000] Device empty286client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT287worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface288worker2 # [ 0.000000] Movable zone start for each node289lb # [ 0.005535] CPU features: detected: Data independent timing control (DIT)290worker1 # [ 0.000000] CPU features: detected: Spectre-v4291worker2 # [ 0.000000] Early memory node ranges292client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]293lb # [ 0.005538] CPU features: detected: E0PD294worker1 # [ 0.000000] CPU features: detected: Spectre-BHB295client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]296worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]297lb # [ 0.005540] CPU features: detected: Enhanced Counter Virtualization298client # [ 0.000000] Zone ranges:299worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38300worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]301lb # [ 0.005543] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)302client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]303worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23304client # [ 0.000000] DMA32 empty305worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000306lb # [ 0.005547] CPU features: detected: Enhanced Virtualization Traps307client # [ 0.000000] Normal empty308worker1 # [ 0.000000] alternatives: applying boot alternatives309worker2 # [ 0.000000] psci: probing for conduit method from DT.310client # [ 0.000000] Device empty311lb # [ 0.005549] CPU features: detected: Fine Grained Traps312client # [ 0.000000] Movable zone start for each node313worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.314client # [ 0.000000] Early memory node ranges315lb # [ 0.005553] CPU features: detected: Generic authentication (architected QARMA5 algorithm)316worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs317client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]318worker2 # [ 0.000000] psci: Trusted OS migration not required319lb # [ 0.005557] CPU features: detected: RCpc load-acquire (LDAPR)320worker2 # [ 0.000000] psci: SMC Calling Convention v1.1321lb # [ 0.005560] CPU features: detected: LSE atomic instructions322client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]323lb # [ 0.005562] CPU features: detected: Privileged Access Never324client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000325worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)326worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/waj0blgjl9bpqbg9b6xiyknfyf8i56g7-nixos-system-worker1-test/init regInfo=/nix/store/036mdri2akx3g3s2sm0y9jgc449gc7mb-closure-info/registration console=ttyAMA0,115200n8 console=tty0327lb # [ 0.005565] CPU features: detected: PMUv3328client # [ 0.000000] psci: probing for conduit method from DT.329worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296330lb # [ 0.005568] CPU features: detected: RAS Extension Support331client # [ 0.000000] psci: PSCIv1.3 detected in firmware.332worker2 # [ 0.000000] Detected PIPT I-cache on CPU0333client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs334lb # [ 0.005571] CPU features: detected: RASv1p1 Extension Support335client # [ 0.000000] psci: Trusted OS migration not required336worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)337lb # [ 0.005573] CPU features: detected: Random Number Generator338worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/036mdri2akx3g3s2sm0y9jgc449gc7mb-closure-info/registration", will be passed to user space.339client # [ 0.000000] psci: SMC Calling Convention v1.1340worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface341lb # [ 0.005576] CPU features: detected: Speculation barrier (SB)342worker2 # [ 0.000000] CPU features: detected: Spectre-v4343worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes344lb # [ 0.005578] CPU features: detected: Stage-2 Force Write-Back345client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)346worker2 # [ 0.000000] CPU features: detected: Spectre-BHB347worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)348lb # [ 0.005581] CPU features: detected: TLB range maintenance instructions349client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296350worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38351client # [ 0.000000] Detected PIPT I-cache on CPU0352lb # [ 0.005586] CPU features: detected: Speculative Store Bypassing Safe (SSBS)353worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)354worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23355lb # [ 0.005626] alternatives: applying system-wide alternatives356client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)357worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB358worker2 # [ 0.000000] alternatives: applying boot alternatives359worker1 # [ 0.000000] software IO TLB: area num 1.360client # [ 0.000000] CPU features: detected: GICv3 CPU interface361lb # [ 0.008507] CPU features: detected: BBM Level 2 without TLB conflict abort362client # [ 0.000000] CPU features: detected: Spectre-v4363worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB364client # [ 0.000000] CPU features: detected: Spectre-BHB365worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)366lb # [ 0.008664] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)367client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38368lb # [ 0.009035] devtmpfs: initialized369worker1 # [ 0.000000] Fallback order for Node 0: 0370client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23371worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216372worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yhybdyymi2isy70630z4y9gwglld4pzg-nixos-system-worker2-test/init regInfo=/nix/store/p2vhx3y5597gdgv9d5wglk0drd0sdzdn-closure-info/registration console=ttyAMA0,115200n8 console=tty0373lb # [ 0.010889] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)374client # [ 0.000000] alternatives: applying boot alternatives375worker1 # [ 0.000000] Policy zone: DMA376lb # [ 0.010911] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).377worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off378lb # [ 0.011094] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL379worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/p2vhx3y5597gdgv9d5wglk0drd0sdzdn-closure-info/registration", will be passed to user space.380worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1381lb # [ 0.011098] 0 pages in range for non-PLT usage382worker1 # [ 0.000000] allocated 3145728 bytes of page_ext383lb # [ 0.011099] 508272 pages in range for PLT usage384worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages385worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes386lb # [ 0.011187] pinctrl core: initialized pinctrl subsystem387worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups388lb # [ 0.012005] DMI not present or invalid.389client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/1m9h2dn87nn6p37bdfw61yjfigah24f6-nixos-system-client-test/init regInfo=/nix/store/x4pxw657nkinqgkz3xzfplm1jf2xbslh-closure-info/registration console=ttyAMA0,115200n8 console=tty0390worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)391worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.392lb # [ 0.015663] NET: Registered PF_NETLINK/PF_ROUTE protocol family393worker1 # [ 0.000000] rcu: RCU event tracing is enabled.394worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)395lb # [ 0.017833] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations396client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/x4pxw657nkinqgkz3xzfplm1jf2xbslh-closure-info/registration", will be passed to user space.397worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.398worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB399lb # [ 0.018008] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations400worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.401worker2 # [ 0.000000] software IO TLB: area num 1.402client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes403worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.404worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB405lb # [ 0.018209] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations406client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)407worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.408lb # [ 0.018230] audit: initializing netlink subsys (disabled)409worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)410worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.411client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)412worker2 # [ 0.000000] Fallback order for Node 0: 0413lb # [ 0.018828] thermal_sys: Registered thermal governor 'fair_share'414worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1415client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB416worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216417lb # [ 0.018830] thermal_sys: Registered thermal governor 'bang_bang'418client # [ 0.000000] software IO TLB: area num 1.419worker2 # [ 0.000000] Policy zone: DMA420worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.421client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB422worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off423lb # [ 0.018834] thermal_sys: Registered thermal governor 'step_wise'424worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1425client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)426worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.427lb # [ 0.018837] thermal_sys: Registered thermal governor 'user_space'428client # [ 0.000000] Fallback order for Node 0: 0429worker2 # [ 0.000000] allocated 3145728 bytes of page_ext430lb # [ 0.018842] thermal_sys: Registered thermal governor 'power_allocator'431worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.432worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages433client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216434client # [ 0.000000] Policy zone: DMA435worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups436worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0437lb # [ 0.018867] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1438worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.439worker1 # [ 0.000000] GICv3: 256 SPIs implemented440client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off441lb # [ 0.018875] cpuidle: using governor ladder442worker2 # [ 0.000000] rcu: RCU event tracing is enabled.443worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented444lb # [ 0.018881] cpuidle: using governor menu445client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1446worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq447worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.448client # [ 0.000000] allocated 3145728 bytes of page_ext449lb # [ 0.019079] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.450worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI451worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.452lb # [ 0.019095] ASID allocator initialised with 65536 entries453client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages454worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.455worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0456lb # [ 0.020348] Serial: AMBA PL011 UART driver457client # [ 0.000000] ftrace: allocated 294 pages with 4 groups458worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.459worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000460client # [ 0.000000] rcu: Hierarchical RCU implementation.461lb # [ 0.025892] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1462client # [ 0.000000] rcu: RCU event tracing is enabled.463worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]464worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.465lb # [ 0.026049] printk: console [ttyAMA0] enabled466client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.467worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1468worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)469client # [ 0.000000] Trampoline variant of Tasks RCU enabled.470worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.471client # [ 0.000000] Rude variant of Tasks RCU enabled.472worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)473client # [ 0.000000] Tracing variant of Tasks RCU enabled.474worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.475worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000476client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.477worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000478worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.479client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1480worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0481worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.482worker2 # [ 0.000000] GICv3: 256 SPIs implemented483client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.484worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented485lb # [ 0.143779] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages486worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns487worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq488client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.489worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).490lb # [ 0.143794] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page491worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI492client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.493lb # [ 0.143800] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages494worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0495client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0496worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns497lb # [ 0.143804] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page498worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000499client # [ 0.000000] GICv3: 256 SPIs implemented500worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]501client # [ 0.000000] GICv3: 0 Extended SPIs implemented502worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns503lb # [ 0.143808] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages504worker1 # [ 0.000033] arm-pv: using stolen time PV505client # [ 0.000000] Root IRQ handler: gic_handle_irq506lb # [ 0.143813] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page507worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)508client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI509lb # [ 0.143817] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages510worker1 # [ 0.000522] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)511client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0512worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)513worker1 # [ 0.000693] Console: colour dummy device 80x25514lb # [ 0.143821] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page515client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000516worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000517worker1 # [ 0.000700] printk: legacy console [tty0] enabled518client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]519worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000520worker1 # [ 0.000887] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)521client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)522worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.523worker1 # [ 0.000893] pid_max: default: 32768 minimum: 301524lb # [ 0.150962] fbcon: Taking over console525lb # [ 0.150975] ACPI: Interpreter disabled.526client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)527worker1 # [ 0.000964] LSM: initializing lsm=capability,landlock,yama,bpf,ima528worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns529worker1 # [ 0.001123] landlock: Up and running.530client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000531worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).532worker1 # [ 0.001126] Yama: becoming mindful.533worker1 # [ 0.001655] LSM support for eBPF active534client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000535worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns536client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.537worker1 # [ 0.001805] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)538worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns539worker1 # [ 0.001846] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)540client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns541worker2 # [ 0.000028] arm-pv: using stolen time PV542worker1 # [ 0.003665] rcu: Hierarchical SRCU implementation.543client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).544worker1 # [ 0.003671] rcu: Max phase no-delay instances is 1000.545worker2 # [ 0.000370] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)546worker1 # [ 0.004913] fsl-mc MSI: its@8080000 domain created547worker2 # [ 0.000593] Console: colour dummy device 80x25548client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns549worker1 # [ 0.005004] EFI services will not be available.550worker2 # [ 0.000602] printk: legacy console [tty0] enabled551worker1 # [ 0.005117] smp: Bringing up secondary CPUs ...552client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns553worker1 # [ 0.005125] smp: Brought up 1 node, 1 CPU554client # [ 0.000029] arm-pv: using stolen time PV555worker2 # [ 0.000816] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)556worker1 # [ 0.005128] SMP: Total of 1 processors activated.557worker1 # [ 0.005133] CPU: All CPU(s) started at EL1558worker2 # [ 0.000823] pid_max: default: 32768 minimum: 301559client # [ 0.000406] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)560lb # [ 0.159926] iommu: Default domain type: Translated561worker2 # [ 0.000897] LSM: initializing lsm=capability,landlock,yama,bpf,ima562worker1 # [ 0.005143] CPU features: detected: Branch Target Identification563client # [ 0.000572] Console: colour dummy device 80x25564lb # [ 0.159936] iommu: DMA domain TLB invalidation policy: strict mode565worker2 # [ 0.001014] landlock: Up and running.566client # [ 0.000580] printk: legacy console [tty0] enabled567worker1 # [ 0.005148] CPU features: detected: ARMv8.4 Translation Table Level568lb # [ 0.160278] SCSI subsystem initialized569worker2 # [ 0.001016] Yama: becoming mindful.570worker2 # [ 0.001435] LSM support for eBPF active571worker1 # [ 0.005151] CPU features: detected: Instruction cache invalidation not required for I/D coherence572client # [ 0.000772] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)573worker2 # [ 0.001582] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)574lb # [ 0.162166] usbcore: registered new interface driver usbfs575client # [ 0.000780] pid_max: default: 32768 minimum: 301576lb # [ 0.162210] usbcore: registered new interface driver hub577worker1 # [ 0.005154] CPU features: detected: Data cache clean to the PoU not required for I/D coherence578worker2 # [ 0.001615] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)579client # [ 0.000851] LSM: initializing lsm=capability,landlock,yama,bpf,ima580lb # [ 0.162226] usbcore: registered new device driver usb581worker1 # [ 0.005158] CPU features: detected: Common not Private translations582worker2 # [ 0.003474] rcu: Hierarchical SRCU implementation.583client # [ 0.000981] landlock: Up and running.584lb # [ 0.162490] pps_core: LinuxPPS API ver. 1 registered585client # [ 0.000984] Yama: becoming mindful.586worker1 # [ 0.005161] CPU features: detected: CRC32 instructions587worker2 # [ 0.003479] rcu: Max phase no-delay instances is 1000.588client # [ 0.001413] LSM support for eBPF active589worker2 # [ 0.004773] fsl-mc MSI: its@8080000 domain created590lb # [ 0.162496] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>591worker1 # [ 0.005164] CPU features: detected: Data cache clean to Point of Deep Persistence592client # [ 0.001563] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)593worker2 # [ 0.004863] EFI services will not be available.594lb # [ 0.162507] PTP clock support registered595lb # [ 0.162558] EDAC MC: Ver: 3.0.0596worker1 # [ 0.005168] CPU features: detected: Data cache clean to Point of Persistence597worker2 # [ 0.004927] smp: Bringing up secondary CPUs ...598client # [ 0.001595] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)599worker2 # [ 0.004936] smp: Brought up 1 node, 1 CPU600lb # [ 0.166965] scmi_core: SCMI protocol bus registered601worker1 # [ 0.005170] CPU features: detected: Data independent timing control (DIT)602client # [ 0.003362] rcu: Hierarchical SRCU implementation.603worker2 # [ 0.004939] SMP: Total of 1 processors activated.604worker1 # [ 0.005174] CPU features: detected: E0PD605lb # [ 0.167886] FPGA manager framework606worker2 # [ 0.004944] CPU: All CPU(s) started at EL1607client # [ 0.003367] rcu: Max phase no-delay instances is 1000.608worker1 # [ 0.005176] CPU features: detected: Enhanced Counter Virtualization609client # [ 0.004613] fsl-mc MSI: its@8080000 domain created610lb # [ 0.168793] vgaarb: loaded611worker2 # [ 0.004954] CPU features: detected: Branch Target Identification612client # [ 0.004705] EFI services will not be available.613worker1 # [ 0.005179] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)614worker2 # [ 0.004959] CPU features: detected: ARMv8.4 Translation Table Level615client # [ 0.004812] smp: Bringing up secondary CPUs ...616lb # [ 0.169383] clocksource: Switched to clocksource arch_sys_counter617worker1 # [ 0.005183] CPU features: detected: Enhanced Virtualization Traps618client # [ 0.004821] smp: Brought up 1 node, 1 CPU619worker2 # [ 0.004962] CPU features: detected: Instruction cache invalidation not required for I/D coherence620client # [ 0.004824] SMP: Total of 1 processors activated.621worker1 # [ 0.005186] CPU features: detected: Fine Grained Traps622client # [ 0.004829] CPU: All CPU(s) started at EL1623worker2 # [ 0.004966] CPU features: detected: Data cache clean to the PoU not required for I/D coherence624worker1 # [ 0.005190] CPU features: detected: Generic authentication (architected QARMA5 algorithm)625client # [ 0.004838] CPU features: detected: Branch Target Identification626worker2 # [ 0.004969] CPU features: detected: Common not Private translations627worker1 # [ 0.005195] CPU features: detected: RCpc load-acquire (LDAPR)628client # [ 0.004843] CPU features: detected: ARMv8.4 Translation Table Level629worker2 # [ 0.004973] CPU features: detected: CRC32 instructions630worker1 # [ 0.005198] CPU features: detected: LSE atomic instructions631client # [ 0.004846] CPU features: detected: Instruction cache invalidation not required for I/D coherence632worker2 # [ 0.004975] CPU features: detected: Data cache clean to Point of Deep Persistence633worker1 # [ 0.005200] CPU features: detected: Privileged Access Never634worker1 # [ 0.005203] CPU features: detected: PMUv3635worker2 # [ 0.004979] CPU features: detected: Data cache clean to Point of Persistence636client # [ 0.004850] CPU features: detected: Data cache clean to the PoU not required for I/D coherence637lb # [ 0.174586] VFS: Disk quotas dquot_6.6.0638worker1 # [ 0.005206] CPU features: detected: RAS Extension Support639worker2 # [ 0.004982] CPU features: detected: Data independent timing control (DIT)640client # [ 0.004854] CPU features: detected: Common not Private translations641lb # [ 0.174621] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)642worker2 # [ 0.004985] CPU features: detected: E0PD643worker1 # [ 0.005209] CPU features: detected: RASv1p1 Extension Support644client # [ 0.004857] CPU features: detected: CRC32 instructions645worker1 # [ 0.005212] CPU features: detected: Random Number Generator646worker2 # [ 0.004988] CPU features: detected: Enhanced Counter Virtualization647client # [ 0.004860] CPU features: detected: Data cache clean to Point of Deep Persistence648worker1 # [ 0.005214] CPU features: detected: Speculation barrier (SB)649worker2 # [ 0.004991] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)650client # [ 0.004863] CPU features: detected: Data cache clean to Point of Persistence651worker1 # [ 0.005217] CPU features: detected: Stage-2 Force Write-Back652worker2 # [ 0.004995] CPU features: detected: Enhanced Virtualization Traps653lb # [ 0.178162] netfs: FS-Cache loaded654client # [ 0.004866] CPU features: detected: Data independent timing control (DIT)655worker1 # [ 0.005220] CPU features: detected: TLB range maintenance instructions656worker2 # [ 0.004998] CPU features: detected: Fine Grained Traps657lb # [ 0.178274] pnp: PnP ACPI: disabled658client # [ 0.004869] CPU features: detected: E0PD659worker1 # [ 0.005225] CPU features: detected: Speculative Store Bypassing Safe (SSBS)660client # [ 0.004872] CPU features: detected: Enhanced Counter Virtualization661worker2 # [ 0.005001] CPU features: detected: Generic authentication (architected QARMA5 algorithm)662worker1 # [ 0.005263] alternatives: applying system-wide alternatives663worker2 # [ 0.005005] CPU features: detected: RCpc load-acquire (LDAPR)664client # [ 0.004874] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)665worker1 # [ 0.008194] CPU features: detected: BBM Level 2 without TLB conflict abort666worker2 # [ 0.005008] CPU features: detected: LSE atomic instructions667client # [ 0.004878] CPU features: detected: Enhanced Virtualization Traps668worker2 # [ 0.005011] CPU features: detected: Privileged Access Never669client # [ 0.004881] CPU features: detected: Fine Grained Traps670worker2 # [ 0.005013] CPU features: detected: PMUv3671worker1 # [ 0.008379] Memory: 1405668K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)672client # [ 0.004884] CPU features: detected: Generic authentication (architected QARMA5 algorithm)673worker2 # [ 0.005016] CPU features: detected: RAS Extension Support674client # [ 0.004888] CPU features: detected: RCpc load-acquire (LDAPR)675worker2 # [ 0.005019] CPU features: detected: RASv1p1 Extension Support676client # [ 0.004891] CPU features: detected: LSE atomic instructions677worker2 # [ 0.005022] CPU features: detected: Random Number Generator678client # [ 0.004894] CPU features: detected: Privileged Access Never679worker2 # [ 0.005024] CPU features: detected: Speculation barrier (SB)680client # [ 0.004897] CPU features: detected: PMUv3681worker2 # [ 0.005027] CPU features: detected: Stage-2 Force Write-Back682lb # [ 0.182127] NET: Registered PF_INET protocol family683client # [ 0.004899] CPU features: detected: RAS Extension Support684client # [ 0.004902] CPU features: detected: RASv1p1 Extension Support685lb # [ 0.182399] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)686client # [ 0.004904] CPU features: detected: Random Number Generator687client # [ 0.004907] CPU features: detected: Speculation barrier (SB)688client # [ 0.004909] CPU features: detected: Stage-2 Force Write-Back689client # [ 0.004912] CPU features: detected: TLB range maintenance instructions690client # [ 0.004916] CPU features: detected: Speculative Store Bypassing Safe (SSBS)691client # [ 0.004952] alternatives: applying system-wide alternatives692worker1 # [ 0.008757] devtmpfs: initialized693client # [ 0.007779] CPU features: detected: BBM Level 2 without TLB conflict abort694worker1 # [ 0.010485] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)695worker1 # [ 0.010506] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).696client # [ 0.007954] Memory: 1405616K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)697worker1 # [ 0.010682] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL698client # [ 0.008332] devtmpfs: initialized699worker1 # [ 0.010687] 0 pages in range for non-PLT usage700worker2 # [ 0.005030] CPU features: detected: TLB range maintenance instructions701client # [ 0.010123] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)702worker1 # [ 0.010688] 508272 pages in range for PLT usage703worker2 # [ 0.005034] CPU features: detected: Speculative Store Bypassing Safe (SSBS)704worker1 # [ 0.010808] pinctrl core: initialized pinctrl subsystem705client # [ 0.010146] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).706worker2 # [ 0.005070] alternatives: applying system-wide alternatives707worker1 # [ 0.011653] DMI not present or invalid.708client # [ 0.010340] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL709worker2 # [ 0.007990] CPU features: detected: BBM Level 2 without TLB conflict abort710worker1 # [ 0.014784] NET: Registered PF_NETLINK/PF_ROUTE protocol family711client # [ 0.010344] 0 pages in range for non-PLT usage712client # [ 0.010345] 508272 pages in range for PLT usage713worker1 # [ 0.016991] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations714client # [ 0.010435] pinctrl core: initialized pinctrl subsystem715worker2 # [ 0.008248] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)716worker1 # [ 0.017181] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations717client # [ 0.011143] DMI not present or invalid.718worker2 # [ 0.008637] devtmpfs: initialized719client # [ 0.014144] NET: Registered PF_NETLINK/PF_ROUTE protocol family720worker1 # [ 0.017380] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations721worker2 # [ 0.010413] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)722client # [ 0.016473] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations723worker1 # [ 0.017401] audit: initializing netlink subsys (disabled)724worker2 # [ 0.010435] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).725worker1 # [ 0.018004] thermal_sys: Registered thermal governor 'fair_share'726client # [ 0.016658] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations727worker2 # [ 0.010612] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL728worker1 # [ 0.018006] thermal_sys: Registered thermal governor 'bang_bang'729worker2 # [ 0.010617] 0 pages in range for non-PLT usage730client # [ 0.016848] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations731worker1 # [ 0.018009] thermal_sys: Registered thermal governor 'step_wise'732worker2 # [ 0.010618] 508272 pages in range for PLT usage733client # [ 0.016869] audit: initializing netlink subsys (disabled)734worker2 # [ 0.010713] pinctrl core: initialized pinctrl subsystem735worker1 # [ 0.018012] thermal_sys: Registered thermal governor 'user_space'736client # [ 0.017458] thermal_sys: Registered thermal governor 'fair_share'737worker2 # [ 0.011467] DMI not present or invalid.738worker1 # [ 0.018017] thermal_sys: Registered thermal governor 'power_allocator'739client # [ 0.017460] thermal_sys: Registered thermal governor 'bang_bang'740worker2 # [ 0.014671] NET: Registered PF_NETLINK/PF_ROUTE protocol family741client # [ 0.017464] thermal_sys: Registered thermal governor 'step_wise'742worker1 # [ 0.018049] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1743worker2 # [ 0.016853] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations744worker1 # [ 0.018058] cpuidle: using governor ladder745client # [ 0.017466] thermal_sys: Registered thermal governor 'user_space'746worker1 # [ 0.018064] cpuidle: using governor menu747worker2 # [ 0.017030] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations748client # [ 0.017471] thermal_sys: Registered thermal governor 'power_allocator'749worker1 # [ 0.018249] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.750worker2 # [ 0.017228] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations751client # [ 0.017501] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1752worker1 # [ 0.018265] ASID allocator initialised with 65536 entries753worker2 # [ 0.017248] audit: initializing netlink subsys (disabled)754client # [ 0.017511] cpuidle: using governor ladder755worker1 # [ 0.019548] Serial: AMBA PL011 UART driver756client # [ 0.017517] cpuidle: using governor menu757worker2 # [ 0.017797] thermal_sys: Registered thermal governor 'fair_share'758worker1 # [ 0.024699] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1759client # [ 0.017693] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.760worker2 # [ 0.017800] thermal_sys: Registered thermal governor 'bang_bang'761worker1 # [ 0.024850] printk: console [ttyAMA0] enabled762client # [ 0.017707] ASID allocator initialised with 65536 entries763worker2 # [ 0.017804] thermal_sys: Registered thermal governor 'step_wise'764client # [ 0.018902] Serial: AMBA PL011 UART driver765worker2 # [ 0.017806] thermal_sys: Registered thermal governor 'user_space'766worker1 # [ 0.147114] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages767worker2 # [ 0.017811] thermal_sys: Registered thermal governor 'power_allocator'768worker1 # [ 0.147133] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page769client # [ 0.023974] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1770worker2 # [ 0.017843] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1771worker1 # [ 0.147139] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages772client # [ 0.024144] printk: console [ttyAMA0] enabled773worker2 # [ 0.017851] cpuidle: using governor ladder774worker1 # [ 0.147143] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page775worker2 # [ 0.017858] cpuidle: using governor menu776worker1 # [ 0.147148] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages777worker2 # [ 0.018044] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.778worker2 # [ 0.018060] ASID allocator initialised with 65536 entries779worker1 # [ 0.147152] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page780worker2 # [ 0.019224] Serial: AMBA PL011 UART driver781worker1 # [ 0.147157] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages782worker2 # [ 0.024500] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1783worker1 # [ 0.147161] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page784worker2 # [ 0.024619] printk: console [ttyAMA0] enabled785worker2 # [ 0.144060] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages786worker1 # [ 0.154673] fbcon: Taking over console787worker2 # [ 0.144077] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page788worker1 # [ 0.154686] ACPI: Interpreter disabled.789client # [ 0.141372] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages790worker2 # [ 0.144082] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages791client # [ 0.141386] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page792worker2 # [ 0.144086] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page793client # [ 0.141391] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages794worker2 # [ 0.144090] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages795client # [ 0.141395] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page796worker2 # [ 0.144095] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page797lb # [ 0.212025] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)798client # [ 0.141399] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages799worker2 # [ 0.144099] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages800client # [ 0.141404] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page801lb # [ 0.212065] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)802worker2 # [ 0.144103] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page803client # [ 0.141408] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages804lb # [ 0.212088] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)805client # [ 0.141412] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page806lb # [ 0.212157] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)807worker1 # [ 0.162531] iommu: Default domain type: Translated808worker2 # [ 0.151513] fbcon: Taking over console809lb # [ 0.212300] TCP: Hash tables configured (established 16384 bind 16384)810worker2 # [ 0.151525] ACPI: Interpreter disabled.811worker1 # [ 0.162540] iommu: DMA domain TLB invalidation policy: strict mode812lb # [ 0.212371] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)813worker2 # [ 0.153315] iommu: Default domain type: Translated814client # [ 0.148727] fbcon: Taking over console815worker1 # [ 0.164225] SCSI subsystem initialized816lb # [ 0.212403] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)817worker2 # [ 0.153325] iommu: DMA domain TLB invalidation policy: strict mode818client # [ 0.148741] ACPI: Interpreter disabled.819worker2 # [ 0.154940] SCSI subsystem initialized820lb # [ 0.212438] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)821worker1 # [ 0.164912] usbcore: registered new interface driver usbfs822lb # [ 0.212538] NET: Registered PF_UNIX/PF_LOCAL protocol family823worker1 # [ 0.164951] usbcore: registered new interface driver hub824lb # [ 0.212589] NET: Registered PF_XDP protocol family825worker2 # [ 0.155636] usbcore: registered new interface driver usbfs826worker1 # [ 0.164967] usbcore: registered new device driver usb827lb # [ 0.212604] PCI: CLS 0 bytes, default 64828worker2 # [ 0.155668] usbcore: registered new interface driver hub829worker1 # [ 0.165237] pps_core: LinuxPPS API ver. 1 registered830lb # [ 0.212827] Trying to unpack rootfs image as initramfs...831worker2 # [ 0.155684] usbcore: registered new device driver usb832worker1 # [ 0.165243] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>833worker2 # [ 0.155944] pps_core: LinuxPPS API ver. 1 registered834worker1 # [ 0.165254] PTP clock support registered835worker1 # [ 0.165310] EDAC MC: Ver: 3.0.0836worker2 # [ 0.155951] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>837worker2 # [ 0.155960] PTP clock support registered838worker1 # [ 0.169967] scmi_core: SCMI protocol bus registered839worker2 # [ 0.156009] EDAC MC: Ver: 3.0.0840worker1 # [ 0.170976] FPGA manager framework841worker1 # [ 0.171910] vgaarb: loaded842client # [ 0.157547] iommu: Default domain type: Translated843worker1 # [ 0.172532] clocksource: Switched to clocksource arch_sys_counter844client # [ 0.157556] iommu: DMA domain TLB invalidation policy: strict mode845client # [ 0.158005] SCSI subsystem initialized846lb # [ 0.230863] kvm [1]: HYP mode not available847client # [ 0.159965] usbcore: registered new interface driver usbfs848client # [ 0.159996] usbcore: registered new interface driver hub849client # [ 0.160012] usbcore: registered new device driver usb850client # [ 0.160276] pps_core: LinuxPPS API ver. 1 registered851worker1 # [ 0.177071] VFS: Disk quotas dquot_6.6.0852worker1 # [ 0.177106] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)853client # [ 0.160283] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>854worker2 # [ 0.167538] scmi_core: SCMI protocol bus registered855client # [ 0.160292] PTP clock support registered856worker1 # [ 0.178790] netfs: FS-Cache loaded857client # [ 0.160344] EDAC MC: Ver: 3.0.0858worker1 # [ 0.178914] pnp: PnP ACPI: disabled859worker2 # [ 0.168520] FPGA manager framework860worker2 # [ 0.169421] vgaarb: loaded861client # [ 0.164956] scmi_core: SCMI protocol bus registered862client # [ 0.165899] FPGA manager framework863client # [ 0.166809] vgaarb: loaded864worker2 # [ 0.170061] clocksource: Switched to clocksource arch_sys_counter865worker2 # [ 0.170648] VFS: Disk quotas dquot_6.6.0866client # [ 0.167424] clocksource: Switched to clocksource arch_sys_counter867worker2 # [ 0.170679] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)868worker2 # [ 0.173059] netfs: FS-Cache loaded869worker2 # [ 0.173150] pnp: PnP ACPI: disabled870worker1 # [ 0.182774] NET: Registered PF_INET protocol family871worker1 # [ 0.183042] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)872client # [ 0.171201] VFS: Disk quotas dquot_6.6.0873client # [ 0.171240] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)874client # [ 0.174841] netfs: FS-Cache loaded875client # [ 0.174954] pnp: PnP ACPI: disabled876worker2 # [ 0.178966] NET: Registered PF_INET protocol family877worker2 # [ 0.179238] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)878client # [ 0.178763] NET: Registered PF_INET protocol family879client # [ 0.179012] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)880worker1 # [ 0.213085] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881worker1 # [ 0.213128] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882worker1 # [ 0.213154] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883worker1 # [ 0.213229] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884worker1 # [ 0.213371] TCP: Hash tables configured (established 16384 bind 16384)885worker2 # [ 0.208034] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)886worker1 # [ 0.213450] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)887worker2 # [ 0.208076] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)888worker1 # [ 0.213486] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)889worker2 # [ 0.208102] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)890worker1 # [ 0.213521] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)891worker2 # [ 0.208173] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)892worker1 # [ 0.213614] NET: Registered PF_UNIX/PF_LOCAL protocol family893worker1 # [ 0.213636] NET: Registered PF_XDP protocol family894worker2 # [ 0.208317] TCP: Hash tables configured (established 16384 bind 16384)895worker1 # [ 0.213653] PCI: CLS 0 bytes, default 64896worker2 # [ 0.208403] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)897worker1 # [ 0.213878] Trying to unpack rootfs image as initramfs...898client # [ 0.208888] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)899worker2 # [ 0.208436] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker2 # [ 0.208472] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)901client # [ 0.208929] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)902worker2 # [ 0.208593] NET: Registered PF_UNIX/PF_LOCAL protocol family903client # [ 0.208956] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)904worker2 # [ 0.208615] NET: Registered PF_XDP protocol family905worker2 # [ 0.208631] PCI: CLS 0 bytes, default 64906client # [ 0.209030] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)907worker2 # [ 0.208866] Trying to unpack rootfs image as initramfs...908client # [ 0.209172] TCP: Hash tables configured (established 16384 bind 16384)909client # [ 0.209252] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)910client # [ 0.209286] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)911worker1 # [ 0.230468] kvm [1]: HYP mode not available912client # [ 0.209324] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)913client # [ 0.209449] NET: Registered PF_UNIX/PF_LOCAL protocol family914client # [ 0.209469] NET: Registered PF_XDP protocol family915client # [ 0.209485] PCI: CLS 0 bytes, default 64916client # [ 0.209706] Trying to unpack rootfs image as initramfs...917worker2 # [ 0.224034] kvm [1]: HYP mode not available918client # [ 0.225080] kvm [1]: HYP mode not available919lb # [ 0.314274] Initialise system trusted keyrings920lb # [ 0.314998] workingset: timestamp_bits=42 max_order=19 bucket_order=0921lb # [ 0.316211] squashfs: version 4.0 (2009/01/31) Phillip Lougher922lb # [ 0.321412] 9p: Installing v9fs 9p2000 file system support923lb # [ 0.342076] Key type asymmetric registered924lb # [ 0.342093] Asymmetric key parser 'x509' registered925lb # [ 0.342151] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)926lb # [ 0.344210] io scheduler mq-deadline registered927lb # [ 0.344221] io scheduler kyber registered928lb # [ 0.353519] pl061_gpio 9030000.pl061: PL061 GPIO chip registered929lb # [ 0.354868] ledtrig-cpu: registered to indicate activity on CPUs930lb # [ 0.355250] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:931lb # [ 0.355267] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000932lb # [ 0.355279] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000933lb # [ 0.355288] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000934lb # [ 0.355307] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits935lb # [ 0.355330] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]936lb # [ 0.355400] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00937lb # [ 0.355409] pci_bus 0000:00: root bus resource [bus 00-ff]938lb # [ 0.355414] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]939lb # [ 0.355419] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]940lb # [ 0.355424] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]941lb # [ 0.355477] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint942lb # [ 0.355908] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint943lb # [ 0.356089] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]944lb # [ 0.356105] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]945lb # [ 0.356135] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]946lb # [ 0.356151] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]947lb # [ 0.356601] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint948lb # [ 0.356779] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]949lb # [ 0.356795] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]950lb # [ 0.356824] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]951lb # [ 0.357264] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint952worker1 # [ 0.321113] Initialise system trusted keyrings953worker1 # [ 0.321847] workingset: timestamp_bits=42 max_order=19 bucket_order=0954worker1 # [ 0.323083] squashfs: version 4.0 (2009/01/31) Phillip Lougher955worker1 # [ 0.323889] 9p: Installing v9fs 9p2000 file system support956worker2 # [ 0.314566] Initialise system trusted keyrings957worker2 # [ 0.315307] workingset: timestamp_bits=42 max_order=19 bucket_order=0958worker2 # [ 0.316555] squashfs: version 4.0 (2009/01/31) Phillip Lougher959worker2 # [ 0.317341] 9p: Installing v9fs 9p2000 file system support960client # [ 0.315928] Initialise system trusted keyrings961lb # [ 0.385508] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]962client # [ 0.316664] workingset: timestamp_bits=42 max_order=19 bucket_order=0963lb # [ 0.385536] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]964lb # [ 0.385566] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]965client # [ 0.317961] squashfs: version 4.0 (2009/01/31) Phillip Lougher966lb # [ 0.386058] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint967lb # [ 0.386244] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]968client # [ 0.318732] 9p: Installing v9fs 9p2000 file system support969lb # [ 0.386260] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]970lb # [ 0.386290] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]971lb # [ 0.386309] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]972lb # [ 0.386765] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint973lb # [ 0.386946] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]974lb # [ 0.386975] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]975lb # [ 0.387422] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint976lb # [ 0.387608] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]977lb # [ 0.387639] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]978lb # [ 0.388062] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint979lb # [ 0.388243] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]980lb # [ 0.388491] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint981lb # [ 0.388676] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]982lb # [ 0.388706] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]983lb # [ 0.389149] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint984lb # [ 0.389330] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]985lb # [ 0.389359] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]986worker2 # [ 0.338045] Key type asymmetric registered987lb # [ 0.389831] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint988worker2 # [ 0.338063] Asymmetric key parser 'x509' registered989lb # [ 0.390016] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]990lb # [ 0.390046] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]991lb # [ 0.390502] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint992lb # [ 0.390760] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]993worker1 # [ 0.352591] Key type asymmetric registered994lb # [ 0.390777] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]995worker1 # [ 0.352614] Asymmetric key parser 'x509' registered996lb # [ 0.390806] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]997client # [ 0.339407] Key type asymmetric registered998worker1 # [ 0.352681] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)999client # [ 0.339423] Asymmetric key parser 'x509' registered1000lb # [ 0.391252] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1001worker1 # [ 0.354796] io scheduler mq-deadline registered1002lb # [ 0.391433] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1003worker1 # [ 0.354806] io scheduler kyber registered1004lb # [ 0.391449] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1005lb # [ 0.391478] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006worker2 # [ 0.346181] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1007lb # [ 0.392036] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1008worker2 # [ 0.347230] io scheduler mq-deadline registered1009worker2 # [ 0.347241] io scheduler kyber registered1010lb # [ 0.392047] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1011lb # [ 0.392052] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1012lb # [ 0.392095] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1013lb # [ 0.392140] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1014lb # [ 0.392186] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1015client # [ 0.347516] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1016client # [ 0.348549] io scheduler mq-deadline registered1017lb # [ 0.392231] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1018client # [ 0.348559] io scheduler kyber registered1019lb # [ 0.392276] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1020lb # [ 0.392321] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1021worker1 # [ 0.364671] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1022lb # [ 0.392366] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1023lb # [ 0.392410] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1024lb # [ 0.392455] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1025worker1 # [ 0.366058] ledtrig-cpu: registered to indicate activity on CPUs1026lb # [ 0.392521] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1027worker2 # [ 0.358231] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1028worker1 # [ 0.366445] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1029lb # [ 0.392565] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1030worker1 # [ 0.366461] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001031lb # [ 0.392587] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1032lb # [ 0.392608] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1033worker1 # [ 0.366474] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001034lb # [ 0.392629] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1035worker2 # [ 0.359552] ledtrig-cpu: registered to indicate activity on CPUs1036worker1 # [ 0.366482] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001037lb # [ 0.392650] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1038worker2 # [ 0.359939] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1039worker1 # [ 0.366508] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1040lb # [ 0.392671] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1041worker2 # [ 0.359956] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001042lb # [ 0.392692] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1043worker1 # [ 0.366530] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1044worker2 # [ 0.359982] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001045lb # [ 0.392716] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1046worker1 # [ 0.366612] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001047lb # [ 0.392737] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1048worker1 # [ 0.366621] pci_bus 0000:00: root bus resource [bus 00-ff]1049worker2 # [ 0.359991] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001050lb # [ 0.392759] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1051worker1 # [ 0.366627] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1052worker2 # [ 0.360017] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1053lb # [ 0.392781] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1054worker1 # [ 0.366632] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1055worker2 # [ 0.360040] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1056lb # [ 0.392803] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1057worker1 # [ 0.366637] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1058worker2 # [ 0.360112] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001059lb # [ 0.392824] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1060worker1 # [ 0.366697] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1061worker2 # [ 0.360121] pci_bus 0000:00: root bus resource [bus 00-ff]1062lb # [ 0.392845] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1063worker2 # [ 0.360127] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1064worker1 # [ 0.367144] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1065lb # [ 0.392866] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1066worker1 # [ 0.367329] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1067worker2 # [ 0.360133] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1068lb # [ 0.392886] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1069client # [ 0.359793] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1070worker1 # [ 0.367345] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1071lb # [ 0.392907] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1072worker2 # [ 0.360138] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1073client # [ 0.360398] ledtrig-cpu: registered to indicate activity on CPUs1074worker1 # [ 0.367374] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1075lb # [ 0.392928] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1076worker2 # [ 0.360191] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1077client # [ 0.360745] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1078lb # [ 0.392953] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1079worker1 # [ 0.367390] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1080worker2 # [ 0.360641] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1081lb # [ 0.392962] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1082client # [ 0.360762] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001083worker1 # [ 0.367843] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1084worker2 # [ 0.360829] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1085lb # [ 0.392966] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1086worker1 # [ 0.368025] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1087client # [ 0.360774] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001088worker2 # [ 0.360845] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1089worker1 # [ 0.368040] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1090worker2 # [ 0.360875] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1091client # [ 0.360782] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001092worker1 # [ 0.368069] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093worker2 # [ 0.360891] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1094client # [ 0.360802] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1095worker1 # [ 0.368527] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1096worker2 # [ 0.361362] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1097client # [ 0.360826] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1098worker2 # [ 0.361546] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1099worker2 # [ 0.361562] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1100client # [ 0.360898] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001101client # [ 0.360907] pci_bus 0000:00: root bus resource [bus 00-ff]1102worker2 # [ 0.361591] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103client # [ 0.360913] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1104worker2 # [ 0.362039] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1105client # [ 0.360919] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1106worker1 # [ 0.388830] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1107worker2 # [ 0.381739] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1108client # [ 0.360924] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1109worker2 # [ 0.381759] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1110worker1 # [ 0.388849] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1111lb # [ 0.453847] pci 0000:00:07.0: enabling device (0000 -> 0002)1112client # [ 0.361022] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1113worker2 # [ 0.381789] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114worker1 # [ 0.388879] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115client # [ 0.361461] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1116worker1 # [ 0.389338] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1117client # [ 0.361646] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1118worker1 # [ 0.389521] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1119client # [ 0.361663] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1120worker1 # [ 0.389536] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1121client # [ 0.361693] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1122worker2 # [ 0.384328] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1123worker1 # [ 0.389566] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124client # [ 0.361709] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1125worker2 # [ 0.384519] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1126worker1 # [ 0.389584] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1127worker2 # [ 0.384535] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1128client # [ 0.362168] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1129worker1 # [ 0.390034] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1130worker2 # [ 0.384564] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1131client # [ 0.362357] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1132worker1 # [ 0.390221] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1133worker2 # [ 0.384583] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1134client # [ 0.362374] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1135client # [ 0.362404] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1136worker1 # [ 0.390251] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137worker2 # [ 0.385037] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1138worker2 # [ 0.385224] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1139client # [ 0.362853] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1140worker1 # [ 0.390700] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1141client # [ 0.363036] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1142worker2 # [ 0.385254] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143worker1 # [ 0.390886] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1144client # [ 0.363051] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1145worker1 # [ 0.390915] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146worker2 # [ 0.385728] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1147client # [ 0.363081] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148worker2 # [ 0.385914] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1149worker1 # [ 0.391307] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1150client # [ 0.363548] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1151worker2 # [ 0.385943] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1152worker1 # [ 0.391488] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1153client # [ 0.363733] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1154worker1 # [ 0.391774] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1155client # [ 0.363749] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1156worker1 # [ 0.391977] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1157client # [ 0.363778] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158worker1 # [ 0.392008] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1159client # [ 0.363797] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1160worker1 # [ 0.392468] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1161client # [ 0.364256] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1162client # [ 0.364455] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1163client # [ 0.364493] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164client # [ 0.364947] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1165client # [ 0.365133] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1166client # [ 0.365163] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167worker2 # [ 0.398406] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1168client # [ 0.365548] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1169worker2 # [ 0.398603] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1170client # [ 0.365729] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1171worker2 # [ 0.398854] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1172client # [ 0.366012] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1173worker2 # [ 0.399039] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1174lb # [ 0.474721] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1175client # [ 0.366200] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1176lb # [ 0.476752] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1177worker2 # [ 0.399069] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1178client # [ 0.366230] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1179worker1 # [ 0.412703] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1180worker2 # [ 0.399513] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1181worker1 # [ 0.412744] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1182client # [ 0.366677] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1183worker2 # [ 0.399698] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1184client # [ 0.366865] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1185worker1 # [ 0.413226] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1186worker2 # [ 0.399728] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1187client # [ 0.366895] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1188worker1 # [ 0.413417] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1189worker2 # [ 0.400186] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1190worker1 # [ 0.413447] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1191client # [ 0.367346] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1192worker2 # [ 0.400372] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1193worker1 # [ 0.413905] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1194worker2 # [ 0.400402] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1195worker1 # [ 0.414168] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1196worker2 # [ 0.400866] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1197worker1 # [ 0.414185] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1198worker2 # [ 0.401117] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1199worker1 # [ 0.414214] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200worker2 # [ 0.401134] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1201worker1 # [ 0.414679] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1202worker2 # [ 0.401164] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1203worker1 # [ 0.414862] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1204worker2 # [ 0.401626] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1205worker1 # [ 0.414877] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1206worker2 # [ 0.401807] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1207worker1 # [ 0.414907] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1208worker2 # [ 0.401823] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1209lb # [ 0.487692] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1210worker1 # [ 0.415481] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1211worker2 # [ 0.401852] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1212worker1 # [ 0.415491] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1213worker1 # [ 0.415497] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1214client # [ 0.411595] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1215worker1 # [ 0.415541] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1216client # [ 0.411636] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1217worker1 # [ 0.415587] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1218client # [ 0.412127] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1219worker1 # [ 0.415633] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1220client # [ 0.412416] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1221worker1 # [ 0.415679] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1222client # [ 0.412434] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1223client # [ 0.412464] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1224worker1 # [ 0.415725] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1225client # [ 0.412925] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1226worker1 # [ 0.415771] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1227client # [ 0.413109] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1228lb # [ 0.493585] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1229worker1 # [ 0.415817] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1230client # [ 0.413125] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1231lb # [ 0.495530] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1232worker2 # [ 0.422519] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1233worker1 # [ 0.415863] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1234client # [ 0.413154] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1235worker2 # [ 0.422542] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1236client # [ 0.413723] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1237worker1 # [ 0.415908] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1238worker2 # [ 0.422548] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1239client # [ 0.413734] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1240worker1 # [ 0.415977] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1241worker2 # [ 0.422596] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1242worker1 # [ 0.416022] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1243client # [ 0.413739] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1244worker2 # [ 0.422645] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1245worker1 # [ 0.416043] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1246client # [ 0.413785] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1247worker2 # [ 0.422691] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1248worker1 # [ 0.416064] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1249client # [ 0.413832] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1250worker2 # [ 0.422739] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1251worker1 # [ 0.416085] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1252client # [ 0.413879] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1253worker1 # [ 0.416105] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1254worker2 # [ 0.422786] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1255client # [ 0.413927] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1256worker1 # [ 0.416126] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1257worker2 # [ 0.422833] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1258client # [ 0.413973] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1259worker1 # [ 0.416147] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1260worker2 # [ 0.422879] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1261client # [ 0.414021] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1262worker1 # [ 0.416171] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1263worker2 # [ 0.422925] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1264worker1 # [ 0.416192] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1265client # [ 0.414069] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1266worker2 # [ 0.422972] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1267worker1 # [ 0.416214] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1268lb # [ 0.505552] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1269client # [ 0.414115] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1270worker2 # [ 0.423073] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1271worker1 # [ 0.416235] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1272lb # [ 0.507429] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1273client # [ 0.414163] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1274worker2 # [ 0.423127] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1275lb # [ 0.509167] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1276worker1 # [ 0.416257] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1277client # [ 0.414234] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1278worker2 # [ 0.423149] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1279worker1 # [ 0.416277] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1280client # [ 0.414280] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1281worker2 # [ 0.423170] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1282worker1 # [ 0.416298] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1283client # [ 0.414301] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1284worker2 # [ 0.423192] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1285worker1 # [ 0.416318] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1286client # [ 0.414323] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1287worker2 # [ 0.423213] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1288worker1 # [ 0.416339] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1289worker1 # [ 0.416359] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1290client # [ 0.414344] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1291worker2 # [ 0.423234] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1292worker1 # [ 0.416380] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1293client # [ 0.414365] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1294worker2 # [ 0.423255] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1295worker1 # [ 0.416406] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1296client # [ 0.414386] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1297worker2 # [ 0.423283] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1298worker1 # [ 0.416415] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1299client # [ 0.414407] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1300worker2 # [ 0.423304] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1301worker1 # [ 0.416419] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1302client # [ 0.414433] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1303worker2 # [ 0.423326] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1304client # [ 0.414455] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1305worker2 # [ 0.423348] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1306client # [ 0.414477] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1307worker2 # [ 0.423370] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1308client # [ 0.414498] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1309worker2 # [ 0.423391] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1310worker2 # [ 0.423411] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1311client # [ 0.414520] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1312lb # [ 0.519179] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1313worker2 # [ 0.423432] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1314client # [ 0.414541] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1315lb # [ 0.521037] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1316client # [ 0.414562] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1317worker2 # [ 0.423453] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1318client # [ 0.414583] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1319worker2 # [ 0.423474] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1320client # [ 0.414604] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1321worker2 # [ 0.423496] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1322client # [ 0.414625] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1323worker2 # [ 0.423521] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1324worker1 # [ 0.461290] pci 0000:00:07.0: enabling device (0000 -> 0002)1325client # [ 0.414646] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1326worker2 # [ 0.423530] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1327client # [ 0.414672] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1328worker2 # [ 0.423535] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1329client # [ 0.414681] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1330worker2 # [ 0.424341] pci 0000:00:07.0: enabling device (0000 -> 0002)1331client # [ 0.414686] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1332lb # [ 0.532045] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1333client # [ 0.459560] pci 0000:00:07.0: enabling device (0000 -> 0002)1334lb # [ 0.542579] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1335lb # [ 0.545172] msm_serial: driver initialized1336lb # [ 0.545339] SuperH (H)SCI(F) driver initialized1337lb # [ 0.546480] STM32 USART driver initialized1338worker1 # [ 0.486895] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1339worker2 # [ 0.480274] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1340worker1 # [ 0.493171] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1341client # [ 0.480311] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1342worker1 # [ 0.495130] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1343client # [ 0.482407] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1344worker2 # [ 0.490281] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1345worker1 # [ 0.505084] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1346worker2 # [ 0.493611] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1347worker1 # [ 0.507133] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1348worker2 # [ 0.495575] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1349client # [ 0.493400] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1350worker2 # [ 0.497610] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1351client # [ 0.495316] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1352lb # [ 0.578123] loop: module loaded1353lb # [ 0.578290] virtio_blk virtio2: 1/0/0 default/read/poll queues1354lb # [ 0.579074] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1355worker2 # [ 0.507935] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1356worker2 # [ 0.509664] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1357client # [ 0.505085] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1358worker1 # [ 0.516700] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1359client # [ 0.508246] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1360worker1 # [ 0.518981] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1361client # [ 0.510130] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1362worker1 # [ 0.520751] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1363worker1 # [ 0.522505] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1364worker1 # [ 0.524294] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1365worker2 # [ 0.520581] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1366lb # [ 0.589986] megasas: 07.734.00.00-rc11367lb # [ 0.590637] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1368worker2 # [ 0.523309] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1369lb # [ 0.592845] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001370worker2 # [ 0.525053] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1371lb # [ 0.592868] Intel/Sharp Extended Query Table at 0x00311372client # [ 0.519934] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1373lb # [ 0.594614] Using buffer write method1374client # [ 0.521700] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1375lb # [ 0.594675] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1376worker1 # [ 0.538032] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1377lb # [ 0.596446] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001378lb # [ 0.596466] Intel/Sharp Extended Query Table at 0x00311379client # [ 0.525035] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1380client # [ 0.527212] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1381worker2 # [ 0.535466] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1382lb # [ 0.616853] Using buffer write method1383worker1 # [ 0.553808] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1384lb # [ 0.616887] Concatenating MTD devices:1385lb # [ 0.616891] (0): "0.flash"1386lb # [ 0.616895] (1): "0.flash"1387worker2 # [ 0.544549] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1388lb # [ 0.616906] into device "0.flash"1389worker1 # [ 0.556328] msm_serial: driver initialized1390worker1 # [ 0.556488] SuperH (H)SCI(F) driver initialized1391client # [ 0.543516] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1392worker2 # [ 0.547034] msm_serial: driver initialized1393worker2 # [ 0.547173] SuperH (H)SCI(F) driver initialized1394worker2 # [ 0.547227] STM32 USART driver initialized1395client # [ 0.546053] msm_serial: driver initialized1396client # [ 0.546184] SuperH (H)SCI(F) driver initialized1397client # [ 0.546237] STM32 USART driver initialized1398worker1 # [ 0.557666] STM32 USART driver initialized1399worker1 # [ 0.587352] loop: module loaded1400worker1 # [ 0.587533] virtio_blk virtio2: 1/0/0 default/read/poll queues1401worker1 # [ 0.588303] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1402client # [ 0.578320] loop: module loaded1403worker2 # [ 0.582684] loop: module loaded1404client # [ 0.578543] virtio_blk virtio2: 1/0/0 default/read/poll queues1405worker2 # [ 0.582876] virtio_blk virtio2: 1/0/0 default/read/poll queues1406client # [ 0.579268] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1407worker2 # [ 0.583613] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1408worker1 # [ 0.593146] megasas: 07.734.00.00-rc11409worker1 # [ 0.593829] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1410worker2 # [ 0.590690] megasas: 07.734.00.00-rc11411worker1 # [ 0.604574] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412worker2 # [ 0.591395] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1413worker1 # [ 0.604601] Intel/Sharp Extended Query Table at 0x00311414worker1 # [ 0.606319] Using buffer write method1415worker2 # [ 0.593317] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001416worker1 # [ 0.606404] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1417worker2 # [ 0.593344] Intel/Sharp Extended Query Table at 0x00311418worker2 # [ 0.595019] Using buffer write method1419worker2 # [ 0.595085] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1420client # [ 0.592036] megasas: 07.734.00.00-rc11421client # [ 0.592815] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1422worker2 # [ 0.597025] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001423worker2 # [ 0.597047] Intel/Sharp Extended Query Table at 0x00311424client # [ 0.594978] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001425client # [ 0.595000] Intel/Sharp Extended Query Table at 0x00311426client # [ 0.596548] Using buffer write method1427client # [ 0.596620] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1428worker1 # [ 0.617116] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001429worker1 # [ 0.617139] Intel/Sharp Extended Query Table at 0x00311430client # [ 0.598320] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001431worker1 # [ 0.618398] Using buffer write method1432client # [ 0.598365] Intel/Sharp Extended Query Table at 0x00311433worker1 # [ 0.618423] Concatenating MTD devices:1434worker1 # [ 0.618427] (0): "0.flash"1435worker1 # [ 0.618431] (1): "0.flash"1436worker1 # [ 0.618439] into device "0.flash"1437client # [ 0.611940] Using buffer write method1438client # [ 0.611972] Concatenating MTD devices:1439client # [ 0.611976] (0): "0.flash"1440client # [ 0.611980] (1): "0.flash"1441worker2 # [ 0.617403] Using buffer write method1442client # [ 0.611987] into device "0.flash"1443worker2 # [ 0.617438] Concatenating MTD devices:1444worker2 # [ 0.617442] (0): "0.flash"1445worker2 # [ 0.617446] (1): "0.flash"1446worker2 # [ 0.617455] into device "0.flash"1447lb # [ 0.850867] Freeing initrd memory: 26948K1448lb # [ 0.856783] tun: Universal TUN/TAP device driver, 1.61449lb # [ 0.860408] thunder_xcv, ver 1.01450lb # [ 0.860455] thunder_bgx, ver 1.01451lb # [ 0.860477] nicpf, ver 1.01452lb # [ 0.861026] e1000: Intel(R) PRO/1000 Network Driver1453lb # [ 0.861033] e1000: Copyright (c) 1999-2006 Intel Corporation.1454lb # [ 0.861056] e1000e: Intel(R) PRO/1000 Network Driver1455lb # [ 0.861064] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1456lb # [ 0.861092] igb: Intel(R) Gigabit Ethernet Network Driver1457lb # [ 0.861097] igb: Copyright (c) 2007-2014 Intel Corporation.1458lb # [ 0.861120] igbvf: Intel(R) Gigabit Virtual Function Network Driver1459lb # [ 0.861126] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1460lb # [ 0.861257] sky2: driver version 1.301461lb # [ 0.869911] usbcore: registered new interface driver usb-storage1462lb # [ 0.870003] usbcore: registered new interface driver usbserial_generic1463lb # [ 0.870016] usbserial: USB Serial support registered for generic1464lb # [ 0.870600] hv_vmbus: registering driver hyperv_keyboard1465lb # [ 0.874741] ehci-pci 0000:00:07.0: EHCI Host Controller1466lb # [ 0.874771] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11467lb # [ 0.875014] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001468lb # [ 0.877662] rtc-pl031 9010000.pl031: registered as rtc01469lb # [ 0.877691] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T06:59:00 UTC (1790146740)1470lb # [ 0.878028] i2c_dev: i2c /dev entries driver1471lb # [ 0.883341] sdhci: Secure Digital Host Controller Interface driver1472lb # [ 0.883352] sdhci: Copyright(c) Pierre Ossman1473lb # [ 0.883615] Synopsys Designware Multimedia Card Interface Driver1474lb # [ 0.883986] sdhci-pltfm: SDHCI platform and OF driver helper1475lb # [ 0.885419] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001476lb # [ 0.885719] hub 1-0:1.0: USB hub found1477lb # [ 0.885750] hub 1-0:1.0: 6 ports detected1478lb # [ 0.890621] hid: raw HID events driver (C) Jiri Kosina1479lb # [ 0.890870] usbcore: registered new interface driver usbhid1480lb # [ 0.890876] usbhid: USB HID core driver1481lb # [ 0.893796] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1482lb # [ 0.895295] drop_monitor: Initializing network drop monitor service1483lb # [ 0.895432] NET: Registered PF_INET6 protocol family1484lb # [ 0.898434] Segment Routing with IPv61485lb # [ 0.898451] In-situ OAM (IOAM) with IPv61486lb # [ 0.898478] NET: Registered PF_PACKET protocol family1487lb # [ 0.900139] 9pnet: Installing 9P2000 support1488lb # [ 0.900189] Key type dns_resolver registered1489lb # [ 0.906876] registered taskstats version 11490lb # [ 0.907012] Loading compiled-in X.509 certificates1491lb # [ 0.915820] Demotion targets for Node 0: null1492lb # [ 0.915934] Key type .fscrypt registered1493lb # [ 0.915941] Key type fscrypt-provisioning registered1494lb # [ 0.916034] ima: No TPM chip found, activating TPM-bypass!1495client # [ 0.839888] Freeing initrd memory: 26972K1496lb # [ 0.916053] ima: Allocated hash algorithm: sha11497lb # [ 0.916075] ima: No architecture policies found1498lb # [ 0.920070] input: gpio-keys as /devices/platform/gpio-keys/input/input01499client # [ 0.845864] tun: Universal TUN/TAP device driver, 1.61500worker1 # [ 0.860953] Freeing initrd memory: 26976K1501worker2 # [ 0.852432] Freeing initrd memory: 26944K1502client # [ 0.849552] thunder_xcv, ver 1.01503client # [ 0.849595] thunder_bgx, ver 1.01504client # [ 0.849617] nicpf, ver 1.01505client # [ 0.850179] e1000: Intel(R) PRO/1000 Network Driver1506worker1 # [ 0.867180] tun: Universal TUN/TAP device driver, 1.61507client # [ 0.850186] e1000: Copyright (c) 1999-2006 Intel Corporation.1508client # [ 0.850210] e1000e: Intel(R) PRO/1000 Network Driver1509worker2 # [ 0.858369] tun: Universal TUN/TAP device driver, 1.61510client # [ 0.850218] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1511client # [ 0.850247] igb: Intel(R) Gigabit Ethernet Network Driver1512client # [ 0.850253] igb: Copyright (c) 2007-2014 Intel Corporation.1513client # [ 0.850276] igbvf: Intel(R) Gigabit Virtual Function Network Driver1514client # [ 0.850282] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1515worker1 # [ 0.870913] thunder_xcv, ver 1.01516worker2 # [ 0.862063] thunder_xcv, ver 1.01517client # [ 0.850417] sky2: driver version 1.301518worker1 # [ 0.870977] thunder_bgx, ver 1.01519worker1 # [ 0.871000] nicpf, ver 1.01520worker1 # [ 0.871567] e1000: Intel(R) PRO/1000 Network Driver1521client # [ 0.859090] usbcore: registered new interface driver usb-storage1522worker1 # [ 0.871574] e1000: Copyright (c) 1999-2006 Intel Corporation.1523client # [ 0.859178] usbcore: registered new interface driver usbserial_generic1524worker1 # [ 0.871598] e1000e: Intel(R) PRO/1000 Network Driver1525client # [ 0.859192] usbserial: USB Serial support registered for generic1526worker1 # [ 0.871606] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1527worker1 # [ 0.871635] igb: Intel(R) Gigabit Ethernet Network Driver1528worker1 # [ 0.871641] igb: Copyright (c) 2007-2014 Intel Corporation.1529client # [ 0.862388] ehci-pci 0000:00:07.0: EHCI Host Controller1530lb # [ 0.937952] clk: Disabling unused clocks1531worker2 # [ 0.862575] thunder_bgx, ver 1.01532worker2 # [ 0.862606] nicpf, ver 1.01533worker1 # [ 0.871689] igbvf: Intel(R) Gigabit Virtual Function Network Driver1534lb # [ 0.937980] PM: genpd: Disabling unused power domains1535worker2 # [ 0.863149] e1000: Intel(R) PRO/1000 Network Driver1536client # [ 0.862418] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11537worker1 # [ 0.871695] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1538client # [ 0.862711] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001539worker2 # [ 0.863156] e1000: Copyright (c) 1999-2006 Intel Corporation.1540worker1 # [ 0.871833] sky2: driver version 1.301541worker2 # [ 0.863180] e1000e: Intel(R) PRO/1000 Network Driver1542lb # [ 0.942089] Freeing unused kernel memory: 4736K1543worker2 # [ 0.863188] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1544lb # [ 0.942298] Run /init as init process1545client # [ 0.865387] hv_vmbus: registering driver hyperv_keyboard1546worker2 # [ 0.863221] igb: Intel(R) Gigabit Ethernet Network Driver1547worker2 # [ 0.863226] igb: Copyright (c) 2007-2014 Intel Corporation.1548worker1 # [ 0.881212] ehci-pci 0000:00:07.0: EHCI Host Controller1549client # [ 0.866906] rtc-pl031 9010000.pl031: registered as rtc01550worker2 # [ 0.863250] igbvf: Intel(R) Gigabit Virtual Function Network Driver1551worker1 # [ 0.881244] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11552client # [ 0.866933] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T06:59:00 UTC (1790146740)1553worker2 # [ 0.863256] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1554worker1 # [ 0.881510] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001555worker2 # [ 0.863392] sky2: driver version 1.301556client # [ 0.867226] i2c_dev: i2c /dev entries driver1557worker2 # [ 0.865029] usbcore: registered new interface driver usb-storage1558worker1 # [ 0.884009] usbcore: registered new interface driver usb-storage1559worker2 # [ 0.865151] usbcore: registered new interface driver usbserial_generic1560worker1 # [ 0.884105] usbcore: registered new interface driver usbserial_generic1561worker2 # [ 0.865166] usbserial: USB Serial support registered for generic1562worker1 # [ 0.884120] usbserial: USB Serial support registered for generic1563worker2 # [ 0.865750] hv_vmbus: registering driver hyperv_keyboard1564worker2 # [ 0.866812] ehci-pci 0000:00:07.0: EHCI Host Controller1565worker1 # [ 0.887068] hv_vmbus: registering driver hyperv_keyboard1566worker2 # [ 0.866856] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11567worker2 # [ 0.867045] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001568worker1 # [ 0.888605] rtc-pl031 9010000.pl031: registered as rtc01569worker2 # [ 0.878832] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001570client # [ 0.872459] sdhci: Secure Digital Host Controller Interface driver1571worker2 # [ 0.879155] hub 1-0:1.0: USB hub found1572client # [ 0.872469] sdhci: Copyright(c) Pierre Ossman1573worker1 # [ 0.888629] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T06:59:00 UTC (1790146740)1574worker2 # [ 0.879176] hub 1-0:1.0: 6 ports detected1575worker1 # [ 0.888936] i2c_dev: i2c /dev entries driver1576client # [ 0.872731] Synopsys Designware Multimedia Card Interface Driver1577client # [ 0.873094] sdhci-pltfm: SDHCI platform and OF driver helper1578worker2 # [ 0.881720] rtc-pl031 9010000.pl031: registered as rtc01579client # [ 0.874591] hid: raw HID events driver (C) Jiri Kosina1580client # [ 0.874833] usbcore: registered new interface driver usbhid1581worker2 # [ 0.881750] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T06:59:00 UTC (1790146740)1582client # [ 0.874839] usbhid: USB HID core driver1583client # [ 0.879864] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001584worker2 # [ 0.883809] i2c_dev: i2c /dev entries driver1585client # [ 0.880907] hub 1-0:1.0: USB hub found1586client # [ 0.881382] hub 1-0:1.0: 6 ports detected1587worker1 # [ 0.894389] sdhci: Secure Digital Host Controller Interface driver1588lb # [ 0.959584] systemd[1]: Successfully made /usr/ read-only.1589worker1 # [ 0.894401] sdhci: Copyright(c) Pierre Ossman1590worker1 # [ 0.894678] Synopsys Designware Multimedia Card Interface Driver1591client # [ 0.882751] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1592worker1 # [ 0.895065] sdhci-pltfm: SDHCI platform and OF driver helper1593worker1 # [ 0.899311] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001594worker1 # [ 0.900320] hub 1-0:1.0: USB hub found1595client # [ 0.885397] drop_monitor: Initializing network drop monitor service1596worker2 # [ 0.887467] sdhci: Secure Digital Host Controller Interface driver1597client # [ 0.885543] NET: Registered PF_INET6 protocol family1598worker1 # [ 0.900829] hub 1-0:1.0: 6 ports detected1599worker2 # [ 0.887479] sdhci: Copyright(c) Pierre Ossman1600client # [ 0.887391] Segment Routing with IPv61601worker2 # [ 0.887735] Synopsys Designware Multimedia Card Interface Driver1602client # [ 0.887420] In-situ OAM (IOAM) with IPv61603worker2 # [ 0.888099] sdhci-pltfm: SDHCI platform and OF driver helper1604client # [ 0.888415] NET: Registered PF_PACKET protocol family1605worker1 # [ 0.901798] hid: raw HID events driver (C) Jiri Kosina1606worker2 # [ 0.889589] hid: raw HID events driver (C) Jiri Kosina1607worker1 # [ 0.902063] usbcore: registered new interface driver usbhid1608worker2 # [ 0.889835] usbcore: registered new interface driver usbhid1609worker1 # [ 0.902069] usbhid: USB HID core driver1610worker2 # [ 0.889842] usbhid: USB HID core driver1611client # [ 0.889116] 9pnet: Installing 9P2000 support1612client # [ 0.889163] Key type dns_resolver registered1613worker1 # [ 0.905041] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1614worker1 # [ 0.906603] drop_monitor: Initializing network drop monitor service1615worker2 # [ 0.895423] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1616worker1 # [ 0.906781] NET: Registered PF_INET6 protocol family1617worker2 # [ 0.896943] drop_monitor: Initializing network drop monitor service1618worker2 # [ 0.897079] NET: Registered PF_INET6 protocol family1619worker1 # [ 0.909834] Segment Routing with IPv61620worker1 # [ 0.909854] In-situ OAM (IOAM) with IPv61621worker2 # [ 0.900101] Segment Routing with IPv61622worker1 # [ 0.909884] NET: Registered PF_PACKET protocol family1623worker2 # [ 0.900120] In-situ OAM (IOAM) with IPv61624worker2 # [ 0.900176] NET: Registered PF_PACKET protocol family1625worker1 # [ 0.911546] 9pnet: Installing 9P2000 support1626worker2 # [ 0.901817] 9pnet: Installing 9P2000 support1627worker1 # [ 0.911594] Key type dns_resolver registered1628worker2 # [ 0.901867] Key type dns_resolver registered1629client # [ 0.895777] registered taskstats version 11630client # [ 0.895947] Loading compiled-in X.509 certificates1631client # [ 0.904583] Demotion targets for Node 0: null1632client # [ 0.904684] Key type .fscrypt registered1633worker2 # [ 0.908559] registered taskstats version 11634client # [ 0.904690] Key type fscrypt-provisioning registered1635worker2 # [ 0.908721] Loading compiled-in X.509 certificates1636client # [ 0.904788] ima: No TPM chip found, activating TPM-bypass!1637client # [ 0.904807] ima: Allocated hash algorithm: sha11638client # [ 0.904829] ima: No architecture policies found1639client # [ 0.908840] input: gpio-keys as /devices/platform/gpio-keys/input/input01640worker1 # [ 0.918402] registered taskstats version 11641worker1 # [ 0.918546] Loading compiled-in X.509 certificates1642worker1 # [ 0.927059] Demotion targets for Node 0: null1643worker1 # [ 0.927168] Key type .fscrypt registered1644worker2 # [ 0.917254] Demotion targets for Node 0: null1645worker1 # [ 0.927174] Key type fscrypt-provisioning registered1646worker2 # [ 0.917365] Key type .fscrypt registered1647worker1 # [ 0.927271] ima: No TPM chip found, activating TPM-bypass!1648worker2 # [ 0.917371] Key type fscrypt-provisioning registered1649worker1 # [ 0.927290] ima: Allocated hash algorithm: sha11650worker2 # [ 0.917467] ima: No TPM chip found, activating TPM-bypass!1651worker1 # [ 0.927310] ima: No architecture policies found1652worker2 # [ 0.917486] ima: Allocated hash algorithm: sha11653worker2 # [ 0.917508] ima: No architecture policies found1654worker1 # [ 0.931820] input: gpio-keys as /devices/platform/gpio-keys/input/input01655worker2 # [ 0.921752] input: gpio-keys as /devices/platform/gpio-keys/input/input01656client # [ 0.926398] clk: Disabling unused clocks1657client # [ 0.926425] PM: genpd: Disabling unused power domains1658client # [ 0.930594] Freeing unused kernel memory: 4736K1659client # [ 0.930803] Run /init as init process1660worker1 # [ 0.950124] clk: Disabling unused clocks1661worker2 # [ 0.940389] clk: Disabling unused clocks1662worker1 # [ 0.950155] PM: genpd: Disabling unused power domains1663worker2 # [ 0.940415] PM: genpd: Disabling unused power domains1664worker1 # [ 0.954365] Freeing unused kernel memory: 4736K1665worker1 # [ 0.954571] Run /init as init process1666worker2 # [ 0.944616] Freeing unused kernel memory: 4736K1667worker2 # [ 0.944822] Run /init as init process1668client # [ 0.945736] systemd[1]: Successfully made /usr/ read-only.1669worker2 # [ 0.960254] systemd[1]: Successfully made /usr/ read-only.1670worker1 # [ 0.970860] systemd[1]: Successfully made /usr/ read-only.1671lb # [ 1.133476] usb 1-1: new high-speed USB device number 2 using ehci-pci1672worker2 # [ 1.126143] usb 1-1: new high-speed USB device number 2 using ehci-pci1673client # [ 1.127513] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker1 # [ 1.148615] usb 1-1: new high-speed USB device number 2 using ehci-pci1675lb # [ 1.285980] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676lb # [ 1.294382] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1677lb # [ 1.306655] systemd[1]: Detected virtualization qemu.1678lb # [ 1.308706] systemd[1]: Detected architecture arm64.1679lb # [ 1.310678] systemd[1]: Running in initrd.1680lb # [ 1.313261] systemd[1]: Initializing machine ID from random generator.1681lb # [ 1.316244] systemd[1]: Hostname set to <lb>.1682worker2 # [ 1.278647] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11683client # [ 1.280051] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684lb # [ 1.365651] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01685worker1 # [ 1.301026] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11686client # [ 1.286068] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1687client # [ 1.298454] systemd[1]: Detected virtualization qemu.1688worker2 # [ 1.295123] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1689client # [ 1.298546] systemd[1]: Detected architecture arm64.1690worker1 # [ 1.307269] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1691client # [ 1.298571] systemd[1]: Running in initrd.1692worker2 # [ 1.307300] systemd[1]: Detected virtualization qemu.1693worker2 # [ 1.309400] systemd[1]: Detected architecture arm64.1694client # [ 1.305136] systemd[1]: Initializing machine ID from random generator.1695worker1 # [ 1.319861] systemd[1]: Detected virtualization qemu.1696worker2 # [ 1.311382] systemd[1]: Running in initrd.1697client # [ 1.307987] systemd[1]: Hostname set to <client>.1698worker1 # [ 1.319950] systemd[1]: Detected architecture arm64.1699worker1 # [ 1.319973] systemd[1]: Running in initrd.1700worker2 # [ 1.313967] systemd[1]: Initializing machine ID from random generator.1701worker1 # [ 1.326626] systemd[1]: Initializing machine ID from random generator.1702worker2 # [ 1.316959] systemd[1]: Hostname set to <worker2>.1703worker1 # [ 1.329655] systemd[1]: Hostname set to <worker1>.1704worker2 # [ 1.362386] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705client # [ 1.359691] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker1 # [ 1.380795] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707lb # [ 1.489430] usb 1-2: new high-speed USB device number 3 using ehci-pci1708worker2 # [ 1.482144] usb 1-2: new high-speed USB device number 3 using ehci-pci1709client # [ 1.483474] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker1 # [ 1.504620] usb 1-2: new high-speed USB device number 3 using ehci-pci1711lb # [ 1.632869] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712lb # [ 1.645838] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21713lb # [ 1.653686] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01714client # [ 1.619337] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker1 # [ 1.640154] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716worker2 # [ 1.630651] systemd[1]: bpf-restrict-fs: LSM BPF program attached1717client # [ 1.642531] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718worker2 # [ 1.646862] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719client # [ 1.648267] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720worker2 # [ 1.647203] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01721worker1 # [ 1.665308] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21722worker1 # [ 1.671162] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723lb # [ 1.743171] systemd[1]: Queued start job for default target Initrd Default Target.1724lb # [ 1.751473] systemd[1]: Created slice Slice /system/modprobe.1725lb # [ 1.752502] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1726lb # [ 1.753754] systemd[1]: Expecting device /dev/disk/by-label/nixos...1727lb # [ 1.754662] systemd[1]: Reached target Path Units.1728lb # [ 1.755359] systemd[1]: Reached target Slice Units.1729lb # [ 1.756073] systemd[1]: Reached target Swaps.1730lb # [ 1.756716] systemd[1]: Reached target Timer Units.1731lb # [ 1.757610] systemd[1]: Listening on D-Bus System Message Bus Socket.1732lb # [ 1.758665] systemd[1]: Listening on Journal Socket (/dev/log).1733lb # [ 1.759647] systemd[1]: Listening on Journal Sockets.1734lb # [ 1.760491] systemd[1]: Listening on udev Control Socket.1735lb # [ 1.761360] systemd[1]: Listening on udev Kernel Socket.1736lb # [ 1.762235] systemd[1]: Reached target Socket Units.1737lb # [ 1.764380] systemd[1]: Starting Create List of Static Device Nodes...1738lb # [ 1.765360] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1739lb # [ 1.773572] systemd[1]: Mounting Kernel Configuration File System...1740lb # [ 1.781779] systemd[1]: Starting Journal Service...1741lb # [ 1.797236] systemd[1]: Starting Load Kernel Modules...1742lb # [ 1.801682] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1743client # [ 1.734539] systemd[1]: Queued start job for default target Initrd Default Target.1744worker2 # [ 1.740792] systemd[1]: Queued start job for default target Initrd Default Target.1745client # [ 1.741625] systemd[1]: Created slice Slice /system/modprobe.1746worker1 # [ 1.755969] systemd[1]: Queued start job for default target Initrd Default Target.1747client # [ 1.742681] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1748lb # [ 1.819343] systemd[1]: Starting Coldplug All udev Devices...1749client # [ 1.743921] systemd[1]: Expecting device /dev/disk/by-label/nixos...1750lb # [ 1.823478] systemd[1]: Finished Create List of Static Device Nodes.1751client # [ 1.743954] systemd[1]: Reached target Path Units.1752client # [ 1.743971] systemd[1]: Reached target Slice Units.1753lb # [ 1.824933] systemd[1]: Mounted Kernel Configuration File System.1754client # [ 1.743988] systemd[1]: Reached target Swaps.1755client # [ 1.744004] systemd[1]: Reached target Timer Units.1756worker2 # [ 1.752816] systemd[1]: Created slice Slice /system/modprobe.1757client # [ 1.744173] systemd[1]: Listening on D-Bus System Message Bus Socket.1758client # [ 1.744308] systemd[1]: Listening on Journal Socket (/dev/log).1759client # [ 1.744442] systemd[1]: Listening on Journal Sockets.1760client # [ 1.744565] systemd[1]: Listening on udev Control Socket.1761client # [ 1.744667] systemd[1]: Listening on udev Kernel Socket.1762worker1 # [ 1.767020] systemd[1]: Created slice Slice /system/modprobe.1763client # [ 1.744687] systemd[1]: Reached target Socket Units.1764worker2 # [ 1.754352] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1765worker1 # [ 1.768216] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1766worker2 # [ 1.754403] systemd[1]: Expecting device /dev/disk/by-label/nixos...1767client # [ 1.754376] systemd[1]: Starting Create List of Static Device Nodes...1768worker2 # [ 1.754438] systemd[1]: Reached target Path Units.1769client # [ 1.755395] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1770worker2 # [ 1.754460] systemd[1]: Reached target Slice Units.1771worker2 # [ 1.754482] systemd[1]: Reached target Swaps.1772worker1 # [ 1.769686] systemd[1]: Expecting device /dev/disk/by-label/nixos...1773worker2 # [ 1.754504] systemd[1]: Reached target Timer Units.1774worker1 # [ 1.769723] systemd[1]: Reached target Path Units.1775worker1 # [ 1.769743] systemd[1]: Reached target Slice Units.1776worker2 # [ 1.754712] systemd[1]: Listening on D-Bus System Message Bus Socket.1777worker1 # [ 1.769762] systemd[1]: Reached target Swaps.1778worker2 # [ 1.754881] systemd[1]: Listening on Journal Socket (/dev/log).1779worker1 # [ 1.769783] systemd[1]: Reached target Timer Units.1780worker2 # [ 1.755047] systemd[1]: Listening on Journal Sockets.1781worker1 # [ 1.769972] systemd[1]: Listening on D-Bus System Message Bus Socket.1782worker2 # [ 1.755195] systemd[1]: Listening on udev Control Socket.1783worker1 # [ 1.770129] systemd[1]: Listening on Journal Socket (/dev/log).1784worker2 # [ 1.755328] systemd[1]: Listening on udev Kernel Socket.1785worker1 # [ 1.770273] systemd[1]: Listening on Journal Sockets.1786client # [ 1.763589] systemd[1]: Mounting Kernel Configuration File System...1787worker2 # [ 1.755353] systemd[1]: Reached target Socket Units.1788worker1 # [ 1.770406] systemd[1]: Listening on udev Control Socket.1789lb # [ 1.839335] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1790worker1 # [ 1.770518] systemd[1]: Listening on udev Kernel Socket.1791worker2 # [ 1.769583] systemd[1]: Starting Create List of Static Device Nodes...1792lb # [ 1.839854] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1793worker1 # [ 1.770540] systemd[1]: Reached target Socket Units.1794worker1 # [ 1.781811] systemd[1]: Starting Create List of Static Device Nodes...1795worker2 # [ 1.770968] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1796worker1 # [ 1.783009] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1797client # [ 1.771682] systemd[1]: Starting Journal Service...1798worker2 # [ 1.778986] systemd[1]: Mounting Kernel Configuration File System...1799worker1 # [ 1.792708] systemd[1]: Mounting Kernel Configuration File System...1800lb # [ 1.857482] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1801worker1 # [ 1.800833] systemd[1]: Starting Journal Service...1802worker2 # [ 1.790404] systemd[1]: Starting Journal Service...1803lb # [ 1.871264] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01804lb # [ 1.871491] [drm] features: -virgl +edid -resource_blob -host_visible1805lb # [ 1.871499] [drm] features: -context_init1806lb # [ 1.872174] [drm] number of scanouts: 11807lb # [ 1.872191] [drm] number of cap sets: 01808client # [ 1.799587] systemd[1]: Starting Load Kernel Modules...1809client # [ 1.800415] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1810client # [ 1.803785] systemd[1]: Starting Coldplug All udev Devices...1811lb # [ 1.882589] systemd-journald[72]: Collecting audit messages is disabled.1812worker1 # [ 1.824771] systemd[1]: Starting Load Kernel Modules...1813client # [ 1.813419] systemd-journald[72]: Collecting audit messages is disabled.1814client # [ 1.814460] systemd[1]: Finished Create List of Static Device Nodes.1815worker1 # [ 1.828792] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1816lb # [ 1.890246] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1817lb # [ 1.890263] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01818worker2 # [ 1.822369] systemd[1]: Starting Load Kernel Modules...1819lb # [ 1.895714] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1820worker2 # [ 1.822502] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1821lb # [ 1.898318] systemd[1]: Starting Create Static Device Nodes in /dev...1822client # [ 1.820008] systemd[1]: Mounted Kernel Configuration File System.1823client # [ 1.825955] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1824worker1 # [ 1.844763] systemd[1]: Starting Coldplug All udev Devices...1825worker2 # [ 1.837705] systemd[1]: Starting Coldplug All udev Devices...1826worker2 # [ 1.839997] systemd-journald[72]: Collecting audit messages is disabled.1827worker1 # [ 1.855371] systemd-journald[72]: Collecting audit messages is disabled.1828worker2 # [ 1.850225] systemd[1]: Finished Create List of Static Device Nodes.1829worker2 # [ 1.850914] systemd[1]: Mounted Kernel Configuration File System.1830worker2 # [ 1.855678] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1831worker1 # [ 1.868737] systemd[1]: Finished Create List of Static Device Nodes.1832lb # [ 1.921678] Console: switching to colour frame buffer device 160x501833worker1 # [ 1.869404] systemd[1]: Mounted Kernel Configuration File System.1834lb # [ 1.928109] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1835worker1 # [ 1.884926] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1836lb # [ 1.950604] systemd[1]: Finished Create Static Device Nodes in /dev.1837lb # [ 1.950799] systemd[1]: Reached target Preparation for Local File Systems.1838lb # [ 1.950830] systemd[1]: Reached target Local File Systems.1839client # [ 1.877664] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1840lb # [ 1.955566] systemd[1]: Starting Rule-based Manager for Device Events and Files...1841client # [ 1.887550] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1842lb # [ 1.965601] systemd[1]: Finished Load Kernel Modules.1843client # [ 1.890021] systemd[1]: Starting Create Static Device Nodes in /dev...1844lb # [ 1.969808] systemd[1]: Starting Apply Kernel Variables...1845worker1 # [ 1.910553] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1846client # [ 1.899603] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1847worker2 # [ 1.904112] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1848worker2 # [ 1.906423] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1849worker2 # [ 1.910452] systemd[1]: Starting Create Static Device Nodes in /dev...1850worker1 # [ 1.925162] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1851worker1 # [ 1.927570] systemd[1]: Starting Create Static Device Nodes in /dev...1852client # [ 1.916979] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01853client # [ 1.917212] [drm] features: -virgl +edid -resource_blob -host_visible1854client # [ 1.917221] [drm] features: -context_init1855client # [ 1.917913] [drm] number of scanouts: 11856client # [ 1.917930] [drm] number of cap sets: 01857worker1 # [ 1.936696] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1858client # [ 1.934230] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1859client # [ 1.934248] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01860worker1 # [ 1.950384] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01861worker1 # [ 1.950616] [drm] features: -virgl +edid -resource_blob -host_visible1862client # [ 1.937800] systemd[1]: Finished Create Static Device Nodes in /dev.1863worker1 # [ 1.950625] [drm] features: -context_init1864worker2 # [ 1.930866] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1865worker1 # [ 1.951331] [drm] number of scanouts: 11866client # [ 1.938903] systemd[1]: Reached target Preparation for Local File Systems.1867worker2 # [ 1.937081] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01868worker1 # [ 1.951349] [drm] number of cap sets: 01869client # [ 1.939811] systemd[1]: Reached target Local File Systems.1870worker2 # [ 1.937311] [drm] features: -virgl +edid -resource_blob -host_visible1871worker2 # [ 1.937322] [drm] features: -context_init1872worker2 # [ 1.938022] [drm] number of scanouts: 11873worker2 # [ 1.938041] [drm] number of cap sets: 01874lb # [ 2.017986] systemd[1]: Finished Apply Kernel Variables.1875worker1 # [ 1.967671] systemd[1]: Finished Create Static Device Nodes in /dev.1876worker1 # [ 1.967975] systemd[1]: Reached target Preparation for Local File Systems.1877client # [ 1.951806] systemd[1]: Starting Rule-based Manager for Device Events and Files...1878worker1 # [ 1.968001] systemd[1]: Reached target Local File Systems.1879worker2 # [ 1.958677] systemd[1]: Finished Create Static Device Nodes in /dev.1880worker2 # [ 1.958997] systemd[1]: Reached target Preparation for Local File Systems.1881worker1 # [ 1.971920] systemd[1]: Starting Rule-based Manager for Device Events and Files...1882worker2 # [ 1.959024] systemd[1]: Reached target Local File Systems.1883lb # [ 2.037111] systemd[1]: Started Journal Service.1884worker2 # [ 1.962807] systemd[1]: Starting Rule-based Manager for Device Events and Files...1885client # [ 1.963756] Console: switching to colour frame buffer device 160x501886worker2 # [ 1.970528] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1887worker2 # [ 1.970544] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01888worker1 # [ 1.984763] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1889worker1 # [ 1.984779] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01890lb # [ 2.040294] systemd-modules-load[73]: Inserted module 'dm_mod'1891lb # [ 2.041548] systemd-modules-load[73]: Module 'virtio_balloon' is built in1892lb # [ 2.042601] systemd-modules-load[73]: Module 'virtio_console' is built in1893lb # [ 2.043648] systemd-modules-load[73]: Inserted module 'virtio_gpu'1894client # [ 1.981703] systemd[1]: Started Journal Service.1895lb # [ 2.056117] systemd-modules-load[73]: Module 'virtio_rng' is built in1896worker2 # [ 1.990307] Console: switching to colour frame buffer device 160x501897worker1 # [ 2.000926] Console: switching to colour frame buffer device 160x501898lb # [ 2.058812] systemd[1]: Starting Create System Files and Directories...1899worker1 # [ 2.007263] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1900client # [ 2.000029] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1901client # [ 1.987744] systemd-modules-load[74]: Inserted module 'dm_mod'1902client # [ 1.989095] systemd-modules-load[74]: Module 'virtio_balloon' is built in1903lb # [ 2.068859] systemd-udevd[79]: Using default interface naming scheme 'v261'.1904client # [ 1.990485] systemd-modules-load[74]: Module 'virtio_console' is built in1905client # [ 1.991513] systemd[1]: Starting Create System Files and Directories...1906lb # [ 2.071691] systemd[1]: Finished Create System Files and Directories.1907worker2 # [ 2.018640] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1908worker2 # [ 2.023332] systemd[1]: Finished Load Kernel Modules.1909worker1 # [ 2.033870] systemd[1]: Finished Load Kernel Modules.1910worker2 # [ 2.012533] systemd-modules-load[73]: Inserted module 'dm_mod'1911client # [ 2.008479] systemd-modules-load[74]: Inserted module 'virtio_gpu'1912client # [ 2.009543] systemd-modules-load[74]: Module 'virtio_rng' is built in1913client # [ 2.010538] systemd[1]: Finished Load Kernel Modules.1914worker1 # [ 2.030520] systemd-modules-load[74]: Inserted module 'dm_mod'1915lb # [ 2.092214] systemd[1]: Started Rule-based Manager for Device Events and Files.1916worker1 # [ 2.044790] systemd[1]: Starting Apply Kernel Variables...1917worker2 # [ 2.034574] systemd[1]: Starting Apply Kernel Variables...1918worker2 # [ 2.034796] systemd[1]: Started Journal Service.1919worker1 # [ 2.046978] systemd[1]: Started Journal Service.1920worker2 # [ 2.023165] systemd-modules-load[73]: Module 'virtio_balloon' is built in1921client # [ 2.020617] systemd[1]: Starting Apply Kernel Variables...1922worker1 # [ 2.044241] systemd-modules-load[74]: Module 'virtio_balloon' is built in1923worker2 # [ 2.032356] systemd-modules-load[73]: Module 'virtio_console' is built in1924worker2 # [ 2.033533] systemd-modules-load[73]: Inserted module 'virtio_gpu'1925worker1 # [ 2.045462] systemd-modules-load[74]: Module 'virtio_console' is built in1926worker2 # [ 2.034558] systemd-modules-load[73]: Module 'virtio_rng' is built in1927worker1 # [ 2.046779] systemd-modules-load[74]: Inserted module 'virtio_gpu'1928worker1 # [ 2.047772] systemd-modules-load[74]: Module 'virtio_rng' is built in1929client # [ 2.033460] systemd[1]: Finished Create System Files and Directories.1930worker2 # [ 2.052123] systemd[1]: Starting Create System Files and Directories...1931client # [ 2.049019] systemd[1]: Finished Apply Kernel Variables.1932worker2 # [ 2.054490] systemd-udevd[79]: Using default interface naming scheme 'v261'.1933client # [ 2.052478] systemd-udevd[79]: Using default interface naming scheme 'v261'.1934worker1 # [ 2.072149] systemd[1]: Starting Create System Files and Directories...1935worker1 # [ 2.082627] systemd-udevd[79]: Using default interface naming scheme 'v261'.1936worker1 # [ 2.089465] systemd[1]: Finished Apply Kernel Variables.1937lb # [ 2.152204] systemd[1]: Starting Virtual Console Setup...1938client # [ 2.078242] systemd[1]: Started Rule-based Manager for Device Events and Files.1939worker2 # [ 2.089191] systemd[1]: Finished Create System Files and Directories.1940worker2 # [ 2.100316] systemd[1]: Started Rule-based Manager for Device Events and Files.1941worker1 # [ 2.113171] systemd[1]: Finished Create System Files and Directories.1942worker2 # [ 2.102213] systemd[1]: Finished Apply Kernel Variables.1943worker1 # [ 2.123494] systemd[1]: Started Rule-based Manager for Device Events and Files.1944lb # [ 2.200511] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1945lb # [ 2.203870] systemd[1]: Finished Virtual Console Setup.1946client # [ 2.131669] systemd[1]: Starting Virtual Console Setup...1947worker2 # [ 2.164156] systemd[1]: Starting Virtual Console Setup...1948worker1 # [ 2.182039] systemd[1]: Starting Virtual Console Setup...1949client # [ 2.184498] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1950client # [ 2.187985] systemd[1]: Finished Virtual Console Setup.1951worker2 # [ 2.208422] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1952worker2 # [ 2.211792] systemd[1]: Finished Virtual Console Setup.1953worker1 # [ 2.228457] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954worker1 # [ 2.231764] systemd[1]: Finished Virtual Console Setup.1955lb # [ 2.801138] systemd[1]: Finished Coldplug All udev Devices.1956lb # [ 2.802085] systemd[1]: Reached target System Initialization.1957lb # [ 2.802893] systemd[1]: Reached target Basic System.1958client # [ 2.790654] systemd[1]: Finished Coldplug All udev Devices.1959client # [ 2.791930] systemd[1]: Reached target System Initialization.1960client # [ 2.796459] systemd[1]: Reached target Basic System.1961worker2 # [ 2.809901] systemd[1]: Finished Coldplug All udev Devices.1962worker2 # [ 2.810886] systemd[1]: Reached target System Initialization.1963worker2 # [ 2.816296] systemd[1]: Reached target Basic System.1964worker1 # [ 2.834121] systemd[1]: Finished Coldplug All udev Devices.1965worker1 # [ 2.836146] systemd[1]: Reached target System Initialization.1966worker1 # [ 2.840101] systemd[1]: Reached target Basic System.1967lb # [ 2.939386] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1968lb # [ 2.977563] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969lb # [ 2.982080] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1970worker2 # [ 2.939725] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1971client # [ 2.947953] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972client # [ 2.959537] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1973lb # [ 3.040122] systemd[1]: Found device /dev/disk/by-label/nixos.1974lb # [ 3.044411] systemd[1]: Reached target Initrd Root Device.1975lb # [ 3.047827] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1976client # [ 2.990237] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1977worker1 # [ 3.007811] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1978worker1 # [ 3.011917] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1979worker2 # [ 3.000775] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1980worker2 # [ 3.004150] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1981worker1 # [ 3.017960] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1982lb # [ 3.103210] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1983lb # [ 3.108244] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1984lb # [ 3.118877] systemd[1]: Mounting /sysroot...1985worker2 # [ 3.068397] systemd[1]: Found device /dev/disk/by-label/nixos.1986client # [ 3.065212] systemd[1]: Found device /dev/disk/by-label/nixos.1987client # [ 3.067010] systemd[1]: Reached target Initrd Root Device.1988client # [ 3.069987] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989worker2 # [ 3.075762] systemd[1]: Reached target Initrd Root Device.1990lb # [ 3.163899] EXT4-fs (vda): mounted filesystem 4e796412-306a-4bfb-ae60-e56a025ef008 r/w with ordered data mode. Quota mode: none.1991worker2 # [ 3.082149] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992lb # [ 3.157017] systemd[1]: Mounted /sysroot.1993lb # [ 3.159002] systemd[1]: Reached target Initrd Root File System.1994lb # [ 3.162508] systemd[1]: Starting Mountpoints Configured in the Real Root...1995worker1 # [ 3.106861] systemd[1]: Found device /dev/disk/by-label/nixos.1996worker1 # [ 3.108638] systemd[1]: Reached target Initrd Root Device.1997worker1 # [ 3.116099] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1998lb # [ 3.188570] systemd-sysroot-fstab-check[118]: /sysroot should be mounted in the initrd, will request daemon-reload.1999lb # [ 3.195988] systemd[1]: Reload requested from client PID 118 ('systemd-sysroot') (unit initrd-parse-etc.service)...2000lb # [ 3.197673] systemd[1]: Reloading...
loading 2,154 hidden lines…
4155worker1 # [ 74.859966] nix-daemon[852]: reaped child process 941, status = succeeded4156lb # [ 74.920923] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received uploads request method=POST path=/api/pending_closures4157worker1 # [ 74.872537] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.493Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4158worker1 # [ 74.874054] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.493Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4159lb # [ 74.960385] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4160worker1 # [ 74.908883] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.532Z level=INFO msg="Uploading 1 narinfos"4161lb # [ 74.975440] niks3-server[1015]: 2026/09/23 07:00:14 INFO Signed narinfos id=10 count=14162lb # [ 74.986274] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4163worker1 # [ 74.943391] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.567Z level=INFO msg="Upload complete. (278ms)"4164lb # [ 75.004826] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4165lb # [ 75.014777] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4166lb # [ 75.019825] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4167worker1 # [ 74.959675] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14168lb # [ 75.027145] niks3-server[1015]: 2026/09/23 07:00:14 INFO Completed upload id=104169lb # [ 75.030192] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4170worker1 # [ 74.976711] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04171worker1 # [ 74.982581] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14172worker1 # [ 74.987327] nix-daemon[852]: reaped child process 920, status = succeeded4173worker1 # [ 75.014823] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864174worker1 # [ 75.029871] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864175worker1 # [ 75.043324] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.667Z level=INFO msg="All 1 paths already cached"4176client # copying 1 paths to 'lb:50051'...4177client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4178worker1 # [ 75.053883] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491864179worker1 # [ 75.070318] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4180worker1 # [ 75.212913] nix-daemon[852]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4181worker1 # [ 75.232453] nix-daemon[852]: reaped child process 950, status = succeeded4182lb # [ 75.297928] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received uploads request method=POST path=/api/pending_closures4183worker1 # [ 75.245410] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.868Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4184worker1 # [ 75.247106] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.868Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4185lb # [ 75.328090] niks3-server[1015]: 2026/09/23 07:00:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4186worker1 # [ 75.275023] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.898Z level=INFO msg="Uploading 1 narinfos"4187lb # [ 75.347511] niks3-server[1015]: 2026/09/23 07:00:14 INFO Signed narinfos id=11 count=14188lb # [ 75.357354] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4189lb # [ 75.373490] niks3-server[1015]: 2026/09/23 07:00:14 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4190worker1 # [ 75.318724] nix-grpc-daemon[902]: time=2026-09-23T07:00:14.942Z level=INFO msg="Upload complete. (236ms)"4191lb # [ 75.381563] niks3-server[1015]: 2026/09/23 07:00:15 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4192lb # [ 75.385368] niks3-server[1015]: 2026/09/23 07:00:15 INFO Completed upload id=114193lb # [ 75.388239] niks3-server[1015]: 2026/09/23 07:00:15 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4194worker1 # [ 75.328342] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=6244195worker1 # [ 75.348569] nix-grpc-daemon[765]: ts=2026-09-23T07:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864196worker1 # [ 75.353420] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4197worker1 # [ 75.364252] nix-daemon[852]: reaped child process 945, status = succeeded4198client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4199worker1 # [ 75.372154] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4200client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4201client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4202client # farm-b> LOG-farm-b-sig4203worker1 # [ 76.198045] nix-daemon[852]: reaped child process 954, status = succeeded4204worker1 # [ 76.308474] nix-daemon[852]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4205worker1 # [ 76.324390] nix-daemon[852]: reaped child process 973, status = succeeded4206lb # [ 76.394219] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received uploads request method=POST path=/api/pending_closures4207worker1 # [ 76.340932] nix-grpc-daemon[902]: time=2026-09-23T07:00:15.963Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4208worker1 # [ 76.342360] nix-grpc-daemon[902]: time=2026-09-23T07:00:15.963Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4209worker1 # [ 76.376419] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.000Z level=INFO msg="Uploading 1 narinfos"4210lb # [ 76.428797] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4211lb # [ 76.442272] niks3-server[1015]: 2026/09/23 07:00:16 INFO Signed narinfos id=12 count=14212lb # [ 76.453637] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4213worker1 # [ 76.406815] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.030Z level=INFO msg="Upload complete. (225ms)"4214lb # [ 76.470152] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4215lb # [ 76.477936] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4216lb # [ 76.485025] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4217worker1 # [ 76.421653] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14218lb # [ 76.489855] niks3-server[1015]: 2026/09/23 07:00:16 INFO Completed upload id=124219lb # [ 76.492485] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4220worker1 # [ 76.437027] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04221worker1 # [ 76.442344] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14222worker1 # [ 76.446901] nix-daemon[852]: reaped child process 952, status = succeeded4223worker1 # [ 76.482766] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864224worker1 # [ 76.497258] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864225worker1 # [ 76.510618] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.134Z level=INFO msg="All 2 paths already cached"4226client # copying 1 paths to 'lb:50051'...4227client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4228worker1 # [ 76.522490] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491864229worker1 # [ 76.539208] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4230worker1 # [ 76.685289] nix-daemon[852]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4231worker1 # [ 76.702191] nix-daemon[852]: reaped child process 982, status = succeeded4232lb # [ 76.770394] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received uploads request method=POST path=/api/pending_closures4233worker1 # [ 76.717165] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.341Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4234worker1 # [ 76.719247] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.342Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4235lb # [ 76.800154] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4236worker1 # [ 76.747456] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.370Z level=INFO msg="Uploading 1 narinfos"4237lb # [ 76.811650] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4238lb # [ 76.823430] niks3-server[1015]: 2026/09/23 07:00:16 INFO Signed narinfos id=13 count=14239lb # [ 76.833888] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4240worker1 # [ 76.778754] nix-grpc-daemon[902]: time=2026-09-23T07:00:16.402Z level=INFO msg="Upload complete. (226ms)"4241lb # [ 76.845228] niks3-server[1015]: 2026/09/23 07:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4242lb # [ 76.850501] niks3-server[1015]: 2026/09/23 07:00:16 INFO Completed upload id=134243worker1 # [ 76.788270] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=7524244lb # [ 76.853471] niks3-server[1015]: 2026/09/23 07:00:16 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4245worker1 # [ 76.798874] nix-grpc-daemon[765]: ts=2026-09-23T07:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864246worker1 # [ 76.808399] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4247worker1 # [ 76.811757] nix-daemon[852]: reaped child process 977, status = succeeded4248client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4249worker1 # [ 76.822434] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4250client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4251client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4252client # farm-top> LOG-farm-top-sig4253lb # [ 78.264275] envoy[895]: conn peer=[2001:db8:1::1]:34270 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9562 tx=4389 ms=58084254lb # [ 78.264604] envoy[895]: conn peer=[2001:db8:1::1]:34282 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=664255lb # [ 78.264726] envoy[895]: conn peer=[2001:db8:1::1]:52174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1893 ms=17764256worker1 # [ 78.289445] nix-daemon[852]: reaped child process 986, status = succeeded4257worker1 # [ 78.392851] nix-daemon[852]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4258worker1 # [ 78.410663] nix-daemon[852]: reaped child process 1005, status = succeeded4259lb # [ 78.479069] niks3-server[1015]: 2026/09/23 07:00:18 INFO Received uploads request method=POST path=/api/pending_closures4260worker1 # [ 78.427838] nix-grpc-daemon[902]: time=2026-09-23T07:00:18.049Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4261worker1 # [ 78.429525] nix-grpc-daemon[902]: time=2026-09-23T07:00:18.049Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4262lb # [ 78.512571] niks3-server[1015]: 2026/09/23 07:00:18 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4263lb # [ 78.526613] niks3-server[1015]: 2026/09/23 07:00:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4264worker1 # [ 78.471577] nix-grpc-daemon[902]: time=2026-09-23T07:00:18.095Z level=INFO msg="Uploading 1 narinfos"4265lb # [ 78.540531] niks3-server[1015]: 2026/09/23 07:00:18 INFO Signed narinfos id=14 count=14266lb # [ 78.546492] niks3-server[1015]: 2026/09/23 07:00:18 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4267lb # [ 78.556434] niks3-server[1015]: 2026/09/23 07:00:18 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4268worker1 # [ 78.501023] nix-grpc-daemon[902]: time=2026-09-23T07:00:18.124Z level=INFO msg="Upload complete. (266ms)"4269lb # [ 78.563061] niks3-server[1015]: 2026/09/23 07:00:18 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4270lb # [ 78.567235] niks3-server[1015]: 2026/09/23 07:00:18 INFO Completed upload id=144271lb # [ 78.572846] niks3-server[1015]: 2026/09/23 07:00:18 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4272worker1 # [ 78.517055] nix-grpc-daemon[765]: ts=2026-09-23T07:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14273worker1 # [ 78.530839] nix-daemon[852]: reaped child process 984, status = succeeded4274client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.43 seconds)4275client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4276client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.18 seconds)4277client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4278worker1 # [ 78.862384] nix-grpc-daemon[765]: ts=2026-09-23T07:00:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:333604279worker1 # [ 78.870619] nix-grpc-daemon[765]: ts=2026-09-23T07:00:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:333604280client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.20 seconds)4281client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4282worker2 # [ 79.063498] nix-grpc-daemon[770]: ts=2026-09-23T07:00:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:338744283client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4284client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4285client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4286client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4287worker2 # [ 79.110284] nix-grpc-daemon[770]: ts=2026-09-23T07:00:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:338744288worker2 # [ 79.119247] nix-grpc-daemon[770]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4289worker2 # [ 79.135625] nix-grpc-daemon[770]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4290worker2 # [ 79.151402] nix-grpc-daemon[770]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4291client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.28 seconds)4292client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4293client # finding garbage collector roots...4294client # removing stale temporary roots file "/nix/var/nix/temproots/837"4295client # removing stale temporary roots file "/nix/var/nix/temproots/804"4296client # removing stale temporary roots file "/nix/var/nix/temproots/866"4297client # removing stale temporary roots file "/nix/var/nix/temproots/881"4298client # removing stale temporary roots file "/nix/var/nix/temproots/858"4299client # removing stale temporary roots file "/nix/var/nix/temproots/825"4300client # removing stale temporary roots file "/nix/var/nix/temproots/845"4301client # removing stale temporary roots file "/nix/var/nix/temproots/789"4302client # deleting specified paths...4303client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4304client # deleting unused links...4305client # note: hard linking is currently saving 0.0 KiB4306worker1 # [ 79.541542] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:491864307worker2 # [ 79.532179] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244308worker2 # [ 79.543481] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.174Z level=INFO msg="All 1 paths already cached"4309worker2 # [ 79.551334] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:434244310worker1 # [ 79.572263] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:491864311client # copying 3 paths...4312worker2 # [ 79.573648] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:434244313client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4314worker2 # [ 79.599930] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:434244315worker2 # [ 79.607296] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=2324316worker2 # [ 79.616320] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:434244317worker2 # [ 79.621300] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=1764318worker2 # [ 79.630062] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:434244319worker2 # [ 79.635142] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=1284320client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4321client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4322client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.53 seconds)4323(finished: subtest: outputs read back through any worker carry the cache signature, in 6.61 seconds)4324subtest: two clients wanting the same drv build it once4325client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4326client # [ 79.775333] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4327client # Running as unit: dedup1.service; invocation ID: 213b01d5f2ab4ba384a40626201ef0134328client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4329client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4330worker1 # [ 79.976467] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:491864331client # [ 79.966074] nix[934]: don't know how to build these paths:4332client # [ 79.967848] nix[934]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4333worker1 # [ 79.994433] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04334worker1 # [ 80.000580] nix-grpc-daemon[765]: ts=2026-09-23T07:00:19Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14335worker2 # [ 80.031898] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244336worker2 # [ 80.044819] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244337client # [ 80.052150] nix[934]: copying 1 paths to 'lb:50051'...4338worker2 # [ 80.058808] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:434244339client # [ 80.056168] nix[934]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4340worker2 # [ 80.075027] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4341worker2 # [ 80.237872] nix-daemon[842]: accepted connection from pid 1003, user nix-grpc-daemon (trusted)4342worker2 # [ 80.259750] nix-daemon[842]: reaped child process 1006, status = succeeded4343lb # [ 80.332912] niks3-server[1015]: 2026/09/23 07:00:19 INFO Received uploads request method=POST path=/api/pending_closures4344worker2 # [ 80.268951] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.900Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4345worker2 # [ 80.270468] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.900Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4346lb # [ 80.369089] niks3-server[1015]: 2026/09/23 07:00:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4347worker2 # [ 80.304560] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.936Z level=INFO msg="Uploading 1 narinfos"4348lb # [ 80.379582] niks3-server[1015]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4349lb # [ 80.392337] niks3-server[1015]: 2026/09/23 07:00:20 INFO Signed narinfos id=15 count=14350lb # [ 80.397462] niks3-server[1015]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4351lb # [ 80.407157] niks3-server[1015]: 2026/09/23 07:00:20 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4352lb # [ 80.412838] niks3-server[1015]: 2026/09/23 07:00:20 INFO Completed upload id=154353worker2 # [ 80.338487] nix-grpc-daemon[854]: time=2026-09-23T07:00:19.970Z level=INFO msg="Upload complete. (245ms)"4354lb # [ 80.417686] niks3-server[1015]: 2026/09/23 07:00:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4355worker2 # [ 80.349264] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=4324356worker2 # [ 80.367825] nix-grpc-daemon[770]: ts=2026-09-23T07:00:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244357worker2 # [ 80.379714] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4358worker2 # [ 80.384638] nix-daemon[842]: reaped child process 1001, status = succeeded4359client # [ 80.390259] nix[934]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4360worker2 # [ 80.400162] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4361client # [ 80.870667] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4362client # Running as unit: dedup2.service; invocation ID: 76c0ac2a1cd5466f9c88faa9998a90104363client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4364client: must succeed: sleep 24365worker2 # [ 81.118894] nix-grpc-daemon[770]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:434244366client # [ 81.121512] nix[950]: this derivation will be built:4367worker2 # [ 81.129381] nix-grpc-daemon[770]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:434244368client # [ 81.127288] nix[950]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4369worker1 # [ 81.155878] nix-grpc-daemon[765]: ts=2026-09-23T07:00:20Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04370worker2 # [ 81.148111] nix-grpc-daemon[770]: ts=2026-09-23T07:00:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244371client # [ 81.147701] nix[950]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4372client: (finished: must succeed: sleep 2, in 2.05 seconds)4373client: waiting for success: ! systemctl is-active dedup1 dedup24374worker2 # [ 83.484848] nix-daemon[842]: reaped child process 1010, status = succeeded4375worker2 # [ 83.611293] nix-daemon[842]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)4376worker2 # [ 83.636097] nix-daemon[842]: reaped child process 1039, status = succeeded4377lb # [ 83.715749] niks3-server[1015]: 2026/09/23 07:00:23 INFO Received uploads request method=POST path=/api/pending_closures4378worker2 # [ 83.656427] nix-grpc-daemon[854]: time=2026-09-23T07:00:23.285Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4379worker2 # [ 83.657958] nix-grpc-daemon[854]: time=2026-09-23T07:00:23.285Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4380lb # [ 83.777831] niks3-server[1015]: 2026/09/23 07:00:23 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4381worker2 # [ 83.710912] nix-grpc-daemon[854]: time=2026-09-23T07:00:23.342Z level=INFO msg="Uploading 1 narinfos"4382lb # [ 83.790659] niks3-server[1015]: 2026/09/23 07:00:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4383lb # [ 83.798157] niks3-server[1015]: 2026/09/23 07:00:23 INFO Signed narinfos id=16 count=14384lb # [ 83.805116] niks3-server[1015]: 2026/09/23 07:00:23 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4385lb # [ 83.814501] niks3-server[1015]: 2026/09/23 07:00:23 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4386worker2 # [ 83.740617] nix-grpc-daemon[854]: time=2026-09-23T07:00:23.372Z level=INFO msg="Upload complete. (325ms)"4387lb # [ 83.818088] niks3-server[1015]: 2026/09/23 07:00:23 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4388lb # [ 83.823394] niks3-server[1015]: 2026/09/23 07:00:23 INFO Completed upload id=164389lb # [ 83.826352] niks3-server[1015]: 2026/09/23 07:00:23 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4390worker2 # [ 83.753283] nix-grpc-daemon[770]: ts=2026-09-23T07:00:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14391worker2 # [ 83.773197] nix-grpc-daemon[770]: ts=2026-09-23T07:00:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=2 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14392worker2 # [ 83.785161] nix-daemon[842]: reaped child process 1008, status = succeeded4393client # [ 83.816505] systemd[1]: dedup2.service: Deactivated successfully.4394client # [ 83.821777] systemd[1]: dedup2.service: Consumed 217ms CPU time over 2.941s wall clock time, 35.2M memory peak, 6.6K incoming IP traffic, 6K outgoing IP traffic.4395client # [ 83.854125] systemd[1]: dedup1.service: Deactivated successfully.4396client # [ 83.857873] systemd[1]: dedup1.service: Consumed 173ms CPU time over 4.082s wall clock time, 41.4M memory peak, 7.4K incoming IP traffic, 8.5K outgoing IP traffic.4397client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.14 seconds)4398client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4399client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4400worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4401worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4402worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4403worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4404worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4405worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4406worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4407worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4408(finished: subtest: two clients wanting the same drv build it once, in 5.00 seconds)4409subtest: --timeout and --max-silent-time reach the builder4410worker2 # [ 84.954265] nix-grpc-daemon[770]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:434244411worker1 # [ 84.986528] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04412worker1 # [ 84.986917] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14413worker1 # [ 85.001990] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864414worker1 # [ 85.013022] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864415worker1 # [ 85.027147] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491864416worker1 # [ 85.042562] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4417worker1 # [ 85.196958] nix-daemon[852]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4418worker1 # [ 85.217256] nix-daemon[852]: reaped child process 1035, status = succeeded4419lb # [ 85.285161] niks3-server[1015]: 2026/09/23 07:00:24 INFO Received uploads request method=POST path=/api/pending_closures4420worker1 # [ 85.234724] nix-grpc-daemon[902]: time=2026-09-23T07:00:24.857Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4421worker1 # [ 85.236254] nix-grpc-daemon[902]: time=2026-09-23T07:00:24.857Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4422lb # [ 85.315837] niks3-server[1015]: 2026/09/23 07:00:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4423worker1 # [ 85.263979] nix-grpc-daemon[902]: time=2026-09-23T07:00:24.887Z level=INFO msg="Uploading 1 narinfos"4424lb # [ 85.331523] niks3-server[1015]: 2026/09/23 07:00:24 INFO Signed narinfos id=17 count=14425lb # [ 85.338592] niks3-server[1015]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4426worker1 # [ 85.289135] nix-grpc-daemon[902]: time=2026-09-23T07:00:24.912Z level=INFO msg="Upload complete. (227ms)"4427lb # [ 85.350695] niks3-server[1015]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4428lb # [ 85.360096] niks3-server[1015]: 2026/09/23 07:00:24 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4429lb # [ 85.365236] niks3-server[1015]: 2026/09/23 07:00:24 INFO Completed upload id=174430lb # [ 85.367671] niks3-server[1015]: 2026/09/23 07:00:24 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4431worker1 # [ 85.296243] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=4244432worker1 # [ 85.310415] nix-grpc-daemon[765]: ts=2026-09-23T07:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864433worker1 # [ 85.316544] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4434worker1 # [ 85.321405] nix-daemon[852]: reaped child process 1030, status = succeeded4435worker1 # [ 85.334237] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4436lb # [ 88.265383] envoy[895]: conn peer=[2001:db8:1::1]:52178 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9485 tx=4349 ms=51894437lb # [ 88.270289] envoy[895]: conn peer=[2001:db8:1::1]:52186 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1784438lb # [ 88.275225] envoy[895]: conn peer=[2001:db8:1::1]:52196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=1074439lb # [ 88.280079] envoy[895]: conn peer=[2001:db8:1::1]:52210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=794440lb # [ 88.285635] envoy[895]: conn peer=[2001:db8:1::1]:52198 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=944441lb # [ 88.290803] envoy[895]: conn peer=[2001:db8:1::1]:38274 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1597 ms=26554442lb # [ 88.296503] envoy[895]: conn peer=[2001:db8:1::1]:52226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1564 ms=38514443worker1 # [ 91.548505] nix-grpc-daemon[765]: ts=2026-09-23T07:00:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04444worker1 # [ 91.571212] nix-daemon[852]: reaped child process 1037, status = succeeded4445worker1 # [ 91.573004] nix-daemon[852]: reaped child process 1039, status = succeeded4446worker1 # [ 91.791073] nix-grpc-daemon[765]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:491864447worker1 # [ 91.807596] nix-grpc-daemon[765]: ts=2026-09-23T07:00:31Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04448worker2 # [ 91.801264] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244449worker1 # [ 91.814123] nix-grpc-daemon[765]: ts=2026-09-23T07:00:31Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14450worker2 # [ 91.817945] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244451worker2 # [ 91.832672] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:434244452worker2 # [ 91.847708] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4453worker2 # [ 92.014803] nix-daemon[842]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)4454worker2 # [ 92.034376] nix-daemon[842]: reaped child process 1055, status = succeeded4455lb # [ 92.113987] niks3-server[1015]: 2026/09/23 07:00:31 INFO Received uploads request method=POST path=/api/pending_closures4456worker2 # [ 92.051613] nix-grpc-daemon[854]: time=2026-09-23T07:00:31.682Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4457worker2 # [ 92.053721] nix-grpc-daemon[854]: time=2026-09-23T07:00:31.682Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4458lb # [ 92.149384] niks3-server[1015]: 2026/09/23 07:00:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4459worker2 # [ 92.086915] nix-grpc-daemon[854]: time=2026-09-23T07:00:31.718Z level=INFO msg="Uploading 1 narinfos"4460lb # [ 92.163870] niks3-server[1015]: 2026/09/23 07:00:31 INFO Signed narinfos id=18 count=14461lb # [ 92.172335] niks3-server[1015]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4462lb # [ 92.181320] niks3-server[1015]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4463lb # [ 92.195906] niks3-server[1015]: 2026/09/23 07:00:31 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4464lb # [ 92.202834] niks3-server[1015]: 2026/09/23 07:00:31 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4465worker2 # [ 92.129380] nix-grpc-daemon[854]: time=2026-09-23T07:00:31.761Z level=INFO msg="Upload complete. (262ms)"4466lb # [ 92.206783] niks3-server[1015]: 2026/09/23 07:00:31 INFO Completed upload id=184467worker2 # [ 92.139678] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=4244468worker2 # [ 92.153697] nix-grpc-daemon[770]: ts=2026-09-23T07:00:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244469worker2 # [ 92.165693] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4470worker2 # [ 92.172423] nix-daemon[842]: reaped child process 1050, status = succeeded4471worker2 # [ 92.184940] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4472worker2 # [ 98.002011] nix-grpc-daemon[770]: ts=2026-09-23T07:00:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04473worker2 # [ 98.042026] nix-daemon[842]: reaped child process 1057, status = succeeded4474worker2 # [ 98.047412] nix-daemon[842]: reaped child process 1059, status = succeeded4475lb # [ 98.264755] envoy[895]: conn peer=[2001:db8:1::1]:38276 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2984 tx=957 ms=66064476(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.51 seconds)4477subtest: the first client leaving does not fail the second4478client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4479client # [ 98.296982] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4480client # Running as unit: share1.service; invocation ID: f219f3c637ce498399b0ba4af614ee544481client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4482client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4483worker2 # [ 98.501816] nix-grpc-daemon[770]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:434244484client # [ 98.503401] nix[1022]: don't know how to build these paths:4485client # [ 98.504538] nix[1022]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4486worker1 # [ 98.536754] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04487worker1 # [ 98.537050] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14488worker1 # [ 98.549411] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864489worker1 # [ 98.565736] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864490client # [ 98.559800] nix[1022]: copying 1 paths to 'lb:50051'...4491client # [ 98.563925] nix[1022]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4492worker1 # [ 98.581547] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491864493worker1 # [ 98.596247] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4494worker1 # [ 98.750352] nix-daemon[852]: accepted connection from pid 1066, user nix-grpc-daemon (trusted)4495worker1 # [ 98.769215] nix-daemon[852]: reaped child process 1069, status = succeeded4496lb # [ 98.270115] envoy[895]: conn peer=[2001:db8:1::1]:40694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3002 tx=953 ms=62834497lb # [ 98.836785] niks3-server[1015]: 2026/09/23 07:00:38 INFO Received uploads request method=POST path=/api/pending_closures4498worker1 # [ 98.789979] nix-grpc-daemon[902]: time=2026-09-23T07:00:38.413Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4499worker1 # [ 98.791639] nix-grpc-daemon[902]: time=2026-09-23T07:00:38.413Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4500lb # [ 98.875496] niks3-server[1015]: 2026/09/23 07:00:38 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4501worker1 # [ 98.829363] nix-grpc-daemon[902]: time=2026-09-23T07:00:38.452Z level=INFO msg="Uploading 1 narinfos"4502lb # [ 98.890716] niks3-server[1015]: 2026/09/23 07:00:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4503lb # [ 98.901779] niks3-server[1015]: 2026/09/23 07:00:38 INFO Signed narinfos id=19 count=14504lb # [ 98.907579] niks3-server[1015]: 2026/09/23 07:00:38 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4505lb # [ 98.922543] niks3-server[1015]: 2026/09/23 07:00:38 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4506lb # [ 98.928488] niks3-server[1015]: 2026/09/23 07:00:38 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4507worker1 # [ 98.866573] nix-grpc-daemon[902]: time=2026-09-23T07:00:38.490Z level=INFO msg="Upload complete. (254ms)"4508lb # [ 98.932188] niks3-server[1015]: 2026/09/23 07:00:38 INFO Completed upload id=194509worker1 # [ 98.876987] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=4324510worker1 # [ 98.892971] nix-grpc-daemon[765]: ts=2026-09-23T07:00:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864511worker1 # [ 98.907832] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4512worker1 # [ 98.911684] nix-daemon[852]: reaped child process 1064, status = succeeded4513worker1 # [ 98.924422] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4514client # [ 98.905602] nix[1022]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4515client # [ 99.402744] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4516client # Running as unit: share2.service; invocation ID: 7bb2a0380c28405a9f2053d2b1716a6b4517client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4518worker2 # [ 99.598239] nix-grpc-daemon[770]: ts=2026-09-23T07:00:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:434244519client # [ 99.599790] nix[1038]: don't know how to build these paths:4520client # [ 99.601532] nix[1038]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4521worker1 # [ 99.630065] nix-grpc-daemon[765]: ts=2026-09-23T07:00:39Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04522client # [ 99.620505] nix[1038]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4523worker1 # [ 99.637461] nix-grpc-daemon[765]: ts=2026-09-23T07:00:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864524client: must succeed: sleep 2; systemctl kill -s INT share14525client # [ 102.777225] systemd[1]: share1.service: Sent signal SIGINT to main process 1022 (nix) on client request.4526client # [ 102.786497] nix[1022]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4527client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4528client: waiting for success: ! systemctl is-active share14529client # [ 102.800539] nix[1022]: error: interrupted by the user4530client # [ 102.864317] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4531client # [ 102.866264] systemd[1]: share1.service: Failed with result 'exit-code'.4532client # [ 102.869643] systemd[1]: share1.service: Consumed 170ms CPU time over 4.572s wall clock time, 43.5M memory peak, 6.2K incoming IP traffic, 8.2K outgoing IP traffic.4533client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4534worker1 # [ 104.158021] nix-daemon[852]: reaped child process 1073, status = succeeded4535client: waiting for success: ! systemctl is-active share24536worker1 # [ 104.245840] nix-daemon[852]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4537worker1 # [ 104.262228] nix-daemon[852]: reaped child process 1112, status = succeeded4538lb # [ 104.332733] niks3-server[1015]: 2026/09/23 07:00:43 INFO Received uploads request method=POST path=/api/pending_closures4539worker1 # [ 104.284367] nix-grpc-daemon[902]: time=2026-09-23T07:00:43.907Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4540worker1 # [ 104.285830] nix-grpc-daemon[902]: time=2026-09-23T07:00:43.907Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4541lb # [ 104.379663] niks3-server[1015]: 2026/09/23 07:00:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4542worker1 # [ 104.327523] nix-grpc-daemon[902]: time=2026-09-23T07:00:43.951Z level=INFO msg="Uploading 1 narinfos"4543lb # [ 104.393304] niks3-server[1015]: 2026/09/23 07:00:44 INFO Signed narinfos id=20 count=14544lb # [ 104.402837] niks3-server[1015]: 2026/09/23 07:00:44 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4545lb # [ 104.411080] niks3-server[1015]: 2026/09/23 07:00:44 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4546lb # [ 104.423085] niks3-server[1015]: 2026/09/23 07:00:44 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4547lb # [ 104.436468] niks3-server[1015]: 2026/09/23 07:00:44 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4548worker1 # [ 104.374537] nix-grpc-daemon[902]: time=2026-09-23T07:00:43.998Z level=INFO msg="Upload complete. (234ms)"4549lb # [ 104.445040] niks3-server[1015]: 2026/09/23 07:00:44 INFO Completed upload id=204550lb # [ 104.449318] niks3-server[1015]: 2026/09/23 07:00:44 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4551worker1 # [ 104.392247] nix-grpc-daemon[765]: ts=2026-09-23T07:00:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14552worker1 # [ 104.405772] nix-grpc-daemon[765]: ts=2026-09-23T07:00:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14553worker1 # [ 104.412697] nix-daemon[852]: reaped child process 1071, status = succeeded4554client # [ 104.404620] systemd[1]: share2.service: Deactivated successfully.4555client # [ 104.409617] systemd[1]: share2.service: Consumed 173ms CPU time over 5.003s wall clock time, 33.3M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4556client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4557client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4558client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4559(finished: subtest: the first client leaving does not fail the second, in 7.11 seconds)4560subtest: build hook: nix-daemon with builders = grpc://lb4561client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4562client # [ 105.475383] systemd[1]: Started Nix Daemon.4563client # [ 105.592622] nix-daemon[1090]: accepted connection from pid 1087, user root (trusted)4564worker1 # [ 105.752681] nix-grpc-daemon[765]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:491864565worker2 # [ 105.745671] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244566worker1 # [ 105.768277] nix-grpc-daemon[765]: ts=2026-09-23T07:00:45Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04567worker1 # [ 105.774354] nix-grpc-daemon[765]: ts=2026-09-23T07:00:45Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14568worker2 # [ 105.765093] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244569worker2 # [ 105.780304] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244570worker2 # [ 105.793787] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:434244571worker2 # [ 105.808894] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4572worker2 # [ 105.962747] nix-daemon[842]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)4573worker2 # [ 105.981824] nix-daemon[842]: reaped child process 1101, status = succeeded4574lb # [ 106.061159] niks3-server[1015]: 2026/09/23 07:00:45 INFO Received uploads request method=POST path=/api/pending_closures4575worker2 # [ 105.997889] nix-grpc-daemon[854]: time=2026-09-23T07:00:45.629Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4576worker2 # [ 105.999515] nix-grpc-daemon[854]: time=2026-09-23T07:00:45.629Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4577lb # [ 106.090051] niks3-server[1015]: 2026/09/23 07:00:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4578worker2 # [ 106.025717] nix-grpc-daemon[854]: time=2026-09-23T07:00:45.657Z level=INFO msg="Uploading 1 narinfos"4579lb # [ 106.104721] niks3-server[1015]: 2026/09/23 07:00:45 INFO Signed narinfos id=21 count=14580lb # [ 106.113128] niks3-server[1015]: 2026/09/23 07:00:45 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4581lb # [ 106.121460] niks3-server[1015]: 2026/09/23 07:00:45 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4582lb # [ 106.132733] niks3-server[1015]: 2026/09/23 07:00:45 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4583worker2 # [ 106.064308] nix-grpc-daemon[854]: time=2026-09-23T07:00:45.696Z level=INFO msg="Upload complete. (238ms)"4584lb # [ 106.139306] niks3-server[1015]: 2026/09/23 07:00:45 INFO Completed upload id=214585lb # [ 106.142364] niks3-server[1015]: 2026/09/23 07:00:45 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4586worker2 # [ 106.076858] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=5124587worker2 # [ 106.088815] nix-grpc-daemon[770]: ts=2026-09-23T07:00:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244588worker2 # [ 106.097749] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4589worker2 # [ 106.107488] nix-daemon[842]: reaped child process 1096, status = succeeded4590worker2 # [ 106.114057] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4591worker2 # [ 107.376788] nix-daemon[842]: reaped child process 1105, status = succeeded4592worker2 # [ 107.470879] nix-daemon[842]: accepted connection from pid 1121, user nix-grpc-daemon (trusted)4593worker2 # [ 107.489006] nix-daemon[842]: reaped child process 1124, status = succeeded4594lb # [ 107.568893] niks3-server[1015]: 2026/09/23 07:00:47 INFO Received uploads request method=POST path=/api/pending_closures4595worker2 # [ 107.505146] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.136Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4596worker2 # [ 107.506669] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.136Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4597lb # [ 107.607548] niks3-server[1015]: 2026/09/23 07:00:47 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4598lb # [ 107.624402] niks3-server[1015]: 2026/09/23 07:00:47 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4599worker2 # [ 107.559376] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.191Z level=INFO msg="Uploading 1 narinfos"4600lb # [ 107.636456] niks3-server[1015]: 2026/09/23 07:00:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4601lb # [ 107.639874] niks3-server[1015]: 2026/09/23 07:00:47 INFO Signed narinfos id=22 count=14602lb # [ 107.647999] niks3-server[1015]: 2026/09/23 07:00:47 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4603lb # [ 107.656164] niks3-server[1015]: 2026/09/23 07:00:47 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4604worker2 # [ 107.582677] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.214Z level=INFO msg="Upload complete. (257ms)"4605lb # [ 107.659795] niks3-server[1015]: 2026/09/23 07:00:47 INFO Completed upload id=224606lb # [ 107.662164] niks3-server[1015]: 2026/09/23 07:00:47 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4607worker2 # [ 107.596464] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14608worker2 # [ 107.611055] nix-daemon[842]: reaped child process 1103, status = succeeded4609worker2 # [ 107.642438] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:434244610worker1 # [ 107.690424] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:491864611worker1 # [ 107.704590] nix-grpc-daemon[765]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4612worker1 # [ 107.722354] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_out=1284613worker1 # [ 108.003637] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:491864614worker2 # [ 107.995949] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244615worker2 # [ 108.005380] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.637Z level=INFO msg="All 1 paths already cached"4616worker1 # [ 108.033357] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04617worker1 # [ 108.038999] nix-grpc-daemon[765]: ts=2026-09-23T07:00:47Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14618worker2 # [ 108.067235] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244619worker2 # [ 108.082618] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244620worker2 # [ 108.097146] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.729Z level=INFO msg="All 1 paths already cached"4621worker2 # [ 108.107036] nix-grpc-daemon[770]: ts=2026-09-23T07:00:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:434244622worker2 # [ 108.121005] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4623lb # [ 108.268792] envoy[895]: conn peer=[2001:db8:1::1]:40696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3029 tx=787 ms=43104624lb # [ 108.269127] envoy[895]: conn peer=[2001:db8:1::1]:40700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=47954625lb # [ 108.269248] envoy[895]: conn peer=[2001:db8:1::1]:48186 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3410 tx=1926 ms=20274626lb # [ 108.269347] envoy[895]: conn peer=[2001:db8:1::1]:48196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=334 ms=1004627worker2 # [ 108.280708] nix-daemon[842]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)4628worker2 # [ 108.302295] nix-daemon[842]: reaped child process 1131, status = succeeded4629lb # [ 108.373546] niks3-server[1015]: 2026/09/23 07:00:48 INFO Received uploads request method=POST path=/api/pending_closures4630worker2 # [ 108.309124] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.940Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4631worker2 # [ 108.310563] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.941Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4632worker2 # [ 108.340815] nix-grpc-daemon[854]: time=2026-09-23T07:00:47.972Z level=INFO msg="Uploading 1 narinfos"4633lb # [ 108.405106] niks3-server[1015]: 2026/09/23 07:00:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4634lb # [ 108.423841] niks3-server[1015]: 2026/09/23 07:00:48 INFO Signed narinfos id=23 count=14635lb # [ 108.431652] niks3-server[1015]: 2026/09/23 07:00:48 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4636lb # [ 108.442620] niks3-server[1015]: 2026/09/23 07:00:48 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4637worker2 # [ 108.379262] nix-grpc-daemon[854]: time=2026-09-23T07:00:48.011Z level=INFO msg="Upload complete. (241ms)"4638lb # [ 108.453946] niks3-server[1015]: 2026/09/23 07:00:48 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4639lb # [ 108.462282] niks3-server[1015]: 2026/09/23 07:00:48 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4640lb # [ 108.467272] niks3-server[1015]: 2026/09/23 07:00:48 INFO Completed upload id=234641worker2 # [ 108.388294] nix-grpc-daemon[770]: ts=2026-09-23T07:00:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=6244642worker2 # [ 108.401191] nix-grpc-daemon[770]: ts=2026-09-23T07:00:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244643worker2 # [ 108.415816] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4644worker2 # [ 108.424203] nix-daemon[842]: reaped child process 1126, status = succeeded4645worker2 # [ 108.433828] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4646worker2 # [ 110.139520] nix-daemon[842]: reaped child process 1135, status = succeeded4647worker2 # [ 110.214401] nix-daemon[842]: accepted connection from pid 1151, user nix-grpc-daemon (trusted)4648worker2 # [ 110.232963] nix-daemon[842]: reaped child process 1154, status = succeeded4649lb # [ 110.313007] niks3-server[1015]: 2026/09/23 07:00:49 INFO Received uploads request method=POST path=/api/pending_closures4650worker2 # [ 110.251944] nix-grpc-daemon[854]: time=2026-09-23T07:00:49.880Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4651worker2 # [ 110.253860] nix-grpc-daemon[854]: time=2026-09-23T07:00:49.880Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4652lb # [ 110.352471] niks3-server[1015]: 2026/09/23 07:00:49 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4653lb # [ 110.366607] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4654worker2 # [ 110.298279] nix-grpc-daemon[854]: time=2026-09-23T07:00:49.930Z level=INFO msg="Uploading 1 narinfos"4655lb # [ 110.376300] niks3-server[1015]: 2026/09/23 07:00:50 INFO Signed narinfos id=24 count=14656lb # [ 110.384090] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4657worker2 # [ 110.318681] nix-grpc-daemon[854]: time=2026-09-23T07:00:49.950Z level=INFO msg="Upload complete. (273ms)"4658lb # [ 110.393166] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4659lb # [ 110.400855] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4660lb # [ 110.405038] niks3-server[1015]: 2026/09/23 07:00:50 INFO Completed upload id=244661worker2 # [ 110.329143] nix-grpc-daemon[770]: ts=2026-09-23T07:00:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14662lb # [ 110.407255] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4663worker2 # [ 110.347859] nix-grpc-daemon[770]: ts=2026-09-23T07:00:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:434244664worker2 # [ 110.353346] nix-daemon[842]: reaped child process 1133, status = succeeded4665worker1 # [ 110.389246] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:491864666worker1 # [ 110.402353] nix-grpc-daemon[765]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4667worker1 # [ 110.419118] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_out=1764668worker2 # [ 110.686380] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:434244669worker1 # [ 110.701969] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864670worker1 # [ 110.712390] nix-grpc-daemon[902]: time=2026-09-23T07:00:50.335Z level=INFO msg="All 2 paths already cached"4671worker1 # [ 110.729681] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04672worker2 # [ 110.721353] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244673worker1 # [ 110.736145] nix-grpc-daemon[765]: ts=2026-09-23T07:00:50Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14674worker2 # [ 110.736649] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244675worker2 # [ 110.752617] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.384Z level=INFO msg="All 2 paths already cached"4676worker2 # [ 110.762316] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:434244677worker2 # [ 110.776128] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4678worker2 # [ 110.934616] nix-daemon[842]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)4679worker2 # [ 110.954279] nix-daemon[842]: reaped child process 1161, status = succeeded4680lb # [ 111.026728] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received uploads request method=POST path=/api/pending_closures4681worker2 # [ 110.962512] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.594Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4682worker2 # [ 110.964262] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.594Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4683lb # [ 111.056618] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4684worker2 # [ 110.990762] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.622Z level=INFO msg="Uploading 1 narinfos"4685lb # [ 111.071564] niks3-server[1015]: 2026/09/23 07:00:50 INFO Signed narinfos id=25 count=14686lb # [ 111.079748] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4687lb # [ 111.085787] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4688worker2 # [ 111.024036] nix-grpc-daemon[854]: time=2026-09-23T07:00:50.656Z level=INFO msg="Upload complete. (230ms)"4689lb # [ 111.098817] niks3-server[1015]: 2026/09/23 07:00:50 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4690lb # [ 111.103150] niks3-server[1015]: 2026/09/23 07:00:50 INFO Completed upload id=254691lb # [ 111.106304] niks3-server[1015]: 2026/09/23 07:00:50 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4692worker2 # [ 111.032288] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_in=7524693worker2 # [ 111.045175] nix-grpc-daemon[770]: ts=2026-09-23T07:00:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244694worker2 # [ 111.056459] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4695worker2 # [ 111.060506] nix-daemon[842]: reaped child process 1156, status = succeeded4696worker2 # [ 111.073830] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4697worker2 # [ 111.707425] nix-daemon[842]: reaped child process 1165, status = succeeded4698worker2 # [ 111.813716] nix-daemon[842]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)4699worker2 # [ 111.832179] nix-daemon[842]: reaped child process 1184, status = succeeded4700lb # [ 111.905707] niks3-server[1015]: 2026/09/23 07:00:51 INFO Received uploads request method=POST path=/api/pending_closures4701worker2 # [ 111.842636] nix-grpc-daemon[854]: time=2026-09-23T07:00:51.473Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4702worker2 # [ 111.844181] nix-grpc-daemon[854]: time=2026-09-23T07:00:51.473Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4703lb # [ 111.947224] niks3-server[1015]: 2026/09/23 07:00:51 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4704worker2 # [ 111.886834] nix-grpc-daemon[854]: time=2026-09-23T07:00:51.518Z level=INFO msg="Uploading 1 narinfos"4705lb # [ 111.963329] niks3-server[1015]: 2026/09/23 07:00:51 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4706lb # [ 111.973768] niks3-server[1015]: 2026/09/23 07:00:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4707lb # [ 111.980279] niks3-server[1015]: 2026/09/23 07:00:51 INFO Signed narinfos id=26 count=14708worker2 # [ 111.913772] nix-grpc-daemon[854]: time=2026-09-23T07:00:51.545Z level=INFO msg="Upload complete. (230ms)"4709lb # [ 111.988010] niks3-server[1015]: 2026/09/23 07:00:51 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4710lb # [ 111.991481] niks3-server[1015]: 2026/09/23 07:00:51 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4711lb # [ 111.996552] niks3-server[1015]: 2026/09/23 07:00:51 INFO Completed upload id=264712lb # [ 111.998889] niks3-server[1015]: 2026/09/23 07:00:51 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4713worker2 # [ 111.924982] nix-grpc-daemon[770]: ts=2026-09-23T07:00:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14714worker2 # [ 111.941226] nix-grpc-daemon[770]: ts=2026-09-23T07:00:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:434244715worker2 # [ 111.946767] nix-daemon[842]: reaped child process 1163, status = succeeded4716worker1 # [ 111.973055] nix-grpc-daemon[765]: ts=2026-09-23T07:00:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:491864717worker1 # [ 111.985418] nix-grpc-daemon[765]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4718worker1 # [ 112.002214] nix-grpc-daemon[765]: ts=2026-09-23T07:00:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_out=2324719client # [ 112.084389] nix-daemon[1090]: reaped child process 1092, status = succeeded4720client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.76 seconds)4721client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4722client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4723client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4724client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4725client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4726client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4727(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.88 seconds)4728subtest: an input only one worker has still reaches the builder4729client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4730client # this derivation will be built:4731client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4732client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4733client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)4734client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4735client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4736worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4737worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)4738worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4739worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4740client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24741client # [ 113.031207] nix-daemon[1090]: accepted connection from pid 1173, user root (trusted)4742client # this derivation will be built:4743client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4744client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4745worker1 # [ 113.195595] nix-grpc-daemon[765]: ts=2026-09-23T07:00:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:491864746worker1 # [ 113.203433] nix-grpc-daemon[765]: ts=2026-09-23T07:00:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864747worker1 # [ 113.365573] nix-daemon[852]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)4748worker1 # [ 113.383687] nix-daemon[852]: reaped child process 1131, status = succeeded4749lb # [ 113.454019] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received uploads request method=POST path=/api/pending_closures4750worker1 # [ 113.402722] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.025Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4751worker1 # [ 113.404564] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.025Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4752lb # [ 113.486666] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4753worker1 # [ 113.435593] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.059Z level=INFO msg="Uploading 1 narinfos"4754lb # [ 113.500672] niks3-server[1015]: 2026/09/23 07:00:53 INFO Signed narinfos id=27 count=14755lb # [ 113.509583] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4756worker1 # [ 113.463387] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.087Z level=INFO msg="Upload complete. (260ms)"4757lb # [ 113.525615] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4758lb # [ 113.535585] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4759lb # [ 113.541712] niks3-server[1015]: 2026/09/23 07:00:53 INFO Completed upload id=274760lb # [ 113.544693] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4761worker1 # [ 113.490060] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04762worker1 # [ 113.497316] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14763worker1 # [ 113.506821] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864764worker1 # [ 113.518387] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864765client # copying 2 paths to 'lb:50051'...4766client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4767worker1 # [ 113.536951] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491864768client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4769worker1 # [ 113.552491] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4770worker1 # [ 113.704120] nix-daemon[852]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)4771worker1 # [ 113.722513] nix-daemon[852]: reaped child process 1140, status = succeeded4772lb # [ 113.792179] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received uploads request method=POST path=/api/pending_closures4773lb # [ 113.802627] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received uploads request method=POST path=/api/pending_closures4774worker1 # [ 113.752100] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.373Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4775worker1 # [ 113.755615] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.373Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4776worker1 # [ 113.758336] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.373Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4777lb # [ 113.841276] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4778lb # [ 113.863488] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4779worker1 # [ 113.813222] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.436Z level=INFO msg="Uploading 3 narinfos"4780lb # [ 113.875173] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4781lb # [ 113.884816] niks3-server[1015]: 2026/09/23 07:00:53 INFO Signed narinfos id=28 count=14782lb # [ 113.889708] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4783lb # [ 113.895681] niks3-server[1015]: 2026/09/23 07:00:53 INFO Signed narinfos id=29 count=24784lb # [ 113.899903] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4785lb # [ 113.909343] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4786lb # [ 113.917737] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4787lb # [ 113.928512] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4788lb # [ 113.931088] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4789lb # [ 113.935913] niks3-server[1015]: 2026/09/23 07:00:53 INFO Completed upload id=284790lb # [ 113.938926] niks3-server[1015]: 2026/09/23 07:00:53 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4791lb # [ 113.942878] niks3-server[1015]: 2026/09/23 07:00:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4792lb # [ 113.945697] niks3-server[1015]: 2026/09/23 07:00:53 INFO Completed upload id=294793worker1 # [ 113.881185] nix-grpc-daemon[902]: time=2026-09-23T07:00:53.504Z level=INFO msg="Upload complete. (311ms)"4794worker1 # [ 113.891922] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=2 nar_bytes_in=9844795worker1 # [ 113.906938] nix-grpc-daemon[765]: ts=2026-09-23T07:00:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864796worker1 # [ 113.921557] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4797worker1 # [ 113.928575] nix-daemon[852]: reaped child process 1135, status = succeeded4798client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4799client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4800worker1 # [ 113.947896] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4801client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4802worker1 # [ 115.175305] nix-daemon[852]: reaped child process 1144, status = succeeded4803worker1 # [ 115.272111] nix-daemon[852]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)4804worker1 # [ 115.286852] nix-daemon[852]: reaped child process 1163, status = succeeded4805lb # [ 115.359321] niks3-server[1015]: 2026/09/23 07:00:54 INFO Received uploads request method=POST path=/api/pending_closures4806worker1 # [ 115.313605] nix-grpc-daemon[902]: time=2026-09-23T07:00:54.935Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4807lb # [ 115.400802] niks3-server[1015]: 2026/09/23 07:00:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign4808worker1 # [ 115.348582] nix-grpc-daemon[902]: time=2026-09-23T07:00:54.972Z level=INFO msg="Uploading 1 narinfos"4809lb # [ 115.411987] niks3-server[1015]: 2026/09/23 07:00:55 INFO Signed narinfos id=30 count=14810lb # [ 115.418720] niks3-server[1015]: 2026/09/23 07:00:55 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4811lb # [ 115.431087] niks3-server[1015]: 2026/09/23 07:00:55 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4812lb # [ 115.441183] niks3-server[1015]: 2026/09/23 07:00:55 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete4813worker1 # [ 115.388543] nix-grpc-daemon[902]: time=2026-09-23T07:00:55.012Z level=INFO msg="Upload complete. (227ms)"4814lb # [ 115.450421] niks3-server[1015]: 2026/09/23 07:00:55 INFO Completed upload id=304815lb # [ 115.454681] niks3-server[1015]: 2026/09/23 07:00:55 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4816client # copying 1 paths...4817worker1 # [ 115.401368] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=14818worker1 # [ 115.419295] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:491864819worker1 # [ 115.424966] nix-daemon[852]: reaped child process 1142, status = succeeded4820client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4821worker2 # [ 115.443834] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:434244822worker2 # [ 115.455858] nix-grpc-daemon[770]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4823worker2 # [ 115.471145] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=1204824client # [ 115.593429] nix-daemon[1090]: reaped child process 1176, status = succeeded4825client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.76 seconds)4826client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24827client # [ 115.766516] nix-daemon[1090]: accepted connection from pid 1195, user root (trusted)4828client # this derivation will be built:4829client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4830worker1 # [ 115.931229] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:491864831client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4832worker1 # [ 115.938821] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864833worker1 # [ 115.946056] nix-grpc-daemon[902]: time=2026-09-23T07:00:55.568Z level=INFO msg="All 1 paths already cached"4834worker1 # [ 115.957158] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04835worker1 # [ 115.963266] nix-grpc-daemon[765]: ts=2026-09-23T07:00:55Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14836worker2 # [ 115.990414] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244837worker2 # [ 116.005797] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:434244838client # copying 1 paths to 'lb:50051'...4839client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4840worker2 # [ 116.027686] nix-grpc-daemon[770]: ts=2026-09-23T07:00:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:434244841worker2 # [ 116.043502] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4842worker2 # [ 116.067364] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 16 ms (attempt 1/5)4843worker2 # [ 116.091958] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5)4844worker2 # [ 116.126765] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 315 ms (attempt 3/5)4845worker2 # [ 116.444913] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 596 ms (attempt 4/5)4846worker2 # [ 117.043256] nix-grpc-daemon[770]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4847worker2 # [ 117.052784] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4848worker2 # [ 117.060260] nix-grpc-daemon[770]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4849worker2 # [ 117.226983] nix-daemon[842]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)4850worker2 # [ 117.250480] nix-daemon[842]: reaped child process 1200, status = succeeded4851lb # [ 117.322801] niks3-server[1015]: 2026/09/23 07:00:56 INFO Received uploads request method=POST path=/api/pending_closures4852worker2 # [ 117.259332] nix-grpc-daemon[854]: time=2026-09-23T07:00:56.890Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4853worker2 # [ 117.261542] nix-grpc-daemon[854]: time=2026-09-23T07:00:56.890Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4854worker2 # [ 117.288369] nix-grpc-daemon[854]: time=2026-09-23T07:00:56.920Z level=INFO msg="Uploading 1 narinfos"4855lb # [ 117.352529] niks3-server[1015]: 2026/09/23 07:00:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign4856lb # [ 117.368368] niks3-server[1015]: 2026/09/23 07:00:56 INFO Signed narinfos id=31 count=14857lb # [ 117.371613] niks3-server[1015]: 2026/09/23 07:00:57 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4858lb # [ 117.390171] niks3-server[1015]: 2026/09/23 07:00:57 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4859worker2 # [ 117.323831] nix-grpc-daemon[854]: time=2026-09-23T07:00:56.955Z level=INFO msg="Upload complete. (249ms)"4860lb # [ 117.398327] niks3-server[1015]: 2026/09/23 07:00:57 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete4861lb # [ 117.407714] niks3-server[1015]: 2026/09/23 07:00:57 INFO Completed upload id=314862lb # [ 117.411215] niks3-server[1015]: 2026/09/23 07:00:57 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4863worker2 # [ 117.332315] nix-grpc-daemon[770]: ts=2026-09-23T07:00:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 paths=1 nar_bytes_in=5684864worker2 # [ 117.343532] nix-grpc-daemon[770]: ts=2026-09-23T07:00:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:434244865worker2 # [ 117.356308] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4866worker2 # [ 117.361654] nix-daemon[842]: reaped child process 1190, status = succeeded4867worker2 # [ 117.376991] nix-grpc-daemon[770]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4868client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4869worker2 # [ 117.393842] nix-daemon[842]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4870client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4871client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4872lb # [ 118.271309] envoy[895]: conn peer=[2001:db8:1::1]:48200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3887 tx=2182 ms=24754873lb # [ 118.276391] envoy[895]: conn peer=[2001:db8:1::1]:48210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=376 ms=954874lb # [ 118.281578] envoy[895]: conn peer=[2001:db8:1::1]:48220 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4322 tx=2374 ms=13214875lb # [ 118.287219] envoy[895]: conn peer=[2001:db8:1::1]:42596 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=407 ms=524876lb # [ 118.292090] envoy[895]: conn peer=[2001:db8:1::1]:42602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3872 tx=2067 ms=23484877lb # [ 118.297757] envoy[895]: conn peer=[2001:db8:1::1]:42616 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=964878worker2 # [ 118.484575] nix-daemon[842]: reaped child process 1205, status = succeeded4879worker2 # [ 118.585628] nix-daemon[842]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)4880worker2 # [ 118.601685] nix-daemon[842]: reaped child process 1224, status = succeeded4881lb # [ 118.683100] niks3-server[1015]: 2026/09/23 07:00:58 INFO Received uploads request method=POST path=/api/pending_closures4882worker2 # [ 118.619693] nix-grpc-daemon[854]: time=2026-09-23T07:00:58.250Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4883lb # [ 118.715713] niks3-server[1015]: 2026/09/23 07:00:58 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4884worker2 # [ 118.651185] nix-grpc-daemon[854]: time=2026-09-23T07:00:58.283Z level=INFO msg="Uploading 1 narinfos"4885lb # [ 118.729011] niks3-server[1015]: 2026/09/23 07:00:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign4886lb # [ 118.739626] niks3-server[1015]: 2026/09/23 07:00:58 INFO Signed narinfos id=32 count=14887lb # [ 118.743988] niks3-server[1015]: 2026/09/23 07:00:58 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4888worker2 # [ 118.678049] nix-grpc-daemon[854]: time=2026-09-23T07:00:58.310Z level=INFO msg="Upload complete. (269ms)"4889lb # [ 118.754570] niks3-server[1015]: 2026/09/23 07:00:58 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete4890lb # [ 118.761539] niks3-server[1015]: 2026/09/23 07:00:58 INFO Completed upload id=324891lb # [ 118.764825] niks3-server[1015]: 2026/09/23 07:00:58 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4892worker2 # [ 118.689103] nix-grpc-daemon[770]: ts=2026-09-23T07:00:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=14893client # copying 1 paths...4894worker1 # [ 118.714284] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:491864895worker2 # [ 118.707692] nix-daemon[842]: reaped child process 1202, status = succeeded4896worker1 # [ 118.726786] nix-grpc-daemon[765]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4897client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4898worker2 # [ 118.758289] nix-grpc-daemon[770]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:434244899worker2 # [ 118.765460] nix-grpc-daemon[770]: ts=2026-09-23T07:00:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:43424 duration_s=0 paths=1 nar_bytes_out=1204900client # [ 118.842546] nix-daemon[1090]: reaped child process 1198, status = succeeded4901client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.25 seconds)4902client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24903client # [ 119.009413] nix-daemon[1090]: accepted connection from pid 1217, user root (trusted)4904client # this derivation will be built:4905client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4906worker2 # [ 119.158426] nix-grpc-daemon[770]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:434244907client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4908worker1 # [ 119.173892] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864909worker1 # [ 119.185465] nix-grpc-daemon[902]: time=2026-09-23T07:00:58.808Z level=INFO msg="All 1 paths already cached"4910worker1 # [ 119.202331] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04911worker1 # [ 119.211660] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14912worker1 # [ 119.220617] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864913worker1 # [ 119.227920] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:491864914worker1 # [ 119.238202] nix-grpc-daemon[902]: time=2026-09-23T07:00:58.861Z level=INFO msg="All 1 paths already cached"4915client # copying 1 paths to 'lb:50051'...4916client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4917worker1 # [ 119.248357] nix-grpc-daemon[765]: ts=2026-09-23T07:00:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:491864918worker1 # [ 119.265787] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4919worker1 # [ 119.418360] nix-daemon[852]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)4920worker1 # [ 119.438262] nix-daemon[852]: reaped child process 1175, status = succeeded4921lb # [ 119.505257] niks3-server[1015]: 2026/09/23 07:00:59 INFO Received uploads request method=POST path=/api/pending_closures4922worker1 # [ 119.453138] nix-grpc-daemon[902]: time=2026-09-23T07:00:59.076Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4923worker1 # [ 119.455096] nix-grpc-daemon[902]: time=2026-09-23T07:00:59.078Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4924lb # [ 119.537981] niks3-server[1015]: 2026/09/23 07:00:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign4925worker1 # [ 119.483275] nix-grpc-daemon[902]: time=2026-09-23T07:00:59.106Z level=INFO msg="Uploading 1 narinfos"4926lb # [ 119.553621] niks3-server[1015]: 2026/09/23 07:00:59 INFO Signed narinfos id=33 count=14927lb # [ 119.559998] niks3-server[1015]: 2026/09/23 07:00:59 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4928lb # [ 119.571915] niks3-server[1015]: 2026/09/23 07:00:59 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4929worker1 # [ 119.515942] nix-grpc-daemon[902]: time=2026-09-23T07:00:59.139Z level=INFO msg="Upload complete. (236ms)"4930lb # [ 119.578361] niks3-server[1015]: 2026/09/23 07:00:59 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete4931worker1 # [ 119.524138] nix-grpc-daemon[765]: ts=2026-09-23T07:00:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_in=5684932lb # [ 119.589106] niks3-server[1015]: 2026/09/23 07:00:59 INFO Completed upload id=334933lb # [ 119.592250] niks3-server[1015]: 2026/09/23 07:00:59 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4934worker1 # [ 119.540217] nix-grpc-daemon[765]: ts=2026-09-23T07:00:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:491864935worker1 # [ 119.547864] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4936worker1 # [ 119.553518] nix-daemon[852]: reaped child process 1170, status = succeeded4937client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4938worker1 # [ 119.567455] nix-daemon[852]: accepted connection from pid 765, user nix-grpc-daemon (trusted)4939client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4940client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4941worker1 # [ 120.860529] nix-daemon[852]: reaped child process 1179, status = succeeded4942worker1 # [ 120.944349] nix-daemon[852]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)4943worker1 # [ 120.961160] nix-daemon[852]: reaped child process 1198, status = succeeded4944lb # [ 121.029129] niks3-server[1015]: 2026/09/23 07:01:00 INFO Received uploads request method=POST path=/api/pending_closures4945worker1 # [ 120.978482] nix-grpc-daemon[902]: time=2026-09-23T07:01:00.600Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4946lb # [ 121.062860] niks3-server[1015]: 2026/09/23 07:01:00 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4947worker1 # [ 121.011246] nix-grpc-daemon[902]: time=2026-09-23T07:01:00.634Z level=INFO msg="Uploading 1 narinfos"4948lb # [ 121.074792] niks3-server[1015]: 2026/09/23 07:01:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign4949lb # [ 121.086420] niks3-server[1015]: 2026/09/23 07:01:00 INFO Signed narinfos id=34 count=14950lb # [ 121.091628] niks3-server[1015]: 2026/09/23 07:01:00 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4951lb # [ 121.102115] niks3-server[1015]: 2026/09/23 07:01:00 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete4952lb # [ 121.107263] niks3-server[1015]: 2026/09/23 07:01:00 INFO Completed upload id=344953worker1 # [ 121.045643] nix-grpc-daemon[902]: time=2026-09-23T07:01:00.669Z level=INFO msg="Upload complete. (267ms)"4954lb # [ 121.110367] niks3-server[1015]: 2026/09/23 07:01:00 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4955client # copying 1 paths...4956worker1 # [ 121.057679] nix-grpc-daemon[765]: ts=2026-09-23T07:01:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=14957worker2 # [ 121.054699] nix-grpc-daemon[770]: ts=2026-09-23T07:01:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:434244958worker1 # [ 121.076508] nix-daemon[852]: reaped child process 1177, status = succeeded4959client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4960worker1 # [ 121.112290] nix-grpc-daemon[765]: ts=2026-09-23T07:01:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:491864961worker1 # [ 121.119476] nix-grpc-daemon[765]: ts=2026-09-23T07:01:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49186 duration_s=0 paths=1 nar_bytes_out=1204962client # [ 121.182957] nix-daemon[1090]: reaped child process 1220, status = succeeded4963client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.34 seconds)4964(finished: subtest: an input only one worker has still reaches the builder, in 8.97 seconds)4965subtest: upload whose reference the worker lacks is completed from the cache4966client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4967client # this derivation will be built:4968client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4969client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4970client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)4971client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4972client # this derivation will be built:4973client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4974worker2 # [ 121.062616] nix-grpc-daemon[770]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4975client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4976client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)4977client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4978client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds)4979worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4980worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.18 seconds)4981worker2 # [ 122.176827] systemd[1]: Stopping Nix worker-protocol over gRPC...4982worker2 # [ 123.056612] nix-grpc-daemon[770]: ts=2026-09-23T07:01:02Z level=info event=signal_exit4983worker1 # [ 125.061815] nix-grpc-daemon[765]: ts=2026-09-23T07:01:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:534324984worker2 # [ 125.051303] nix-grpc-daemon[770]: ts=2026-09-23T07:01:04Z level=info event=scheduler_restarting4985worker1 # [ 125.069745] nix-grpc-daemon[765]: ts=2026-09-23T07:01:04Z level=info event=worker_gone addr=192.168.1.4:500514986worker2 # [ 125.080825] nix-grpc-daemon[770]: ts=2026-09-23T07:01:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4987worker2 # [ 125.108215] nix-daemon[842]: reaped child process 844, status = succeeded4988worker2 # [ 125.227365] nix-grpc-daemon[770]: E0000 00:00:1790146864.859355 816 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true4989worker2 # [ 126.802649] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4990worker2 # [ 126.807506] systemd[1]: Stopped Nix worker-protocol over gRPC.4991worker2 # [ 126.813420] systemd[1]: nix-grpc-daemon.service: Consumed 3.609s CPU time over 1min 47.956s wall clock time, 66M memory peak, 93.7K incoming IP traffic, 179.8K outgoing IP traffic.4992worker2 # [ 126.829245] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.4993worker2 # [ 126.835356] systemd[1]: Closed Nix worker-protocol over gRPC.4994worker2 # [ 126.838009] systemd[1]: nix-grpc-daemon.socket: Consumed 623us CPU time over 1min 56.348s wall clock time, 256K memory peak, 104.5K incoming IP traffic, 92.4K outgoing IP traffic.4995lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux4996lb: must succeed: curl -sf localhost:9901/clusters4997lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4998lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.07 seconds)4999client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25000worker1 # [ 127.149502] nix-grpc-daemon[765]: ts=2026-09-23T07:01:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:491865001worker1 # [ 127.156219] nix-grpc-daemon[765]: ts=2026-09-23T07:01:06Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:491865002client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5003client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5004worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5005worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5006worker2 # [ 127.265963] systemd[1]: Listening on Nix worker-protocol over gRPC.5007worker1 # [ 127.351960] systemd[1]: Stopping Nix worker-protocol over gRPC...5008worker1 # [ 127.466193] nix-grpc-daemon[765]: ts=2026-09-23T07:01:07Z level=info event=signal_exit5009worker2 # [ 127.800079] systemd[1]: Starting scheduler-token.service...5010worker2 # [ 127.891023] systemd[1]: scheduler-token.service: Deactivated successfully.5011worker2 # [ 127.901387] systemd[1]: Finished scheduler-token.service.5012worker2 # [ 127.903478] systemd[1]: scheduler-token.service: Consumed 57ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5013worker2 # [ 127.911629] systemd[1]: Starting Nix worker-protocol over gRPC...5014lb # [ 128.121323] niks3-server[1015]: 2026/09/23 07:01:07 INFO lead: released remote=192.168.1.3:591825015worker2 # [ 128.052471] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5016worker2 # [ 128.054855] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5017worker2 # [ 128.058791] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15018worker2 # [ 128.064057] nix-grpc-daemon[1249]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5019worker2 # [ 128.065519] nix-grpc-daemon[1249]: E0000 00:00:1790146867.695715 1249 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5020lb # [ 128.153116] niks3-server[1015]: 2026/09/23 07:01:07 INFO lead: acquired remote=192.168.1.4:331005021worker2 # [ 128.104053] nix-grpc-daemon[1249]: E0000 00:00:1790146867.701978 1249 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5022worker2 # [ 128.106405] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5023worker2 # [ 128.115956] nix-grpc-daemon[1249]: E0000 00:00:1790146867.711310 1292 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5024worker2 # [ 128.119012] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_take_over5025worker2 # [ 128.120260] nix-grpc-daemon[1249]: E0000 00:00:1790146867.728461 1292 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5026worker2 # [ 128.122528] nix-grpc-daemon[1249]: E0000 00:00:1790146867.728599 1292 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5027worker2 # [ 128.127787] systemd[1]: Started Nix worker-protocol over gRPC.5028worker2 # [ 128.130815] nix-grpc-daemon[1249]: E0000 00:00:1790146867.747477 1292 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5029worker2 # [ 128.133340] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_connected addr=lb:500515030worker2 # [ 128.134599] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5031lb # [ 128.275719] envoy[895]: conn peer=[2001:db8:1::1]:42618 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3726 tx=2151 ms=28615032lb # [ 128.280880] envoy[895]: conn peer=[2001:db8:1::1]:42630 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=285033lb # [ 128.285634] envoy[895]: conn peer=[2001:db8:1::1]:42636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2123 ms=19645034lb # [ 128.290475] envoy[895]: conn peer=[2001:db8:1::1]:43648 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=285035lb # [ 128.295133] envoy[895]: conn peer=[2001:db8:1::4]:52508 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=19063 tx=1863 ms=913165036lb # [ 128.300153] envoy[895]: conn peer=[2001:db8:1::1]:43652 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=523 ms=295037worker2 # [ 128.332394] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_connected addr=lb:500515038worker2 # [ 128.337444] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5039worker2 # [ 128.538404] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_connected addr=lb:500515040worker2 # [ 128.543600] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5041worker2 # [ 128.744342] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_connected addr=lb:500515042worker2 # [ 128.749328] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5043worker2 # [ 128.950211] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_connected addr=lb:500515044worker2 # [ 128.955289] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045worker1 # [ 129.062341] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=scheduler_restarting5046lb # [ 129.121029] envoy[895]: [2026-09-23 07:01:08.756][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5047worker1 # [ 129.062727] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=scheduler_restarting addr=lb:500515048worker1 # [ 129.071828] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:534325049worker1 # [ 129.081629] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=worker_gone addr=192.168.1.3:500515050worker1 # [ 129.112426] nix-daemon[852]: reaped child process 854, status = succeeded5051worker2 # [ 129.129503] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:516385052worker2 # [ 129.156073] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=scheduler_connected addr=lb:500515053worker2 # [ 129.172138] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=oidc_jwks provider=mock keys=15054worker2 # [ 129.176001] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:516445055worker2 # [ 129.192061] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:08Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05056worker1 # [ 129.287712] nix-grpc-daemon[765]: ts=2026-09-23T07:01:08Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5057worker1 # [ 129.656475] nix-grpc-daemon[765]: E0000 00:00:1790146869.280063 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true5058worker1 # [ 130.759002] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5059worker1 # [ 130.763881] systemd[1]: Stopped Nix worker-protocol over gRPC.5060worker1 # [ 130.772301] systemd[1]: nix-grpc-daemon.service: Consumed 2.894s CPU time over 1min 52.563s wall clock time, 66.9M memory peak, 82.4K incoming IP traffic, 161.4K outgoing IP traffic.5061worker1 # [ 130.784634] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5062worker1 # [ 130.786760] systemd[1]: Closed Nix worker-protocol over gRPC.5063worker1 # [ 130.793419] systemd[1]: nix-grpc-daemon.socket: Consumed 695us CPU time over 2min 253ms wall clock time, 256K memory peak, 294.5K incoming IP traffic, 102.5K outgoing IP traffic.5064lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5065lb: must succeed: curl -sf localhost:9901/clusters5066lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5067lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.08 seconds)5068client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25069worker2 # [ 131.082135] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404705070worker2 # [ 131.095918] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5071worker2 # [ 131.129547] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 6 ms (attempt 1/5)5072worker2 # [ 131.143311] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 151 ms (attempt 2/5)5073worker2 # [ 131.301958] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 3/5)5074worker2 # [ 131.395975] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 437 ms (attempt 4/5)5075worker2 # [ 131.835288] nix-grpc-daemon[1249]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5076worker2 # [ 131.841946] nix-grpc-daemon[1249]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5077client # copying 2 paths...5078client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5079worker2 # [ 131.864657] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404705080client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5081worker2 # [ 131.881809] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5082worker2 # [ 131.910933] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:11Z level=info event=niks3_push_started pid=13105083worker2 # [ 132.057592] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.688Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5084worker2 # [ 132.166681] nix-daemon[842]: accepted connection from pid 1323, user nix-grpc-daemon (trusted)5085worker2 # [ 132.186268] nix-daemon[842]: reaped child process 1326, status = succeeded5086lb # [ 132.259209] niks3-server[1015]: 2026/09/23 07:01:11 INFO Received uploads request method=POST path=/api/pending_closures5087lb # [ 132.268367] niks3-server[1015]: 2026/09/23 07:01:11 INFO Received uploads request method=POST path=/api/pending_closures5088worker2 # [ 132.207157] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.837Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5089worker2 # [ 132.210747] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.837Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5090worker2 # [ 132.213133] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.837Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5091lb # [ 132.329091] niks3-server[1015]: 2026/09/23 07:01:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5092worker2 # [ 132.269512] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.899Z level=INFO msg="Uploading 3 narinfos"5093lb # [ 132.344320] niks3-server[1015]: 2026/09/23 07:01:11 INFO Signed narinfos id=35 count=15094lb # [ 132.346443] niks3-server[1015]: 2026/09/23 07:01:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5095lb # [ 132.362278] niks3-server[1015]: 2026/09/23 07:01:11 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5096lb # [ 132.372707] niks3-server[1015]: 2026/09/23 07:01:11 INFO Signed narinfos id=36 count=25097lb # [ 132.374633] niks3-server[1015]: 2026/09/23 07:01:11 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5098lb # [ 132.388767] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5099lb # [ 132.397222] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5100lb # [ 132.403464] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5101lb # [ 132.412802] niks3-server[1015]: 2026/09/23 07:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5102lb # [ 132.417454] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5103lb # [ 132.421078] niks3-server[1015]: 2026/09/23 07:01:12 INFO Completed upload id=355104lb # [ 132.423636] niks3-server[1015]: 2026/09/23 07:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5105lb # [ 132.427637] niks3-server[1015]: 2026/09/23 07:01:12 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5106lb # [ 132.430509] niks3-server[1015]: 2026/09/23 07:01:12 INFO Completed upload id=365107worker2 # [ 132.357139] nix-grpc-daemon[1310]: time=2026-09-23T07:01:11.989Z level=INFO msg="Upload complete. (296ms)"5108worker2 # [ 132.369109] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=0 paths=2 nar_bytes_in=3125109worker2 # [ 132.386312] nix-daemon[842]: reaped child process 1308, status = succeeded5110client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.54 seconds)5111worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5112worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5113worker1 # [ 132.519104] systemd[1]: Listening on Nix worker-protocol over gRPC.5114worker1 # [ 132.530246] systemd[1]: Starting Nix worker-protocol over gRPC...5115worker1 # [ 132.682167] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5116worker1 # [ 132.684502] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5117worker1 # [ 132.688447] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15118worker1 # [ 132.695332] nix-grpc-daemon[1221]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5119worker1 # [ 132.705538] nix-grpc-daemon[1221]: E0000 00:00:1790146872.316902 1221 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5120lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5121lb: must succeed: curl -sf localhost:9901/clusters5122worker1 # [ 132.736176] nix-grpc-daemon[1221]: E0000 00:00:1790146872.319018 1221 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5123worker1 # [ 132.738489] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5124worker1 # [ 132.744796] nix-grpc-daemon[1221]: E0000 00:00:1790146872.329786 1261 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5125worker1 # [ 132.747675] nix-grpc-daemon[1221]: E0000 00:00:1790146872.348980 1261 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5126worker1 # [ 132.750324] nix-grpc-daemon[1221]: E0000 00:00:1790146872.349155 1261 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5127worker1 # [ 132.752737] systemd[1]: Started Nix worker-protocol over gRPC.5128worker1 # [ 132.753546] nix-grpc-daemon[1221]: E0000 00:00:1790146872.367722 1261 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5129worker1 # [ 132.764422] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=scheduler_connected addr=lb:500515130worker2 # [ 132.757130] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:12Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:516445131worker2 # [ 132.763965] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:12Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05132lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5133lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.09 seconds)5134(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.60 seconds)5135subtest: a niks3 restart does not cost the leader its role5136lb: must succeed: curl -sf localhost:9901/clusters5137lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5138lb: must succeed: curl -sf localhost:9901/clusters5139lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5140lb # [ 132.980544] systemd[1]: Stopping niks3 server...5141lb # [ 132.981701] niks3-server[1015]: 2026/09/23 07:01:12 INFO Shutdown signal received, draining in-flight requests timeout=10s5142worker1 # [ 132.918945] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:12Z level=info event=scheduler_lock_lost status=2005143worker2 # [ 132.916140] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:12Z level=info event=scheduler_lock_lost status=2005144lb # [ 133.000337] niks3-server[1015]: 2026/09/23 07:01:12 INFO lead: released remote=192.168.1.4:331005145lb # [ 133.009615] systemd[1]: niks3.service: Deactivated successfully.5146lb # [ 133.012098] systemd[1]: Stopped niks3 server.5147lb # [ 133.012838] systemd[1]: niks3.service: Consumed 628ms CPU time over 1min 44.122s wall clock time, 27.6M memory peak, 98.8K incoming IP traffic, 142.3K outgoing IP traffic.5148lb # [ 133.022589] systemd[1]: Starting niks3 server...5149lb # [ 133.079595] niks3-server[1210]: 2026/09/23 07:01:12 goose: no migrations to run. current version: 202609200000005150lb # [ 133.086288] niks3-server[1210]: 2026/09/23 07:01:12 OK 1_commit_pending_closure.sql (4.8ms)5151lb # [ 133.090987] niks3-server[1210]: 2026/09/23 07:01:12 OK 2_object_stats_trigger.sql (4.31ms)5152lb # [ 133.092978] niks3-server[1210]: 2026/09/23 07:01:12 goose: up to current file version: 25153lb # [ 133.094277] niks3-server[1210]: 2026/09/23 07:01:12 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5154lb # [ 133.104854] niks3-server[1210]: 2026/09/23 07:01:12 INFO Read proxy enabled — serving cache objects from S35155lb # [ 133.109851] niks3-server[1210]: 2026/09/23 07:01:12 INFO Using socket-activated listener address=0.0.0.0:57515156lb # [ 133.111417] niks3-server[1210]: 2026/09/23 07:01:12 INFO systemd watchdog enabled interval=15s5157lb # [ 133.113563] niks3-server[1210]: 2026/09/23 07:01:12 INFO Starting HTTP server address=0.0.0.0:57515158lb: waiting for unit niks3.service5159lb # [ 133.115493] systemd[1]: Started niks3 server.5160lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5161lb: must succeed: curl -sf localhost:9901/clusters5162lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5163worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5164worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5165lb: must succeed: curl -sf localhost:9901/clusters5166lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5167lb: must succeed: curl -sf localhost:9901/clusters5168lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5169worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5170worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5171client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25172client # don't know how to build these paths:5173client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5174worker2 # [ 133.505664] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:404705175worker1 # [ 133.525433] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531745176worker1 # [ 133.548233] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5177worker1 # [ 133.566542] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 1/5)5178worker1 # [ 133.667509] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 124 ms (attempt 2/5)5179worker1 # [ 133.794510] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 3/5)5180lb # [ 133.994981] niks3-server[1210]: 2026/09/23 07:01:13 INFO lead: acquired remote=192.168.1.4:511085181worker1 # [ 133.931148] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 263 ms (attempt 4/5)5182worker1 # [ 134.204352] nix-grpc-daemon[1221]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5183worker1 # [ 134.211991] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5184worker2 # [ 134.248496] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05185worker2 # [ 134.255242] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15186worker2 # [ 134.303593] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404705187worker2 # [ 134.321226] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404705188client # copying 1 paths to 'lb:50051'...5189client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5190worker2 # [ 134.335523] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404705191worker2 # [ 134.350512] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5192worker2 # [ 134.507003] nix-daemon[842]: accepted connection from pid 1345, user nix-grpc-daemon (trusted)5193worker2 # [ 134.526565] nix-daemon[842]: reaped child process 1348, status = succeeded5194lb # [ 134.605472] niks3-server[1210]: 2026/09/23 07:01:14 INFO Received uploads request method=POST path=/api/pending_closures5195worker2 # [ 134.556229] nix-grpc-daemon[1310]: time=2026-09-23T07:01:14.188Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5196worker2 # [ 134.559414] nix-grpc-daemon[1310]: time=2026-09-23T07:01:14.190Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5197lb # [ 134.658835] niks3-server[1210]: 2026/09/23 07:01:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5198lb # [ 134.682022] niks3-server[1210]: 2026/09/23 07:01:14 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5199lb # [ 134.700613] niks3-server[1210]: 2026/09/23 07:01:14 INFO Signed narinfos id=37 count=15200worker2 # [ 134.631906] nix-grpc-daemon[1310]: time=2026-09-23T07:01:14.262Z level=INFO msg="Uploading 1 narinfos"5201lb # [ 134.712622] niks3-server[1210]: 2026/09/23 07:01:14 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5202lb # [ 134.720399] niks3-server[1210]: 2026/09/23 07:01:14 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5203lb # [ 134.727693] niks3-server[1210]: 2026/09/23 07:01:14 INFO Completed upload id=375204worker2 # [ 134.653051] nix-grpc-daemon[1310]: time=2026-09-23T07:01:14.285Z level=INFO msg="Upload complete. (285ms)"5205lb # [ 134.734607] niks3-server[1210]: 2026/09/23 07:01:14 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5206worker2 # [ 134.665079] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=0 paths=1 nar_bytes_in=5445207worker2 # [ 134.686724] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404705208worker2 # [ 134.691236] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5209worker2 # [ 134.700305] nix-daemon[842]: reaped child process 1343, status = succeeded5210client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5211worker2 # [ 134.709449] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5212client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5213client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5214client # farm-a> LOG-farm-a-after-niks3-restart5215worker2 # [ 135.796573] nix-daemon[842]: reaped child process 1352, status = succeeded5216worker2 # [ 135.886158] nix-daemon[842]: accepted connection from pid 1368, user nix-grpc-daemon (trusted)5217worker2 # [ 135.907764] nix-daemon[842]: reaped child process 1371, status = succeeded5218lb # [ 135.981426] niks3-server[1210]: 2026/09/23 07:01:15 INFO Received uploads request method=POST path=/api/pending_closures5219worker2 # [ 135.923629] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.553Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5220worker2 # [ 135.925693] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.553Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5221lb # [ 136.037838] niks3-server[1210]: 2026/09/23 07:01:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5222worker2 # [ 135.974062] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.606Z level=INFO msg="Uploading 1 narinfos"5223lb # [ 136.050022] niks3-server[1210]: 2026/09/23 07:01:15 INFO Signed narinfos id=38 count=15224lb # [ 136.056397] niks3-server[1210]: 2026/09/23 07:01:15 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5225lb # [ 136.067554] niks3-server[1210]: 2026/09/23 07:01:15 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5226lb # [ 136.077250] niks3-server[1210]: 2026/09/23 07:01:15 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5227lb # [ 136.089564] niks3-server[1210]: 2026/09/23 07:01:15 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5228lb # [ 136.096188] niks3-server[1210]: 2026/09/23 07:01:15 INFO Completed upload id=385229worker2 # [ 136.022225] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.654Z level=INFO msg="Upload complete. (300ms)"5230lb # [ 136.099475] niks3-server[1210]: 2026/09/23 07:01:15 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5231worker2 # [ 136.045420] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15232worker2 # [ 136.064377] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05233worker2 # [ 136.069319] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15234worker2 # [ 136.074221] nix-daemon[842]: reaped child process 1350, status = succeeded5235worker2 # [ 136.087199] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404705236worker2 # [ 136.102923] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404705237worker2 # [ 136.120258] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.752Z level=INFO msg="All 1 paths already cached"5238client # copying 1 paths to 'lb:50051'...5239client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5240worker2 # [ 136.130966] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404705241worker2 # [ 136.145388] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5242worker2 # [ 136.300637] nix-daemon[842]: accepted connection from pid 1377, user nix-grpc-daemon (trusted)5243worker2 # [ 136.318539] nix-daemon[842]: reaped child process 1380, status = succeeded5244lb # [ 136.396784] niks3-server[1210]: 2026/09/23 07:01:16 INFO Received uploads request method=POST path=/api/pending_closures5245worker2 # [ 136.334670] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.966Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5246worker2 # [ 136.336821] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.966Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5247lb # [ 136.431066] niks3-server[1210]: 2026/09/23 07:01:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5248worker2 # [ 136.363776] nix-grpc-daemon[1310]: time=2026-09-23T07:01:15.995Z level=INFO msg="Uploading 1 narinfos"5249lb # [ 136.444629] niks3-server[1210]: 2026/09/23 07:01:16 INFO Signed narinfos id=39 count=15250lb # [ 136.453863] niks3-server[1210]: 2026/09/23 07:01:16 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5251lb # [ 136.464563] niks3-server[1210]: 2026/09/23 07:01:16 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5252worker2 # [ 136.404857] nix-grpc-daemon[1310]: time=2026-09-23T07:01:16.037Z level=INFO msg="Upload complete. (244ms)"5253lb # [ 136.480124] niks3-server[1210]: 2026/09/23 07:01:16 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5254lb # [ 136.487960] niks3-server[1210]: 2026/09/23 07:01:16 INFO Completed upload id=395255lb # [ 136.490828] niks3-server[1210]: 2026/09/23 07:01:16 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5256worker2 # [ 136.416371] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=0 paths=1 nar_bytes_in=6565257worker2 # [ 136.436255] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404705258worker2 # [ 136.440605] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5259worker2 # [ 136.448314] nix-daemon[842]: reaped child process 1375, status = succeeded5260client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5261worker2 # [ 136.458896] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5262client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5263client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5264client # farm-b> LOG-farm-b-after-niks3-restart5265worker2 # [ 137.918318] nix-daemon[842]: reaped child process 1384, status = succeeded5266worker2 # [ 138.028478] nix-daemon[842]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)5267worker2 # [ 138.048615] nix-daemon[842]: reaped child process 1403, status = succeeded5268lb # [ 138.121738] niks3-server[1210]: 2026/09/23 07:01:17 INFO Received uploads request method=POST path=/api/pending_closures5269worker2 # [ 138.058718] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.689Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5270worker2 # [ 138.060545] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.689Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5271lb # [ 138.163761] niks3-server[1210]: 2026/09/23 07:01:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5272worker2 # [ 138.099442] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.731Z level=INFO msg="Uploading 1 narinfos"5273lb # [ 138.178929] niks3-server[1210]: 2026/09/23 07:01:17 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5274lb # [ 138.188588] niks3-server[1210]: 2026/09/23 07:01:17 INFO Signed narinfos id=40 count=15275lb # [ 138.194396] niks3-server[1210]: 2026/09/23 07:01:17 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5276lb # [ 138.209274] niks3-server[1210]: 2026/09/23 07:01:17 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5277worker2 # [ 138.142042] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.774Z level=INFO msg="Upload complete. (273ms)"5278lb # [ 138.216229] niks3-server[1210]: 2026/09/23 07:01:17 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5279lb # [ 138.220116] niks3-server[1210]: 2026/09/23 07:01:17 INFO Completed upload id=405280lb # [ 138.224581] niks3-server[1210]: 2026/09/23 07:01:17 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5281worker2 # [ 138.159823] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15282worker2 # [ 138.178654] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05283worker2 # [ 138.183771] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15284worker2 # [ 138.188901] nix-daemon[842]: reaped child process 1382, status = succeeded5285lb # [ 138.280177] envoy[895]: conn peer=192.168.1.3:49072 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1994 tx=920 ms=944815286lb # [ 138.285091] envoy[895]: conn peer=[2001:db8:1::1]:50456 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1004 tx=282 ms=13035287worker2 # [ 138.222522] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404705288worker2 # [ 138.237645] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:404705289worker2 # [ 138.252432] nix-grpc-daemon[1310]: time=2026-09-23T07:01:17.884Z level=INFO msg="All 2 paths already cached"5290client # copying 1 paths to 'lb:50051'...5291client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5292worker2 # [ 138.263728] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:404705293worker2 # [ 138.280361] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5294worker2 # [ 138.427266] nix-daemon[842]: accepted connection from pid 1409, user nix-grpc-daemon (trusted)5295worker2 # [ 138.444329] nix-daemon[842]: reaped child process 1412, status = succeeded5296lb # [ 138.522195] niks3-server[1210]: 2026/09/23 07:01:18 INFO Received uploads request method=POST path=/api/pending_closures5297worker2 # [ 138.460164] nix-grpc-daemon[1310]: time=2026-09-23T07:01:18.092Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5298worker2 # [ 138.463540] nix-grpc-daemon[1310]: time=2026-09-23T07:01:18.095Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5299lb # [ 138.556156] niks3-server[1210]: 2026/09/23 07:01:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5300worker2 # [ 138.490590] nix-grpc-daemon[1310]: time=2026-09-23T07:01:18.122Z level=INFO msg="Uploading 1 narinfos"5301lb # [ 138.570688] niks3-server[1210]: 2026/09/23 07:01:18 INFO Signed narinfos id=41 count=15302lb # [ 138.580194] niks3-server[1210]: 2026/09/23 07:01:18 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5303lb # [ 138.592494] niks3-server[1210]: 2026/09/23 07:01:18 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5304worker2 # [ 138.527278] nix-grpc-daemon[1310]: time=2026-09-23T07:01:18.159Z level=INFO msg="Upload complete. (233ms)"5305lb # [ 138.604086] niks3-server[1210]: 2026/09/23 07:01:18 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5306lb # [ 138.608874] niks3-server[1210]: 2026/09/23 07:01:18 INFO Completed upload id=415307worker2 # [ 138.536319] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=0 paths=1 nar_bytes_in=7845308lb # [ 138.612904] niks3-server[1210]: 2026/09/23 07:01:18 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5309worker2 # [ 138.550894] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:404705310worker2 # [ 138.560182] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5311worker2 # [ 138.564594] nix-daemon[842]: reaped child process 1407, status = succeeded5312client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5313client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5314worker2 # [ 138.579532] nix-daemon[842]: accepted connection from pid 1249, user nix-grpc-daemon (trusted)5315client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5316client # farm-top> LOG-farm-top-after-niks3-restart5317worker2 # [ 139.411769] nix-daemon[842]: reaped child process 1416, status = succeeded5318worker2 # [ 139.519717] nix-daemon[842]: accepted connection from pid 1432, user nix-grpc-daemon (trusted)5319worker2 # [ 139.535150] nix-daemon[842]: reaped child process 1435, status = succeeded5320lb # [ 139.617067] niks3-server[1210]: 2026/09/23 07:01:19 INFO Received uploads request method=POST path=/api/pending_closures5321worker2 # [ 139.554930] nix-grpc-daemon[1310]: time=2026-09-23T07:01:19.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5322worker2 # [ 139.556608] nix-grpc-daemon[1310]: time=2026-09-23T07:01:19.186Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5323lb # [ 139.659168] niks3-server[1210]: 2026/09/23 07:01:19 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5324worker2 # [ 139.596532] nix-grpc-daemon[1310]: time=2026-09-23T07:01:19.228Z level=INFO msg="Uploading 1 narinfos"5325lb # [ 139.673975] niks3-server[1210]: 2026/09/23 07:01:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5326lb # [ 139.683805] niks3-server[1210]: 2026/09/23 07:01:19 INFO Signed narinfos id=42 count=15327lb # [ 139.688883] niks3-server[1210]: 2026/09/23 07:01:19 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5328lb # [ 139.694927] niks3-server[1210]: 2026/09/23 07:01:19 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5329lb # [ 139.703082] niks3-server[1210]: 2026/09/23 07:01:19 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5330worker2 # [ 139.629091] nix-grpc-daemon[1310]: time=2026-09-23T07:01:19.261Z level=INFO msg="Upload complete. (240ms)"5331lb # [ 139.706689] niks3-server[1210]: 2026/09/23 07:01:19 INFO Completed upload id=425332lb # [ 139.708668] niks3-server[1210]: 2026/09/23 07:01:19 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5333worker2 # [ 139.644292] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40470 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15334worker2 # [ 139.659823] nix-daemon[842]: reaped child process 1414, status = succeeded5335client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.40 seconds)5336client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5337client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.18 seconds)5338(finished: subtest: a niks3 restart does not cost the leader its role, in 7.07 seconds)5339subtest: scheduler down: the other node takes the lock and keeps it5340lb: must succeed: curl -sf localhost:9901/clusters5341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5342lb: must succeed: curl -sf localhost:9901/clusters5343lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5344lb: must succeed: curl -sf localhost:9901/clusters5345lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5346lb: must succeed: curl -sf localhost:9901/clusters5347lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5348worker2 # [ 140.120904] systemd[1]: Stopping Nix worker-protocol over gRPC...5349worker2 # [ 141.081452] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:20Z level=info event=signal_exit5350lb # [ 142.000729] niks3-server[1210]: 2026/09/23 07:01:21 INFO lead: released remote=192.168.1.4:511085351worker2 # [ 142.932709] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=scheduler_restarting5352worker1 # [ 142.944841] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_restarting addr=lb:500515353lb # [ 143.004158] envoy[895]: [2026-09-23 07:01:22.636][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5354worker2 # [ 142.940552] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=scheduler_restarting addr=lb:500515355worker2 # [ 142.946192] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:516445356worker2 # [ 142.951490] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=worker_gone addr=192.168.1.3:500515357worker2 # [ 142.956682] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:516445358worker2 # [ 142.961740] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=worker_gone addr=192.168.1.4:500515359worker2 # [ 142.987425] nix-daemon[842]: reaped child process 1302, status = succeeded5360worker2 # [ 143.126917] nix-grpc-daemon[1249]: E0000 00:00:1790146882.758917 1295 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5361worker1 # [ 143.144352] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_connected addr=lb:500515362worker1 # [ 143.149816] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5363worker2 # [ 143.154973] nix-grpc-daemon[1249]: ts=2026-09-23T07:01:22Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5364worker2 # [ 143.200346] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5365worker2 # [ 143.207054] systemd[1]: Stopped Nix worker-protocol over gRPC.5366worker2 # [ 143.215079] systemd[1]: nix-grpc-daemon.service: Consumed 1.372s CPU time over 15.293s wall clock time, 39.8M memory peak, 47.1K incoming IP traffic, 83.5K outgoing IP traffic.5367worker2 # [ 143.229157] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5368worker2 # [ 143.231047] systemd[1]: Closed Nix worker-protocol over gRPC.5369worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5370worker2 # [ 143.235787] systemd[1]: nix-grpc-daemon.socket: Consumed 857us CPU time over 15.967s wall clock time, 256K memory peak, 271.5K incoming IP traffic, 32.4K outgoing IP traffic.5371worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.07 seconds)5372worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5373worker1 # [ 143.350214] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_connected addr=lb:500515374worker1 # [ 143.353937] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5375worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5376worker1 # [ 143.554613] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:500515377worker1 # [ 143.559917] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5378worker1 # [ 143.760543] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:500515379worker1 # [ 143.765626] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5380worker1 # [ 143.926325] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_take_over5381lb # [ 143.988766] niks3-server[1210]: 2026/09/23 07:01:23 INFO lead: acquired remote=192.168.1.3:358745382worker1 # [ 143.967942] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:500515383worker1 # [ 143.971939] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker1 # [ 144.168590] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:500515385worker1 # [ 144.173609] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5386worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5387worker1 # [ 144.375986] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_connected addr=lb:500515388worker1 # [ 144.379969] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5389worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5390worker1 # [ 144.577073] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=scheduler_connected addr=lb:500515391worker1 # [ 144.582412] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5392worker1 # [ 144.782869] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=scheduler_connected addr=lb:500515393worker1 # [ 144.791707] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:446905394worker1 # [ 144.797427] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05395worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5396worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5397client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25398client # don't know how to build these paths:5399client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5400worker1 # [ 145.730548] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531745401worker1 # [ 145.738237] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531745402worker1 # [ 145.763558] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05403worker1 # [ 145.770280] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15404worker1 # [ 145.810471] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531745405worker1 # [ 145.828179] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531745406client # copying 1 paths to 'lb:50051'...5407client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5408worker1 # [ 145.844439] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531745409worker1 # [ 145.859971] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5410worker1 # [ 145.891884] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=info event=niks3_push_started pid=13015411worker1 # [ 146.038759] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.662Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5412worker1 # [ 146.148121] nix-daemon[852]: accepted connection from pid 1314, user nix-grpc-daemon (trusted)5413worker1 # [ 146.169130] nix-daemon[852]: reaped child process 1317, status = succeeded5414lb # [ 146.229908] niks3-server[1210]: 2026/09/23 07:01:25 INFO Received uploads request method=POST path=/api/pending_closures5415worker1 # [ 146.179510] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.802Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5416worker1 # [ 146.184309] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.802Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5417lb # [ 146.274595] niks3-server[1210]: 2026/09/23 07:01:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5418worker1 # [ 146.223988] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.846Z level=INFO msg="Uploading 1 narinfos"5419lb # [ 146.286553] niks3-server[1210]: 2026/09/23 07:01:25 INFO Signed narinfos id=43 count=15420lb # [ 146.306828] niks3-server[1210]: 2026/09/23 07:01:25 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5421lb # [ 146.314110] niks3-server[1210]: 2026/09/23 07:01:25 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5422lb # [ 146.322344] niks3-server[1210]: 2026/09/23 07:01:25 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5423lb # [ 146.329055] niks3-server[1210]: 2026/09/23 07:01:25 INFO Completed upload id=435424worker1 # [ 146.265967] nix-grpc-daemon[1301]: time=2026-09-23T07:01:25.889Z level=INFO msg="Upload complete. (223ms)"5425lb # [ 146.330904] niks3-server[1210]: 2026/09/23 07:01:25 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5426worker1 # [ 146.280325] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=0 paths=1 nar_bytes_in=5205427worker1 # [ 146.292367] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531745428worker1 # [ 146.298121] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5429worker1 # [ 146.302738] nix-daemon[852]: reaped child process 1299, status = succeeded5430client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5431worker1 # [ 146.315115] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5432client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5433client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5434client # farm-a> LOG-farm-a-on-standby5435worker1 # [ 147.380692] nix-daemon[852]: reaped child process 1321, status = succeeded5436worker1 # [ 147.457018] nix-daemon[852]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5437worker1 # [ 147.474587] nix-daemon[852]: reaped child process 1340, status = succeeded5438lb # [ 147.543093] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received uploads request method=POST path=/api/pending_closures5439worker1 # [ 147.492363] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.114Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5440worker1 # [ 147.493835] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.115Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5441lb # [ 147.571335] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5442lb # [ 147.596311] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5443worker1 # [ 147.542775] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.166Z level=INFO msg="Uploading 1 narinfos"5444lb # [ 147.606275] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5445lb # [ 147.614839] niks3-server[1210]: 2026/09/23 07:01:27 INFO Signed narinfos id=44 count=15446lb # [ 147.619270] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5447lb # [ 147.627283] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5448worker1 # [ 147.566833] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.190Z level=INFO msg="Upload complete. (242ms)"5449lb # [ 147.630879] niks3-server[1210]: 2026/09/23 07:01:27 INFO Completed upload id=445450lb # [ 147.633149] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5451worker1 # [ 147.588526] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15452worker1 # [ 147.600259] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05453worker1 # [ 147.603903] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15454worker1 # [ 147.607590] nix-daemon[852]: reaped child process 1319, status = succeeded5455worker1 # [ 147.631726] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531745456worker1 # [ 147.645674] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531745457worker1 # [ 147.659489] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.283Z level=INFO msg="All 1 paths already cached"5458client # copying 1 paths to 'lb:50051'...5459client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5460worker1 # [ 147.670159] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531745461worker1 # [ 147.683545] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5462worker1 # [ 147.837076] nix-daemon[852]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5463worker1 # [ 147.854792] nix-daemon[852]: reaped child process 1349, status = succeeded5464lb # [ 147.919919] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received uploads request method=POST path=/api/pending_closures5465worker1 # [ 147.867378] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.490Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5466worker1 # [ 147.869113] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.490Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5467worker1 # [ 147.899246] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.522Z level=INFO msg="Uploading 1 narinfos"5468lb # [ 147.949419] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5469lb # [ 147.964647] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5470lb # [ 147.975593] niks3-server[1210]: 2026/09/23 07:01:27 INFO Signed narinfos id=45 count=15471lb # [ 147.984349] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5472lb # [ 147.987905] niks3-server[1210]: 2026/09/23 07:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5473lb # [ 147.996938] niks3-server[1210]: 2026/09/23 07:01:27 INFO Completed upload id=455474worker1 # [ 147.934404] nix-grpc-daemon[1301]: time=2026-09-23T07:01:27.558Z level=INFO msg="Upload complete. (236ms)"5475lb # [ 147.999666] niks3-server[1210]: 2026/09/23 07:01:27 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5476worker1 # [ 147.944324] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=0 paths=1 nar_bytes_in=6405477worker1 # [ 147.960622] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531745478worker1 # [ 147.969977] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5479worker1 # [ 147.973260] nix-daemon[852]: reaped child process 1344, status = succeeded5480client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5481client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5482worker1 # [ 147.987798] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5483lb # [ 148.275221] envoy[895]: conn peer=[2001:db8:1::1]:50464 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9736 tx=4476 ms=61525484lb # [ 148.280428] envoy[895]: conn peer=[2001:db8:1::4]:34528 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5513 tx=665 ms=150185485worker1 # [ 149.239605] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:28Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:446925486client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5487client # farm-b> LOG-farm-b-on-standby5488worker1 # [ 149.618426] nix-daemon[852]: reaped child process 1353, status = succeeded5489worker1 # [ 149.712076] nix-daemon[852]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5490worker1 # [ 149.731019] nix-daemon[852]: reaped child process 1372, status = succeeded5491lb # [ 149.792619] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received uploads request method=POST path=/api/pending_closures5492worker1 # [ 149.741737] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.364Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5493worker1 # [ 149.743194] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.364Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5494lb # [ 149.827879] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5495lb # [ 149.844059] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5496lb # [ 149.863584] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5497lb # [ 149.871025] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5498lb # [ 149.875893] niks3-server[1210]: 2026/09/23 07:01:29 INFO Signed narinfos id=46 count=15499worker1 # [ 149.814331] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.437Z level=INFO msg="Uploading 1 narinfos"5500lb # [ 149.884859] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5501lb # [ 149.896478] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5502lb # [ 149.903712] niks3-server[1210]: 2026/09/23 07:01:29 INFO Completed upload id=465503worker1 # [ 149.842481] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.466Z level=INFO msg="Upload complete. (244ms)"5504worker1 # [ 149.856359] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15505worker1 # [ 149.879428] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05506worker1 # [ 149.884528] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15507worker1 # [ 149.889458] nix-daemon[852]: reaped child process 1351, status = succeeded5508worker1 # [ 149.902646] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531745509worker1 # [ 149.917167] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531745510worker1 # [ 149.931827] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.555Z level=INFO msg="All 2 paths already cached"5511client # copying 1 paths to 'lb:50051'...5512client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5513worker1 # [ 149.942817] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531745514worker1 # [ 149.956134] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5515worker1 # [ 150.105404] nix-daemon[852]: accepted connection from pid 1378, user nix-grpc-daemon (trusted)5516worker1 # [ 150.121550] nix-daemon[852]: reaped child process 1381, status = succeeded5517lb # [ 150.190148] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received uploads request method=POST path=/api/pending_closures5518worker1 # [ 150.140089] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.763Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5519worker1 # [ 150.141704] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.763Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5520lb # [ 150.222124] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5521worker1 # [ 150.169735] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.793Z level=INFO msg="Uploading 1 narinfos"5522lb # [ 150.238537] niks3-server[1210]: 2026/09/23 07:01:29 INFO Signed narinfos id=47 count=15523lb # [ 150.246105] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5524lb # [ 150.253188] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5525lb # [ 150.266277] niks3-server[1210]: 2026/09/23 07:01:29 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5526worker1 # [ 150.201812] nix-grpc-daemon[1301]: time=2026-09-23T07:01:29.825Z level=INFO msg="Upload complete. (230ms)"5527lb # [ 150.272084] niks3-server[1210]: 2026/09/23 07:01:29 INFO Completed upload id=475528lb # [ 150.275288] niks3-server[1210]: 2026/09/23 07:01:29 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5529worker1 # [ 150.212460] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=0 paths=1 nar_bytes_in=7685530worker1 # [ 150.224201] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531745531worker1 # [ 150.234131] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5532worker1 # [ 150.237257] nix-daemon[852]: reaped child process 1376, status = succeeded5533client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5534client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5535worker1 # [ 150.250935] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5536client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5537client # farm-top> LOG-farm-top-on-standby5538worker1 # [ 151.561187] nix-daemon[852]: reaped child process 1385, status = succeeded5539worker1 # [ 151.633094] nix-daemon[852]: accepted connection from pid 1401, user nix-grpc-daemon (trusted)5540worker1 # [ 151.653861] nix-daemon[852]: reaped child process 1404, status = succeeded5541lb # [ 151.722122] niks3-server[1210]: 2026/09/23 07:01:31 INFO Received uploads request method=POST path=/api/pending_closures5542worker1 # [ 151.672527] nix-grpc-daemon[1301]: time=2026-09-23T07:01:31.294Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5543worker1 # [ 151.674434] nix-grpc-daemon[1301]: time=2026-09-23T07:01:31.294Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5544lb # [ 151.769662] niks3-server[1210]: 2026/09/23 07:01:31 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5545worker1 # [ 151.721003] nix-grpc-daemon[1301]: time=2026-09-23T07:01:31.344Z level=INFO msg="Uploading 1 narinfos"5546lb # [ 151.786415] niks3-server[1210]: 2026/09/23 07:01:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5547lb # [ 151.793705] niks3-server[1210]: 2026/09/23 07:01:31 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5548lb # [ 151.799847] niks3-server[1210]: 2026/09/23 07:01:31 INFO Signed narinfos id=48 count=15549lb # [ 151.807379] niks3-server[1210]: 2026/09/23 07:01:31 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5550worker1 # [ 151.741427] nix-grpc-daemon[1301]: time=2026-09-23T07:01:31.365Z level=INFO msg="Upload complete. (278ms)"5551lb # [ 151.811277] niks3-server[1210]: 2026/09/23 07:01:31 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5552lb # [ 151.815032] niks3-server[1210]: 2026/09/23 07:01:31 INFO Completed upload id=485553lb # [ 151.817426] niks3-server[1210]: 2026/09/23 07:01:31 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5554worker1 # [ 151.753257] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15555worker1 # [ 151.771247] nix-daemon[852]: reaped child process 1383, status = succeeded5556client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.29 seconds)5557client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5558client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5559worker2 # [ 152.055330] systemd[1]: Listening on Nix worker-protocol over gRPC.5560worker2 # [ 152.065488] systemd[1]: Starting scheduler-token.service...5561worker2 # [ 152.149760] systemd[1]: scheduler-token.service: Deactivated successfully.5562worker2 # [ 152.155137] systemd[1]: Finished scheduler-token.service.5563worker2 # [ 152.160536] systemd[1]: scheduler-token.service: Consumed 48ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5564worker2 # [ 152.167148] systemd[1]: Starting Nix worker-protocol over gRPC...5565worker2 # [ 152.290535] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5566worker2 # [ 152.292867] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5567worker2 # [ 152.296712] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15568worker2 # [ 152.301100] nix-grpc-daemon[1456]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5569worker2 # [ 152.323589] nix-grpc-daemon[1456]: E0000 00:00:1790146891.933328 1456 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5570lb: must succeed: curl -sf localhost:9901/clusters5571worker2 # [ 152.352491] nix-grpc-daemon[1456]: E0000 00:00:1790146891.937441 1456 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5572worker2 # [ 152.354795] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5573worker2 # [ 152.358030] nix-grpc-daemon[1456]: E0000 00:00:1790146891.944746 1496 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5574worker2 # [ 152.361089] systemd[1]: Started Nix worker-protocol over gRPC.5575worker2 # [ 152.362210] nix-grpc-daemon[1456]: E0000 00:00:1790146891.969463 1496 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5576worker2 # [ 152.364576] nix-grpc-daemon[1456]: E0000 00:00:1790146891.969635 1496 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5577worker2 # [ 152.366958] nix-grpc-daemon[1456]: E0000 00:00:1790146891.989457 1496 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5578worker2 # [ 152.375749] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:32Z level=info event=scheduler_connected addr=lb:500515579worker1 # [ 152.397918] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=info event=oidc_jwks provider=mock keys=15580worker1 # [ 152.404349] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:446905581worker1 # [ 152.413242] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05582lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5583worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5584worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5585lb: must succeed: curl -sf localhost:9901/clusters5586lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5587lb: must succeed: curl -sf localhost:9901/clusters5588lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5589worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5590worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5591(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.81 seconds)5592subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5593client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5594client # [ 152.743097] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5595client # Running as unit: rs.service; invocation ID: 9c8bd89e06c0430e83a64a1c000ee6515596client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5597client # [ 152.943612] nix[1344]: don't know how to build these paths:5598client # [ 152.945190] nix[1344]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5599worker1 # [ 152.959928] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:531745600worker1 # [ 152.972271] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05601worker1 # [ 152.978154] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15602worker1 # [ 153.022766] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531745603worker1 # [ 153.039123] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:531745604client # [ 153.036269] nix[1344]: copying 1 paths to 'lb:50051'...5605worker1 # [ 153.054049] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:531745606client # [ 153.038599] nix[1344]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5607worker1 # [ 153.070763] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5608worker1 # [ 153.232983] nix-daemon[852]: accepted connection from pid 1417, user nix-grpc-daemon (trusted)5609worker1 # [ 153.254123] nix-daemon[852]: reaped child process 1420, status = succeeded5610lb # [ 153.319799] niks3-server[1210]: 2026/09/23 07:01:32 INFO Received uploads request method=POST path=/api/pending_closures5611worker1 # [ 153.268067] nix-grpc-daemon[1301]: time=2026-09-23T07:01:32.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5612worker1 # [ 153.269755] nix-grpc-daemon[1301]: time=2026-09-23T07:01:32.891Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5613lb # [ 153.358084] niks3-server[1210]: 2026/09/23 07:01:32 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5614worker1 # [ 153.306422] nix-grpc-daemon[1301]: time=2026-09-23T07:01:32.929Z level=INFO msg="Uploading 1 narinfos"5615lb # [ 153.370052] niks3-server[1210]: 2026/09/23 07:01:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5616lb # [ 153.379375] niks3-server[1210]: 2026/09/23 07:01:33 INFO Signed narinfos id=49 count=15617lb # [ 153.387446] niks3-server[1210]: 2026/09/23 07:01:33 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5618lb # [ 153.399592] niks3-server[1210]: 2026/09/23 07:01:33 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5619lb # [ 153.412134] niks3-server[1210]: 2026/09/23 07:01:33 INFO Completed upload id=495620worker1 # [ 153.349153] nix-grpc-daemon[1301]: time=2026-09-23T07:01:32.972Z level=INFO msg="Upload complete. (261ms)"5621lb # [ 153.414493] niks3-server[1210]: 2026/09/23 07:01:33 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5622worker1 # [ 153.360779] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=0 paths=1 nar_bytes_in=4165623worker1 # [ 153.375508] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:531745624worker1 # [ 153.385904] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5625worker1 # [ 153.390976] nix-daemon[852]: reaped child process 1415, status = succeeded5626client # [ 153.385204] nix[1344]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5627worker1 # [ 153.404247] nix-daemon[852]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5628lb: must succeed: curl -sf localhost:9901/clusters5629lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5630lb: must succeed: curl -sf localhost:9901/clusters5631lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5632lb: must succeed: curl -sf localhost:9901/clusters5633lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5634lb: must succeed: curl -sf localhost:9901/clusters5635lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5636worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5637worker2 # [ 155.535912] systemd-journald[271]: Received client request to rotate journal, rotating.5638worker2 # [ 155.556072] systemd-journald[271]: Vacuuming done, freed 0B of archived journals from /var/log/journal/0ca198ba34754993b8a9f9af0e0d185e.5639worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5640worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5641worker1 # [ 155.619430] systemd[1]: Stopping Nix worker-protocol over gRPC...5642worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5643client: must succeed: sleep 5.05644worker1 # [ 155.709428] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:35Z level=info event=draining5645lb # [ 158.277723] envoy[895]: conn peer=[2001:db8:1::1]:38968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9619 tx=4364 ms=60455646client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5647worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5648worker1 # [ 161.116593] nix-daemon[852]: reaped child process 1424, status = succeeded5649worker1 # [ 161.193214] nix-daemon[852]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5650worker1 # [ 161.213553] nix-daemon[852]: reaped child process 1468, status = succeeded5651lb # [ 161.279165] niks3-server[1210]: 2026/09/23 07:01:40 INFO Received uploads request method=POST path=/api/pending_closures5652worker1 # [ 161.231460] nix-grpc-daemon[1301]: time=2026-09-23T07:01:40.852Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5653worker1 # [ 161.233436] nix-grpc-daemon[1301]: time=2026-09-23T07:01:40.852Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5654lb # [ 161.327661] niks3-server[1210]: 2026/09/23 07:01:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5655worker1 # [ 161.275529] nix-grpc-daemon[1301]: time=2026-09-23T07:01:40.899Z level=INFO msg="Uploading 1 narinfos"5656lb # [ 161.341105] niks3-server[1210]: 2026/09/23 07:01:40 INFO Signed narinfos id=50 count=15657lb # [ 161.354342] niks3-server[1210]: 2026/09/23 07:01:40 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5658lb # [ 161.362824] niks3-server[1210]: 2026/09/23 07:01:40 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5659lb # [ 161.372788] niks3-server[1210]: 2026/09/23 07:01:41 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5660lb # [ 161.378899] niks3-server[1210]: 2026/09/23 07:01:41 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5661lb # [ 161.389046] niks3-server[1210]: 2026/09/23 07:01:41 INFO Completed upload id=505662lb # [ 161.392165] niks3-server[1210]: 2026/09/23 07:01:41 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5663worker1 # [ 161.327310] nix-grpc-daemon[1301]: time=2026-09-23T07:01:40.951Z level=INFO msg="Upload complete. (302ms)"5664worker1 # [ 161.341337] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53174 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15665worker1 # [ 161.360979] nix-daemon[852]: reaped child process 1422, status = succeeded5666client # [ 161.354417] systemd[1]: rs.service: Deactivated successfully.5667client # [ 161.359593] systemd[1]: rs.service: Consumed 191ms CPU time over 8.612s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.5668worker1 # [ 161.713465] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:41Z level=info event=signal_exit5669lb # [ 161.992314] niks3-server[1210]: 2026/09/23 07:01:41 INFO lead: released remote=192.168.1.3:358745670lb # [ 162.388801] niks3-server[1210]: 2026/09/23 07:01:42 INFO lead: acquired remote=192.168.1.4:357365671worker2 # [ 162.320825] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:41Z level=info event=scheduler_take_over5672worker2 # [ 162.921383] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_restarting addr=lb:500515673worker1 # [ 162.934802] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=scheduler_restarting5674lb # [ 162.994447] envoy[895]: [2026-09-23 07:01:42.627][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5675worker1 # [ 162.939186] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:446905676worker1 # [ 162.947323] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=worker_gone addr=192.168.1.4:500515677worker1 # [ 162.952579] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=scheduler_restarting addr=lb:500515678worker1 # [ 162.957433] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:446905679worker1 # [ 162.964853] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=worker_gone addr=192.168.1.3:500515680worker1 # [ 162.987176] nix-daemon[852]: reaped child process 1274, status = succeeded5681worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.17 seconds)5682client: waiting for success: ! systemctl is-active rs5683worker2 # [ 163.122767] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_connected addr=lb:500515684worker2 # [ 163.128133] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5685client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5686client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5687worker1 # [ 163.161906] nix-grpc-daemon[1221]: ts=2026-09-23T07:01:42Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5688client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5689client: must fail: journalctl -u rs | grep -q 'reconnecting'5690client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5691lb: must succeed: curl -sf localhost:9901/clusters5692lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5693worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5694worker2 # [ 163.332758] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_connected addr=lb:500515695worker2 # [ 163.337491] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5696worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5697worker2 # [ 163.531989] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=scheduler_connected addr=lb:500515698worker2 # [ 163.548144] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=oidc_jwks provider=mock keys=15699worker2 # [ 163.553772] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:527825700worker2 # [ 163.559779] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05701worker1 # [ 163.830807] nix-grpc-daemon[1221]: E0000 00:00:1790146903.454412 1267 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 23, started true, shutdown true5702worker1 # [ 163.847934] nix-grpc-daemon[1221]: E0000 00:00:1790146903.471599 1267 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5703worker2 # [ 164.245795] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:43Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:527925704lb: must succeed: curl -sf localhost:9901/clusters5705lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5706worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5707worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5708worker1 # [ 164.551933] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5709worker1 # [ 164.558405] systemd[1]: Stopped Nix worker-protocol over gRPC.5710worker1 # [ 164.563815] systemd[1]: nix-grpc-daemon.service: Consumed 1.570s CPU time over 32.026s wall clock time, 39.1M memory peak, 48.3K incoming IP traffic, 99.8K outgoing IP traffic.5711worker1 # [ 164.581124] systemd[1]: Starting Nix worker-protocol over gRPC...5712worker1 # [ 164.727845] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5713worker1 # [ 164.730244] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5714worker1 # [ 164.733983] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15715worker1 # [ 164.739240] nix-grpc-daemon[1475]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5716worker1 # [ 164.760106] nix-grpc-daemon[1475]: E0000 00:00:1790146904.362237 1475 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5717worker1 # [ 164.762435] nix-grpc-daemon[1475]: E0000 00:00:1790146904.365776 1475 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5718worker1 # [ 164.783427] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5719worker1 # [ 164.785689] nix-grpc-daemon[1475]: E0000 00:00:1790146904.371547 1516 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5720worker1 # [ 164.788210] systemd[1]: Started Nix worker-protocol over gRPC.5721worker1 # [ 164.789076] nix-grpc-daemon[1475]: E0000 00:00:1790146904.391705 1516 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5722worker1 # [ 164.794328] nix-grpc-daemon[1475]: E0000 00:00:1790146904.391848 1516 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5723worker1 # [ 164.798641] nix-grpc-daemon[1475]: E0000 00:00:1790146904.406248 1516 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5724worker1 # [ 164.801245] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:44Z level=info event=scheduler_connected addr=lb:500515725worker2 # [ 164.789873] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:44Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:527825726worker2 # [ 164.795519] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:44Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05727lb: must succeed: curl -sf localhost:9901/clusters5728lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5729worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5730worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5731(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.95 seconds)5732subtest: scheduler restart mid-build: build finishes, no double build5733client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5734client # [ 165.691303] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5735client # Running as unit: mid.service; invocation ID: ff370474f9e443e7b0b853f7a8a58c565736client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.08 seconds)5737worker2 # [ 165.905057] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:528065738worker2 # [ 165.919736] nix-daemon[842]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5739client # [ 165.931717] nix[1380]: don't know how to build these paths:5740client # [ 165.933443] nix[1380]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5741worker2 # [ 165.950950] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:45Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05742worker2 # [ 165.957922] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:45Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15743worker2 # [ 166.003422] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528065744worker2 # [ 166.019234] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 91 ms (attempt 1/5)5745worker2 # [ 166.119601] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 55 ms (attempt 2/5)5746worker2 # [ 166.176523] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 385 ms (attempt 3/5)5747worker2 # [ 166.563382] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 770 ms (attempt 4/5)5748worker2 # [ 167.336924] nix-grpc-daemon[1456]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5749worker2 # [ 167.341847] nix-grpc-daemon[1456]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5750worker2 # [ 167.363480] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:528065751client # [ 167.373206] nix[1380]: copying 1 paths to 'lb:50051'...5752worker2 # [ 167.379210] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:528065753client # [ 167.377667] nix[1380]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5754worker2 # [ 167.394882] nix-daemon[842]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5755worker2 # [ 167.422147] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:47Z level=info event=niks3_push_started pid=15865756worker2 # [ 167.573613] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.204Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5757worker2 # [ 167.679021] nix-daemon[842]: accepted connection from pid 1599, user nix-grpc-daemon (trusted)5758worker2 # [ 167.697193] nix-daemon[842]: reaped child process 1602, status = succeeded5759lb # [ 167.779523] niks3-server[1210]: 2026/09/23 07:01:47 INFO Received uploads request method=POST path=/api/pending_closures5760worker2 # [ 167.717987] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5761worker2 # [ 167.719469] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.349Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5762lb # [ 167.820459] niks3-server[1210]: 2026/09/23 07:01:47 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5763worker2 # [ 167.757311] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.388Z level=INFO msg="Uploading 1 narinfos"5764lb # [ 167.834041] niks3-server[1210]: 2026/09/23 07:01:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5765lb # [ 167.843629] niks3-server[1210]: 2026/09/23 07:01:47 INFO Signed narinfos id=51 count=15766lb # [ 167.853320] niks3-server[1210]: 2026/09/23 07:01:47 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5767lb # [ 167.867182] niks3-server[1210]: 2026/09/23 07:01:47 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5768lb # [ 167.875705] niks3-server[1210]: 2026/09/23 07:01:47 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5769worker2 # [ 167.802366] nix-grpc-daemon[1586]: time=2026-09-23T07:01:47.434Z level=INFO msg="Upload complete. (229ms)"5770lb # [ 167.880383] niks3-server[1210]: 2026/09/23 07:01:47 INFO Completed upload id=515771worker2 # [ 167.812277] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52806 duration_s=0 paths=1 nar_bytes_in=4245772worker2 # [ 167.828861] nix-grpc-daemon[1456]: ts=2026-09-23T07:01:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:528065773worker2 # [ 167.837087] nix-daemon[842]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5774worker2 # [ 167.843521] nix-daemon[842]: reaped child process 1584, status = succeeded5775client # [ 167.850007] nix[1380]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5776worker2 # [ 167.859290] nix-daemon[842]: accepted connection from pid 1456, user nix-grpc-daemon (trusted)5777lb # [ 168.284926] envoy[895]: conn peer=[2001:db8:1::1]:55508 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1516 ms=83935778lb # [ 168.291046] envoy[895]: conn peer=[2001:db8:1::3]:47676 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2582 tx=921 ms=303925779lb: must succeed: curl -sf localhost:9901/clusters5780lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5781lb: must succeed: curl -sf localhost:9901/clusters5782lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5783worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5784worker2 # [ 169.556750] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1456 (nix-grpc-daemon) on client request.5785worker2 # [ 169.558374] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5786lb # [ 169.652827] niks3-server[1210]: 2026/09/23 07:01:49 INFO lead: released remote=192.168.1.4:357365787worker2 # [ 169.582545] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5788worker2 # [ 169.590945] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5789client # [ 169.580995] nix[1380]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5790worker1 # [ 169.598280] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5791lb # [ 169.660474] envoy[895]: [2026-09-23 07:01:49.294][895][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5792client # [ 169.587381] nix[1380]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5793worker2 # [ 169.597244] systemd[1]: nix-grpc-daemon.service: Consumed 506ms CPU time over 17.412s wall clock time, 34.9M memory peak, 15.6K incoming IP traffic, 28.2K outgoing IP traffic.5794worker2 # [ 169.599232] systemd[1]: Starting scheduler-token.service...5795worker2 # [ 169.640313] nix-daemon[842]: reaped child process 1604, status = succeeded5796worker2 # [ 169.641903] nix-daemon[842]: reaped child process 1572, status = succeeded5797worker2 # [ 169.648789] nix-daemon[1606]: unexpected Nix daemon error: error: interrupted by the user5798worker2 # [ 169.661690] nix-daemon[842]: reaped child process 1606, status = failed with exit code 15799worker2 # [ 169.675962] systemd[1]: scheduler-token.service: Deactivated successfully.5800worker2 # [ 169.679895] systemd[1]: Finished scheduler-token.service.5801worker2 # [ 169.682622] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 79ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5802worker2 # [ 169.688175] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5803worker2 # [ 169.689648] systemd[1]: Starting scheduler-token.service...5804worker2 # [ 169.728908] systemd[1]: scheduler-token.service: Deactivated successfully.5805worker2 # [ 169.732090] systemd[1]: Finished scheduler-token.service.5806worker2 # [ 169.732883] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 48ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5807worker2 # [ 169.736725] systemd[1]: Starting Nix worker-protocol over gRPC...5808lb # [ 169.811639] niks3-server[1210]: 2026/09/23 07:01:49 INFO lead: acquired remote=192.168.1.3:549645809worker1 # [ 169.750494] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_take_over5810worker1 # [ 169.798379] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:500515811worker2 # [ 169.837750] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5812worker2 # [ 169.839976] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5813worker2 # [ 169.843694] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15814worker2 # [ 169.849271] nix-grpc-daemon[1649]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5815worker2 # [ 169.850724] nix-grpc-daemon[1649]: E0000 00:00:1790146909.480630 1649 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5816worker1 # [ 169.879948] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5817worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.36 seconds)5818client: must succeed: sleep 5.05819worker2 # [ 169.892145] nix-grpc-daemon[1649]: E0000 00:00:1790146909.486609 1649 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5820worker2 # [ 169.894442] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5821worker2 # [ 169.903577] nix-grpc-daemon[1649]: E0000 00:00:1790146909.499003 1690 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5822worker2 # [ 169.906603] nix-grpc-daemon[1649]: E0000 00:00:1790146909.516255 1690 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5823worker2 # [ 169.909279] nix-grpc-daemon[1649]: E0000 00:00:1790146909.516452 1690 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5824worker2 # [ 169.911636] systemd[1]: Started Nix worker-protocol over gRPC.5825worker2 # [ 169.915167] nix-grpc-daemon[1649]: E0000 00:00:1790146909.535074 1690 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5826worker2 # [ 169.920235] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:500515827worker2 # [ 169.922098] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5828worker1 # [ 170.080830] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:500515829worker1 # [ 170.086213] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5830client # [ 170.085358] nix[1380]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting5831worker2 # [ 170.123406] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:500515832worker2 # [ 170.130061] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5833worker1 # [ 170.286997] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:500515834worker1 # [ 170.292397] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5835worker2 # [ 170.331106] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_connected addr=lb:500515836worker2 # [ 170.337831] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:49Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5837worker1 # [ 170.493133] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515838worker1 # [ 170.498297] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5839worker2 # [ 170.538855] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515840worker2 # [ 170.544110] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5841worker1 # [ 170.700675] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515842worker1 # [ 170.704700] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5843worker2 # [ 170.744969] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515844worker2 # [ 170.750037] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5845worker1 # [ 170.900808] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515846worker1 # [ 170.905799] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker2 # [ 170.952536] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515848worker2 # [ 170.956627] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5849client # [ 171.088864] nix[1380]: scheduler at lb:50051: no healthy upstream, reconnecting5850worker1 # [ 171.106526] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515851worker1 # [ 171.111637] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5852worker2 # [ 171.152980] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515853worker2 # [ 171.157990] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5854worker1 # [ 171.312430] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515855worker1 # [ 171.317413] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5856worker2 # [ 171.359022] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_connected addr=lb:500515857worker2 # [ 171.364130] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5858worker1 # [ 171.518126] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=scheduler_connected addr=lb:500515859worker1 # [ 171.523264] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5860worker2 # [ 171.565068] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:51Z level=info event=scheduler_connected addr=lb:500515861worker1 # [ 171.600120] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=oidc_jwks provider=mock keys=15862worker1 # [ 171.605218] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:455365863worker1 # [ 171.610829] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05864worker1 # [ 171.726472] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=scheduler_connected addr=lb:500515865worker1 # [ 171.730603] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:455365866worker1 # [ 171.735689] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05867worker1 # [ 173.116818] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:52Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05868worker1 # [ 173.117189] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:52Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15869worker2 # [ 173.157058] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:549225870worker2 # [ 173.172655] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)5871worker2 # [ 173.195277] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)5872client # [ 173.205369] nix[1380]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5873worker2 # [ 173.214192] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)5874worker1 # [ 174.264850] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:53Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:455505875client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5876client: waiting for success: ! systemctl is-active mid5877worker2 # [ 175.229979] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:54Z level=info event=niks3_push_started pid=17315878worker2 # [ 175.260266] nix-daemon[842]: reaped child process 1706, status = succeeded5879worker2 # [ 175.361963] nix-grpc-daemon[1731]: time=2026-09-23T07:01:54.993Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5880worker2 # [ 175.468265] nix-daemon[842]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)5881worker2 # [ 175.487137] nix-daemon[842]: reaped child process 1747, status = succeeded5882lb # [ 175.566755] niks3-server[1210]: 2026/09/23 07:01:55 INFO Received uploads request method=POST path=/api/pending_closures5883worker2 # [ 175.504078] nix-grpc-daemon[1731]: time=2026-09-23T07:01:55.135Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5884worker2 # [ 175.505556] nix-grpc-daemon[1731]: time=2026-09-23T07:01:55.135Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5885lb # [ 175.621264] niks3-server[1210]: 2026/09/23 07:01:55 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5886worker2 # [ 175.558900] nix-grpc-daemon[1731]: time=2026-09-23T07:01:55.190Z level=INFO msg="Uploading 1 narinfos"5887lb # [ 175.635597] niks3-server[1210]: 2026/09/23 07:01:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign5888lb # [ 175.653225] niks3-server[1210]: 2026/09/23 07:01:55 INFO Signed narinfos id=52 count=15889lb # [ 175.660479] niks3-server[1210]: 2026/09/23 07:01:55 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete5890lb # [ 175.671156] niks3-server[1210]: 2026/09/23 07:01:55 INFO Completed upload id=525891worker2 # [ 175.595621] nix-grpc-daemon[1731]: time=2026-09-23T07:01:55.227Z level=INFO msg="Upload complete. (230ms)"5892worker2 # [ 175.607921] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54922 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=15893worker2 # [ 175.622630] nix-daemon[842]: reaped child process 1704, status = succeeded5894client # [ 175.638235] systemd[1]: mid.service: Deactivated successfully.5895client # [ 175.642134] systemd[1]: mid.service: Consumed 203ms CPU time over 9.946s wall clock time, 43.4M memory peak, 9.3K incoming IP traffic, 11.2K outgoing IP traffic.5896client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5897client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5898client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5899(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.72 seconds)5900subtest: interrupting the client stops the build on the worker5901client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5902client # [ 176.384623] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5903client # Running as unit: intr.service; invocation ID: bd0bb50dc39d43e9a9cd2817e54d8d765904client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5905worker2 # [ 176.563899] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:549225906client # [ 176.564892] nix[1416]: don't know how to build these paths:5907client # [ 176.566591] nix[1416]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5908worker1 # [ 176.593377] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:56Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05909worker1 # [ 176.599501] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:56Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=15910worker1 # [ 176.662109] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455625911worker1 # [ 176.677036] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5912worker1 # [ 176.699060] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5913worker1 # [ 176.711348] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 2/5)5914worker1 # [ 176.737049] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 3/5)5915worker1 # [ 176.753956] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 779 ms (attempt 4/5)5916worker1 # [ 177.535708] nix-grpc-daemon[1475]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5917worker1 # [ 177.539620] nix-grpc-daemon[1475]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5918worker1 # [ 177.559074] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455625919client # [ 177.556140] nix[1416]: copying 1 paths to 'lb:50051'...5920worker1 # [ 177.574942] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455625921client # [ 177.560148] nix[1416]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5922worker1 # [ 177.592498] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5923worker1 # [ 177.633915] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=info event=niks3_push_started pid=15765924worker1 # [ 177.787364] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.409Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5925worker1 # [ 177.897559] nix-daemon[852]: accepted connection from pid 1589, user nix-grpc-daemon (trusted)5926worker1 # [ 177.916658] nix-daemon[852]: reaped child process 1592, status = succeeded5927lb # [ 177.986758] niks3-server[1210]: 2026/09/23 07:01:57 INFO Received uploads request method=POST path=/api/pending_closures5928worker1 # [ 177.938891] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.561Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5929worker1 # [ 177.942836] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.561Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5930lb # [ 178.028192] niks3-server[1210]: 2026/09/23 07:01:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign5931worker1 # [ 177.978976] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.601Z level=INFO msg="Uploading 1 narinfos"5932lb # [ 178.044577] niks3-server[1210]: 2026/09/23 07:01:57 INFO Signed narinfos id=53 count=15933lb # [ 178.052205] niks3-server[1210]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5934lb # [ 178.065987] niks3-server[1210]: 2026/09/23 07:01:57 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete5935worker1 # [ 178.009858] nix-grpc-daemon[1576]: time=2026-09-23T07:01:57.633Z level=INFO msg="Upload complete. (219ms)"5936lb # [ 178.073084] niks3-server[1210]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5937lb # [ 178.075524] niks3-server[1210]: 2026/09/23 07:01:57 INFO Completed upload id=535938lb # [ 178.078905] niks3-server[1210]: 2026/09/23 07:01:57 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5939worker1 # [ 178.017271] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=4245940worker1 # [ 178.028322] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455625941worker1 # [ 178.042605] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5942worker1 # [ 178.045142] nix-daemon[852]: reaped child process 1573, status = succeeded5943client # [ 178.039221] nix[1416]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5944worker1 # [ 178.059103] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)5945lb # [ 178.287209] envoy[895]: conn peer=[2001:db8:1::4]:59704 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3483 tx=421 ms=172225946lb # [ 178.292954] envoy[895]: conn peer=[2001:db8:1::1]:39946 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4258 tx=2100 ms=97305947client: must succeed: systemctl kill -s INT intr5948worker1 # [ 179.010734] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:58Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5949client # [ 178.989910] systemd[1]: intr.service: Sent signal SIGINT to main process 1416 (nix) on client request.5950client # [ 178.999658] nix[1416]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5951client # [ 179.012486] nix[1416]: error: interrupted by the user5952client: (finished: must succeed: systemctl kill -s INT intr, in 0.07 seconds)5953client # [ 179.037283] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5954client # [ 179.042117] systemd[1]: intr.service: Failed with result 'exit-code'.5955client # [ 179.044352] systemd[1]: intr.service: Consumed 168ms CPU time over 2.655s wall clock time, 43.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5956worker1 # [ 179.097153] nix-daemon[1596]: unexpected Nix daemon error: error: interrupted by the user5957worker1 # [ 179.104623] nix-daemon[852]: reaped child process 1594, status = succeeded5958worker1 # [ 179.107087] nix-daemon[852]: reaped child process 1596, status = failed with exit code 15959(finished: subtest: interrupting the client stops the build on the worker, in 2.80 seconds)5960subtest: a deploy mid-build drains: the build finishes, the new generation takes over5961client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5962client # [ 179.211252] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5963client # Running as unit: sw.service; invocation ID: 189f693ebbf147aba338e74d845eb9105964client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)5965worker2 # [ 179.418523] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:549225966client # [ 179.420227] nix[1437]: don't know how to build these paths:5967client # [ 179.421241] nix[1437]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5968worker1 # [ 179.450172] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:59Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05969worker1 # [ 179.456309] nix-grpc-daemon[1475]: ts=2026-09-23T07:01:59Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=15970worker2 # [ 179.483335] nix-grpc-daemon[1649]: ts=2026-09-23T07:01:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:549225971worker2 # [ 179.498861] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)5972worker2 # [ 179.515899] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 2/5)5973worker2 # [ 179.624355] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 356 ms (attempt 3/5)5974worker2 # [ 179.982177] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 501 ms (attempt 4/5)5975worker2 # [ 180.490592] nix-grpc-daemon[1649]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5976worker2 # [ 180.497847] nix-grpc-daemon[1649]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5977worker2 # [ 180.516805] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:549225978client # [ 180.525162] nix[1437]: copying 1 paths to 'lb:50051'...5979worker2 # [ 180.530219] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:549225980client # [ 180.525418] nix[1437]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5981worker2 # [ 180.549991] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)5982worker2 # [ 180.689817] nix-daemon[842]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)5983worker2 # [ 180.711642] nix-daemon[842]: reaped child process 1782, status = succeeded5984lb # [ 180.787574] niks3-server[1210]: 2026/09/23 07:02:00 INFO Received uploads request method=POST path=/api/pending_closures5985worker2 # [ 180.724355] nix-grpc-daemon[1731]: time=2026-09-23T07:02:00.355Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5986worker2 # [ 180.725829] nix-grpc-daemon[1731]: time=2026-09-23T07:02:00.356Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5987lb # [ 180.819885] niks3-server[1210]: 2026/09/23 07:02:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign5988worker2 # [ 180.758886] nix-grpc-daemon[1731]: time=2026-09-23T07:02:00.390Z level=INFO msg="Uploading 1 narinfos"5989lb # [ 180.836773] niks3-server[1210]: 2026/09/23 07:02:00 INFO Signed narinfos id=54 count=15990lb # [ 180.843897] niks3-server[1210]: 2026/09/23 07:02:00 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5991lb # [ 180.857274] niks3-server[1210]: 2026/09/23 07:02:00 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5992worker2 # [ 180.791329] nix-grpc-daemon[1731]: time=2026-09-23T07:02:00.423Z level=INFO msg="Upload complete. (223ms)"5993lb # [ 180.870573] niks3-server[1210]: 2026/09/23 07:02:00 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete5994lb # [ 180.875645] niks3-server[1210]: 2026/09/23 07:02:00 INFO Completed upload id=545995worker2 # [ 180.800323] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54922 duration_s=0 paths=1 nar_bytes_in=4165996lb # [ 180.879033] niks3-server[1210]: 2026/09/23 07:02:00 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5997worker2 # [ 180.815059] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:549225998worker2 # [ 180.821863] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)5999worker2 # [ 180.826673] nix-daemon[842]: reaped child process 1777, status = succeeded6000worker2 # [ 180.837757] nix-daemon[842]: accepted connection from pid 1649, user nix-grpc-daemon (trusted)6001client # [ 180.831135] nix[1437]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6002worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26003worker2 # Checking switch inhibitors... done6004worker2 # activating the configuration...6005worker2 # [ 183.416729] nixos[1818]: switching to system configuration /nix/store/zgy718pd25v31g01575mcapmgw5qlpcy-nixos-system-worker2-test6006worker2 # [ 183.418514] systemd[1]: Stopped target Local File Systems.6007worker2 # [ 183.419302] systemd[1]: Stopped target Remote File Systems.6008worker2 # setting up /etc...6009worker2 # [ 183.949199] systemd[1]: Reload requested from client PID 1818 ('.switch-to-conf') (unit backdoor.service)...6010worker2 # [ 183.950690] systemd[1]: Reloading...6011lb # [ 188.283847] envoy[895]: conn peer=[2001:db8:1::1]:48344 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=826 ms=24616012worker2 # [ 188.858363] systemd[1]: Reloading finished in 4901 ms.6013worker2 # restarting sysinit-reactivation.target6014worker2 # [ 188.997110] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6015worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6016worker2 # [ 189.120259] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6017worker2 # [ 189.121403] systemd[1]: Reached target Reactivate sysinit units.6018worker2 # [ 189.122193] systemd[1]: Reloading D-Bus System Message Bus...6019worker2 # [ 189.123225] dbus-broker-launch[492]: Caught SIGHUP, trigger reload.6020worker2 # [ 189.140280] systemd[1]: Reloading Nix worker-protocol over gRPC...6021worker2 # [ 189.143604] dbus-broker-launch[492]: Looking up NSS user entry for 'systemd-timesync'...6022worker2 # [ 189.148697] dbus-broker-launch[492]: NSS returned no entry for 'systemd-timesync'6023worker2 # [ 189.150335] dbus-broker-launch[492]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6024worker2 # [ 189.180636] systemd[1]: Reloaded Nix worker-protocol over gRPC.6025worker2 # [ 189.186463] systemd[1]: Reloaded D-Bus System Message Bus.6026worker2 # [ 189.196381] systemd[1]: Reached target Remote File Systems.6027worker2 # [ 189.202273] systemd[1]: Starting linger-users.service...6028worker2 # [ 189.206499] systemd[1]: Reached target Local File Systems.6029worker2 # [ 189.207303] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6030worker2 # [ 189.222890] systemd[1]: Starting Create SUID/SGID Wrappers...6031worker2 # [ 189.223728] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6032worker2 # [ 189.225848] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676033worker2 # [ 189.227973] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6034worker2 # [ 189.238397] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6035worker2 # [ 189.239685] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6036worker2 # [ 189.252083] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6037worker2 # [ 189.255293] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6038worker2 # [ 189.274166] systemd[1]: linger-users.service: Deactivated successfully.6039worker2 # [ 189.275117] systemd[1]: Finished linger-users.service.6040worker2 # [ 189.563954] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6041worker2 # [ 189.568212] systemd[1]: Finished Create SUID/SGID Wrappers.6042worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6043worker2 # [ 189.819082] nixos[1818]: finished switching to system configuration /nix/store/zgy718pd25v31g01575mcapmgw5qlpcy-nixos-system-worker2-test6044worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.84 seconds)6045worker2: must succeed: pgrep -f 'read -t [9]0 x'6046worker2 # [ 189.863708] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:09Z level=info event=draining6047worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6048client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26049client # don't know how to build these paths:6050client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6051worker1 # [ 190.122775] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455626052worker1 # [ 190.130637] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626053worker1 # [ 190.155656] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06054worker1 # [ 190.162307] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16055worker1 # [ 190.212886] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626056worker1 # [ 190.227962] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626057client # copying 1 paths to 'lb:50051'...6058worker1 # [ 190.243103] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626059client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6060worker1 # [ 190.266599] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6061worker1 # [ 190.425039] nix-daemon[852]: accepted connection from pid 1647, user nix-grpc-daemon (trusted)6062worker1 # [ 190.445902] nix-daemon[852]: reaped child process 1650, status = succeeded6063lb # [ 190.513936] niks3-server[1210]: 2026/09/23 07:02:10 INFO Received uploads request method=POST path=/api/pending_closures6064worker1 # [ 190.460400] nix-grpc-daemon[1576]: time=2026-09-23T07:02:10.083Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6065worker1 # [ 190.461989] nix-grpc-daemon[1576]: time=2026-09-23T07:02:10.083Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6066lb # [ 190.544783] niks3-server[1210]: 2026/09/23 07:02:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6067worker1 # [ 190.490428] nix-grpc-daemon[1576]: time=2026-09-23T07:02:10.113Z level=INFO msg="Uploading 1 narinfos"6068lb # [ 190.559145] niks3-server[1210]: 2026/09/23 07:02:10 INFO Signed narinfos id=55 count=16069lb # [ 190.566617] niks3-server[1210]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6070lb # [ 190.583355] niks3-server[1210]: 2026/09/23 07:02:10 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6071worker1 # [ 190.531348] nix-grpc-daemon[1576]: time=2026-09-23T07:02:10.155Z level=INFO msg="Upload complete. (247ms)"6072lb # [ 190.593294] niks3-server[1210]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6073lb # [ 190.600645] niks3-server[1210]: 2026/09/23 07:02:10 INFO Completed upload id=556074lb # [ 190.603648] niks3-server[1210]: 2026/09/23 07:02:10 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6075worker1 # [ 190.540332] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=5286076worker1 # [ 190.559708] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626077worker1 # [ 190.569856] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6078worker1 # [ 190.573582] nix-daemon[852]: reaped child process 1645, status = succeeded6079client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6080client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6081worker1 # [ 190.589353] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6082client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6083client # farm-a> LOG-farm-a-during-drain6084worker1 # [ 191.627847] nix-daemon[852]: reaped child process 1654, status = succeeded6085worker1 # [ 191.724426] nix-daemon[852]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6086worker1 # [ 191.741655] nix-daemon[852]: reaped child process 1673, status = succeeded6087lb # [ 191.811409] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received uploads request method=POST path=/api/pending_closures6088worker1 # [ 191.764389] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6089worker1 # [ 191.765961] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.386Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6090lb # [ 191.856680] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6091worker1 # [ 191.804673] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.428Z level=INFO msg="Uploading 1 narinfos"6092lb # [ 191.872400] niks3-server[1210]: 2026/09/23 07:02:11 INFO Signed narinfos id=56 count=16093lb # [ 191.883779] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6094lb # [ 191.894317] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6095lb # [ 191.901677] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6096worker1 # [ 191.850358] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.474Z level=INFO msg="Upload complete. (250ms)"6097lb # [ 191.912856] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6098lb # [ 191.917525] niks3-server[1210]: 2026/09/23 07:02:11 INFO Completed upload id=566099lb # [ 191.923541] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6100worker1 # [ 191.867747] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16101worker1 # [ 191.886166] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06102worker1 # [ 191.890725] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16103worker1 # [ 191.895389] nix-daemon[852]: reaped child process 1652, status = succeeded6104worker1 # [ 191.922912] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626105worker1 # [ 191.939925] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626106worker1 # [ 191.953740] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.577Z level=INFO msg="All 1 paths already cached"6107client # copying 1 paths to 'lb:50051'...6108worker1 # [ 191.964106] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626109client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6110worker1 # [ 191.986679] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6111worker1 # [ 192.144700] nix-daemon[852]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6112worker1 # [ 192.164190] nix-daemon[852]: reaped child process 1682, status = succeeded6113lb # [ 192.224212] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received uploads request method=POST path=/api/pending_closures6114worker1 # [ 192.171468] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.794Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6115worker1 # [ 192.173205] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.794Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6116lb # [ 192.256276] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6117worker1 # [ 192.201915] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.825Z level=INFO msg="Uploading 1 narinfos"6118lb # [ 192.268338] niks3-server[1210]: 2026/09/23 07:02:11 INFO Signed narinfos id=57 count=16119lb # [ 192.271546] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6120lb # [ 192.288677] niks3-server[1210]: 2026/09/23 07:02:11 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6121lb # [ 192.296550] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6122worker1 # [ 192.239785] nix-grpc-daemon[1576]: time=2026-09-23T07:02:11.863Z level=INFO msg="Upload complete. (235ms)"6123lb # [ 192.301618] niks3-server[1210]: 2026/09/23 07:02:11 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6124lb # [ 192.305469] niks3-server[1210]: 2026/09/23 07:02:11 INFO Completed upload id=576125worker1 # [ 192.250038] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=6406126worker1 # [ 192.266030] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626127worker1 # [ 192.274926] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6128worker1 # [ 192.277920] nix-daemon[852]: reaped child process 1677, status = succeeded6129client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6130client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6131worker1 # [ 192.292363] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6132client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6133client # farm-b> LOG-farm-b-during-drain6134worker1 # [ 193.560546] nix-daemon[852]: reaped child process 1686, status = succeeded6135worker1 # [ 193.661830] nix-daemon[852]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6136worker1 # [ 193.678221] nix-daemon[852]: reaped child process 1705, status = succeeded6137lb # [ 193.748769] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received uploads request method=POST path=/api/pending_closures6138worker1 # [ 193.698132] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.320Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6139worker1 # [ 193.699748] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.320Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6140lb # [ 193.789564] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6141worker1 # [ 193.738148] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.361Z level=INFO msg="Uploading 1 narinfos"6142lb # [ 193.804484] niks3-server[1210]: 2026/09/23 07:02:13 INFO Signed narinfos id=58 count=16143lb # [ 193.807219] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6144lb # [ 193.820977] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6145lb # [ 193.824980] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6146lb # [ 193.832886] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6147lb # [ 193.838122] niks3-server[1210]: 2026/09/23 07:02:13 INFO Completed upload id=586148worker1 # [ 193.775330] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.399Z level=INFO msg="Upload complete. (271ms)"6149lb # [ 193.842091] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6150worker1 # [ 193.791175] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16151worker1 # [ 193.812454] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06152worker1 # [ 193.817914] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16153worker1 # [ 193.823592] nix-daemon[852]: reaped child process 1684, status = succeeded6154worker1 # [ 193.850724] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626155worker1 # [ 193.866066] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626156worker1 # [ 193.881091] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.504Z level=INFO msg="All 2 paths already cached"6157client # copying 1 paths to 'lb:50051'...6158client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6159worker1 # [ 193.892597] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626160worker1 # [ 193.909271] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6161worker1 # [ 194.061197] nix-daemon[852]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6162worker1 # [ 194.079043] nix-daemon[852]: reaped child process 1714, status = succeeded6163lb # [ 194.146951] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received uploads request method=POST path=/api/pending_closures6164worker1 # [ 194.095635] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.719Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6165worker1 # [ 194.097571] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.719Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6166lb # [ 194.174293] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6167worker1 # [ 194.120776] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.744Z level=INFO msg="Uploading 1 narinfos"6168lb # [ 194.189493] niks3-server[1210]: 2026/09/23 07:02:13 INFO Signed narinfos id=59 count=16169lb # [ 194.197535] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6170lb # [ 194.212794] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6171worker1 # [ 194.151077] nix-grpc-daemon[1576]: time=2026-09-23T07:02:13.774Z level=INFO msg="Upload complete. (224ms)"6172lb # [ 194.218926] niks3-server[1210]: 2026/09/23 07:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6173worker1 # [ 194.160499] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=7686174lb # [ 194.224581] niks3-server[1210]: 2026/09/23 07:02:13 INFO Completed upload id=596175lb # [ 194.229613] niks3-server[1210]: 2026/09/23 07:02:13 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6176worker1 # [ 194.175834] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626177worker1 # [ 194.182873] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6178worker1 # [ 194.188869] nix-daemon[852]: reaped child process 1709, status = succeeded6179client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6180client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6181worker1 # [ 194.203317] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6182client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6183client # farm-top> LOG-farm-top-during-drain6184worker1 # [ 195.091917] nix-daemon[852]: reaped child process 1718, status = succeeded6185worker1 # [ 195.161892] nix-daemon[852]: accepted connection from pid 1734, user nix-grpc-daemon (trusted)6186worker1 # [ 195.181369] nix-daemon[852]: reaped child process 1737, status = succeeded6187lb # [ 195.242464] niks3-server[1210]: 2026/09/23 07:02:14 INFO Received uploads request method=POST path=/api/pending_closures6188worker1 # [ 195.192433] nix-grpc-daemon[1576]: time=2026-09-23T07:02:14.814Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6189worker1 # [ 195.194891] nix-grpc-daemon[1576]: time=2026-09-23T07:02:14.814Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6190lb # [ 195.281742] niks3-server[1210]: 2026/09/23 07:02:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6191worker1 # [ 195.229407] nix-grpc-daemon[1576]: time=2026-09-23T07:02:14.853Z level=INFO msg="Uploading 1 narinfos"6192lb # [ 195.295874] niks3-server[1210]: 2026/09/23 07:02:14 INFO Signed narinfos id=60 count=16193lb # [ 195.304295] niks3-server[1210]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6194lb # [ 195.313316] niks3-server[1210]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6195lb # [ 195.319004] niks3-server[1210]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6196worker1 # [ 195.267425] nix-grpc-daemon[1576]: time=2026-09-23T07:02:14.891Z level=INFO msg="Upload complete. (253ms)"6197lb # [ 195.328749] niks3-server[1210]: 2026/09/23 07:02:14 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6198lb # [ 195.333361] niks3-server[1210]: 2026/09/23 07:02:14 INFO Completed upload id=606199lb # [ 195.337081] niks3-server[1210]: 2026/09/23 07:02:14 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6200worker1 # [ 195.281255] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16201worker1 # [ 195.299598] nix-daemon[852]: reaped child process 1716, status = succeeded6202client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.43 seconds)6203client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6204client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6205worker2: must succeed: pkill -f 'read -t [9]0 x'6206worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6207client: waiting for success: ! systemctl is-active sw6208worker2 # [ 195.704095] nix-daemon[842]: reaped child process 1786, status = succeeded6209worker2 # [ 195.837775] nix-daemon[842]: accepted connection from pid 1958, user nix-grpc-daemon (trusted)6210worker2 # [ 195.856846] nix-daemon[842]: reaped child process 1961, status = succeeded6211lb # [ 195.934899] niks3-server[1210]: 2026/09/23 07:02:15 INFO Received uploads request method=POST path=/api/pending_closures6212worker2 # [ 195.871587] nix-grpc-daemon[1731]: time=2026-09-23T07:02:15.502Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6213worker2 # [ 195.873518] nix-grpc-daemon[1731]: time=2026-09-23T07:02:15.502Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6214lb # [ 195.970970] niks3-server[1210]: 2026/09/23 07:02:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6215worker2 # [ 195.905772] nix-grpc-daemon[1731]: time=2026-09-23T07:02:15.537Z level=INFO msg="Uploading 1 narinfos"6216lb # [ 195.986926] niks3-server[1210]: 2026/09/23 07:02:15 INFO Signed narinfos id=61 count=16217lb # [ 195.992583] niks3-server[1210]: 2026/09/23 07:02:15 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6218lb # [ 196.001095] niks3-server[1210]: 2026/09/23 07:02:15 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6219lb # [ 196.010259] niks3-server[1210]: 2026/09/23 07:02:15 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6220worker2 # [ 195.935968] nix-grpc-daemon[1731]: time=2026-09-23T07:02:15.568Z level=INFO msg="Upload complete. (268ms)"6221lb # [ 196.016777] niks3-server[1210]: 2026/09/23 07:02:15 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6222lb # [ 196.021858] niks3-server[1210]: 2026/09/23 07:02:15 INFO Completed upload id=616223lb # [ 196.024390] niks3-server[1210]: 2026/09/23 07:02:15 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6224worker2 # [ 195.949214] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54922 duration_s=15 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16225worker2 # [ 195.966009] nix-daemon[842]: reaped child process 1784, status = succeeded6226client # [ 195.985069] systemd[1]: sw.service: Deactivated successfully.6227client # [ 195.989492] systemd[1]: sw.service: Consumed 200ms CPU time over 16.777s wall clock time, 41.4M memory peak, 7.5K incoming IP traffic, 8.5K outgoing IP traffic.6228client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6229client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6230client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6231worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6232worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.06 seconds)6233lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6234lb: must succeed: curl -sf localhost:9901/clusters6235lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6236worker2 # [ 196.866107] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:16Z level=info event=signal_exit6237lb: must succeed: curl -sf localhost:9901/clusters6238worker1 # [ 197.891818] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:17Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:455366239worker2 # [ 197.880994] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:17Z level=info event=scheduler_restarting6240worker1 # [ 197.899999] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:17Z level=info event=worker_gone addr=192.168.1.4:500516241lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6242worker2 # [ 197.907680] nix-grpc-daemon[1649]: ts=2026-09-23T07:02:17Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6243worker2 # [ 197.927623] nix-daemon[842]: reaped child process 1702, status = succeeded6244lb # [ 198.284749] envoy[895]: conn peer=[2001:db8:1::1]:51426 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9578 tx=4383 ms=51786245lb # [ 198.289789] envoy[895]: conn peer=[2001:db8:1::1]:48350 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3021 tx=1561 ms=165516246lb # [ 198.294668] envoy[895]: conn peer=[2001:db8:1::4]:52180 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7599 tx=679 ms=279716247worker2 # [ 198.433300] nix-grpc-daemon[1649]: E0000 00:00:1790146938.065321 1695 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true6248lb: must succeed: curl -sf localhost:9901/clusters6249lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6250worker2 # [ 199.897242] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6251worker2 # [ 199.903491] systemd[1]: nix-grpc-daemon.service: Consumed 784ms CPU time over 30.318s wall clock time, 36.1M memory peak, 22.6K incoming IP traffic, 36.7K outgoing IP traffic.6252lb: must succeed: curl -sf localhost:9901/clusters6253lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6254worker2 # [ 200.892166] systemd[1]: Starting scheduler-token.service...6255worker2 # [ 200.976402] systemd[1]: scheduler-token.service: Deactivated successfully.6256worker2 # [ 200.984773] systemd[1]: Finished scheduler-token.service.6257worker2 # [ 200.986777] systemd[1]: scheduler-token.service: Consumed 56ms CPU time over 88ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6258worker2 # [ 201.007530] systemd[1]: Starting Nix worker-protocol over gRPC...6259lb: must succeed: curl -sf localhost:9901/clusters6260lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6261worker2 # [ 201.118987] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6262worker2 # [ 201.121293] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6263worker2 # [ 201.125058] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16264worker2 # [ 201.130291] nix-grpc-daemon[1976]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6265worker2 # [ 201.131720] nix-grpc-daemon[1976]: E0000 00:00:1790146940.761719 1976 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6266worker2 # [ 201.175510] nix-grpc-daemon[1976]: E0000 00:00:1790146940.767663 1976 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6267worker2 # [ 201.178434] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6268worker2 # [ 201.180279] nix-grpc-daemon[1976]: E0000 00:00:1790146940.775749 2016 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6269worker2 # [ 201.182590] nix-grpc-daemon[1976]: E0000 00:00:1790146940.792600 2016 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6270worker2 # [ 201.187654] nix-grpc-daemon[1976]: E0000 00:00:1790146940.792732 2016 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6271worker2 # [ 201.192067] nix-grpc-daemon[1976]: E0000 00:00:1790146940.807003 2016 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6272worker2 # [ 201.194576] systemd[1]: Started Nix worker-protocol over gRPC.6273worker2 # [ 201.195412] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:20Z level=info event=scheduler_connected addr=lb:500516274worker1 # [ 201.207869] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:20Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:455366275worker1 # [ 201.217091] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:20Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06276lb: must succeed: curl -sf localhost:9901/clusters6277lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6278lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 5.40 seconds)6279lb: must succeed: curl -sf localhost:9901/clusters6280lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6281worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6282worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6283(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 23.12 seconds)6284subtest: queue gauges show a burst while every build blocks6285lb: must succeed: curl -sf localhost:9901/clusters6286lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6287lb: must succeed: curl -sf localhost:9901/clusters6288lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6289worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6290worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6291client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6292client # [ 202.499382] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6293client # Running as unit: burst.service; invocation ID: 35ae9d3fa2b0405395cb14da9678d6716294client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6295lb: must succeed: curl -sf localhost:9901/clusters6296lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6297lb: must succeed: curl -sf localhost:9901/clusters6298lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6299worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6300worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6301worker1 # [ 202.723052] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455626302client # [ 202.712168] nix[1493]: don't know how to build these paths:6303client # [ 202.713905] nix[1493]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6304client # [ 202.715151] nix[1493]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6305client # [ 202.716898] nix[1493]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6306client # [ 202.718149] nix[1493]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6307worker1 # [ 202.743565] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06308worker1 # [ 202.750683] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16309worker1 # [ 202.756962] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06310worker1 # [ 202.762689] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16311worker1 # [ 202.768073] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06312worker1 # [ 202.772293] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06313worker1 # [ 202.791105] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626314worker2 # [ 202.785870] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498306315worker1 # [ 202.807134] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626316worker2 # [ 202.799441] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6317client # [ 202.804249] nix[1493]: copying 1 paths to 'lb:50051'...6318worker1 # [ 202.821662] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626319client # [ 202.806563] nix[1493]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6320worker1 # [ 202.838201] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6321worker2 # [ 202.824080] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 1/5)6322worker2 # [ 202.920794] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)6323worker2 # [ 202.958622] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 267 ms (attempt 3/5)6324worker1 # [ 202.987520] nix-daemon[852]: accepted connection from pid 1752, user nix-grpc-daemon (trusted)6325worker1 # [ 203.005935] nix-daemon[852]: reaped child process 1755, status = succeeded6326lb # [ 203.075223] niks3-server[1210]: 2026/09/23 07:02:22 INFO Received uploads request method=POST path=/api/pending_closures6327worker1 # [ 203.023705] nix-grpc-daemon[1576]: time=2026-09-23T07:02:22.646Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6328worker1 # [ 203.025451] nix-grpc-daemon[1576]: time=2026-09-23T07:02:22.646Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6329lb # [ 203.108609] niks3-server[1210]: 2026/09/23 07:02:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6330worker1 # [ 203.056452] nix-grpc-daemon[1576]: time=2026-09-23T07:02:22.680Z level=INFO msg="Uploading 1 narinfos"6331lb # [ 203.120625] niks3-server[1210]: 2026/09/23 07:02:22 INFO Signed narinfos id=62 count=16332lb # [ 203.129960] niks3-server[1210]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6333lb # [ 203.146399] niks3-server[1210]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6334lb # [ 203.160183] niks3-server[1210]: 2026/09/23 07:02:22 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6335lb # [ 203.167365] niks3-server[1210]: 2026/09/23 07:02:22 INFO Completed upload id=626336worker1 # [ 203.104638] nix-grpc-daemon[1576]: time=2026-09-23T07:02:22.728Z level=INFO msg="Upload complete. (251ms)"6337lb # [ 203.174831] niks3-server[1210]: 2026/09/23 07:02:22 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6338worker1 # [ 203.116358] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=4166339worker1 # [ 203.132817] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626340worker1 # [ 203.140543] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6341worker1 # [ 203.147076] nix-daemon[852]: reaped child process 1750, status = succeeded6342client # [ 203.140961] nix[1493]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6343worker1 # [ 203.160166] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6344worker2 # [ 203.227620] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 4/5)6345worker2 # [ 203.371208] nix-grpc-daemon[1976]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6346worker2 # [ 203.376923] nix-grpc-daemon[1976]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6347worker2 # [ 203.394437] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498306348client # [ 203.404251] nix[1493]: copying 1 paths to 'lb:50051'...6349worker2 # [ 203.409819] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:498306350client # [ 203.406556] nix[1493]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6351worker2 # [ 203.426187] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6352worker2 # [ 203.451497] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=info event=niks3_push_started pid=20376353worker2 # [ 203.598681] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.230Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6354lb: must succeed: curl -sf localhost:9901/clusters6355worker2 # [ 203.703170] nix-daemon[842]: accepted connection from pid 2050, user nix-grpc-daemon (trusted)6356worker2 # [ 203.721562] nix-daemon[842]: reaped child process 2053, status = succeeded6357lb # [ 203.799378] niks3-server[1210]: 2026/09/23 07:02:23 INFO Received uploads request method=POST path=/api/pending_closures6358worker2 # [ 203.738576] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.369Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6359worker2 # [ 203.742058] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.369Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6360lb # [ 203.854204] niks3-server[1210]: 2026/09/23 07:02:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6361worker2 # [ 203.786714] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.417Z level=INFO msg="Uploading 1 narinfos"6362lb # [ 203.865043] niks3-server[1210]: 2026/09/23 07:02:23 INFO Signed narinfos id=63 count=16363lb # [ 203.870914] niks3-server[1210]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6364lb # [ 203.877516] niks3-server[1210]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6365lb # [ 203.888449] niks3-server[1210]: 2026/09/23 07:02:23 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6366worker2 # [ 203.819745] nix-grpc-daemon[2037]: time=2026-09-23T07:02:23.451Z level=INFO msg="Upload complete. (217ms)"6367lb # [ 203.899420] niks3-server[1210]: 2026/09/23 07:02:23 INFO Completed upload id=636368worker2 # [ 203.829821] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=4166369lb # [ 203.903703] niks3-server[1210]: 2026/09/23 07:02:23 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6370lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.13 seconds)6371lb: must succeed: curl -sf localhost:9901/clusters6372worker2 # [ 203.840475] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498306373worker2 # [ 203.851466] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6374worker2 # [ 203.856262] nix-daemon[842]: reaped child process 2035, status = succeeded6375worker2 # [ 203.865362] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6376client # [ 203.858155] nix[1493]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6378worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6379worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)6380worker1 # [ 205.415065] nix-daemon[852]: reaped child process 1759, status = succeeded6381worker1 # [ 205.542499] nix-daemon[852]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6382worker1 # [ 205.562318] nix-daemon[852]: reaped child process 1790, status = succeeded6383lb # [ 205.629258] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received uploads request method=POST path=/api/pending_closures6384worker2 # [ 205.560954] nix-daemon[842]: reaped child process 2057, status = succeeded6385worker1 # [ 205.580098] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6386worker1 # [ 205.581899] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.200Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6387worker1 # [ 205.619585] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.243Z level=INFO msg="Uploading 1 narinfos"6388lb # [ 205.671996] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6389lb # [ 205.685408] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6390lb # [ 205.696105] niks3-server[1210]: 2026/09/23 07:02:25 INFO Signed narinfos id=64 count=16391lb # [ 205.703694] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6392worker1 # [ 205.647760] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.271Z level=INFO msg="Upload complete. (272ms)"6393lb # [ 205.711487] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6394lb # [ 205.714743] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6395lb # [ 205.719348] niks3-server[1210]: 2026/09/23 07:02:25 INFO Completed upload id=646396worker2 # [ 205.649634] nix-daemon[842]: accepted connection from pid 2082, user nix-grpc-daemon (trusted)6397lb # [ 205.722024] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6398worker1 # [ 205.661485] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=16399worker1 # [ 205.676324] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=16400worker2 # [ 205.668333] nix-daemon[842]: reaped child process 2085, status = succeeded6401worker1 # [ 205.681481] nix-daemon[852]: reaped child process 1757, status = succeeded6402lb # [ 205.745736] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received uploads request method=POST path=/api/pending_closures6403worker2 # [ 205.682329] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.312Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6404worker2 # [ 205.683961] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.312Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6405worker1 # [ 205.703010] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626406lb # [ 205.781107] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6407lb # [ 205.791438] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6408worker1 # [ 205.728246] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626409worker2 # [ 205.726940] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.358Z level=INFO msg="Uploading 1 narinfos"6410lb # [ 205.802039] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6411lb # [ 205.810702] niks3-server[1210]: 2026/09/23 07:02:25 INFO Signed narinfos id=65 count=16412client # [ 205.733117] nix[1493]: copying 1 paths to 'lb:50051'...6413worker1 # [ 205.749283] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626414client # [ 205.733362] nix[1493]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6415lb # [ 205.816270] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6416lb # [ 205.824119] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6417worker1 # [ 205.763028] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6418lb # [ 205.830052] niks3-server[1210]: 2026/09/23 07:02:25 INFO Completed upload id=656419lb # [ 205.832639] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6420worker2 # [ 205.756272] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.388Z level=INFO msg="Upload complete. (278ms)"6421worker1 # [ 205.775826] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=16422worker2 # [ 205.773152] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=16423worker2 # [ 205.788936] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498306424worker2 # [ 205.799865] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498306425worker2 # [ 205.805324] nix-daemon[842]: reaped child process 2055, status = succeeded6426client # [ 205.800790] nix[1493]: copying 1 paths to 'lb:50051'...6427worker2 # [ 205.811326] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:498306428client # [ 205.806268] nix[1493]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6429worker2 # [ 205.825091] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6430worker1 # [ 205.924762] nix-daemon[852]: accepted connection from pid 1794, user nix-grpc-daemon (trusted)6431worker1 # [ 205.945643] nix-daemon[852]: reaped child process 1797, status = succeeded6432lb # [ 206.005718] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received uploads request method=POST path=/api/pending_closures6433worker1 # [ 205.952811] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.576Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6434worker1 # [ 205.954289] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.576Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6435worker2 # [ 205.959385] nix-daemon[842]: accepted connection from pid 2089, user nix-grpc-daemon (trusted)6436lb # [ 206.033463] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6437worker1 # [ 205.977727] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.601Z level=INFO msg="Uploading 1 narinfos"6438worker2 # [ 205.974733] nix-daemon[842]: reaped child process 2092, status = succeeded6439lb # [ 206.047220] niks3-server[1210]: 2026/09/23 07:02:25 INFO Signed narinfos id=66 count=16440lb # [ 206.058722] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6441worker2 # [ 205.991500] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.623Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6442worker2 # [ 205.993159] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.623Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6443lb # [ 206.068317] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received uploads request method=POST path=/api/pending_closures6444lb # [ 206.074662] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6445lb # [ 206.083861] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6446worker2 # [ 206.018935] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.650Z level=INFO msg="Uploading 1 narinfos"6447worker1 # [ 206.029157] nix-grpc-daemon[1576]: time=2026-09-23T07:02:25.652Z level=INFO msg="Upload complete. (242ms)"6448lb # [ 206.093421] niks3-server[1210]: 2026/09/23 07:02:25 INFO Completed upload id=666449lb # [ 206.095501] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6450worker1 # [ 206.040352] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=4166451lb # [ 206.109712] niks3-server[1210]: 2026/09/23 07:02:25 INFO Signed narinfos id=67 count=16452lb # [ 206.111960] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6453worker1 # [ 206.053260] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626454lb # [ 206.118852] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6455lb # [ 206.126192] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6456worker2 # [ 206.056585] nix-grpc-daemon[2037]: time=2026-09-23T07:02:25.688Z level=INFO msg="Upload complete. (222ms)"6457lb # [ 206.131004] niks3-server[1210]: 2026/09/23 07:02:25 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6458worker1 # [ 206.070342] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6459worker1 # [ 206.074822] nix-daemon[852]: reaped child process 1792, status = succeeded6460lb # [ 206.136242] niks3-server[1210]: 2026/09/23 07:02:25 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6461lb # [ 206.140199] niks3-server[1210]: 2026/09/23 07:02:25 INFO Completed upload id=676462worker2 # [ 206.067228] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=4166463client # [ 206.069989] nix[1493]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6464worker1 # [ 206.089414] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6465worker2 # [ 206.080570] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498306466worker2 # [ 206.089429] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6467worker2 # [ 206.092488] nix-daemon[842]: reaped child process 2087, status = succeeded6468client # [ 206.096692] nix[1493]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6469worker2 # [ 206.106015] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6470worker1 # [ 207.730243] nix-daemon[852]: reaped child process 1801, status = succeeded6471worker2 # [ 207.731772] nix-daemon[842]: reaped child process 2096, status = succeeded6472worker2 # [ 207.827428] nix-daemon[842]: accepted connection from pid 2121, user nix-grpc-daemon (trusted)6473worker1 # [ 207.841220] nix-daemon[852]: accepted connection from pid 1826, user nix-grpc-daemon (trusted)6474worker2 # [ 207.843616] nix-daemon[842]: reaped child process 2124, status = succeeded6475worker1 # [ 207.860685] nix-daemon[852]: reaped child process 1829, status = succeeded6476lb # [ 207.925025] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received uploads request method=POST path=/api/pending_closures6477worker2 # [ 207.862751] nix-grpc-daemon[2037]: time=2026-09-23T07:02:27.494Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6478worker2 # [ 207.864400] nix-grpc-daemon[2037]: time=2026-09-23T07:02:27.494Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6479worker1 # [ 207.880000] nix-grpc-daemon[1576]: time=2026-09-23T07:02:27.500Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6480worker1 # [ 207.881571] nix-grpc-daemon[1576]: time=2026-09-23T07:02:27.500Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6481lb # [ 207.931319] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received uploads request method=POST path=/api/pending_closures6482lb # [ 207.988259] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6483worker2 # [ 207.928810] nix-grpc-daemon[2037]: time=2026-09-23T07:02:27.560Z level=INFO msg="Uploading 1 narinfos"6484worker1 # [ 207.945161] nix-grpc-daemon[1576]: time=2026-09-23T07:02:27.568Z level=INFO msg="Uploading 1 narinfos"6485lb # [ 208.011758] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6486lb # [ 208.021186] niks3-server[1210]: 2026/09/23 07:02:27 INFO Signed narinfos id=68 count=16487lb # [ 208.023344] niks3-server[1210]: 2026/09/23 07:02:27 INFO Signed narinfos id=69 count=16488lb # [ 208.029849] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6489worker2 # [ 207.965999] nix-grpc-daemon[2037]: time=2026-09-23T07:02:27.598Z level=INFO msg="Upload complete. (249ms)"6490lb # [ 208.038715] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6491lb # [ 208.049221] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6492worker2 # [ 207.977103] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=16493lb # [ 208.057732] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6494worker1 # [ 207.996533] nix-grpc-daemon[1576]: time=2026-09-23T07:02:27.620Z level=INFO msg="Upload complete. (292ms)"6495lb # [ 208.062849] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6496lb # [ 208.066603] niks3-server[1210]: 2026/09/23 07:02:27 INFO Completed upload id=686497lb # [ 208.068719] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6498lb # [ 208.071725] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6499worker2 # [ 207.998238] nix-daemon[842]: reaped child process 2094, status = succeeded6500lb # [ 208.075598] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6501lb # [ 208.078569] niks3-server[1210]: 2026/09/23 07:02:27 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6502lb # [ 208.081801] niks3-server[1210]: 2026/09/23 07:02:27 INFO Completed upload id=696503lb # [ 208.083496] niks3-server[1210]: 2026/09/23 07:02:27 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6504worker1 # [ 208.012993] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=16505worker1 # [ 208.029423] nix-daemon[852]: reaped child process 1799, status = succeeded6506client # [ 208.028520] systemd[1]: burst.service: Deactivated successfully.6507client # [ 208.034077] systemd[1]: burst.service: Consumed 221ms CPU time over 5.530s wall clock time, 53.6M memory peak, 14.2K incoming IP traffic, 18.4K outgoing IP traffic.6508lb # [ 208.286582] envoy[895]: conn peer=[2001:db8:1::1]:34700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9654 tx=5168 ms=53026509client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6510client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6511lb: must succeed: curl -sf localhost:9901/clusters6512lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6513lb: must succeed: curl -sf localhost:9901/clusters6514lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6515worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6516worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6517(finished: subtest: queue gauges show a burst while every build blocks, in 6.91 seconds)6518subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6519client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16520worker1 # [ 209.410717] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:455626521worker2 # [ 209.403246] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498306522worker1 # [ 209.453083] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06523worker1 # [ 209.459814] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16524worker1 # [ 209.466538] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06525worker1 # [ 209.471831] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06526worker1 # [ 209.499100] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626527worker1 # [ 209.514172] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626528worker1 # [ 209.528880] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626529worker1 # [ 209.543442] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6530worker1 # [ 209.695777] nix-daemon[852]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)6531worker1 # [ 209.716873] nix-daemon[852]: reaped child process 1845, status = succeeded6532lb # [ 209.777539] niks3-server[1210]: 2026/09/23 07:02:29 INFO Received uploads request method=POST path=/api/pending_closures6533worker1 # [ 209.726798] nix-grpc-daemon[1576]: time=2026-09-23T07:02:29.349Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6534worker1 # [ 209.728459] nix-grpc-daemon[1576]: time=2026-09-23T07:02:29.349Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6535lb # [ 209.813035] niks3-server[1210]: 2026/09/23 07:02:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6536worker1 # [ 209.758360] nix-grpc-daemon[1576]: time=2026-09-23T07:02:29.381Z level=INFO msg="Uploading 1 narinfos"6537lb # [ 209.824005] niks3-server[1210]: 2026/09/23 07:02:29 INFO Signed narinfos id=70 count=16538lb # [ 209.826721] niks3-server[1210]: 2026/09/23 07:02:29 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6539worker1 # [ 209.780842] nix-grpc-daemon[1576]: time=2026-09-23T07:02:29.404Z level=INFO msg="Upload complete. (221ms)"6540lb # [ 209.839189] niks3-server[1210]: 2026/09/23 07:02:29 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6541lb # [ 209.848250] niks3-server[1210]: 2026/09/23 07:02:29 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6542lb # [ 209.852582] niks3-server[1210]: 2026/09/23 07:02:29 INFO Completed upload id=706543worker1 # [ 209.788271] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=5126544lb # [ 209.854645] niks3-server[1210]: 2026/09/23 07:02:29 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6545worker1 # [ 209.802808] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626546worker1 # [ 209.811818] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6547worker1 # [ 209.814623] nix-daemon[852]: reaped child process 1840, status = succeeded6548worker1 # [ 209.828129] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6549worker1 # [ 211.326367] nix-daemon[852]: reaped child process 1849, status = succeeded6550worker1 # [ 211.397793] nix-daemon[852]: accepted connection from pid 1865, user nix-grpc-daemon (trusted)6551worker1 # [ 211.413621] nix-daemon[852]: reaped child process 1868, status = succeeded6552lb # [ 211.481956] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received uploads request method=POST path=/api/pending_closures6553worker1 # [ 211.430700] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.053Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6554worker1 # [ 211.432910] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.053Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6555lb # [ 211.524219] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6556worker1 # [ 211.476711] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.100Z level=INFO msg="Uploading 1 narinfos"6557lb # [ 211.539208] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6558lb # [ 211.548324] niks3-server[1210]: 2026/09/23 07:02:31 INFO Signed narinfos id=71 count=16559lb # [ 211.550526] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6560lb # [ 211.560629] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6561lb # [ 211.563928] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6562lb # [ 211.568683] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6563lb # [ 211.572607] niks3-server[1210]: 2026/09/23 07:02:31 INFO Completed upload id=716564worker1 # [ 211.510719] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.134Z level=INFO msg="Upload complete. (271ms)"6565worker1 # [ 211.529205] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=16566worker1 # [ 211.548884] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16567worker1 # [ 211.558448] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626568worker1 # [ 211.564401] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06569worker1 # [ 211.576880] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626570worker1 # [ 211.587233] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626571worker1 # [ 211.593703] nix-daemon[852]: reaped child process 1847, status = succeeded6572worker1 # [ 211.598552] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6573worker1 # [ 211.716567] nix-daemon[852]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6574worker1 # [ 211.732830] nix-daemon[852]: reaped child process 1877, status = succeeded6575lb # [ 211.799566] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received uploads request method=POST path=/api/pending_closures6576worker1 # [ 211.746319] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.369Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6577worker1 # [ 211.747784] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.369Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6578lb # [ 211.828087] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6579worker1 # [ 211.772148] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.395Z level=INFO msg="Uploading 1 narinfos"6580lb # [ 211.842573] niks3-server[1210]: 2026/09/23 07:02:31 INFO Signed narinfos id=72 count=16581lb # [ 211.850175] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6582worker1 # [ 211.801081] nix-grpc-daemon[1576]: time=2026-09-23T07:02:31.424Z level=INFO msg="Upload complete. (208ms)"6583lb # [ 211.863158] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6584lb # [ 211.868269] niks3-server[1210]: 2026/09/23 07:02:31 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6585worker1 # [ 211.808144] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=5126586lb # [ 211.875431] niks3-server[1210]: 2026/09/23 07:02:31 INFO Completed upload id=726587lb # [ 211.877870] niks3-server[1210]: 2026/09/23 07:02:31 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6588worker1 # [ 211.821730] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626589worker1 # [ 211.827342] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6590worker1 # [ 211.831839] nix-daemon[852]: reaped child process 1872, status = succeeded6591worker1 # [ 211.841758] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6592worker1 # [ 213.121197] nix-daemon[852]: reaped child process 1881, status = succeeded6593worker1 # [ 213.230646] nix-daemon[852]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6594worker1 # [ 213.247935] nix-daemon[852]: reaped child process 1900, status = succeeded6595lb # [ 213.316946] niks3-server[1210]: 2026/09/23 07:02:32 INFO Received uploads request method=POST path=/api/pending_closures6596worker1 # [ 213.268428] nix-grpc-daemon[1576]: time=2026-09-23T07:02:32.888Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6597worker1 # [ 213.270714] nix-grpc-daemon[1576]: time=2026-09-23T07:02:32.888Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6598lb # [ 213.359258] niks3-server[1210]: 2026/09/23 07:02:32 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6599worker1 # [ 213.308922] nix-grpc-daemon[1576]: time=2026-09-23T07:02:32.932Z level=INFO msg="Uploading 1 narinfos"6600lb # [ 213.371161] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6601lb # [ 213.382709] niks3-server[1210]: 2026/09/23 07:02:33 INFO Signed narinfos id=73 count=16602lb # [ 213.387445] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6603lb # [ 213.395838] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6604worker1 # [ 213.342974] nix-grpc-daemon[1576]: time=2026-09-23T07:02:32.966Z level=INFO msg="Upload complete. (285ms)"6605lb # [ 213.405917] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6606lb # [ 213.409676] niks3-server[1210]: 2026/09/23 07:02:33 INFO Completed upload id=736607lb # [ 213.414562] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6608worker1 # [ 213.362745] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=16609worker1 # [ 213.378929] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:32Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16610worker1 # [ 213.384595] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:32Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06611worker1 # [ 213.389793] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626612worker1 # [ 213.394720] nix-daemon[852]: reaped child process 1879, status = succeeded6613worker1 # [ 213.396632] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626614worker1 # [ 213.404545] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626615worker1 # [ 213.414633] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6616worker1 # [ 213.546585] nix-daemon[852]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6617worker1 # [ 213.567041] nix-daemon[852]: reaped child process 1909, status = succeeded6618lb # [ 213.629497] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received uploads request method=POST path=/api/pending_closures6619worker1 # [ 213.577046] nix-grpc-daemon[1576]: time=2026-09-23T07:02:33.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6620worker1 # [ 213.578519] nix-grpc-daemon[1576]: time=2026-09-23T07:02:33.200Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6621lb # [ 213.660622] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6622worker1 # [ 213.611018] nix-grpc-daemon[1576]: time=2026-09-23T07:02:33.234Z level=INFO msg="Uploading 1 narinfos"6623lb # [ 213.673661] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6624lb # [ 213.685190] niks3-server[1210]: 2026/09/23 07:02:33 INFO Signed narinfos id=74 count=16625lb # [ 213.687818] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6626lb # [ 213.700831] niks3-server[1210]: 2026/09/23 07:02:33 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6627lb # [ 213.705044] niks3-server[1210]: 2026/09/23 07:02:33 INFO Completed upload id=746628worker1 # [ 213.643210] nix-grpc-daemon[1576]: time=2026-09-23T07:02:33.266Z level=INFO msg="Upload complete. (219ms)"6629lb # [ 213.708262] niks3-server[1210]: 2026/09/23 07:02:33 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6630worker1 # [ 213.652310] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=5126631worker1 # [ 213.667121] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626632worker1 # [ 213.675741] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6633worker1 # [ 213.678928] nix-daemon[852]: reaped child process 1904, status = succeeded6634worker1 # [ 213.690117] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6635worker1 # [ 214.351282] nix-daemon[852]: reaped child process 1913, status = succeeded6636worker1 # [ 214.482575] nix-daemon[852]: accepted connection from pid 1929, user nix-grpc-daemon (trusted)6637worker1 # [ 214.500192] nix-daemon[852]: reaped child process 1932, status = succeeded6638lb # [ 214.569903] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received uploads request method=POST path=/api/pending_closures6639worker1 # [ 214.516432] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.139Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6640worker1 # [ 214.518064] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.139Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6641lb # [ 214.614437] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6642worker1 # [ 214.564753] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.188Z level=INFO msg="Uploading 1 narinfos"6643lb # [ 214.628428] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6644lb # [ 214.640465] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6645lb # [ 214.649979] niks3-server[1210]: 2026/09/23 07:02:34 INFO Signed narinfos id=75 count=16646worker1 # [ 214.598239] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.221Z level=INFO msg="Upload complete. (296ms)"6647lb # [ 214.660555] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6648lb # [ 214.665375] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6649lb # [ 214.673536] niks3-server[1210]: 2026/09/23 07:02:34 INFO Completed upload id=756650lb # [ 214.676517] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6651worker1 # [ 214.614301] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=16652worker1 # [ 214.637579] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16653worker1 # [ 214.644728] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626654worker1 # [ 214.650858] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06655worker1 # [ 214.657342] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626656worker1 # [ 214.662907] nix-daemon[852]: reaped child process 1911, status = succeeded6657worker1 # [ 214.665419] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.285Z level=INFO msg="All 1 paths already cached"6658worker1 # [ 214.680244] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626659worker1 # [ 214.683747] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6660worker1 # [ 214.815532] nix-daemon[852]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)6661worker1 # [ 214.830077] nix-daemon[852]: reaped child process 1941, status = succeeded6662lb # [ 214.899028] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received uploads request method=POST path=/api/pending_closures6663worker1 # [ 214.847527] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.471Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6664worker1 # [ 214.849223] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.471Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6665lb # [ 214.932225] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6666worker1 # [ 214.880717] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.504Z level=INFO msg="Uploading 1 narinfos"6667lb # [ 214.945971] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6668lb # [ 214.960349] niks3-server[1210]: 2026/09/23 07:02:34 INFO Signed narinfos id=76 count=16669worker1 # [ 214.898928] nix-grpc-daemon[1576]: time=2026-09-23T07:02:34.522Z level=INFO msg="Upload complete. (208ms)"6670lb # [ 214.964475] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6671worker1 # [ 214.906513] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=6246672lb # [ 214.971942] niks3-server[1210]: 2026/09/23 07:02:34 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6673lb # [ 214.976970] niks3-server[1210]: 2026/09/23 07:02:34 INFO Completed upload id=766674lb # [ 214.979422] niks3-server[1210]: 2026/09/23 07:02:34 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6675worker1 # [ 214.919646] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626676worker1 # [ 214.925944] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6677worker1 # [ 214.930917] nix-daemon[852]: reaped child process 1936, status = succeeded6678worker1 # [ 214.941861] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6679worker1 # [ 216.408923] nix-daemon[852]: reaped child process 1945, status = succeeded6680worker1 # [ 216.522773] nix-daemon[852]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)6681worker1 # [ 216.542536] nix-daemon[852]: reaped child process 1964, status = succeeded6682lb # [ 216.608834] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received uploads request method=POST path=/api/pending_closures6683worker1 # [ 216.560428] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.181Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6684worker1 # [ 216.562154] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.181Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6685lb # [ 216.666138] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6686worker1 # [ 216.613004] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.236Z level=INFO msg="Uploading 1 narinfos"6687lb # [ 216.678763] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6688lb # [ 216.687068] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6689lb # [ 216.697735] niks3-server[1210]: 2026/09/23 07:02:36 INFO Signed narinfos id=77 count=16690worker1 # [ 216.640526] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.264Z level=INFO msg="Upload complete. (286ms)"6691lb # [ 216.703380] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6692lb # [ 216.707533] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6693lb # [ 216.712446] niks3-server[1210]: 2026/09/23 07:02:36 INFO Completed upload id=776694lb # [ 216.715640] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6695worker1 # [ 216.657714] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=16696worker1 # [ 216.678897] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16697worker1 # [ 216.685552] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626698worker1 # [ 216.691264] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06699worker1 # [ 216.696880] nix-daemon[852]: reaped child process 1943, status = succeeded6700worker1 # [ 216.700854] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626701worker1 # [ 216.706062] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.328Z level=INFO msg="All 1 paths already cached"6702worker1 # [ 216.710165] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626703worker1 # [ 216.726614] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6704worker1 # [ 216.858905] nix-daemon[852]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)6705worker1 # [ 216.874988] nix-daemon[852]: reaped child process 1973, status = succeeded6706lb # [ 216.941738] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received uploads request method=POST path=/api/pending_closures6707worker1 # [ 216.890944] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.514Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6708worker1 # [ 216.892516] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.514Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6709lb # [ 216.974022] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6710worker1 # [ 216.923600] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.547Z level=INFO msg="Uploading 1 narinfos"6711lb # [ 216.990572] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6712lb # [ 216.999535] niks3-server[1210]: 2026/09/23 07:02:36 INFO Signed narinfos id=78 count=16713worker1 # [ 216.945202] nix-grpc-daemon[1576]: time=2026-09-23T07:02:36.569Z level=INFO msg="Upload complete. (213ms)"6714lb # [ 217.008518] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6715worker1 # [ 216.951643] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=6246716lb # [ 217.016819] niks3-server[1210]: 2026/09/23 07:02:36 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6717lb # [ 217.020986] niks3-server[1210]: 2026/09/23 07:02:36 INFO Completed upload id=786718lb # [ 217.023669] niks3-server[1210]: 2026/09/23 07:02:36 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6719worker1 # [ 216.964414] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626720worker1 # [ 216.971000] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6721worker1 # [ 216.975704] nix-daemon[852]: reaped child process 1968, status = succeeded6722worker1 # [ 216.986023] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6723worker1 # [ 217.843162] nix-daemon[852]: reaped child process 1977, status = succeeded6724worker1 # [ 217.975931] nix-daemon[852]: accepted connection from pid 1993, user nix-grpc-daemon (trusted)6725worker1 # [ 217.991934] nix-daemon[852]: reaped child process 1996, status = succeeded6726lb # [ 218.061906] niks3-server[1210]: 2026/09/23 07:02:37 INFO Received uploads request method=POST path=/api/pending_closures6727worker1 # [ 218.008391] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.631Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6728worker1 # [ 218.010014] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.631Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6729lb # [ 218.101744] niks3-server[1210]: 2026/09/23 07:02:37 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6730lb # [ 218.117427] niks3-server[1210]: 2026/09/23 07:02:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6731worker1 # [ 218.063077] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.686Z level=INFO msg="Uploading 1 narinfos"6732lb # [ 218.128885] niks3-server[1210]: 2026/09/23 07:02:37 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6733lb # [ 218.140140] niks3-server[1210]: 2026/09/23 07:02:37 INFO Signed narinfos id=79 count=16734lb # [ 218.145270] niks3-server[1210]: 2026/09/23 07:02:37 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6735worker1 # [ 218.093368] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.717Z level=INFO msg="Upload complete. (296ms)"6736lb # [ 218.155623] niks3-server[1210]: 2026/09/23 07:02:37 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6737lb # [ 218.162085] niks3-server[1210]: 2026/09/23 07:02:37 INFO Completed upload id=796738lb # [ 218.169462] niks3-server[1210]: 2026/09/23 07:02:37 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6739worker1 # [ 218.109831] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=16740worker1 # [ 218.130043] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16741worker1 # [ 218.140793] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626742worker1 # [ 218.146011] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06743worker1 # [ 218.151285] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626744worker1 # [ 218.156326] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.779Z level=INFO msg="All 1 paths already cached"6745worker1 # [ 218.159101] nix-daemon[852]: reaped child process 1975, status = succeeded6746worker1 # [ 218.168275] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6747worker1 # [ 218.170621] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626748worker1 # [ 218.304651] nix-daemon[852]: accepted connection from pid 2002, user nix-grpc-daemon (trusted)6749worker1 # [ 218.321907] nix-daemon[852]: reaped child process 2005, status = succeeded6750lb # [ 218.389768] niks3-server[1210]: 2026/09/23 07:02:38 INFO Received uploads request method=POST path=/api/pending_closures6751worker1 # [ 218.337276] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.960Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6752worker1 # [ 218.339061] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.960Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6753lb # [ 218.419016] niks3-server[1210]: 2026/09/23 07:02:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6754worker1 # [ 218.364107] nix-grpc-daemon[1576]: time=2026-09-23T07:02:37.987Z level=INFO msg="Uploading 1 narinfos"6755lb # [ 218.433980] niks3-server[1210]: 2026/09/23 07:02:38 INFO Signed narinfos id=80 count=16756lb # [ 218.441005] niks3-server[1210]: 2026/09/23 07:02:38 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6757worker1 # [ 218.393214] nix-grpc-daemon[1576]: time=2026-09-23T07:02:38.016Z level=INFO msg="Upload complete. (211ms)"6758lb # [ 218.455557] niks3-server[1210]: 2026/09/23 07:02:38 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6759lb # [ 218.461864] niks3-server[1210]: 2026/09/23 07:02:38 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6760lb # [ 218.466373] niks3-server[1210]: 2026/09/23 07:02:38 INFO Completed upload id=806761worker1 # [ 218.404263] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=6246762lb # [ 218.469473] niks3-server[1210]: 2026/09/23 07:02:38 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6763worker1 # [ 218.414529] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626764worker1 # [ 218.423715] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6765worker1 # [ 218.426561] nix-daemon[852]: reaped child process 2000, status = succeeded6766worker1 # [ 218.437851] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6767worker1 # [ 219.708229] nix-daemon[852]: reaped child process 2009, status = succeeded6768worker1 # [ 219.802417] nix-daemon[852]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)6769worker1 # [ 219.821124] nix-daemon[852]: reaped child process 2028, status = succeeded6770lb # [ 219.881791] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received uploads request method=POST path=/api/pending_closures6771worker1 # [ 219.832224] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.453Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6772worker1 # [ 219.833810] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.453Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6773lb # [ 219.922275] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6774worker1 # [ 219.870560] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.494Z level=INFO msg="Uploading 1 narinfos"6775lb # [ 219.936379] niks3-server[1210]: 2026/09/23 07:02:39 INFO Signed narinfos id=81 count=16776lb # [ 219.944272] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6777lb # [ 219.955818] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6778worker1 # [ 219.900747] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.524Z level=INFO msg="Upload complete. (250ms)"6779lb # [ 219.966747] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6780lb # [ 219.972659] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete6781lb # [ 219.977513] niks3-server[1210]: 2026/09/23 07:02:39 INFO Completed upload id=816782lb # [ 219.981491] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6783worker1 # [ 219.918979] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=16784worker1 # [ 219.935457] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16785worker1 # [ 219.941546] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626786worker1 # [ 219.947550] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06787worker1 # [ 219.953724] nix-daemon[852]: reaped child process 2007, status = succeeded6788worker1 # [ 219.955894] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626789worker1 # [ 219.962896] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.585Z level=INFO msg="All 2 paths already cached"6790worker1 # [ 219.966745] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626791worker1 # [ 219.980142] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6792worker1 # [ 220.109941] nix-daemon[852]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6793worker1 # [ 220.126793] nix-daemon[852]: reaped child process 2037, status = succeeded6794lb # [ 220.195175] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received uploads request method=POST path=/api/pending_closures6795worker1 # [ 220.142368] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.765Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6796worker1 # [ 220.143818] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.765Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6797lb # [ 220.223777] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6798worker1 # [ 220.171860] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.795Z level=INFO msg="Uploading 1 narinfos"6799lb # [ 220.236338] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign6800lb # [ 220.248988] niks3-server[1210]: 2026/09/23 07:02:39 INFO Signed narinfos id=82 count=16801lb # [ 220.251631] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6802lb # [ 220.261956] niks3-server[1210]: 2026/09/23 07:02:39 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete6803worker1 # [ 220.204474] nix-grpc-daemon[1576]: time=2026-09-23T07:02:39.828Z level=INFO msg="Upload complete. (214ms)"6804lb # [ 220.268752] niks3-server[1210]: 2026/09/23 07:02:39 INFO Completed upload id=826805lb # [ 220.272232] niks3-server[1210]: 2026/09/23 07:02:39 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6806worker1 # [ 220.212299] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=7526807worker1 # [ 220.225837] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626808worker1 # [ 220.234150] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6809worker1 # [ 220.237186] nix-daemon[852]: reaped child process 2032, status = succeeded6810worker1 # [ 220.249543] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6811worker1 # [ 220.924400] nix-daemon[852]: reaped child process 2041, status = succeeded6812worker1 # [ 220.990105] nix-daemon[852]: accepted connection from pid 2057, user nix-grpc-daemon (trusted)6813worker1 # [ 221.007206] nix-daemon[852]: reaped child process 2060, status = succeeded6814lb # [ 221.075613] niks3-server[1210]: 2026/09/23 07:02:40 INFO Received uploads request method=POST path=/api/pending_closures6815worker1 # [ 221.024350] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.647Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6816worker1 # [ 221.025822] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.647Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6817worker1 # [ 221.067749] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.691Z level=INFO msg="Uploading 1 narinfos"6818lb # [ 221.120743] niks3-server[1210]: 2026/09/23 07:02:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign6819lb # [ 221.133277] niks3-server[1210]: 2026/09/23 07:02:40 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6820lb # [ 221.143046] niks3-server[1210]: 2026/09/23 07:02:40 INFO Signed narinfos id=83 count=16821lb # [ 221.147867] niks3-server[1210]: 2026/09/23 07:02:40 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6822lb # [ 221.153354] niks3-server[1210]: 2026/09/23 07:02:40 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6823lb # [ 221.162058] niks3-server[1210]: 2026/09/23 07:02:40 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete6824lb # [ 221.166519] niks3-server[1210]: 2026/09/23 07:02:40 INFO Completed upload id=836825lb # [ 221.169625] niks3-server[1210]: 2026/09/23 07:02:40 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6826worker1 # [ 221.104680] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.728Z level=INFO msg="Upload complete. (246ms)"6827worker1 # [ 221.121287] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=16828worker1 # [ 221.136091] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16829worker1 # [ 221.140724] nix-daemon[852]: reaped child process 2039, status = succeeded6830worker1 # [ 221.159068] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626831worker1 # [ 221.175353] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626832worker1 # [ 221.190962] nix-grpc-daemon[1576]: time=2026-09-23T07:02:40.814Z level=INFO msg="All 2 paths already cached"6833worker1 # [ 221.201738] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626834worker1 # [ 221.215445] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6835worker1 # [ 221.369906] nix-daemon[852]: accepted connection from pid 2064, user nix-grpc-daemon (trusted)6836worker1 # [ 221.389731] nix-daemon[852]: reaped child process 2067, status = succeeded6837lb # [ 221.456450] niks3-server[1210]: 2026/09/23 07:02:41 INFO Received uploads request method=POST path=/api/pending_closures6838worker1 # [ 221.404096] nix-grpc-daemon[1576]: time=2026-09-23T07:02:41.027Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6839worker1 # [ 221.405744] nix-grpc-daemon[1576]: time=2026-09-23T07:02:41.027Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6840lb # [ 221.489279] niks3-server[1210]: 2026/09/23 07:02:41 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6841worker1 # [ 221.439136] nix-grpc-daemon[1576]: time=2026-09-23T07:02:41.062Z level=INFO msg="Uploading 1 narinfos"6842lb # [ 221.504085] niks3-server[1210]: 2026/09/23 07:02:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign6843lb # [ 221.513536] niks3-server[1210]: 2026/09/23 07:02:41 INFO Signed narinfos id=84 count=16844lb # [ 221.519264] niks3-server[1210]: 2026/09/23 07:02:41 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6845lb # [ 221.529761] niks3-server[1210]: 2026/09/23 07:02:41 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete6846worker1 # [ 221.474112] nix-grpc-daemon[1576]: time=2026-09-23T07:02:41.097Z level=INFO msg="Upload complete. (242ms)"6847lb # [ 221.538347] niks3-server[1210]: 2026/09/23 07:02:41 INFO Completed upload id=846848lb # [ 221.542932] niks3-server[1210]: 2026/09/23 07:02:41 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6849worker1 # [ 221.484342] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=7526850worker1 # [ 221.504158] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626851worker1 # [ 221.511015] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6852worker1 # [ 221.516944] nix-daemon[852]: reaped child process 2062, status = succeeded6853worker1 # [ 221.531233] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6854worker1 # [ 222.992740] nix-daemon[852]: reaped child process 2071, status = succeeded6855worker1 # [ 223.072877] nix-daemon[852]: accepted connection from pid 2087, user nix-grpc-daemon (trusted)6856worker1 # [ 223.088788] nix-daemon[852]: reaped child process 2090, status = succeeded6857lb # [ 223.156752] niks3-server[1210]: 2026/09/23 07:02:42 INFO Received uploads request method=POST path=/api/pending_closures6858worker1 # [ 223.106225] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.728Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6859worker1 # [ 223.107718] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.728Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6860worker1 # [ 223.145661] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.769Z level=INFO msg="Uploading 1 narinfos"6861lb # [ 223.197807] niks3-server[1210]: 2026/09/23 07:02:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign6862lb # [ 223.209966] niks3-server[1210]: 2026/09/23 07:02:42 INFO Signed narinfos id=85 count=16863lb # [ 223.217945] niks3-server[1210]: 2026/09/23 07:02:42 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6864lb # [ 223.228220] niks3-server[1210]: 2026/09/23 07:02:42 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6865lb # [ 223.242776] niks3-server[1210]: 2026/09/23 07:02:42 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete6866lb # [ 223.248668] niks3-server[1210]: 2026/09/23 07:02:42 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6867worker1 # [ 223.192455] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.816Z level=INFO msg="Upload complete. (249ms)"6868lb # [ 223.255820] niks3-server[1210]: 2026/09/23 07:02:42 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6869lb # [ 223.259710] niks3-server[1210]: 2026/09/23 07:02:42 INFO Completed upload id=856870worker1 # [ 223.208569] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=16871worker1 # [ 223.223183] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16872worker1 # [ 223.228720] nix-daemon[852]: reaped child process 2069, status = succeeded6873worker1 # [ 223.247280] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626874worker1 # [ 223.261935] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:455626875worker1 # [ 223.275622] nix-grpc-daemon[1576]: time=2026-09-23T07:02:42.899Z level=INFO msg="All 2 paths already cached"6876worker1 # [ 223.285690] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:455626877worker1 # [ 223.298996] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6878worker1 # [ 223.445320] nix-daemon[852]: accepted connection from pid 2094, user nix-grpc-daemon (trusted)6879worker1 # [ 223.463937] nix-daemon[852]: reaped child process 2097, status = succeeded6880lb # [ 223.523720] niks3-server[1210]: 2026/09/23 07:02:43 INFO Received uploads request method=POST path=/api/pending_closures6881worker1 # [ 223.471653] nix-grpc-daemon[1576]: time=2026-09-23T07:02:43.095Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6882worker1 # [ 223.473364] nix-grpc-daemon[1576]: time=2026-09-23T07:02:43.095Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6883lb # [ 223.554197] niks3-server[1210]: 2026/09/23 07:02:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign6884worker1 # [ 223.500579] nix-grpc-daemon[1576]: time=2026-09-23T07:02:43.124Z level=INFO msg="Uploading 1 narinfos"6885lb # [ 223.568736] niks3-server[1210]: 2026/09/23 07:02:43 INFO Signed narinfos id=86 count=16886lb # [ 223.576285] niks3-server[1210]: 2026/09/23 07:02:43 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6887lb # [ 223.587298] niks3-server[1210]: 2026/09/23 07:02:43 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6888worker1 # [ 223.533322] nix-grpc-daemon[1576]: time=2026-09-23T07:02:43.157Z level=INFO msg="Upload complete. (219ms)"6889lb # [ 223.595621] niks3-server[1210]: 2026/09/23 07:02:43 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete6890lb # [ 223.604507] niks3-server[1210]: 2026/09/23 07:02:43 INFO Completed upload id=866891worker1 # [ 223.543210] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=0 paths=1 nar_bytes_in=7526892lb # [ 223.609815] niks3-server[1210]: 2026/09/23 07:02:43 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6893worker1 # [ 223.558203] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:455626894worker1 # [ 223.566985] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6895worker1 # [ 223.570466] nix-daemon[852]: reaped child process 2092, status = succeeded6896worker1 # [ 223.580163] nix-daemon[852]: accepted connection from pid 1475, user nix-grpc-daemon (trusted)6897worker1 # [ 224.685802] nix-daemon[852]: reaped child process 2101, status = succeeded6898worker1 # [ 224.757765] nix-daemon[852]: accepted connection from pid 2117, user nix-grpc-daemon (trusted)6899worker1 # [ 224.778646] nix-daemon[852]: reaped child process 2120, status = succeeded6900lb # [ 224.838832] niks3-server[1210]: 2026/09/23 07:02:44 INFO Received uploads request method=POST path=/api/pending_closures6901worker1 # [ 224.787786] nix-grpc-daemon[1576]: time=2026-09-23T07:02:44.410Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6902worker1 # [ 224.789606] nix-grpc-daemon[1576]: time=2026-09-23T07:02:44.410Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6903lb # [ 224.877728] niks3-server[1210]: 2026/09/23 07:02:44 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6904worker1 # [ 224.830403] nix-grpc-daemon[1576]: time=2026-09-23T07:02:44.453Z level=INFO msg="Uploading 1 narinfos"6905lb # [ 224.891899] niks3-server[1210]: 2026/09/23 07:02:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign6906lb # [ 224.902639] niks3-server[1210]: 2026/09/23 07:02:44 INFO Signed narinfos id=87 count=16907lb # [ 224.908639] niks3-server[1210]: 2026/09/23 07:02:44 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6908lb # [ 224.919517] niks3-server[1210]: 2026/09/23 07:02:44 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6909worker1 # [ 224.864187] nix-grpc-daemon[1576]: time=2026-09-23T07:02:44.487Z level=INFO msg="Upload complete. (272ms)"6910lb # [ 224.926552] niks3-server[1210]: 2026/09/23 07:02:44 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete6911lb # [ 224.931593] niks3-server[1210]: 2026/09/23 07:02:44 INFO Completed upload id=876912lb # [ 224.935174] niks3-server[1210]: 2026/09/23 07:02:44 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6913worker1 # [ 224.876994] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45562 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=16914worker1 # [ 224.896892] nix-daemon[852]: reaped child process 2099, status = succeeded6915client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.77 seconds)6916worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6917worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6918client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6919client # [ 225.071963] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6920client # Running as unit: gpu.service; invocation ID: 9b7017b0bf964a1aa976a7c9393340196921client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.08 seconds)6922lb: must succeed: curl -sf localhost:9901/clusters6923lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6924lb: must succeed: curl -sf localhost:9901/clusters6925lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6926worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6927client # [ 225.277720] nix[1563]: don't know how to build these paths:6928client # [ 225.278625] nix[1563]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6929worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6930worker2 # [ 225.282500] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:498306931client # [ 225.304323] nix[1563]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6932worker1 # [ 225.320696] nix-grpc-daemon[1475]: ts=2026-09-23T07:02:44Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06933worker2 # [ 225.290930] nix-grpc-daemon[1976]: ts=2026-09-23T07:02:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498306934lb: must succeed: curl -sf localhost:9901/clusters6935lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6936lb: must succeed: curl -sf localhost:9901/clusters6937lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6938worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6939worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6940client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6941lb # [ 228.293176] envoy[895]: conn peer=[2001:db8:1::1]:34716 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27097 tx=12330 ms=154976942client # [ 255.305824] nix[1563]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6943client # [ 255.317855] nix[1563]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6944client # [ 255.354896] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6945client # [ 255.363048] systemd[1]: gpu.service: Failed with result 'exit-code'.6946client # [ 255.366906] systemd[1]: gpu.service: Consumed 195ms CPU time over 30.288s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic.6947client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.05 seconds)6948client: must succeed: journalctl -u gpu -o cat6949client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6950lb: must succeed: curl -sf localhost:9901/clusters6951lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6952lb: must succeed: curl -sf localhost:9901/clusters6953lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6954worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6955worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6956(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.54 seconds)6957subtest: low disk drains a worker and builds go to the other6958worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6959worker1 # [ 256.793743] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:16Z level=info event=unhealthy reason=min_free6960worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.25 seconds)6961worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6962worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.04 seconds)6963client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26964client # don't know how to build these paths:6965client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6966worker2 # [ 257.231523] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:498306967worker2 # [ 257.240568] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498306968worker1 # [ 257.277660] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:16Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06969worker1 # [ 257.284306] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:16Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16970worker2 # [ 257.310998] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498306971worker2 # [ 257.328500] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498306972client # copying 1 paths to 'lb:50051'...6973client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6974worker2 # [ 257.344841] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:498306975worker2 # [ 257.360969] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6976worker2 # [ 257.522814] nix-daemon[842]: accepted connection from pid 2132, user nix-grpc-daemon (trusted)6977worker2 # [ 257.543329] nix-daemon[842]: reaped child process 2135, status = succeeded6978lb # [ 257.626346] niks3-server[1210]: 2026/09/23 07:03:17 INFO Received uploads request method=POST path=/api/pending_closures6979worker2 # [ 257.562178] nix-grpc-daemon[2037]: time=2026-09-23T07:03:17.194Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6980worker2 # [ 257.564605] nix-grpc-daemon[2037]: time=2026-09-23T07:03:17.196Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6981worker2 # [ 257.602301] nix-grpc-daemon[2037]: time=2026-09-23T07:03:17.234Z level=INFO msg="Uploading 1 narinfos"6982lb # [ 257.664231] niks3-server[1210]: 2026/09/23 07:03:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign6983lb # [ 257.680938] niks3-server[1210]: 2026/09/23 07:03:17 INFO Signed narinfos id=88 count=16984lb # [ 257.688284] niks3-server[1210]: 2026/09/23 07:03:17 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6985lb # [ 257.702100] niks3-server[1210]: 2026/09/23 07:03:17 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6986lb # [ 257.713610] niks3-server[1210]: 2026/09/23 07:03:17 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete6987lb # [ 257.721778] niks3-server[1210]: 2026/09/23 07:03:17 INFO Completed upload id=886988worker2 # [ 257.646421] nix-grpc-daemon[2037]: time=2026-09-23T07:03:17.278Z level=INFO msg="Upload complete. (268ms)"6989worker2 # [ 257.656587] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=5126990lb # [ 257.741305] niks3-server[1210]: 2026/09/23 07:03:17 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6991worker2 # [ 257.671520] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498306992worker2 # [ 257.684844] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6993worker2 # [ 257.690449] nix-daemon[842]: reaped child process 2130, status = succeeded6994client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6995worker2 # [ 257.702069] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)6996client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6997lb # [ 258.292078] envoy[895]: conn peer=[2001:db8:1::1]:56396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300596998client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6999client # farm-a> LOG-farm-a-drain7000worker2 # [ 259.213623] nix-daemon[842]: reaped child process 2139, status = succeeded7001worker2 # [ 259.283854] nix-daemon[842]: accepted connection from pid 2155, user nix-grpc-daemon (trusted)7002worker2 # [ 259.306752] nix-daemon[842]: reaped child process 2158, status = succeeded7003lb # [ 259.379300] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received uploads request method=POST path=/api/pending_closures7004worker2 # [ 259.319759] nix-grpc-daemon[2037]: time=2026-09-23T07:03:18.950Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7005worker2 # [ 259.321519] nix-grpc-daemon[2037]: time=2026-09-23T07:03:18.950Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7006lb # [ 259.419322] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7007worker2 # [ 259.354384] nix-grpc-daemon[2037]: time=2026-09-23T07:03:18.986Z level=INFO msg="Uploading 1 narinfos"7008lb # [ 259.433497] niks3-server[1210]: 2026/09/23 07:03:19 INFO Signed narinfos id=89 count=17009lb # [ 259.435922] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7010lb # [ 259.447028] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7011lb # [ 259.459174] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7012worker2 # [ 259.389575] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.021Z level=INFO msg="Upload complete. (274ms)"7013lb # [ 259.467081] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7014lb # [ 259.472057] niks3-server[1210]: 2026/09/23 07:03:19 INFO Completed upload id=897015lb # [ 259.474643] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7016worker2 # [ 259.401125] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=17017worker1 # [ 259.422109] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:19Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07018worker1 # [ 259.427815] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:19Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17019worker2 # [ 259.419787] nix-daemon[842]: reaped child process 2137, status = succeeded7020worker2 # [ 259.454855] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498307021worker2 # [ 259.471520] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498307022worker2 # [ 259.486668] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.118Z level=INFO msg="All 1 paths already cached"7023client # copying 1 paths to 'lb:50051'...7024client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7025worker2 # [ 259.497927] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:498307026worker2 # [ 259.511749] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7027worker2 # [ 259.661281] nix-daemon[842]: accepted connection from pid 2162, user nix-grpc-daemon (trusted)7028worker2 # [ 259.680993] nix-daemon[842]: reaped child process 2165, status = succeeded7029lb # [ 259.758438] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received uploads request method=POST path=/api/pending_closures7030worker2 # [ 259.695438] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.327Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7031worker2 # [ 259.697209] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.327Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7032lb # [ 259.787453] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7033worker2 # [ 259.724820] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.356Z level=INFO msg="Uploading 1 narinfos"7034lb # [ 259.798171] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7035lb # [ 259.805940] niks3-server[1210]: 2026/09/23 07:03:19 INFO Signed narinfos id=90 count=17036lb # [ 259.810892] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7037lb # [ 259.816779] niks3-server[1210]: 2026/09/23 07:03:19 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7038lb # [ 259.824545] niks3-server[1210]: 2026/09/23 07:03:19 INFO Completed upload id=907039worker2 # [ 259.751090] nix-grpc-daemon[2037]: time=2026-09-23T07:03:19.383Z level=INFO msg="Upload complete. (223ms)"7040lb # [ 259.827078] niks3-server[1210]: 2026/09/23 07:03:19 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7041worker2 # [ 259.760151] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=6327042worker2 # [ 259.773421] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498307043worker2 # [ 259.779335] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7044worker2 # [ 259.783820] nix-daemon[842]: reaped child process 2160, status = succeeded7045client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7046worker2 # [ 259.794254] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7047client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7048client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7049client # farm-b> LOG-farm-b-drain7050worker2 # [ 261.457628] nix-daemon[842]: reaped child process 2169, status = succeeded7051worker2 # [ 261.563706] nix-daemon[842]: accepted connection from pid 2185, user nix-grpc-daemon (trusted)7052worker2 # [ 261.584579] nix-daemon[842]: reaped child process 2188, status = succeeded7053lb # [ 261.662024] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received uploads request method=POST path=/api/pending_closures7054worker2 # [ 261.596361] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.228Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7055worker2 # [ 261.597824] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.228Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7056worker2 # [ 261.639730] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.271Z level=INFO msg="Uploading 1 narinfos"7057lb # [ 261.705177] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7058lb # [ 261.721300] niks3-server[1210]: 2026/09/23 07:03:21 INFO Signed narinfos id=91 count=17059lb # [ 261.731432] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7060lb # [ 261.741137] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7061lb # [ 261.749854] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7062worker2 # [ 261.683535] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.315Z level=INFO msg="Upload complete. (276ms)"7063lb # [ 261.759281] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7064lb # [ 261.765491] niks3-server[1210]: 2026/09/23 07:03:21 INFO Completed upload id=917065lb # [ 261.770447] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7066worker2 # [ 261.697135] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=17067worker1 # [ 261.718578] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:21Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07068worker1 # [ 261.718944] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:21Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17069worker2 # [ 261.716229] nix-daemon[842]: reaped child process 2167, status = succeeded7070worker2 # [ 261.751797] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498307071worker2 # [ 261.766133] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:498307072worker2 # [ 261.780498] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.412Z level=INFO msg="All 2 paths already cached"7073client # copying 1 paths to 'lb:50051'...7074client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7075worker2 # [ 261.791522] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:498307076worker2 # [ 261.805013] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7077worker2 # [ 261.959070] nix-daemon[842]: accepted connection from pid 2192, user nix-grpc-daemon (trusted)7078worker2 # [ 261.977866] nix-daemon[842]: reaped child process 2195, status = succeeded7079lb # [ 262.049332] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received uploads request method=POST path=/api/pending_closures7080worker2 # [ 261.984167] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.616Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7081worker2 # [ 261.985652] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.616Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7082lb # [ 262.082443] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7083worker2 # [ 262.017562] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.649Z level=INFO msg="Uploading 1 narinfos"7084lb # [ 262.096062] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7085lb # [ 262.106545] niks3-server[1210]: 2026/09/23 07:03:21 INFO Signed narinfos id=92 count=17086worker2 # [ 262.040811] nix-grpc-daemon[2037]: time=2026-09-23T07:03:21.672Z level=INFO msg="Upload complete. (219ms)"7087lb # [ 262.114666] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7088lb # [ 262.123761] niks3-server[1210]: 2026/09/23 07:03:21 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7089worker2 # [ 262.050154] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=0 paths=1 nar_bytes_in=7607090lb # [ 262.128322] niks3-server[1210]: 2026/09/23 07:03:21 INFO Completed upload id=927091lb # [ 262.130257] niks3-server[1210]: 2026/09/23 07:03:21 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7092worker2 # [ 262.063334] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:498307093worker2 # [ 262.071852] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7094worker2 # [ 262.079817] nix-daemon[842]: reaped child process 2190, status = succeeded7095client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7096worker2 # [ 262.085636] nix-daemon[842]: accepted connection from pid 1976, user nix-grpc-daemon (trusted)7097client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7098client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7099client # farm-top> LOG-farm-top-drain7100worker2 # [ 263.580434] nix-daemon[842]: reaped child process 2199, status = succeeded7101worker2 # [ 263.666974] nix-daemon[842]: accepted connection from pid 2215, user nix-grpc-daemon (trusted)7102worker2 # [ 263.682364] nix-daemon[842]: reaped child process 2218, status = succeeded7103lb # [ 263.765955] niks3-server[1210]: 2026/09/23 07:03:23 INFO Received uploads request method=POST path=/api/pending_closures7104worker2 # [ 263.703558] nix-grpc-daemon[2037]: time=2026-09-23T07:03:23.332Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7105worker2 # [ 263.705565] nix-grpc-daemon[2037]: time=2026-09-23T07:03:23.332Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7106lb # [ 263.803513] niks3-server[1210]: 2026/09/23 07:03:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7107worker2 # [ 263.741653] nix-grpc-daemon[2037]: time=2026-09-23T07:03:23.373Z level=INFO msg="Uploading 1 narinfos"7108lb # [ 263.821177] niks3-server[1210]: 2026/09/23 07:03:23 INFO Signed narinfos id=93 count=17109lb # [ 263.823854] niks3-server[1210]: 2026/09/23 07:03:23 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7110lb # [ 263.839342] niks3-server[1210]: 2026/09/23 07:03:23 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7111worker2 # [ 263.770816] nix-grpc-daemon[2037]: time=2026-09-23T07:03:23.402Z level=INFO msg="Upload complete. (272ms)"7112lb # [ 263.847563] niks3-server[1210]: 2026/09/23 07:03:23 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7113lb # [ 263.850930] niks3-server[1210]: 2026/09/23 07:03:23 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7114lb # [ 263.855287] niks3-server[1210]: 2026/09/23 07:03:23 INFO Completed upload id=937115lb # [ 263.857847] niks3-server[1210]: 2026/09/23 07:03:23 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7116worker2 # [ 263.785113] nix-grpc-daemon[1976]: ts=2026-09-23T07:03:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49830 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=17117worker2 # [ 263.801123] nix-daemon[842]: reaped child process 2197, status = succeeded7118client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.82 seconds)7119client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7120client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7121worker1: must succeed: rm /nix/.rw-store/fill7122worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7123worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7124worker1 # [ 264.798402] nix-grpc-daemon[1475]: ts=2026-09-23T07:03:24Z level=info event=healthy reason=min_free7125worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7126(finished: subtest: low disk drains a worker and builds go to the other, in 9.63 seconds)7127(finished: run the VM test script, in 266.25 seconds)7128test script finished in 266.51s7129cleanup7130kill QemuMachine (pid 49)7131client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7132client # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7133client # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7134client # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7135kill QemuMachine (pid 50)7136lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7137lb # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7138lb # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7139lb # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7140kill QemuMachine (pid 52)7141worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7142worker1 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7143worker1 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7144worker1 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7145kill QemuMachine (pid 51)7146worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7147worker2 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7148worker2 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7149worker2 # [2026-09-23T07:03:25Z INFO virtiofsd] Client disconnected, shutting down7150(finished: cleanup, in 1.06 seconds)7151additionally exposed symbols:7152 client, lb, worker1, worker2,7153 vlan1,7154 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh