vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #501
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14worker1 # Disk image does not exist, creating the virtualisation disk image...15client: QEMU running (pid 49)16worker2: QEMU running (pid 51)17client # Disk image does not exist, creating the virtualisation disk image...18worker2 # Formatting '/build/vm-state-worker2/tmp.uQSOPTwJsU', fmt=raw size=107374182419worker1: QEMU running (pid 52)20lb # Formatting '/build/vm-state-lb/tmp.TmizxiiMR8', fmt=raw size=107374182421worker1 # Formatting '/build/vm-state-worker1/tmp.tdXdMITBdE', fmt=raw size=107374182422lb: QEMU running (pid 50)23lb # mke2fs 1.47.4 (6-Mar-2025)24worker1 # mke2fs 1.47.4 (6-Mar-2025)25worker2 # mke2fs 1.47.4 (6-Mar-2025)26client # Formatting '/build/vm-state-client/tmp.bxac97Zvk2', fmt=raw size=107374182427lb # Discarding device blocks: 0/262144 done28worker1 # Discarding device blocks: 0/262144 done29worker2 # Discarding device blocks: 0/262144 done30lb # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes32worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes33lb # Filesystem UUID: e86ec5e8-8bca-4b52-9b9c-8ca6e786976c34worker1 # Filesystem UUID: eb5679b0-4df0-438c-9004-2a9d3fefaa7435worker2 # Filesystem UUID: 3184b9ef-249a-49ba-8cf1-b67ca6fecee236lb # Superblock backups stored on blocks:37worker1 # Superblock backups stored on blocks:38worker2 # Superblock backups stored on blocks:39lb # 32768, 98304, 163840, 22937640worker1 # 32768, 98304, 163840, 22937641worker2 # 32768, 98304, 163840, 22937642lb # 43worker1 # 44worker2 # 45lb # Allocating group tables: 0/8 done46worker1 # Allocating group tables: 0/8 done47worker2 # Allocating group tables: 0/8 done48lb # Writing inode tables: 0/8 done49worker1 # Writing inode tables: 0/8 done50worker2 # Writing inode tables: 0/8 done51lb # Creating journal (8192 blocks): done52worker1 # Creating journal (8192 blocks): done53worker2 # Creating journal (8192 blocks): done54lb # Writing superblocks and filesystem accounting information: 0/8 done55worker1 # Writing superblocks and filesystem accounting information: 0/8 done56worker2 # Writing superblocks and filesystem accounting information: 0/8 done57lb # 58worker1 # 59worker2 # 60lb # Virtualisation disk image created.61worker1 # Virtualisation disk image created.62worker2 # Virtualisation disk image created.63lb # Starting virtiofs daemons...64worker1 # Starting virtiofs daemons...65worker2 # Starting virtiofs daemons...66lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)68worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)73worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether76worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79client # mke2fs 1.47.4 (6-Mar-2025)80(finished: start all VMs, in 0.48 seconds)81worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...82worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...83lb # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84client # Discarding device blocks: 0/262144 done85worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...86lb # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...87client # Creating filesystem with 262144 4k blocks and 65536 inodes88worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)89lb # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...90client # Filesystem UUID: 90dd4bde-bcaa-4091-be50-17c845d35dc491worker2 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92lb # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...93client # Superblock backups stored on blocks:94worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...95lb # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests96client # 32768, 98304, 163840, 22937697worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests98lb # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests99client # 100worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests101lb # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests102client # Allocating group tables: 0/8 done103worker2 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests104client: waiting for unit network-addresses-eth1.service105worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...106client: waiting for the VM to finish booting107worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)108client # Writing inode tables: 0/8 done109worker1 # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether110client # Creating journal (8192 blocks): done111client # Writing superblocks and filesystem accounting information: 0/8 done112worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...113client # 114worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests115client # Virtualisation disk image created.116worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests117client # Starting virtiofs daemons...118worker1 # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests119client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...122client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124client # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...125client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126client # [2026-09-22T14:51:03Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127client # [2026-09-22T14:51:03Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-22T14:51:03Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135client # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140client # [ 0.000000] Zone ranges:141client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] DMA32 empty143client # [ 0.000000] Normal empty144client # [ 0.000000] Device empty145client # [ 0.000000] Movable zone start for each node146client # [ 0.000000] Early memory node ranges147client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150client # [ 0.000000] psci: probing for conduit method from DT.151client # [ 0.000000] psci: PSCIv1.3 detected in firmware.152client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153client # [ 0.000000] psci: Trusted OS migration not required154client # [ 0.000000] psci: SMC Calling Convention v1.1155client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157client # [ 0.000000] Detected PIPT I-cache on CPU0158client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159client # [ 0.000000] CPU features: detected: GICv3 CPU interface160client # [ 0.000000] CPU features: detected: Spectre-v4161client # [ 0.000000] CPU features: detected: Spectre-BHB162client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164client # [ 0.000000] alternatives: applying boot alternatives165client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/9ynmmvibq935q3g99kxpcvq4zbn0p3xi-nixos-system-client-test/init regInfo=/nix/store/6a50qpfalbjwv7f1s4agz8m5sg4l2qn4-closure-info/registration console=ttyAMA0,115200n8 console=tty0166client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/6a50qpfalbjwv7f1s4agz8m5sg4l2qn4-closure-info/registration", will be passed to user space.167client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]169client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)170client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)171worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026172worker2 # [ 0.000000] KASLR enabled173client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB174worker2 # [ 0.000000] random: crng init done175client # [ 0.000000] software IO TLB: area num 1.176worker2 # [ 0.000000] Machine model: linux,dummy-virt177client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB178worker2 # [ 0.000000] efi: UEFI not found.179worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT180client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)181client # [ 0.000000] Fallback order for Node 0: 0182worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]183client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216184worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]185client # [ 0.000000] Policy zone: DMA186worker2 # [ 0.000000] Zone ranges:187client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off188worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]189worker2 # [ 0.000000] DMA32 empty190worker2 # [ 0.000000] Normal empty191client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1192worker2 # [ 0.000000] Device empty193client # [ 0.000000] allocated 3145728 bytes of page_ext194worker2 # [ 0.000000] Movable zone start for each node195client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages196worker2 # [ 0.000000] Early memory node ranges197client # [ 0.000000] ftrace: allocated 294 pages with 4 groups198worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]199client # [ 0.000000] rcu: Hierarchical RCU implementation.200worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]201client # [ 0.000000] rcu: RCU event tracing is enabled.202worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000203client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.204worker2 # [ 0.000000] psci: probing for conduit method from DT.205client # [ 0.000000] Trampoline variant of Tasks RCU enabled.206worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.207client # [ 0.000000] Rude variant of Tasks RCU enabled.208worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs209client # [ 0.000000] Tracing variant of Tasks RCU enabled.210worker2 # [ 0.000000] psci: Trusted OS migration not required211worker2 # [ 0.000000] psci: SMC Calling Convention v1.1212client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.213client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1214worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)215worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296216client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.217worker2 # [ 0.000000] Detected PIPT I-cache on CPU0218client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.219worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)220worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface221client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.222worker2 # [ 0.000000] CPU features: detected: Spectre-v4223client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0224worker2 # [ 0.000000] CPU features: detected: Spectre-BHB225client # [ 0.000000] GICv3: 256 SPIs implemented226worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38227client # [ 0.000000] GICv3: 0 Extended SPIs implemented228client # [ 0.000000] Root IRQ handler: gic_handle_irq229worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23230client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI231worker2 # [ 0.000000] alternatives: applying boot alternatives232client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0233client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000234client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]235client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)236worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/7nr90ian7b661w5sws26k1lwc9mbmx4q-nixos-system-worker2-test/init regInfo=/nix/store/6r9fszg6kxsc3k2rsf87aynbkvl8fb2b-closure-info/registration console=ttyAMA0,115200n8 console=tty0237client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)238client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000239worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/6r9fszg6kxsc3k2rsf87aynbkvl8fb2b-closure-info/registration", will be passed to user space.240client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000241worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes242client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.243worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)244client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns245worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)246client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).247worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB248worker2 # [ 0.000000] software IO TLB: area num 1.249client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns250worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB251client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns252client # [ 0.000039] arm-pv: using stolen time PV253worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)254worker2 # [ 0.000000] Fallback order for Node 0: 0255lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]256client # [ 0.000680] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)257worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216258worker2 # [ 0.000000] Policy zone: DMA259client # [ 0.000861] Console: colour dummy device 80x25260lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026261client # [ 0.000869] printk: legacy console [tty0] enabled262lb # [ 0.000000] KASLR enabled263worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off264lb # [ 0.000000] random: crng init done265worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1266lb # [ 0.000000] Machine model: linux,dummy-virt267client # [ 0.001061] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)268worker2 # [ 0.000000] allocated 3145728 bytes of page_ext269lb # [ 0.000000] efi: UEFI not found.270client # [ 0.001068] pid_max: default: 32768 minimum: 301271worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages272lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT273client # [ 0.001141] LSM: initializing lsm=capability,landlock,yama,bpf,ima274worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups275client # [ 0.001315] landlock: Up and running.276lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]277worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.278client # [ 0.001318] Yama: becoming mindful.279worker2 # [ 0.000000] rcu: RCU event tracing is enabled.280client # [ 0.001948] LSM support for eBPF active281lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]282lb # [ 0.000000] Zone ranges:283worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.284client # [ 0.002105] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)285lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]286worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.287lb # [ 0.000000] DMA32 empty288lb # [ 0.000000] Normal empty289client # [ 0.002138] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)290worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.291lb # [ 0.000000] Device empty292client # [ 0.004022] rcu: Hierarchical SRCU implementation.293worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.294lb # [ 0.000000] Movable zone start for each node295client # [ 0.004027] rcu: Max phase no-delay instances is 1000.296lb # [ 0.000000] Early memory node ranges297worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.298client # [ 0.005293] fsl-mc MSI: its@8080000 domain created299lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]300client # [ 0.005384] EFI services will not be available.301worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1302client # [ 0.005450] smp: Bringing up secondary CPUs ...303lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]304client # [ 0.005458] smp: Brought up 1 node, 1 CPU305worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.306lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000307worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]308client # [ 0.005461] SMP: Total of 1 processors activated.309lb # [ 0.000000] psci: probing for conduit method from DT.310client # [ 0.005466] CPU: All CPU(s) started at EL1311worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.312lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.313worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026314client # [ 0.005477] CPU features: detected: Branch Target Identification315lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs316worker1 # [ 0.000000] KASLR enabled317worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.318worker1 # [ 0.000000] random: crng init done319lb # [ 0.000000] psci: Trusted OS migration not required320client # [ 0.005482] CPU features: detected: ARMv8.4 Translation Table Level321worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0322worker1 # [ 0.000000] Machine model: linux,dummy-virt323lb # [ 0.000000] psci: SMC Calling Convention v1.1324worker1 # [ 0.000000] efi: UEFI not found.325worker2 # [ 0.000000] GICv3: 256 SPIs implemented326client # [ 0.005485] CPU features: detected: Instruction cache invalidation not required for I/D coherence327worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented328lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)329worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT330worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq331client # [ 0.005488] CPU features: detected: Data cache clean to the PoU not required for I/D coherence332lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296333worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI334worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]335lb # [ 0.000000] Detected PIPT I-cache on CPU0336client # [ 0.005492] CPU features: detected: Common not Private translations337worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0338worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]339client # [ 0.005495] CPU features: detected: CRC32 instructions340worker1 # [ 0.000000] Zone ranges:341worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000342lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)343worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]344client # [ 0.005498] CPU features: detected: Data cache clean to Point of Deep Persistence345worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]346lb # [ 0.000000] CPU features: detected: GICv3 CPU interface347worker1 # [ 0.000000] DMA32 empty348worker1 # [ 0.000000] Normal empty349lb # [ 0.000000] CPU features: detected: Spectre-v4350client # [ 0.005501] CPU features: detected: Data cache clean to Point of Persistence351worker1 # [ 0.000000] Device empty352worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)353lb # [ 0.000000] CPU features: detected: Spectre-BHB354worker1 # [ 0.000000] Movable zone start for each node355client # [ 0.005505] CPU features: detected: Data independent timing control (DIT)356lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38357worker1 # [ 0.000000] Early memory node ranges358client # [ 0.005508] CPU features: detected: E0PD359worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)360worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]361lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23362client # [ 0.005511] CPU features: detected: Enhanced Counter Virtualization363worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000364lb # [ 0.000000] alternatives: applying boot alternatives365worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]366client # [ 0.005514] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)367worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000368worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000369client # [ 0.005517] CPU features: detected: Enhanced Virtualization Traps370worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.371worker1 # [ 0.000000] psci: probing for conduit method from DT.372client # [ 0.005520] CPU features: detected: Fine Grained Traps373worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.374worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns375worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs376client # [ 0.005524] CPU features: detected: Generic authentication (architected QARMA5 algorithm)377worker1 # [ 0.000000] psci: Trusted OS migration not required378worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).379client # [ 0.005529] CPU features: detected: RCpc load-acquire (LDAPR)380worker1 # [ 0.000000] psci: SMC Calling Convention v1.1381lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rwj7rgrjnik4yfrdy5nxpi16ig7dj0yx-nixos-system-lb-test/init regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration console=ttyAMA0,115200n8 console=tty0382client # [ 0.005532] CPU features: detected: LSE atomic instructions383worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns384worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)385client # [ 0.005535] CPU features: detected: Privileged Access Never386client # [ 0.005538] CPU features: detected: PMUv3387worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296388worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns389worker1 # [ 0.000000] Detected PIPT I-cache on CPU0390worker2 # [ 0.000035] arm-pv: using stolen time PV391client # [ 0.005541] CPU features: detected: RAS Extension Support392lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/y0b8gyps1r0yyzfhbdbfwqvj69pgkv2z-closure-info/registration", will be passed to user space.393client # [ 0.005543] CPU features: detected: RASv1p1 Extension Support394worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)395lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes396client # [ 0.005546] CPU features: detected: Random Number Generator397worker2 # [ 0.000523] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)398worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface399worker2 # [ 0.000722] Console: colour dummy device 80x25400lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)401worker1 # [ 0.000000] CPU features: detected: Spectre-v4402worker2 # [ 0.000729] printk: legacy console [tty0] enabled403client # [ 0.005549] CPU features: detected: Speculation barrier (SB)404worker1 # [ 0.000000] CPU features: detected: Spectre-BHB405lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)406worker2 # [ 0.000918] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)407worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38408client # [ 0.005551] CPU features: detected: Stage-2 Force Write-Back409lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB410worker2 # [ 0.000924] pid_max: default: 32768 minimum: 301411worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23412lb # [ 0.000000] software IO TLB: area num 1.413client # [ 0.005554] CPU features: detected: TLB range maintenance instructions414worker2 # [ 0.001012] LSM: initializing lsm=capability,landlock,yama,bpf,ima415worker1 # [ 0.000000] alternatives: applying boot alternatives416lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB417client # [ 0.005559] CPU features: detected: Speculative Store Bypassing Safe (SSBS)418worker2 # [ 0.001208] landlock: Up and running.419client # [ 0.005600] alternatives: applying system-wide alternatives420worker2 # [ 0.001211] Yama: becoming mindful.421lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)422worker2 # [ 0.001768] LSM support for eBPF active423lb # [ 0.000000] Fallback order for Node 0: 0424client # [ 0.008615] CPU features: detected: BBM Level 2 without TLB conflict abort425worker2 # [ 0.001920] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)426lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216427lb # [ 0.000000] Policy zone: DMA428worker2 # [ 0.001951] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)429client # [ 0.008857] Memory: 1405636K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)430lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off431worker2 # [ 0.004063] rcu: Hierarchical SRCU implementation.432client # [ 0.009290] devtmpfs: initialized433worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/wrqfs8d5q8wxr8pbh005xzrx73np2k9s-nixos-system-worker1-test/init regInfo=/nix/store/bngrdnfx7hv1pjgib4i5amm9vsmkd6h4-closure-info/registration console=ttyAMA0,115200n8 console=tty0434worker2 # [ 0.004068] rcu: Max phase no-delay instances is 1000.435lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1436client # [ 0.011143] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)437lb # [ 0.000000] allocated 3145728 bytes of page_ext438worker2 # [ 0.005323] fsl-mc MSI: its@8080000 domain created439lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages440worker2 # [ 0.005414] EFI services will not be available.441worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bngrdnfx7hv1pjgib4i5amm9vsmkd6h4-closure-info/registration", will be passed to user space.442client # [ 0.011169] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).443worker2 # [ 0.005495] smp: Bringing up secondary CPUs ...444lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups445worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes446client # [ 0.011357] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL447worker2 # [ 0.005505] smp: Brought up 1 node, 1 CPU448lb # [ 0.000000] rcu: Hierarchical RCU implementation.449client # [ 0.011361] 0 pages in range for non-PLT usage450worker2 # [ 0.005508] SMP: Total of 1 processors activated.451lb # [ 0.000000] rcu: RCU event tracing is enabled.452worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)453client # [ 0.011362] 508272 pages in range for PLT usage454worker2 # [ 0.005514] CPU: All CPU(s) started at EL1455lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.456client # [ 0.011469] pinctrl core: initialized pinctrl subsystem457worker2 # [ 0.005525] CPU features: detected: Branch Target Identification458worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)459client # [ 0.012259] DMI not present or invalid.460lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.461worker2 # [ 0.005529] CPU features: detected: ARMv8.4 Translation Table Level462lb # [ 0.000000] Rude variant of Tasks RCU enabled.463client # [ 0.016000] NET: Registered PF_NETLINK/PF_ROUTE protocol family464worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB465lb # [ 0.000000] Tracing variant of Tasks RCU enabled.466worker2 # [ 0.005532] CPU features: detected: Instruction cache invalidation not required for I/D coherence467client # [ 0.018283] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations468worker1 # [ 0.000000] software IO TLB: area num 1.469lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.470worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB471worker2 # [ 0.005536] CPU features: detected: Data cache clean to the PoU not required for I/D coherence472client # [ 0.018457] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations473lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1474worker2 # [ 0.005540] CPU features: detected: Common not Private translations475worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)476client # [ 0.018654] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations477worker1 # [ 0.000000] Fallback order for Node 0: 0478worker2 # [ 0.005543] CPU features: detected: CRC32 instructions479lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.480client # [ 0.018675] audit: initializing netlink subsys (disabled)481worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216482worker2 # [ 0.005546] CPU features: detected: Data cache clean to Point of Deep Persistence483client # [ 0.019248] thermal_sys: Registered thermal governor 'fair_share'484worker1 # [ 0.000000] Policy zone: DMA485lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.486client # [ 0.019250] thermal_sys: Registered thermal governor 'bang_bang'487worker2 # [ 0.005550] CPU features: detected: Data cache clean to Point of Persistence488worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off489lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.490client # [ 0.019254] thermal_sys: Registered thermal governor 'step_wise'491worker2 # [ 0.005553] CPU features: detected: Data independent timing control (DIT)492worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1493lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0494worker2 # [ 0.005556] CPU features: detected: E0PD495client # [ 0.019257] thermal_sys: Registered thermal governor 'user_space'496worker1 # [ 0.000000] allocated 3145728 bytes of page_ext497lb # [ 0.000000] GICv3: 256 SPIs implemented498worker2 # [ 0.005559] CPU features: detected: Enhanced Counter Virtualization499worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages500client # [ 0.019262] thermal_sys: Registered thermal governor 'power_allocator'501lb # [ 0.000000] GICv3: 0 Extended SPIs implemented502lb # [ 0.000000] Root IRQ handler: gic_handle_irq503worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups504worker2 # [ 0.005562] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)505client # [ 0.019289] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1506worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.507lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI508client # [ 0.019297] cpuidle: using governor ladder509worker2 # [ 0.005565] CPU features: detected: Enhanced Virtualization Traps510worker1 # [ 0.000000] rcu: RCU event tracing is enabled.511lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0512client # [ 0.019303] cpuidle: using governor menu513worker2 # [ 0.005568] CPU features: detected: Fine Grained Traps514worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.515lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000516client # [ 0.019496] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.517lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]518worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.519worker2 # [ 0.005571] CPU features: detected: Generic authentication (architected QARMA5 algorithm)520client # [ 0.019512] ASID allocator initialised with 65536 entries521worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.522worker2 # [ 0.005576] CPU features: detected: RCpc load-acquire (LDAPR)523client # [ 0.020799] Serial: AMBA PL011 UART driver524worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.525lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)526worker2 # [ 0.005579] CPU features: detected: LSE atomic instructions527client # [ 0.026055] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1528worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.529worker2 # [ 0.005582] CPU features: detected: Privileged Access Never530lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)531client # [ 0.026218] printk: console [ttyAMA0] enabled532worker2 # [ 0.005585] CPU features: detected: PMUv3533worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1534lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000535worker2 # [ 0.005588] CPU features: detected: RAS Extension Support536worker2 # [ 0.005591] CPU features: detected: RASv1p1 Extension Support537worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.538lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a10000539worker2 # [ 0.005593] CPU features: detected: Random Number Generator540lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.541worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.542worker2 # [ 0.005596] CPU features: detected: Speculation barrier (SB)543worker2 # [ 0.005599] CPU features: detected: Stage-2 Force Write-Back544lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns545worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.546lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).547worker2 # [ 0.005602] CPU features: detected: TLB range maintenance instructions548worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0549worker1 # [ 0.000000] GICv3: 256 SPIs implemented550worker2 # [ 0.005606] CPU features: detected: Speculative Store Bypassing Safe (SSBS)551worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented552lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns553worker2 # [ 0.005647] alternatives: applying system-wide alternatives554worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq555worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI556lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns557worker2 # [ 0.008655] CPU features: detected: BBM Level 2 without TLB conflict abort558worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0559client # [ 0.147021] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages560lb # [ 0.000033] arm-pv: using stolen time PV561client # [ 0.147039] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page562worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000563worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]564lb # [ 0.000456] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)565worker2 # [ 0.008829] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)566client # [ 0.147044] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages567worker2 # [ 0.009212] devtmpfs: initialized568lb # [ 0.000647] Console: colour dummy device 80x25569client # [ 0.147048] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page570worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)571lb # [ 0.000655] printk: legacy console [tty0] enabled572worker2 # [ 0.011034] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)573client # [ 0.147053] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages574worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)575lb # [ 0.000867] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)576worker2 # [ 0.011077] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).577client # [ 0.147057] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page578lb # [ 0.000875] pid_max: default: 32768 minimum: 301579worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000580worker2 # [ 0.011283] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL581client # [ 0.147061] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages582lb # [ 0.000971] LSM: initializing lsm=capability,landlock,yama,bpf,ima583worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000584lb # [ 0.001099] landlock: Up and running.585client # [ 0.147065] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page586worker2 # [ 0.011288] 0 pages in range for non-PLT usage587lb # [ 0.001102] Yama: becoming mindful.588worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.589worker2 # [ 0.011289] 508272 pages in range for PLT usage590lb # [ 0.001580] LSM support for eBPF active591worker2 # [ 0.011403] pinctrl core: initialized pinctrl subsystem592worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns593lb # [ 0.001731] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)594worker2 # [ 0.012188] DMI not present or invalid.595client # [ 0.154410] fbcon: Taking over console596worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).597worker2 # [ 0.015577] NET: Registered PF_NETLINK/PF_ROUTE protocol family598lb # [ 0.001764] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)599client # [ 0.154426] ACPI: Interpreter disabled.600lb # [ 0.003560] rcu: Hierarchical SRCU implementation.601worker2 # [ 0.017823] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations602worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns603lb # [ 0.003566] rcu: Max phase no-delay instances is 1000.604worker2 # [ 0.018020] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations605lb # [ 0.004799] fsl-mc MSI: its@8080000 domain created606worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns607lb # [ 0.004897] EFI services will not be available.608worker1 # [ 0.000029] arm-pv: using stolen time PV609worker2 # [ 0.018227] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations610lb # [ 0.005026] smp: Bringing up secondary CPUs ...611worker2 # [ 0.018250] audit: initializing netlink subsys (disabled)612lb # [ 0.005035] smp: Brought up 1 node, 1 CPU613worker1 # [ 0.000386] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)614lb # [ 0.005038] SMP: Total of 1 processors activated.615worker2 # [ 0.018816] thermal_sys: Registered thermal governor 'fair_share'616worker1 # [ 0.000570] Console: colour dummy device 80x25617lb # [ 0.005043] CPU: All CPU(s) started at EL1618worker1 # [ 0.000577] printk: legacy console [tty0] enabled619worker2 # [ 0.018818] thermal_sys: Registered thermal governor 'bang_bang'620lb # [ 0.005054] CPU features: detected: Branch Target Identification621worker2 # [ 0.018821] thermal_sys: Registered thermal governor 'step_wise'622lb # [ 0.005059] CPU features: detected: ARMv8.4 Translation Table Level623worker1 # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)624worker2 # [ 0.018824] thermal_sys: Registered thermal governor 'user_space'625worker1 # [ 0.000781] pid_max: default: 32768 minimum: 301626lb # [ 0.005062] CPU features: detected: Instruction cache invalidation not required for I/D coherence627worker2 # [ 0.018830] thermal_sys: Registered thermal governor 'power_allocator'628worker1 # [ 0.000853] LSM: initializing lsm=capability,landlock,yama,bpf,ima629worker1 # [ 0.000972] landlock: Up and running.630worker2 # [ 0.018856] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1631lb # [ 0.005066] CPU features: detected: Data cache clean to the PoU not required for I/D coherence632worker1 # [ 0.000975] Yama: becoming mindful.633worker2 # [ 0.018865] cpuidle: using governor ladder634worker1 # [ 0.001420] LSM support for eBPF active635lb # [ 0.005070] CPU features: detected: Common not Private translations636client # [ 0.163427] iommu: Default domain type: Translated637worker2 # [ 0.018872] cpuidle: using governor menu638lb # [ 0.005073] CPU features: detected: CRC32 instructions639worker1 # [ 0.001567] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)640client # [ 0.163437] iommu: DMA domain TLB invalidation policy: strict mode641worker2 # [ 0.019129] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.642client # [ 0.163782] SCSI subsystem initialized643lb # [ 0.005076] CPU features: detected: Data cache clean to Point of Deep Persistence644worker1 # [ 0.001599] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)645worker2 # [ 0.019146] ASID allocator initialised with 65536 entries646worker2 # [ 0.020434] Serial: AMBA PL011 UART driver647worker1 # [ 0.003517] rcu: Hierarchical SRCU implementation.648lb # [ 0.005079] CPU features: detected: Data cache clean to Point of Persistence649client # [ 0.165705] usbcore: registered new interface driver usbfs650worker1 # [ 0.003521] rcu: Max phase no-delay instances is 1000.651worker2 # [ 0.025882] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1652client # [ 0.165737] usbcore: registered new interface driver hub653lb # [ 0.005083] CPU features: detected: Data independent timing control (DIT)654worker1 # [ 0.004757] fsl-mc MSI: its@8080000 domain created655worker2 # [ 0.026039] printk: console [ttyAMA0] enabled656lb # [ 0.005086] CPU features: detected: E0PD657client # [ 0.165752] usbcore: registered new device driver usb658worker1 # [ 0.004849] EFI services will not be available.659client # [ 0.166012] pps_core: LinuxPPS API ver. 1 registered660worker1 # [ 0.004913] smp: Bringing up secondary CPUs ...661lb # [ 0.005088] CPU features: detected: Enhanced Counter Virtualization662worker1 # [ 0.004921] smp: Brought up 1 node, 1 CPU663lb # [ 0.005091] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)664client # [ 0.166019] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>665worker1 # [ 0.004924] SMP: Total of 1 processors activated.666client # [ 0.166029] PTP clock support registered667worker1 # [ 0.004930] CPU: All CPU(s) started at EL1668lb # [ 0.005094] CPU features: detected: Enhanced Virtualization Traps669client # [ 0.166078] EDAC MC: Ver: 3.0.0670lb # [ 0.005097] CPU features: detected: Fine Grained Traps671worker1 # [ 0.004940] CPU features: detected: Branch Target Identification672client # [ 0.170567] scmi_core: SCMI protocol bus registered673worker1 # [ 0.004944] CPU features: detected: ARMv8.4 Translation Table Level674lb # [ 0.005101] CPU features: detected: Generic authentication (architected QARMA5 algorithm)675client # [ 0.171525] FPGA manager framework676lb # [ 0.005107] CPU features: detected: RCpc load-acquire (LDAPR)677worker1 # [ 0.004948] CPU features: detected: Instruction cache invalidation not required for I/D coherence678client # [ 0.172425] vgaarb: loaded679lb # [ 0.005110] CPU features: detected: LSE atomic instructions680worker1 # [ 0.004951] CPU features: detected: Data cache clean to the PoU not required for I/D coherence681lb # [ 0.005112] CPU features: detected: Privileged Access Never682client # [ 0.173047] clocksource: Switched to clocksource arch_sys_counter683lb # [ 0.005115] CPU features: detected: PMUv3684worker1 # [ 0.004955] CPU features: detected: Common not Private translations685worker2 # [ 0.144329] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages686lb # [ 0.005118] CPU features: detected: RAS Extension Support687worker1 # [ 0.004959] CPU features: detected: CRC32 instructions688worker2 # [ 0.144345] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page689lb # [ 0.005121] CPU features: detected: RASv1p1 Extension Support690worker1 # [ 0.004961] CPU features: detected: Data cache clean to Point of Deep Persistence691worker2 # [ 0.144351] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages692lb # [ 0.005123] CPU features: detected: Random Number Generator693worker1 # [ 0.004965] CPU features: detected: Data cache clean to Point of Persistence694worker2 # [ 0.144355] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page695lb # [ 0.005126] CPU features: detected: Speculation barrier (SB)696worker1 # [ 0.004968] CPU features: detected: Data independent timing control (DIT)697worker2 # [ 0.144359] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages698lb # [ 0.005128] CPU features: detected: Stage-2 Force Write-Back699worker1 # [ 0.004971] CPU features: detected: E0PD700worker2 # [ 0.144364] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page701lb # [ 0.005131] CPU features: detected: TLB range maintenance instructions702worker1 # [ 0.004973] CPU features: detected: Enhanced Counter Virtualization703worker2 # [ 0.144368] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages704lb # [ 0.005136] CPU features: detected: Speculative Store Bypassing Safe (SSBS)705worker1 # [ 0.004977] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)706client # [ 0.179211] VFS: Disk quotas dquot_6.6.0707worker2 # [ 0.144372] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page708lb # [ 0.005173] alternatives: applying system-wide alternatives709worker1 # [ 0.004980] CPU features: detected: Enhanced Virtualization Traps710client # [ 0.179252] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)711worker1 # [ 0.004984] CPU features: detected: Fine Grained Traps712lb # [ 0.008085] CPU features: detected: BBM Level 2 without TLB conflict abort713client # [ 0.180887] netfs: FS-Cache loaded714worker1 # [ 0.004988] CPU features: detected: Generic authentication (architected QARMA5 algorithm)715client # [ 0.180986] pnp: PnP ACPI: disabled716worker2 # [ 0.151708] fbcon: Taking over console717worker1 # [ 0.004993] CPU features: detected: RCpc load-acquire (LDAPR)718lb # [ 0.008274] Memory: 1405692K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)719worker2 # [ 0.151723] ACPI: Interpreter disabled.720lb # [ 0.008660] devtmpfs: initialized721worker1 # [ 0.004996] CPU features: detected: LSE atomic instructions722worker1 # [ 0.004999] CPU features: detected: Privileged Access Never723lb # [ 0.010532] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)724worker1 # [ 0.005002] CPU features: detected: PMUv3725lb # [ 0.010554] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).726worker1 # [ 0.005005] CPU features: detected: RAS Extension Support727client # [ 0.184824] NET: Registered PF_INET protocol family728worker1 # [ 0.005008] CPU features: detected: RASv1p1 Extension Support729lb # [ 0.010746] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL730worker1 # [ 0.005010] CPU features: detected: Random Number Generator731worker1 # [ 0.005013] CPU features: detected: Speculation barrier (SB)732worker1 # [ 0.005015] CPU features: detected: Stage-2 Force Write-Back733worker1 # [ 0.005018] CPU features: detected: TLB range maintenance instructions734worker1 # [ 0.005024] CPU features: detected: Speculative Store Bypassing Safe (SSBS)735worker1 # [ 0.005063] alternatives: applying system-wide alternatives736client # [ 0.185722] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)737worker1 # [ 0.007952] CPU features: detected: BBM Level 2 without TLB conflict abort738worker2 # [ 0.160699] iommu: Default domain type: Translated739worker2 # [ 0.160709] iommu: DMA domain TLB invalidation policy: strict mode740worker2 # [ 0.161036] SCSI subsystem initialized741worker1 # [ 0.008174] Memory: 1405612K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)742worker1 # [ 0.008543] devtmpfs: initialized743worker2 # [ 0.163087] usbcore: registered new interface driver usbfs744worker1 # [ 0.010421] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)745worker2 # [ 0.163120] usbcore: registered new interface driver hub746lb # [ 0.010750] 0 pages in range for non-PLT usage747worker1 # [ 0.010443] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).748lb # [ 0.010751] 508272 pages in range for PLT usage749worker2 # [ 0.163135] usbcore: registered new device driver usb750worker2 # [ 0.163393] pps_core: LinuxPPS API ver. 1 registered751lb # [ 0.010864] pinctrl core: initialized pinctrl subsystem752worker1 # [ 0.010627] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL753lb # [ 0.011630] DMI not present or invalid.754worker1 # [ 0.010631] 0 pages in range for non-PLT usage755worker2 # [ 0.163399] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>756worker1 # [ 0.010633] 508272 pages in range for PLT usage757lb # [ 0.014813] NET: Registered PF_NETLINK/PF_ROUTE protocol family758worker2 # [ 0.163408] PTP clock support registered759worker1 # [ 0.010749] pinctrl core: initialized pinctrl subsystem760worker2 # [ 0.163459] EDAC MC: Ver: 3.0.0761lb # [ 0.017019] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations762worker1 # [ 0.011517] DMI not present or invalid.763worker2 # [ 0.167990] scmi_core: SCMI protocol bus registered764worker1 # [ 0.014588] NET: Registered PF_NETLINK/PF_ROUTE protocol family765lb # [ 0.017195] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations766worker1 # [ 0.016748] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations767worker2 # [ 0.168973] FPGA manager framework768lb # [ 0.017393] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations769worker2 # [ 0.169887] vgaarb: loaded770lb # [ 0.017415] audit: initializing netlink subsys (disabled)771lb # [ 0.017947] thermal_sys: Registered thermal governor 'fair_share'772worker2 # [ 0.170514] clocksource: Switched to clocksource arch_sys_counter773lb # [ 0.017949] thermal_sys: Registered thermal governor 'bang_bang'774lb # [ 0.017952] thermal_sys: Registered thermal governor 'step_wise'775lb # [ 0.017955] thermal_sys: Registered thermal governor 'user_space'776lb # [ 0.017960] thermal_sys: Registered thermal governor 'power_allocator'777lb # [ 0.018005] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1778lb # [ 0.018013] cpuidle: using governor ladder779lb # [ 0.018019] cpuidle: using governor menu780lb # [ 0.018205] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.781worker2 # [ 0.176731] VFS: Disk quotas dquot_6.6.0782lb # [ 0.018220] ASID allocator initialised with 65536 entries783worker2 # [ 0.176777] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)784lb # [ 0.019395] Serial: AMBA PL011 UART driver785worker1 # [ 0.016915] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations786worker2 # [ 0.178416] netfs: FS-Cache loaded787lb # [ 0.024642] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1788worker2 # [ 0.178961] pnp: PnP ACPI: disabled789worker1 # [ 0.017108] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations790lb # [ 0.024760] printk: console [ttyAMA0] enabled791worker1 # [ 0.017128] audit: initializing netlink subsys (disabled)792worker1 # [ 0.017643] thermal_sys: Registered thermal governor 'fair_share'793lb # [ 0.146049] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages794worker1 # [ 0.017645] thermal_sys: Registered thermal governor 'bang_bang'795lb # [ 0.146068] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page796worker1 # [ 0.017649] thermal_sys: Registered thermal governor 'step_wise'797lb # [ 0.146073] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages798worker2 # [ 0.182396] NET: Registered PF_INET protocol family799worker1 # [ 0.017652] thermal_sys: Registered thermal governor 'user_space'800lb # [ 0.146077] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page801worker1 # [ 0.017657] thermal_sys: Registered thermal governor 'power_allocator'802lb # [ 0.146082] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages803worker1 # [ 0.017681] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1804lb # [ 0.146086] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page805worker1 # [ 0.017689] cpuidle: using governor ladder806worker1 # [ 0.017695] cpuidle: using governor menu807lb # [ 0.146091] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages808worker1 # [ 0.017887] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.809lb # [ 0.146095] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page810worker1 # [ 0.017902] ASID allocator initialised with 65536 entries811worker2 # [ 0.183297] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)812worker1 # [ 0.019127] Serial: AMBA PL011 UART driver813worker1 # [ 0.024325] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1814lb # [ 0.153759] fbcon: Taking over console815worker1 # [ 0.024493] printk: console [ttyAMA0] enabled816lb # [ 0.153773] ACPI: Interpreter disabled.817lb # [ 0.155677] iommu: Default domain type: Translated818worker1 # [ 0.144509] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages819lb # [ 0.155687] iommu: DMA domain TLB invalidation policy: strict mode820worker1 # [ 0.144527] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page821lb # [ 0.157492] SCSI subsystem initialized822worker1 # [ 0.144532] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages823client # [ 0.214239] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)824worker1 # [ 0.144537] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page825client # [ 0.214286] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)826worker1 # [ 0.144541] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages827worker1 # [ 0.144545] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page828client # [ 0.214315] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)829worker1 # [ 0.144549] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages830client # [ 0.214388] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)831worker1 # [ 0.144553] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page832client # [ 0.214530] TCP: Hash tables configured (established 16384 bind 16384)833client # [ 0.214612] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)834client # [ 0.214647] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)835worker1 # [ 0.152096] fbcon: Taking over console836client # [ 0.214682] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)837worker1 # [ 0.152111] ACPI: Interpreter disabled.838client # [ 0.214772] NET: Registered PF_UNIX/PF_LOCAL protocol family839worker1 # [ 0.154056] iommu: Default domain type: Translated840client # [ 0.214799] NET: Registered PF_XDP protocol family841lb # [ 0.165197] usbcore: registered new interface driver usbfs842worker1 # [ 0.154066] iommu: DMA domain TLB invalidation policy: strict mode843client # [ 0.214817] PCI: CLS 0 bytes, default 64844lb # [ 0.165223] usbcore: registered new interface driver hub845client # [ 0.215067] Trying to unpack rootfs image as initramfs...846worker1 # [ 0.155821] SCSI subsystem initialized847lb # [ 0.165266] usbcore: registered new device driver usb848lb # [ 0.165553] pps_core: LinuxPPS API ver. 1 registered849lb # [ 0.165559] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>850lb # [ 0.165569] PTP clock support registered851lb # [ 0.165608] EDAC MC: Ver: 3.0.0852lb # [ 0.170227] scmi_core: SCMI protocol bus registered853lb # [ 0.171234] FPGA manager framework854lb # [ 0.172233] vgaarb: loaded855lb # [ 0.172870] clocksource: Switched to clocksource arch_sys_counter856client # [ 0.230840] kvm [1]: HYP mode not available857lb # [ 0.174274] VFS: Disk quotas dquot_6.6.0858worker1 # [ 0.162963] usbcore: registered new interface driver usbfs859lb # [ 0.174304] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)860worker1 # [ 0.162998] usbcore: registered new interface driver hub861worker1 # [ 0.163015] usbcore: registered new device driver usb862lb # [ 0.175871] netfs: FS-Cache loaded863lb # [ 0.175993] pnp: PnP ACPI: disabled864worker1 # [ 0.163309] pps_core: LinuxPPS API ver. 1 registered865worker1 # [ 0.163316] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>866worker1 # [ 0.163334] PTP clock support registered867worker1 # [ 0.163393] EDAC MC: Ver: 3.0.0868worker1 # [ 0.168027] scmi_core: SCMI protocol bus registered869worker1 # [ 0.169014] FPGA manager framework870worker1 # [ 0.170200] vgaarb: loaded871worker1 # [ 0.170824] clocksource: Switched to clocksource arch_sys_counter872worker1 # [ 0.172879] VFS: Disk quotas dquot_6.6.0873worker1 # [ 0.172907] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)874worker2 # [ 0.212512] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)875lb # [ 0.182054] NET: Registered PF_INET protocol family876lb # [ 0.182315] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)877worker2 # [ 0.212558] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)878worker2 # [ 0.212585] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)879worker2 # [ 0.212656] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)880worker1 # [ 0.176674] netfs: FS-Cache loaded881worker1 # [ 0.176796] pnp: PnP ACPI: disabled882worker2 # [ 0.212799] TCP: Hash tables configured (established 16384 bind 16384)883worker2 # [ 0.212884] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)884worker2 # [ 0.212920] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)885worker2 # [ 0.212955] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)886worker2 # [ 0.213053] NET: Registered PF_UNIX/PF_LOCAL protocol family887worker2 # [ 0.213073] NET: Registered PF_XDP protocol family888worker2 # [ 0.213090] PCI: CLS 0 bytes, default 64889worker2 # [ 0.213348] Trying to unpack rootfs image as initramfs...890worker1 # [ 0.180733] NET: Registered PF_INET protocol family891worker1 # [ 0.180996] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)892worker2 # [ 0.228298] kvm [1]: HYP mode not available893lb # [ 0.211452] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)894lb # [ 0.211503] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)895worker1 # [ 0.210794] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)896lb # [ 0.211530] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)897lb # [ 0.211606] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)898lb # [ 0.211749] TCP: Hash tables configured (established 16384 bind 16384)899lb # [ 0.211832] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)900lb # [ 0.211875] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)901lb # [ 0.211911] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)902lb # [ 0.212008] NET: Registered PF_UNIX/PF_LOCAL protocol family903lb # [ 0.212029] NET: Registered PF_XDP protocol family904lb # [ 0.212046] PCI: CLS 0 bytes, default 64905lb # [ 0.212300] Trying to unpack rootfs image as initramfs...906worker1 # [ 0.212069] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)907worker1 # [ 0.212099] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)908worker1 # [ 0.212170] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)909worker1 # [ 0.212314] TCP: Hash tables configured (established 16384 bind 16384)910worker1 # [ 0.212400] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)911worker1 # [ 0.212437] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)912worker1 # [ 0.212473] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)913worker1 # [ 0.212565] NET: Registered PF_UNIX/PF_LOCAL protocol family914worker1 # [ 0.212585] NET: Registered PF_XDP protocol family915lb # [ 0.230667] kvm [1]: HYP mode not available916worker1 # [ 0.212605] PCI: CLS 0 bytes, default 64917worker1 # [ 0.212834] Trying to unpack rootfs image as initramfs...918worker1 # [ 0.234854] kvm [1]: HYP mode not available919client # [ 0.321603] Initialise system trusted keyrings920client # [ 0.322442] workingset: timestamp_bits=42 max_order=19 bucket_order=0921client # [ 0.323687] squashfs: version 4.0 (2009/01/31) Phillip Lougher922client # [ 0.324465] 9p: Installing v9fs 9p2000 file system support923worker2 # [ 0.323244] Initialise system trusted keyrings924client # [ 0.353246] Key type asymmetric registered925client # [ 0.353271] Asymmetric key parser 'x509' registered926worker2 # [ 0.323976] workingset: timestamp_bits=42 max_order=19 bucket_order=0927client # [ 0.353338] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)928worker2 # [ 0.325646] squashfs: version 4.0 (2009/01/31) Phillip Lougher929client # [ 0.355466] io scheduler mq-deadline registered930worker2 # [ 0.326418] 9p: Installing v9fs 9p2000 file system support931client # [ 0.355477] io scheduler kyber registered932client # [ 0.365189] pl061_gpio 9030000.pl061: PL061 GPIO chip registered933client # [ 0.366597] ledtrig-cpu: registered to indicate activity on CPUs934client # [ 0.366987] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:935client # [ 0.367004] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000936client # [ 0.367016] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000937client # [ 0.367025] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000938client # [ 0.367046] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits939client # [ 0.367071] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]940client # [ 0.367144] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00941client # [ 0.367154] pci_bus 0000:00: root bus resource [bus 00-ff]942client # [ 0.367160] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]943client # [ 0.367165] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]944client # [ 0.367170] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]945client # [ 0.367256] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint946lb # [ 0.317850] Initialise system trusted keyrings947client # [ 0.367685] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint948client # [ 0.367866] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]949client # [ 0.367882] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]950client # [ 0.367911] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]951client # [ 0.367927] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]952worker2 # [ 0.355179] Key type asymmetric registered953worker2 # [ 0.355204] Asymmetric key parser 'x509' registered954client # [ 0.368377] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint955client # [ 0.368581] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]956worker2 # [ 0.355274] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)957client # [ 0.368596] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]958worker2 # [ 0.357396] io scheduler mq-deadline registered959worker2 # [ 0.357406] io scheduler kyber registered960client # [ 0.368625] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]961lb # [ 0.324945] workingset: timestamp_bits=42 max_order=19 bucket_order=0962client # [ 0.388098] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint963lb # [ 0.326368] squashfs: version 4.0 (2009/01/31) Phillip Lougher964client # [ 0.388296] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]965client # [ 0.388313] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]966client # [ 0.388343] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]967lb # [ 0.327177] 9p: Installing v9fs 9p2000 file system support968client # [ 0.388898] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint969worker2 # [ 0.366663] pl061_gpio 9030000.pl061: PL061 GPIO chip registered970worker1 # [ 0.327386] Initialise system trusted keyrings971worker1 # [ 0.328128] workingset: timestamp_bits=42 max_order=19 bucket_order=0972worker2 # [ 0.368162] ledtrig-cpu: registered to indicate activity on CPUs973worker1 # [ 0.329402] squashfs: version 4.0 (2009/01/31) Phillip Lougher974worker2 # [ 0.368585] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:975worker1 # [ 0.330316] 9p: Installing v9fs 9p2000 file system support976worker2 # [ 0.368603] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000977worker2 # [ 0.368615] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000978worker2 # [ 0.368624] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000979worker2 # [ 0.368648] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits980client # [ 0.401170] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]981client # [ 0.401201] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]982worker2 # [ 0.368674] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]983client # [ 0.401232] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]984worker2 # [ 0.368747] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00985lb # [ 0.347894] Key type asymmetric registered986client # [ 0.401251] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]987worker2 # [ 0.368756] pci_bus 0000:00: root bus resource [bus 00-ff]988lb # [ 0.347915] Asymmetric key parser 'x509' registered989worker2 # [ 0.368762] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]990client # [ 0.401764] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint991lb # [ 0.348000] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)992worker2 # [ 0.368767] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]993client # [ 0.401953] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]994worker2 # [ 0.368772] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]995client # [ 0.401983] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]996worker2 # [ 0.368859] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint997client # [ 0.402441] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint998client # [ 0.402627] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]999worker2 # [ 0.369288] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1000client # [ 0.402656] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1001worker2 # [ 0.369468] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1002worker2 # [ 0.369485] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1003client # [ 0.403044] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1004worker2 # [ 0.369514] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1005client # [ 0.403225] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1006worker2 # [ 0.369531] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1007client # [ 0.403515] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1008worker2 # [ 0.369998] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1009client # [ 0.403701] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1010lb # [ 0.357004] io scheduler mq-deadline registered1011worker2 # [ 0.370177] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1012lb # [ 0.357022] io scheduler kyber registered1013client # [ 0.403731] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1014worker2 # [ 0.370192] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1015client # [ 0.404179] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1016worker2 # [ 0.370222] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017client # [ 0.404366] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1018worker2 # [ 0.370688] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1019client # [ 0.404421] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1020worker2 # [ 0.370872] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1021client # [ 0.404874] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1022worker2 # [ 0.370888] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1023client # [ 0.405077] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1024worker2 # [ 0.370917] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1025client # [ 0.405107] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1026lb # [ 0.362086] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1027worker2 # [ 0.371379] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1028client # [ 0.405566] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1029worker2 # [ 0.371560] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1030client # [ 0.405879] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1031worker2 # [ 0.371575] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1032client # [ 0.405898] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1033lb # [ 0.363377] ledtrig-cpu: registered to indicate activity on CPUs1034worker2 # [ 0.371605] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035client # [ 0.405927] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1036lb # [ 0.363755] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1037worker2 # [ 0.371623] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1038client # [ 0.406394] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1039worker2 # [ 0.372072] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1040lb # [ 0.363772] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001041client # [ 0.406576] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1042worker2 # [ 0.372259] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1043lb # [ 0.363785] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001044client # [ 0.406592] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1045worker2 # [ 0.372289] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046client # [ 0.406622] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1047lb # [ 0.363794] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001048worker2 # [ 0.372743] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1049client # [ 0.407212] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1050lb # [ 0.363814] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1051worker2 # [ 0.372932] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1052worker1 # [ 0.359063] Key type asymmetric registered1053client # [ 0.407224] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1054worker1 # [ 0.359090] Asymmetric key parser 'x509' registered1055worker2 # [ 0.372964] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1056lb # [ 0.363848] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1057client # [ 0.407229] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1058worker1 # [ 0.359169] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1059worker2 # [ 0.373374] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1060lb # [ 0.363930] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001061worker1 # [ 0.361200] io scheduler mq-deadline registered1062client # [ 0.407275] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1063worker2 # [ 0.373558] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1064lb # [ 0.363939] pci_bus 0000:00: root bus resource [bus 00-ff]1065worker1 # [ 0.361211] io scheduler kyber registered1066lb # [ 0.363945] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1067client # [ 0.407321] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1068worker2 # [ 0.373829] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1069lb # [ 0.363950] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1070worker2 # [ 0.374016] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1071client # [ 0.407368] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1072lb # [ 0.363955] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1073worker2 # [ 0.374046] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074client # [ 0.407414] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1075worker2 # [ 0.374490] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1076lb # [ 0.364055] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1077client # [ 0.407460] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1078lb # [ 0.364499] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1079client # [ 0.407506] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1080lb # [ 0.364689] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1081client # [ 0.407552] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1082lb # [ 0.364705] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1083client # [ 0.407598] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1084lb # [ 0.364736] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085lb # [ 0.364752] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1086client # [ 0.407645] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1087client # [ 0.407718] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1088client # [ 0.407763] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1089client # [ 0.407786] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1090worker1 # [ 0.370977] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1091client # [ 0.407807] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1092client # [ 0.407828] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1093client # [ 0.407849] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1094client # [ 0.407871] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1095worker1 # [ 0.372397] ledtrig-cpu: registered to indicate activity on CPUs1096client # [ 0.407893] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1097worker1 # [ 0.372809] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1098client # [ 0.407917] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1099worker1 # [ 0.372827] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001100client # [ 0.407939] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1101worker1 # [ 0.372839] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001102client # [ 0.407961] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1103lb # [ 0.385324] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1104lb # [ 0.385530] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1105client # [ 0.407984] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1106worker1 # [ 0.372848] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001107lb # [ 0.385548] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1108client # [ 0.408005] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1109worker1 # [ 0.372874] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1110lb # [ 0.385578] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1111client # [ 0.408027] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1112worker1 # [ 0.372899] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1113client # [ 0.408048] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1114lb # [ 0.386050] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1115worker1 # [ 0.372971] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001116client # [ 0.408069] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1117lb # [ 0.386236] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1118worker1 # [ 0.372981] pci_bus 0000:00: root bus resource [bus 00-ff]1119client # [ 0.408090] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1120lb # [ 0.386252] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1121worker1 # [ 0.372987] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1122client # [ 0.408111] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1123lb # [ 0.386282] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124client # [ 0.408132] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1125worker1 # [ 0.372992] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1126lb # [ 0.386735] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1127worker2 # [ 0.422782] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1128client # [ 0.408159] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1129worker1 # [ 0.372997] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1130lb # [ 0.386919] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1131worker2 # [ 0.422828] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1132client # [ 0.408169] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1133lb # [ 0.386935] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1134worker1 # [ 0.373095] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1135client # [ 0.408174] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1136worker2 # [ 0.423378] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1137lb # [ 0.386965] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1138worker1 # [ 0.373529] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1139client # [ 0.408986] pci 0000:00:07.0: enabling device (0000 -> 0002)1140worker2 # [ 0.423566] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1141lb # [ 0.386984] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1142worker1 # [ 0.373715] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1143worker2 # [ 0.423596] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144worker1 # [ 0.373731] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1145lb # [ 0.387444] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1146worker2 # [ 0.424063] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1147lb # [ 0.387632] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1148worker1 # [ 0.373761] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker2 # [ 0.424371] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1150worker1 # [ 0.373777] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1151lb # [ 0.387662] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1152worker2 # [ 0.424389] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1153worker1 # [ 0.374251] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1154lb # [ 0.388123] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1155worker2 # [ 0.424419] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1156worker1 # [ 0.374433] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1157lb # [ 0.388312] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1158worker2 # [ 0.424877] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1159worker1 # [ 0.374449] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1160lb # [ 0.388343] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1161worker2 # [ 0.425060] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1162client # [ 0.465206] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 54906 usecs1163worker1 # [ 0.374479] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164worker2 # [ 0.425076] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1165lb # [ 0.388736] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1166worker2 # [ 0.425106] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167worker2 # [ 0.425682] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1168worker1 # [ 0.393413] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1169worker2 # [ 0.425694] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1170worker1 # [ 0.393601] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1171worker1 # [ 0.393617] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1172worker2 # [ 0.425699] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1173worker1 # [ 0.393647] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1174worker2 # [ 0.425749] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1175worker1 # [ 0.394112] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1176worker2 # [ 0.425800] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1177worker1 # [ 0.394294] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1178worker2 # [ 0.425850] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1179worker1 # [ 0.394310] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1180worker1 # [ 0.394339] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1181worker2 # [ 0.425900] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1182worker1 # [ 0.394357] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1183worker2 # [ 0.425947] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1184lb # [ 0.409000] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1185worker1 # [ 0.394805] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1186worker2 # [ 0.425995] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1187lb # [ 0.409345] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1188worker2 # [ 0.426043] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1189lb # [ 0.409540] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1190lb # [ 0.409571] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1191worker2 # [ 0.426089] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1192lb # [ 0.410031] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1193worker2 # [ 0.426135] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1194lb # [ 0.410220] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1195worker2 # [ 0.426213] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1196lb # [ 0.410249] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1197worker2 # [ 0.426260] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1198lb # [ 0.410705] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1199worker2 # [ 0.426282] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1200lb # [ 0.410893] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1201worker2 # [ 0.426303] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1202lb # [ 0.410924] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1203worker2 # [ 0.426324] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1204lb # [ 0.411384] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1205worker2 # [ 0.426346] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1206lb # [ 0.411686] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1207worker2 # [ 0.426367] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1208lb # [ 0.411704] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1209worker2 # [ 0.426389] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1210lb # [ 0.411734] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1211worker2 # [ 0.426415] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1212lb # [ 0.412204] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1213worker2 # [ 0.426437] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1214lb # [ 0.412389] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1215client # [ 0.485943] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1216worker1 # [ 0.407050] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1217worker2 # [ 0.426459] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1218client # [ 0.488146] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1219lb # [ 0.412405] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1220worker1 # [ 0.407091] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1221worker2 # [ 0.426481] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1222lb # [ 0.412435] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1223worker2 # [ 0.426503] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1224worker1 # [ 0.407574] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1225worker2 # [ 0.426550] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1226worker1 # [ 0.407762] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1227worker2 # [ 0.426572] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1228worker1 # [ 0.407791] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1229worker2 # [ 0.426593] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1230worker1 # [ 0.408178] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1231worker2 # [ 0.426614] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1232worker1 # [ 0.408360] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1233worker2 # [ 0.426635] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1234worker1 # [ 0.408660] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1235worker2 # [ 0.426656] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1236worker1 # [ 0.408852] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1237worker2 # [ 0.426682] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1238worker2 # [ 0.426692] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1239worker1 # [ 0.408882] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1240worker2 # [ 0.426696] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1241worker1 # [ 0.409341] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1242worker2 # [ 0.427519] pci 0000:00:07.0: enabling device (0000 -> 0002)1243worker1 # [ 0.409527] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1244worker1 # [ 0.409557] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1245worker1 # [ 0.410018] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1246lb # [ 0.433116] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1247worker1 # [ 0.410205] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1248client # [ 0.499343] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1249lb # [ 0.433140] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1250worker1 # [ 0.410234] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1251lb # [ 0.433145] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1252worker1 # [ 0.410695] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1253lb # [ 0.433199] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1254worker1 # [ 0.410994] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1255worker1 # [ 0.411010] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1256lb # [ 0.433248] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1257worker1 # [ 0.411040] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1258lb # [ 0.433297] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1259worker1 # [ 0.411499] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1260lb # [ 0.433346] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1261worker1 # [ 0.411681] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1262lb # [ 0.433394] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1263worker1 # [ 0.411697] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1264worker1 # [ 0.411726] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1265lb # [ 0.433443] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1266worker1 # [ 0.412296] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1267lb # [ 0.433490] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1268worker1 # [ 0.412308] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1269lb # [ 0.433538] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1270worker1 # [ 0.412313] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1271lb # [ 0.433585] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1272worker1 # [ 0.412358] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1273lb # [ 0.433710] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1274worker1 # [ 0.412405] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1275lb # [ 0.433759] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1276worker1 # [ 0.412452] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1277lb # [ 0.433781] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1278lb # [ 0.433804] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1279worker1 # [ 0.412498] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1280lb # [ 0.433825] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1281worker1 # [ 0.412545] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1282lb # [ 0.433847] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1283worker1 # [ 0.412591] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1284client # [ 0.509330] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1285lb # [ 0.433869] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1286client # [ 0.513303] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1287worker1 # [ 0.412637] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1288lb # [ 0.433891] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1289client # [ 0.515466] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1290worker1 # [ 0.412682] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1291lb # [ 0.433919] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1292worker1 # [ 0.412729] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1293lb # [ 0.433942] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1294worker2 # [ 0.488469] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1295worker1 # [ 0.412814] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1296lb # [ 0.433968] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1297lb # [ 0.433991] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1298worker1 # [ 0.412875] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1299worker1 # [ 0.412897] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1300lb # [ 0.434014] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1301lb # [ 0.434036] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1302worker1 # [ 0.412918] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1303lb # [ 0.434058] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1304worker1 # [ 0.412939] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1305lb # [ 0.434081] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1306worker1 # [ 0.412960] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1307lb # [ 0.434103] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1308worker1 # [ 0.412981] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1309lb # [ 0.434124] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1310worker1 # [ 0.413002] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1311lb # [ 0.434147] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1312worker1 # [ 0.413026] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1313lb # [ 0.434175] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1314worker1 # [ 0.413048] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1315lb # [ 0.434186] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1316worker1 # [ 0.413070] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1317client # [ 0.525259] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1318lb # [ 0.434191] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1319worker1 # [ 0.413092] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1320lb # [ 0.435030] pci 0000:00:07.0: enabling device (0000 -> 0002)1321client # [ 0.527322] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1322worker1 # [ 0.413114] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1323worker1 # [ 0.413134] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1324worker2 # [ 0.498737] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1325worker1 # [ 0.413155] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1326worker1 # [ 0.413176] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1327worker2 # [ 0.501627] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1328worker1 # [ 0.413197] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1329worker1 # [ 0.413218] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1330worker1 # [ 0.413239] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1331worker1 # [ 0.413264] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1332worker1 # [ 0.413273] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1333worker1 # [ 0.413278] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1334worker1 # [ 0.414093] pci 0000:00:07.0: enabling device (0000 -> 0002)1335worker2 # [ 0.511591] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1336client # [ 0.537216] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1337worker2 # [ 0.514424] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1338client # [ 0.539095] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1339client # [ 0.541601] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1340worker2 # [ 0.518125] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1341lb # [ 0.485941] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1342lb # [ 0.488116] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1343client # [ 0.554655] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1344worker2 # [ 0.523716] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1345worker2 # [ 0.526147] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1346client # [ 0.557138] msm_serial: driver initialized1347client # [ 0.557321] SuperH (H)SCI(F) driver initialized1348client # [ 0.557375] STM32 USART driver initialized1349lb # [ 0.499240] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1350worker1 # [ 0.490709] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1351worker2 # [ 0.535842] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1352worker2 # [ 0.538135] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1353worker1 # [ 0.500237] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1354worker2 # [ 0.547920] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1355lb # [ 0.509121] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1356lb # [ 0.512276] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1357lb # [ 0.514495] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1358lb # [ 0.516402] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1359worker2 # [ 0.561778] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1360lb # [ 0.526423] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1361client # [ 0.591856] loop: module loaded1362lb # [ 0.528316] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1363worker1 # [ 0.511441] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1364client # [ 0.592034] virtio_blk virtio2: 1/0/0 default/read/poll queues1365worker1 # [ 0.513608] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1366client # [ 0.592848] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1367worker1 # [ 0.515667] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1368worker1 # [ 0.517776] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1369worker2 # [ 0.564328] msm_serial: driver initialized1370worker2 # [ 0.564512] SuperH (H)SCI(F) driver initialized1371worker2 # [ 0.564565] STM32 USART driver initialized1372client # [ 0.597675] megasas: 07.734.00.00-rc11373client # [ 0.598378] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1374lb # [ 0.539566] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1375worker1 # [ 0.528229] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1376worker1 # [ 0.530141] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1377lb # [ 0.546072] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1378client # [ 0.609761] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001379client # [ 0.609794] Intel/Sharp Extended Query Table at 0x00311380client # [ 0.611416] Using buffer write method1381client # [ 0.611482] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1382worker1 # [ 0.541478] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1383worker1 # [ 0.544246] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1384worker1 # [ 0.546489] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1385client # [ 0.621098] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001386client # [ 0.621128] Intel/Sharp Extended Query Table at 0x00311387client # [ 0.624390] Using buffer write method1388lb # [ 0.562140] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1389client # [ 0.624428] Concatenating MTD devices:1390client # [ 0.624432] (0): "0.flash"1391client # [ 0.624441] (1): "0.flash"1392client # [ 0.624445] into device "0.flash"1393lb # [ 0.564783] msm_serial: driver initialized1394worker2 # [ 0.598760] loop: module loaded1395worker2 # [ 0.598947] virtio_blk virtio2: 1/0/0 default/read/poll queues1396worker2 # [ 0.599854] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1397lb # [ 0.565524] SuperH (H)SCI(F) driver initialized1398lb # [ 0.565585] STM32 USART driver initialized1399worker1 # [ 0.562892] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1400worker1 # [ 0.565448] msm_serial: driver initialized1401worker1 # [ 0.565628] SuperH (H)SCI(F) driver initialized1402worker1 # [ 0.565683] STM32 USART driver initialized1403worker2 # [ 0.607231] megasas: 07.734.00.00-rc11404worker2 # [ 0.607976] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1405worker2 # [ 0.609873] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406worker2 # [ 0.609896] Intel/Sharp Extended Query Table at 0x00311407worker2 # [ 0.611705] Using buffer write method1408worker2 # [ 0.611762] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1409worker2 # [ 0.613633] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001410worker2 # [ 0.613656] Intel/Sharp Extended Query Table at 0x00311411lb # [ 0.595386] loop: module loaded1412lb # [ 0.595572] virtio_blk virtio2: 1/0/0 default/read/poll queues1413lb # [ 0.596328] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1414worker2 # [ 0.633285] Using buffer write method1415worker2 # [ 0.633320] Concatenating MTD devices:1416worker2 # [ 0.633324] (0): "0.flash"1417worker2 # [ 0.633327] (1): "0.flash"1418worker2 # [ 0.633336] into device "0.flash"1419lb # [ 0.601509] megasas: 07.734.00.00-rc11420lb # [ 0.602208] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1421lb # [ 0.604443] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001422lb # [ 0.604494] Intel/Sharp Extended Query Table at 0x00311423worker1 # [ 0.599066] loop: module loaded1424worker1 # [ 0.599264] virtio_blk virtio2: 1/0/0 default/read/poll queues1425worker1 # [ 0.600055] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1426lb # [ 0.613895] Using buffer write method1427lb # [ 0.613973] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1428lb # [ 0.616857] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001429worker1 # [ 0.607420] megasas: 07.734.00.00-rc11430worker1 # [ 0.608123] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1431lb # [ 0.624933] Intel/Sharp Extended Query Table at 0x00311432lb # [ 0.626452] Using buffer write method1433lb # [ 0.626482] Concatenating MTD devices:1434worker1 # [ 0.610331] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001435lb # [ 0.626486] (0): "0.flash"1436worker1 # [ 0.610357] Intel/Sharp Extended Query Table at 0x00311437lb # [ 0.626490] (1): "0.flash"1438worker1 # [ 0.611854] Using buffer write method1439lb # [ 0.626500] into device "0.flash"1440worker1 # [ 0.611927] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1441worker1 # [ 0.613579] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001442worker1 # [ 0.613601] Intel/Sharp Extended Query Table at 0x00311443worker1 # [ 0.632425] Using buffer write method1444worker1 # [ 0.632460] Concatenating MTD devices:1445worker1 # [ 0.632465] (0): "0.flash"1446worker1 # [ 0.632469] (1): "0.flash"1447worker1 # [ 0.632478] into device "0.flash"1448client # [ 0.876776] Freeing initrd memory: 26972K1449client # [ 0.882807] tun: Universal TUN/TAP device driver, 1.61450client # [ 0.886763] thunder_xcv, ver 1.01451client # [ 0.886814] thunder_bgx, ver 1.01452client # [ 0.886836] nicpf, ver 1.01453client # [ 0.887407] e1000: Intel(R) PRO/1000 Network Driver1454client # [ 0.887415] e1000: Copyright (c) 1999-2006 Intel Corporation.1455client # [ 0.887440] e1000e: Intel(R) PRO/1000 Network Driver1456client # [ 0.887447] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457client # [ 0.887476] igb: Intel(R) Gigabit Ethernet Network Driver1458client # [ 0.887482] igb: Copyright (c) 2007-2014 Intel Corporation.1459client # [ 0.887505] igbvf: Intel(R) Gigabit Virtual Function Network Driver1460client # [ 0.887511] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1461client # [ 0.887645] sky2: driver version 1.301462client # [ 0.889799] ehci-pci 0000:00:07.0: EHCI Host Controller1463client # [ 0.889831] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11464client # [ 0.890108] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001465client # [ 0.899826] usbcore: registered new interface driver usb-storage1466client # [ 0.899917] usbcore: registered new interface driver usbserial_generic1467client # [ 0.899931] usbserial: USB Serial support registered for generic1468client # [ 0.900535] hv_vmbus: registering driver hyperv_keyboard1469client # [ 0.901094] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001470client # [ 0.901467] hub 1-0:1.0: USB hub found1471client # [ 0.901495] hub 1-0:1.0: 6 ports detected1472client # [ 0.906797] rtc-pl031 9010000.pl031: registered as rtc01473client # [ 0.906828] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T14:51:05 UTC (1790088665)1474client # [ 0.907187] i2c_dev: i2c /dev entries driver1475client # [ 0.912554] sdhci: Secure Digital Host Controller Interface driver1476client # [ 0.912572] sdhci: Copyright(c) Pierre Ossman1477client # [ 0.912836] Synopsys Designware Multimedia Card Interface Driver1478worker2 # [ 0.884051] Freeing initrd memory: 26944K1479client # [ 0.915219] sdhci-pltfm: SDHCI platform and OF driver helper1480client # [ 0.916795] hid: raw HID events driver (C) Jiri Kosina1481client # [ 0.917037] usbcore: registered new interface driver usbhid1482client # [ 0.917044] usbhid: USB HID core driver1483worker2 # [ 0.890182] tun: Universal TUN/TAP device driver, 1.61484client # [ 0.920674] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1485worker2 # [ 0.893935] thunder_xcv, ver 1.01486client # [ 0.922222] drop_monitor: Initializing network drop monitor service1487worker2 # [ 0.893978] thunder_bgx, ver 1.01488client # [ 0.922377] NET: Registered PF_INET6 protocol family1489worker2 # [ 0.894001] nicpf, ver 1.01490client # [ 0.925588] Segment Routing with IPv61491client # [ 0.925610] In-situ OAM (IOAM) with IPv61492client # [ 0.925637] NET: Registered PF_PACKET protocol family1493worker2 # [ 0.895797] e1000: Intel(R) PRO/1000 Network Driver1494client # [ 0.927232] 9pnet: Installing 9P2000 support1495client # [ 0.927283] Key type dns_resolver registered1496worker2 # [ 0.895809] e1000: Copyright (c) 1999-2006 Intel Corporation.1497worker2 # [ 0.895835] e1000e: Intel(R) PRO/1000 Network Driver1498worker2 # [ 0.895845] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1499worker2 # [ 0.895875] igb: Intel(R) Gigabit Ethernet Network Driver1500worker2 # [ 0.895881] igb: Copyright (c) 2007-2014 Intel Corporation.1501worker2 # [ 0.895925] igbvf: Intel(R) Gigabit Virtual Function Network Driver1502worker2 # [ 0.895931] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1503worker2 # [ 0.896072] sky2: driver version 1.301504worker2 # [ 0.897668] usbcore: registered new interface driver usb-storage1505worker2 # [ 0.897717] usbcore: registered new interface driver usbserial_generic1506worker2 # [ 0.897736] usbserial: USB Serial support registered for generic1507client # [ 0.934009] registered taskstats version 11508worker2 # [ 0.898334] hv_vmbus: registering driver hyperv_keyboard1509client # [ 0.934175] Loading compiled-in X.509 certificates1510worker2 # [ 0.899236] ehci-pci 0000:00:07.0: EHCI Host Controller1511worker2 # [ 0.899263] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11512worker2 # [ 0.899475] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001513worker2 # [ 0.910842] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001514worker2 # [ 0.911265] hub 1-0:1.0: USB hub found1515worker2 # [ 0.911285] hub 1-0:1.0: 6 ports detected1516client # [ 0.942841] Demotion targets for Node 0: null1517lb # [ 0.878692] Freeing initrd memory: 26948K1518client # [ 0.942956] Key type .fscrypt registered1519worker2 # [ 0.914053] rtc-pl031 9010000.pl031: registered as rtc01520client # [ 0.942962] Key type fscrypt-provisioning registered1521worker1 # [ 0.870326] Freeing initrd memory: 26976K1522worker2 # [ 0.914082] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T14:51:05 UTC (1790088665)1523client # [ 0.943063] ima: No TPM chip found, activating TPM-bypass!1524worker2 # [ 0.914389] i2c_dev: i2c /dev entries driver1525client # [ 0.943083] ima: Allocated hash algorithm: sha11526client # [ 0.943106] ima: No architecture policies found1527client # [ 0.947218] input: gpio-keys as /devices/platform/gpio-keys/input/input01528lb # [ 0.884729] tun: Universal TUN/TAP device driver, 1.61529worker1 # [ 0.876362] tun: Universal TUN/TAP device driver, 1.61530worker2 # [ 0.919727] sdhci: Secure Digital Host Controller Interface driver1531lb # [ 0.888559] thunder_xcv, ver 1.01532worker2 # [ 0.919738] sdhci: Copyright(c) Pierre Ossman1533lb # [ 0.888605] thunder_bgx, ver 1.01534lb # [ 0.888627] nicpf, ver 1.01535worker2 # [ 0.920005] Synopsys Designware Multimedia Card Interface Driver1536worker2 # [ 0.920376] sdhci-pltfm: SDHCI platform and OF driver helper1537worker2 # [ 0.921894] hid: raw HID events driver (C) Jiri Kosina1538worker2 # [ 0.922175] usbcore: registered new interface driver usbhid1539worker2 # [ 0.922181] usbhid: USB HID core driver1540lb # [ 0.890465] e1000: Intel(R) PRO/1000 Network Driver1541lb # [ 0.890475] e1000: Copyright (c) 1999-2006 Intel Corporation.1542worker1 # [ 0.880241] thunder_xcv, ver 1.01543worker1 # [ 0.880284] thunder_bgx, ver 1.01544lb # [ 0.890500] e1000e: Intel(R) PRO/1000 Network Driver1545worker1 # [ 0.880308] nicpf, ver 1.01546lb # [ 0.890508] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1547worker1 # [ 0.880877] e1000: Intel(R) PRO/1000 Network Driver1548worker2 # [ 0.927854] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1549lb # [ 0.890538] igb: Intel(R) Gigabit Ethernet Network Driver1550worker1 # [ 0.880884] e1000: Copyright (c) 1999-2006 Intel Corporation.1551worker2 # [ 0.929406] drop_monitor: Initializing network drop monitor service1552lb # [ 0.890544] igb: Copyright (c) 2007-2014 Intel Corporation.1553worker1 # [ 0.880909] e1000e: Intel(R) PRO/1000 Network Driver1554worker2 # [ 0.929576] NET: Registered PF_INET6 protocol family1555lb # [ 0.890568] igbvf: Intel(R) Gigabit Virtual Function Network Driver1556worker1 # [ 0.880919] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1557worker2 # [ 0.932561] Segment Routing with IPv61558worker1 # [ 0.880949] igb: Intel(R) Gigabit Ethernet Network Driver1559lb # [ 0.890573] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1560worker2 # [ 0.932582] In-situ OAM (IOAM) with IPv61561lb # [ 0.890711] sky2: driver version 1.301562worker1 # [ 0.880955] igb: Copyright (c) 2007-2014 Intel Corporation.1563worker2 # [ 0.932627] NET: Registered PF_PACKET protocol family1564lb # [ 0.892286] usbcore: registered new interface driver usb-storage1565worker1 # [ 0.880999] igbvf: Intel(R) Gigabit Virtual Function Network Driver1566worker2 # [ 0.934229] 9pnet: Installing 9P2000 support1567lb # [ 0.892380] usbcore: registered new interface driver usbserial_generic1568worker1 # [ 0.881005] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1569worker2 # [ 0.934276] Key type dns_resolver registered1570worker1 # [ 0.881142] sky2: driver version 1.301571lb # [ 0.892394] usbserial: USB Serial support registered for generic1572client # [ 0.964846] clk: Disabling unused clocks1573worker1 # [ 0.882776] usbcore: registered new interface driver usb-storage1574lb # [ 0.893650] ehci-pci 0000:00:07.0: EHCI Host Controller1575client # [ 0.964877] PM: genpd: Disabling unused power domains1576worker1 # [ 0.883570] ehci-pci 0000:00:07.0: EHCI Host Controller1577lb # [ 0.893676] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11578worker1 # [ 0.883596] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11579lb # [ 0.893842] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001580worker1 # [ 0.883763] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001581lb # [ 0.904717] hv_vmbus: registering driver hyperv_keyboard1582client # [ 0.968994] Freeing unused kernel memory: 4736K1583lb # [ 0.905600] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001584client # [ 0.969783] Run /init as init process1585lb # [ 0.906635] hub 1-0:1.0: USB hub found1586worker1 # [ 0.893897] usbcore: registered new interface driver usbserial_generic1587lb # [ 0.907114] hub 1-0:1.0: 6 ports detected1588worker1 # [ 0.893922] usbserial: USB Serial support registered for generic1589worker1 # [ 0.894580] hv_vmbus: registering driver hyperv_keyboard1590worker2 # [ 0.941011] registered taskstats version 11591worker1 # [ 0.894877] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001592worker2 # [ 0.941169] Loading compiled-in X.509 certificates1593lb # [ 0.908840] rtc-pl031 9010000.pl031: registered as rtc01594worker1 # [ 0.895185] hub 1-0:1.0: USB hub found1595worker1 # [ 0.895205] hub 1-0:1.0: 6 ports detected1596lb # [ 0.909539] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T14:51:05 UTC (1790088665)1597lb # [ 0.909855] i2c_dev: i2c /dev entries driver1598worker1 # [ 0.899944] rtc-pl031 9010000.pl031: registered as rtc01599worker1 # [ 0.899972] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T14:51:05 UTC (1790088665)1600worker1 # [ 0.900271] i2c_dev: i2c /dev entries driver1601worker2 # [ 0.949760] Demotion targets for Node 0: null1602worker2 # [ 0.949872] Key type .fscrypt registered1603lb # [ 0.914632] sdhci: Secure Digital Host Controller Interface driver1604worker2 # [ 0.949878] Key type fscrypt-provisioning registered1605lb # [ 0.914642] sdhci: Copyright(c) Pierre Ossman1606worker2 # [ 0.949975] ima: No TPM chip found, activating TPM-bypass!1607worker1 # [ 0.905652] sdhci: Secure Digital Host Controller Interface driver1608lb # [ 0.914906] Synopsys Designware Multimedia Card Interface Driver1609worker2 # [ 0.949994] ima: Allocated hash algorithm: sha11610worker1 # [ 0.905664] sdhci: Copyright(c) Pierre Ossman1611lb # [ 0.915279] sdhci-pltfm: SDHCI platform and OF driver helper1612worker2 # [ 0.950016] ima: No architecture policies found1613worker1 # [ 0.905947] Synopsys Designware Multimedia Card Interface Driver1614lb # [ 0.916780] hid: raw HID events driver (C) Jiri Kosina1615worker1 # [ 0.906318] sdhci-pltfm: SDHCI platform and OF driver helper1616client # [ 0.983282] systemd[1]: Successfully made /usr/ read-only.1617worker2 # [ 0.954267] input: gpio-keys as /devices/platform/gpio-keys/input/input01618lb # [ 0.920350] usbcore: registered new interface driver usbhid1619lb # [ 0.920362] usbhid: USB HID core driver1620worker1 # [ 0.910501] hid: raw HID events driver (C) Jiri Kosina1621worker1 # [ 0.910768] usbcore: registered new interface driver usbhid1622worker1 # [ 0.910775] usbhid: USB HID core driver1623lb # [ 0.922709] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1624lb # [ 0.924248] drop_monitor: Initializing network drop monitor service1625worker1 # [ 0.913669] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1626lb # [ 0.924391] NET: Registered PF_INET6 protocol family1627lb # [ 0.927569] Segment Routing with IPv61628worker1 # [ 0.916386] drop_monitor: Initializing network drop monitor service1629lb # [ 0.927587] In-situ OAM (IOAM) with IPv61630worker1 # [ 0.916535] NET: Registered PF_INET6 protocol family1631lb # [ 0.927616] NET: Registered PF_PACKET protocol family1632worker1 # [ 0.918447] Segment Routing with IPv61633worker1 # [ 0.918466] In-situ OAM (IOAM) with IPv61634worker1 # [ 0.918492] NET: Registered PF_PACKET protocol family1635lb # [ 0.929516] 9pnet: Installing 9P2000 support1636lb # [ 0.929564] Key type dns_resolver registered1637worker1 # [ 0.920135] 9pnet: Installing 9P2000 support1638worker1 # [ 0.920180] Key type dns_resolver registered1639lb # [ 0.936045] registered taskstats version 11640lb # [ 0.936198] Loading compiled-in X.509 certificates1641worker2 # [ 0.972401] clk: Disabling unused clocks1642worker2 # [ 0.972426] PM: genpd: Disabling unused power domains1643worker2 # [ 0.976617] Freeing unused kernel memory: 4736K1644worker2 # [ 0.976858] Run /init as init process1645lb # [ 0.944807] Demotion targets for Node 0: null1646lb # [ 0.945660] Key type .fscrypt registered1647worker1 # [ 0.927286] registered taskstats version 11648worker1 # [ 0.927428] Loading compiled-in X.509 certificates1649lb # [ 0.945672] Key type fscrypt-provisioning registered1650lb # [ 0.945770] ima: No TPM chip found, activating TPM-bypass!1651worker1 # [ 0.936033] Demotion targets for Node 0: null1652lb # [ 0.945789] ima: Allocated hash algorithm: sha11653worker1 # [ 0.936145] Key type .fscrypt registered1654lb # [ 0.945810] ima: No architecture policies found1655worker1 # [ 0.936151] Key type fscrypt-provisioning registered1656worker1 # [ 0.936245] ima: No TPM chip found, activating TPM-bypass!1657lb # [ 0.949329] input: gpio-keys as /devices/platform/gpio-keys/input/input01658worker1 # [ 0.936265] ima: Allocated hash algorithm: sha11659worker1 # [ 0.936286] ima: No architecture policies found1660worker1 # [ 0.940557] input: gpio-keys as /devices/platform/gpio-keys/input/input01661worker2 # [ 0.992855] systemd[1]: Successfully made /usr/ read-only.1662lb # [ 0.967757] clk: Disabling unused clocks1663lb # [ 0.967786] PM: genpd: Disabling unused power domains1664worker1 # [ 0.957886] clk: Disabling unused clocks1665worker1 # [ 0.957908] PM: genpd: Disabling unused power domains1666lb # [ 0.972005] Freeing unused kernel memory: 4736K1667lb # [ 0.972223] Run /init as init process1668worker1 # [ 0.962090] Freeing unused kernel memory: 4736K1669worker1 # [ 0.962277] Run /init as init process1670lb # [ 0.987097] systemd[1]: Successfully made /usr/ read-only.1671worker1 # [ 0.977011] systemd[1]: Successfully made /usr/ read-only.1672client # [ 1.149120] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker2 # [ 1.158605] usb 1-1: new high-speed USB device number 2 using ehci-pci1674lb # [ 1.152961] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker1 # [ 1.146909] usb 1-1: new high-speed USB device number 2 using ehci-pci1676client # [ 1.301614] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677client # [ 1.317887] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678client # [ 1.329994] systemd[1]: Detected virtualization qemu.1679client # [ 1.331975] systemd[1]: Detected architecture arm64.1680client # [ 1.333965] systemd[1]: Running in initrd.1681client # [ 1.336582] systemd[1]: Initializing machine ID from random generator.1682worker2 # [ 1.308597] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11683client # [ 1.339540] systemd[1]: Hostname set to <client>.1684lb # [ 1.302716] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685worker2 # [ 1.327779] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1686worker2 # [ 1.340020] systemd[1]: Detected virtualization qemu.1687worker2 # [ 1.342005] systemd[1]: Detected architecture arm64.1688worker1 # [ 1.296688] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11689worker2 # [ 1.344035] systemd[1]: Running in initrd.1690worker2 # [ 1.346759] systemd[1]: Initializing machine ID from random generator.1691worker2 # [ 1.349719] systemd[1]: Hostname set to <worker2>.1692client # [ 1.389340] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01693lb # [ 1.321973] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1694worker1 # [ 1.311935] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1695worker1 # [ 1.324171] systemd[1]: Detected virtualization qemu.1696lb # [ 1.334272] systemd[1]: Detected virtualization qemu.1697worker1 # [ 1.326182] systemd[1]: Detected architecture arm64.1698lb # [ 1.334379] systemd[1]: Detected architecture arm64.1699worker1 # [ 1.328188] systemd[1]: Running in initrd.1700lb # [ 1.334404] systemd[1]: Running in initrd.1701lb # [ 1.335394] systemd[1]: Initializing machine ID from random generator.1702worker1 # [ 1.330951] systemd[1]: Initializing machine ID from random generator.1703lb # [ 1.335842] systemd[1]: Hostname set to <lb>.1704worker1 # [ 1.334051] systemd[1]: Hostname set to <worker1>.1705worker2 # [ 1.394833] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706lb # [ 1.389186] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker1 # [ 1.379125] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708client # [ 1.509099] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.518602] usb 1-2: new high-speed USB device number 3 using ehci-pci1710lb # [ 1.508921] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker1 # [ 1.502917] usb 1-2: new high-speed USB device number 3 using ehci-pci1712client # [ 1.648007] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713client # [ 1.669769] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21714client # [ 1.674972] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01715worker2 # [ 1.655283] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716worker1 # [ 1.618522] systemd[1]: bpf-restrict-fs: LSM BPF program attached1717worker2 # [ 1.673848] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718worker2 # [ 1.679748] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01719lb # [ 1.658303] systemd[1]: bpf-restrict-fs: LSM BPF program attached1720worker1 # [ 1.658724] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721worker1 # [ 1.661392] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722lb # [ 1.678613] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21723lb # [ 1.678968] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01724client # [ 1.759593] systemd[1]: Queued start job for default target Initrd Default Target.1725client # [ 1.770956] systemd[1]: Created slice Slice /system/modprobe.1726client # [ 1.771896] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1727client # [ 1.772901] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728client # [ 1.773777] systemd[1]: Reached target Path Units.1729client # [ 1.774387] systemd[1]: Reached target Slice Units.1730client # [ 1.775123] systemd[1]: Reached target Swaps.1731worker1 # [ 1.700170] systemd[1]: Queued start job for default target Initrd Default Target.1732client # [ 1.775700] systemd[1]: Reached target Timer Units.1733client # [ 1.776490] systemd[1]: Listening on D-Bus System Message Bus Socket.1734client # [ 1.777462] systemd[1]: Listening on Journal Socket (/dev/log).1735client # [ 1.778321] systemd[1]: Listening on Journal Sockets.1736client # [ 1.779070] systemd[1]: Listening on udev Control Socket.1737client # [ 1.779852] systemd[1]: Listening on udev Kernel Socket.1738client # [ 1.780553] systemd[1]: Reached target Socket Units.1739client # [ 1.782538] systemd[1]: Starting Create List of Static Device Nodes...1740worker1 # [ 1.709304] systemd[1]: Created slice Slice /system/modprobe.1741client # [ 1.782605] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1742worker1 # [ 1.710483] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1743worker1 # [ 1.711879] systemd[1]: Expecting device /dev/disk/by-label/nixos...1744worker1 # [ 1.712883] systemd[1]: Reached target Path Units.1745worker1 # [ 1.713658] systemd[1]: Reached target Slice Units.1746worker1 # [ 1.714454] systemd[1]: Reached target Swaps.1747worker1 # [ 1.715209] systemd[1]: Reached target Timer Units.1748worker1 # [ 1.716167] systemd[1]: Listening on D-Bus System Message Bus Socket.1749worker1 # [ 1.717469] systemd[1]: Listening on Journal Socket (/dev/log).1750client # [ 1.792571] systemd[1]: Mounting Kernel Configuration File System...1751worker1 # [ 1.718563] systemd[1]: Listening on Journal Sockets.1752worker1 # [ 1.719561] systemd[1]: Listening on udev Control Socket.1753worker1 # [ 1.719692] systemd[1]: Listening on udev Kernel Socket.1754worker1 # [ 1.719716] systemd[1]: Reached target Socket Units.1755worker2 # [ 1.769526] systemd[1]: Queued start job for default target Initrd Default Target.1756worker1 # [ 1.723613] systemd[1]: Starting Create List of Static Device Nodes...1757worker1 # [ 1.723691] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1758worker1 # [ 1.734779] systemd[1]: Mounting Kernel Configuration File System...1759worker2 # [ 1.780956] systemd[1]: Created slice Slice /system/modprobe.1760worker2 # [ 1.782177] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1761worker2 # [ 1.783628] systemd[1]: Expecting device /dev/disk/by-label/nixos...1762client # [ 1.813366] systemd[1]: Starting Journal Service...1763worker2 # [ 1.784723] systemd[1]: Reached target Path Units.1764worker2 # [ 1.785553] systemd[1]: Reached target Slice Units.1765worker2 # [ 1.786404] systemd[1]: Reached target Swaps.1766client # [ 1.815725] systemd[1]: Starting Load Kernel Modules...1767worker2 # [ 1.787287] systemd[1]: Reached target Timer Units.1768client # [ 1.816658] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1769worker2 # [ 1.788334] systemd[1]: Listening on D-Bus System Message Bus Socket.1770worker2 # [ 1.789599] systemd[1]: Listening on Journal Socket (/dev/log).1771worker2 # [ 1.790774] systemd[1]: Listening on Journal Sockets.1772worker2 # [ 1.790918] systemd[1]: Listening on udev Control Socket.1773worker2 # [ 1.791057] systemd[1]: Listening on udev Kernel Socket.1774worker2 # [ 1.791080] systemd[1]: Reached target Socket Units.1775worker2 # [ 1.796318] systemd[1]: Starting Create List of Static Device Nodes...1776worker2 # [ 1.797485] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1777client # [ 1.828031] systemd[1]: Starting Coldplug All udev Devices...1778worker1 # [ 1.756006] systemd[1]: Starting Journal Service...1779lb # [ 1.768998] systemd[1]: Queued start job for default target Initrd Default Target.1780worker1 # [ 1.758582] systemd[1]: Starting Load Kernel Modules...1781worker1 # [ 1.758713] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1782worker2 # [ 1.806712] systemd[1]: Mounting Kernel Configuration File System...1783worker1 # [ 1.767066] systemd[1]: Starting Coldplug All udev Devices...1784lb # [ 1.779107] systemd[1]: Created slice Slice /system/modprobe.1785client # [ 1.842860] systemd[1]: Finished Create List of Static Device Nodes.1786lb # [ 1.780161] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1787worker2 # [ 1.814821] systemd[1]: Starting Journal Service...1788client # [ 1.843545] systemd[1]: Mounted Kernel Configuration File System.1789lb # [ 1.781390] systemd[1]: Expecting device /dev/disk/by-label/nixos...1790lb # [ 1.782325] systemd[1]: Reached target Path Units.1791lb # [ 1.783095] systemd[1]: Reached target Slice Units.1792lb # [ 1.783814] systemd[1]: Reached target Swaps.1793lb # [ 1.784461] systemd[1]: Reached target Timer Units.1794lb # [ 1.785391] systemd[1]: Listening on D-Bus System Message Bus Socket.1795lb # [ 1.786452] systemd[1]: Listening on Journal Socket (/dev/log).1796lb # [ 1.787451] systemd[1]: Listening on Journal Sockets.1797lb # [ 1.788317] systemd[1]: Listening on udev Control Socket.1798lb # [ 1.789247] systemd[1]: Listening on udev Kernel Socket.1799lb # [ 1.789272] systemd[1]: Reached target Socket Units.1800lb # [ 1.792143] systemd[1]: Starting Create List of Static Device Nodes...1801worker1 # [ 1.781403] systemd[1]: Finished Create List of Static Device Nodes.1802worker1 # [ 1.782256] systemd[1]: Mounted Kernel Configuration File System.1803lb # [ 1.793229] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1804client # [ 1.857399] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1805lb # [ 1.801042] systemd[1]: Mounting Kernel Configuration File System...1806worker1 # [ 1.795535] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1807lb # [ 1.809291] systemd[1]: Starting Journal Service...1808worker2 # [ 1.846767] systemd[1]: Starting Load Kernel Modules...1809worker2 # [ 1.846906] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1810worker2 # [ 1.864639] systemd-journald[72]: Collecting audit messages is disabled.1811worker2 # [ 1.865810] systemd[1]: Starting Coldplug All udev Devices...1812client # [ 1.900345] systemd-journald[72]: Collecting audit messages is disabled.1813lb # [ 1.841555] systemd[1]: Starting Load Kernel Modules...1814worker1 # [ 1.830820] systemd-journald[72]: Collecting audit messages is disabled.1815lb # [ 1.841682] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1816client # [ 1.906560] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1817worker2 # [ 1.880110] systemd[1]: Finished Create List of Static Device Nodes.1818worker2 # [ 1.880762] systemd[1]: Mounted Kernel Configuration File System.1819lb # [ 1.849320] systemd[1]: Starting Coldplug All udev Devices...1820client # [ 1.913459] systemd[1]: Starting Create Static Device Nodes in /dev...1821worker1 # [ 1.841662] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1822client # [ 1.921482] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1823worker2 # [ 1.894900] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1824lb # [ 1.867490] systemd-journald[72]: Collecting audit messages is disabled.1825lb # [ 1.869947] systemd[1]: Finished Create List of Static Device Nodes.1826client # [ 1.937218] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1827worker1 # [ 1.863411] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1828worker1 # [ 1.866351] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1829lb # [ 1.882176] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1830worker1 # [ 1.871292] systemd[1]: Starting Create Static Device Nodes in /dev...1831client # [ 1.946540] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01832client # [ 1.946789] [drm] features: -virgl +edid -resource_blob -host_visible1833client # [ 1.946799] [drm] features: -context_init1834client # [ 1.947490] [drm] number of scanouts: 11835client # [ 1.947508] [drm] number of cap sets: 01836worker1 # [ 1.884120] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01837worker1 # [ 1.884364] [drm] features: -virgl +edid -resource_blob -host_visible1838worker1 # [ 1.884374] [drm] features: -context_init1839worker1 # [ 1.885105] [drm] number of scanouts: 11840worker1 # [ 1.885123] [drm] number of cap sets: 01841worker2 # [ 1.931170] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1842worker2 # [ 1.932795] systemd[1]: Starting Create Static Device Nodes in /dev...1843lb # [ 1.900790] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1844lb # [ 1.902752] systemd[1]: Mounted Kernel Configuration File System.1845client # [ 1.965166] systemd[1]: Finished Create Static Device Nodes in /dev.1846client # [ 1.965378] systemd[1]: Reached target Preparation for Local File Systems.1847client # [ 1.965406] systemd[1]: Reached target Local File Systems.1848client # [ 1.969151] systemd[1]: Starting Rule-based Manager for Device Events and Files...1849worker2 # [ 1.943428] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1850lb # [ 1.913100] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1851client # [ 1.973533] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1852client # [ 1.973553] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01853worker1 # [ 1.903141] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1854worker1 # [ 1.903161] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01855lb # [ 1.926169] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01856lb # [ 1.926404] [drm] features: -virgl +edid -resource_blob -host_visible1857lb # [ 1.926414] [drm] features: -context_init1858lb # [ 1.927126] [drm] number of scanouts: 11859lb # [ 1.927144] [drm] number of cap sets: 01860worker2 # [ 1.958691] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1861worker2 # [ 1.964806] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01862worker2 # [ 1.965046] [drm] features: -virgl +edid -resource_blob -host_visible1863worker2 # [ 1.965056] [drm] features: -context_init1864worker2 # [ 1.965778] [drm] number of scanouts: 11865lb # [ 1.933624] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1866worker2 # [ 1.965797] [drm] number of cap sets: 01867lb # [ 1.937260] systemd[1]: Starting Create Static Device Nodes in /dev...1868worker1 # [ 1.931172] Console: switching to colour frame buffer device 160x501869client # [ 2.005333] Console: switching to colour frame buffer device 160x501870client # [ 2.012065] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1871worker2 # [ 1.987194] systemd[1]: Finished Create Static Device Nodes in /dev.1872worker2 # [ 1.987549] systemd[1]: Reached target Preparation for Local File Systems.1873worker2 # [ 1.987577] systemd[1]: Reached target Local File Systems.1874worker1 # [ 1.947559] systemd[1]: Finished Create Static Device Nodes in /dev.1875worker1 # [ 1.947770] systemd[1]: Reached target Preparation for Local File Systems.1876worker1 # [ 1.947798] systemd[1]: Reached target Local File Systems.1877lb # [ 1.957235] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1878lb # [ 1.957254] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01879worker2 # [ 1.994906] systemd[1]: Starting Rule-based Manager for Device Events and Files...1880worker1 # [ 1.955209] systemd[1]: Starting Rule-based Manager for Device Events and Files...1881worker2 # [ 1.999999] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1882worker2 # [ 2.000017] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01883worker1 # [ 1.960773] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1884lb # [ 1.977623] systemd[1]: Finished Create Static Device Nodes in /dev.1885client # [ 2.041716] systemd[1]: Finished Load Kernel Modules.1886lb # [ 1.977815] systemd[1]: Reached target Preparation for Local File Systems.1887lb # [ 1.977841] systemd[1]: Reached target Local File Systems.1888worker1 # [ 1.972124] systemd[1]: Finished Load Kernel Modules.1889lb # [ 1.985243] systemd[1]: Starting Rule-based Manager for Device Events and Files...1890worker2 # [ 2.019015] Console: switching to colour frame buffer device 160x501891lb # [ 1.990259] Console: switching to colour frame buffer device 160x501892client # [ 2.057490] systemd[1]: Starting Apply Kernel Variables...1893worker2 # [ 2.039147] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1894lb # [ 2.006709] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1895worker1 # [ 1.995092] systemd[1]: Starting Apply Kernel Variables...1896worker2 # [ 2.054831] systemd[1]: Started Journal Service.1897lb # [ 2.021523] systemd[1]: Finished Load Kernel Modules.1898lb # [ 2.025069] systemd[1]: Starting Apply Kernel Variables...1899worker1 # [ 2.015075] systemd[1]: Started Journal Service.1900worker2 # [ 2.048335] systemd-modules-load[73]: Inserted module 'dm_mod'1901client # [ 2.090208] systemd[1]: Started Journal Service.1902worker2 # [ 2.049603] systemd-modules-load[73]: Module 'virtio_balloon' is built in1903client # [ 2.076449] systemd-modules-load[73]: Inserted module 'dm_mod'1904worker2 # [ 2.050750] systemd-modules-load[73]: Module 'virtio_console' is built in1905worker2 # [ 2.051802] systemd-modules-load[73]: Inserted module 'virtio_gpu'1906client # [ 2.080469] systemd-modules-load[73]: Module 'virtio_balloon' is built in1907client # [ 2.081582] systemd-modules-load[73]: Module 'virtio_console' is built in1908client # [ 2.082628] systemd-modules-load[73]: Inserted module 'virtio_gpu'1909client # [ 2.083572] systemd-modules-load[73]: Module 'virtio_rng' is built in1910worker1 # [ 2.007454] systemd-modules-load[73]: Inserted module 'dm_mod'1911client # [ 2.091092] systemd-udevd[79]: Using default interface naming scheme 'v261'.1912worker2 # [ 2.064408] systemd-modules-load[73]: Module 'virtio_rng' is built in1913worker2 # [ 2.065495] systemd[1]: Finished Load Kernel Modules.1914worker2 # [ 2.066448] systemd[1]: Starting Apply Kernel Variables...1915worker1 # [ 2.016929] systemd-modules-load[73]: Module 'virtio_balloon' is built in1916worker1 # [ 2.018061] systemd-modules-load[73]: Module 'virtio_console' is built in1917worker1 # [ 2.019130] systemd-modules-load[73]: Inserted module 'virtio_gpu'1918client # [ 2.098323] systemd[1]: Starting Create System Files and Directories...1919client # [ 2.099406] systemd[1]: Finished Apply Kernel Variables.1920worker1 # [ 2.024211] systemd-modules-load[73]: Module 'virtio_rng' is built in1921lb # [ 2.044415] systemd-modules-load[73]: Inserted module 'dm_mod'1922worker1 # [ 2.032372] systemd[1]: Starting Create System Files and Directories...1923worker2 # [ 2.081904] systemd[1]: Starting Create System Files and Directories...1924lb # [ 2.045527] systemd-modules-load[73]: Module 'virtio_balloon' is built in1925worker1 # [ 2.034713] systemd[1]: Finished Apply Kernel Variables.1926lb # [ 2.063471] systemd[1]: Started Journal Service.1927worker2 # [ 2.092450] systemd-udevd[79]: Using default interface naming scheme 'v261'.1928lb # [ 2.056431] systemd-modules-load[73]: Module 'virtio_console' is built in1929lb # [ 2.063970] systemd-modules-load[73]: Inserted module 'virtio_gpu'1930worker1 # [ 2.051604] systemd-udevd[78]: Using default interface naming scheme 'v261'.1931worker1 # [ 2.055016] systemd[1]: Finished Create System Files and Directories.1932client # [ 2.133073] systemd[1]: Finished Create System Files and Directories.1933client # [ 2.135480] systemd[1]: Started Rule-based Manager for Device Events and Files.1934lb # [ 2.072373] systemd-modules-load[73]: Module 'virtio_rng' is built in1935lb # [ 2.075902] systemd[1]: Starting Create System Files and Directories...1936lb # [ 2.081428] systemd[1]: Finished Apply Kernel Variables.1937worker2 # [ 2.125409] systemd[1]: Finished Create System Files and Directories.1938lb # [ 2.092411] systemd-udevd[79]: Using default interface naming scheme 'v261'.1939worker2 # [ 2.132233] systemd[1]: Finished Apply Kernel Variables.1940worker1 # [ 2.086219] systemd[1]: Started Rule-based Manager for Device Events and Files.1941lb # [ 2.099583] systemd[1]: Finished Create System Files and Directories.1942worker2 # [ 2.144355] systemd[1]: Started Rule-based Manager for Device Events and Files.1943lb # [ 2.125167] systemd[1]: Started Rule-based Manager for Device Events and Files.1944client # [ 2.196511] systemd[1]: Starting Virtual Console Setup...1945worker1 # [ 2.142453] systemd[1]: Starting Virtual Console Setup...1946worker2 # [ 2.200767] systemd[1]: Starting Virtual Console Setup...1947lb # [ 2.184144] systemd[1]: Starting Virtual Console Setup...1948client # [ 2.248515] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1949client # [ 2.251978] systemd[1]: Finished Virtual Console Setup.1950worker1 # [ 2.196517] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1951worker1 # [ 2.200131] systemd[1]: Finished Virtual Console Setup.1952worker2 # [ 2.256483] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1953worker2 # [ 2.260000] systemd[1]: Finished Virtual Console Setup.1954lb # [ 2.232482] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1955lb # [ 2.235913] systemd[1]: Finished Virtual Console Setup.1956client # [ 2.827761] systemd[1]: Finished Coldplug All udev Devices.1957client # [ 2.828783] systemd[1]: Reached target System Initialization.1958client # [ 2.829585] systemd[1]: Reached target Basic System.1959worker1 # [ 2.797384] systemd[1]: Finished Coldplug All udev Devices.1960worker1 # [ 2.798357] systemd[1]: Reached target System Initialization.1961worker1 # [ 2.799187] systemd[1]: Reached target Basic System.1962lb # [ 2.840182] systemd[1]: Finished Coldplug All udev Devices.1963lb # [ 2.841773] systemd[1]: Reached target System Initialization.1964lb # [ 2.844120] systemd[1]: Reached target Basic System.1965worker2 # [ 2.899993] systemd[1]: Finished Coldplug All udev Devices.1966worker2 # [ 2.901000] systemd[1]: Reached target System Initialization.1967worker2 # [ 2.901807] systemd[1]: Reached target Basic System.1968client # [ 3.000452] (udev-worker)[104]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969client # [ 3.004725] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1970client # [ 3.008640] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 2.946141] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 2.982759] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1973worker1 # [ 2.978438] (udev-worker)[89]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1974worker1 # [ 2.983091] (udev-worker)[89]: Network interface NamePolicy= disabled on kernel command line.1975client # [ 3.069798] systemd[1]: Found device /dev/disk/by-label/nixos.1976client # [ 3.072727] systemd[1]: Reached target Initrd Root Device.1977client # [ 3.076466] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1978worker2 # [ 3.052404] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1979lb # [ 3.026675] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1980lb # [ 3.032381] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1981worker2 # [ 3.092482] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1982worker1 # [ 3.049663] systemd[1]: Found device /dev/disk/by-label/nixos.1983worker2 # [ 3.099315] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1984client # [ 3.130727] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1985worker1 # [ 3.056273] systemd[1]: Reached target Initrd Root Device.1986client # [ 3.137811] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1987worker1 # [ 3.061959] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1988client # [ 3.142476] systemd[1]: Mounting /sysroot...1989lb # [ 3.096410] systemd[1]: Found device /dev/disk/by-label/nixos.1990lb # [ 3.101455] systemd[1]: Reached target Initrd Root Device.1991lb # [ 3.105532] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992client # [ 3.195567] EXT4-fs (vda): mounted filesystem 90dd4bde-bcaa-4091-be50-17c845d35dc4 r/w with ordered data mode. Quota mode: none.1993worker2 # [ 3.156137] systemd[1]: Found device /dev/disk/by-label/nixos.1994client # [ 3.188096] systemd[1]: Mounted /sysroot.1995worker2 # [ 3.162377] systemd[1]: Reached target Initrd Root Device.1996client # [ 3.189591] systemd[1]: Reached target Initrd Root File System.1997worker1 # [ 3.114851] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1998worker2 # [ 3.167237] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1999client # [ 3.196108] systemd[1]: Starting Mountpoints Configured in the Real Root...2000worker1 # [ 3.122383] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.
loading 2,257 hidden lines…
4258worker2 # [ 46.631037] nix-grpc-daemon[852]: time=2026-09-22T14:51:51.231Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4259worker2 # [ 46.632634] nix-grpc-daemon[852]: time=2026-09-22T14:51:51.231Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4260worker2 # [ 46.668739] nix-grpc-daemon[852]: time=2026-09-22T14:51:51.268Z level=INFO msg="Uploading 1 narinfos"4261lb # [ 46.618988] niks3-server[1007]: 2026/09/22 14:51:51 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4262lb # [ 46.637818] niks3-server[1007]: 2026/09/22 14:51:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4263lb # [ 46.649943] niks3-server[1007]: 2026/09/22 14:51:51 INFO Signed narinfos id=5 count=14264lb # [ 46.656682] niks3-server[1007]: 2026/09/22 14:51:51 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4265lb # [ 46.671140] niks3-server[1007]: 2026/09/22 14:51:51 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4266worker2 # [ 46.714874] nix-grpc-daemon[852]: time=2026-09-22T14:51:51.315Z level=INFO msg="Upload complete. (375ms)"4267lb # [ 46.678166] niks3-server[1007]: 2026/09/22 14:51:51 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4268lb # [ 46.682253] niks3-server[1007]: 2026/09/22 14:51:51 INFO Completed upload id=54269worker2 # [ 46.725331] nix-grpc-daemon[770]: ts=2026-09-22T14:51:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=7524270worker2 # [ 46.740732] nix-grpc-daemon[770]: ts=2026-09-22T14:51:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304271worker2 # [ 46.747293] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4272worker2 # [ 46.756419] nix-daemon[839]: reaped child process 934, status = succeeded4273client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4274worker2 # [ 46.766173] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4275client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4276client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4277client # farm-top> LOG-farm-top-t14278worker2 # [ 47.998060] nix-daemon[839]: reaped child process 948, status = succeeded4279worker2 # [ 48.097693] nix-daemon[839]: accepted connection from pid 964, user nix-grpc-daemon (trusted)4280worker2 # [ 48.118041] nix-daemon[839]: reaped child process 967, status = succeeded4281lb # [ 48.089184] niks3-server[1007]: 2026/09/22 14:51:52 INFO Received uploads request method=POST path=/api/pending_closures4282worker2 # [ 48.140321] nix-grpc-daemon[852]: time=2026-09-22T14:51:52.738Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4283worker2 # [ 48.141840] nix-grpc-daemon[852]: time=2026-09-22T14:51:52.738Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4284lb # [ 48.103713] envoy[897]: conn peer=[2001:db8:1::1]:47150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=20734285lb # [ 48.114977] envoy[897]: conn peer=[2001:db8:1::1]:49806 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=844286lb # [ 48.124475] envoy[897]: conn peer=[2001:db8:1::1]:49822 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114287lb # [ 48.133738] envoy[897]: conn peer=[2001:db8:1::1]:49828 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134288lb # [ 48.153566] niks3-server[1007]: 2026/09/22 14:51:52 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4289lb # [ 48.164511] niks3-server[1007]: 2026/09/22 14:51:52 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4290worker2 # [ 48.211820] nix-grpc-daemon[852]: time=2026-09-22T14:51:52.811Z level=INFO msg="Uploading 1 narinfos"4291lb # [ 48.175584] niks3-server[1007]: 2026/09/22 14:51:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4292lb # [ 48.183746] niks3-server[1007]: 2026/09/22 14:51:52 INFO Signed narinfos id=6 count=14293lb # [ 48.189234] niks3-server[1007]: 2026/09/22 14:51:52 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4294lb # [ 48.200769] niks3-server[1007]: 2026/09/22 14:51:52 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4295lb # [ 48.210352] niks3-server[1007]: 2026/09/22 14:51:52 INFO Completed upload id=64296worker2 # [ 48.243933] nix-grpc-daemon[852]: time=2026-09-22T14:51:52.844Z level=INFO msg="Upload complete. (268ms)"4297lb # [ 48.212803] niks3-server[1007]: 2026/09/22 14:51:52 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4298worker2 # [ 48.257328] nix-grpc-daemon[770]: ts=2026-09-22T14:51:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14299worker2 # [ 48.277153] nix-daemon[839]: reaped child process 946, status = succeeded4300client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 5.77 seconds)4301client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4302worker1 # [ 48.349269] nix-grpc-daemon[766]: E0000 00:00:1790088712.963320 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4303worker1 # [ 48.402328] nix-grpc-daemon[766]: E0000 00:00:1790088713.019372 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4304worker1 # [ 48.433121] nix-grpc-daemon[766]: E0000 00:00:1790088713.026196 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4305worker1 # [ 48.436516] nix-grpc-daemon[766]: E0000 00:00:1790088713.050365 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4306client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds)4307client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4308client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4309client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4310client # copying 3 paths...4311client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4312client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4313client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4314client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)4315(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.31 seconds)4316subtest: repeat is answered Cached by the scheduler without building4317worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4318worker1 # finding garbage collector roots...4319worker1 # deleting specified paths...4320worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4321worker1 # deleting unused links...4322worker1 # note: hard linking is currently saving 0.0 KiB4323worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)4324worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4325worker2 # finding garbage collector roots...4326worker2 # deleting specified paths...4327worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4328worker2 # deleting unused links...4329worker2 # note: hard linking is currently saving 0.0 KiB4330worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4331client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24332client # don't know how to build these paths:4333client # /nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4334worker1 # [ 49.476715] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461484335worker1 # [ 49.485484] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484336worker1 # [ 49.511691] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14337client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.28 seconds)4338client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4339client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.14 seconds)4340lb: must succeed: curl -sf localhost:9901/clusters4341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4342lb: must succeed: curl -sf localhost:9901/clusters4343lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4344worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4345worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4346(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.16 seconds)4347subtest: inputs already in the cache are not sent to the scheduler4348lb: must succeed: curl -sf localhost:9901/clusters4349lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4350lb: must succeed: curl -sf localhost:9901/clusters4351lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4352worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4353worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4354client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24355client # don't know how to build these paths:4356client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4357worker2 # [ 50.402905] nix-grpc-daemon[770]: ts=2026-09-22T14:51:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500304358worker1 # [ 50.360357] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484359worker1 # [ 50.377956] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04360worker1 # [ 50.384322] nix-grpc-daemon[766]: ts=2026-09-22T14:51:54Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14361worker2 # [ 50.473380] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304362worker2 # [ 50.491868] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500304363worker2 # [ 50.508715] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.107Z level=INFO msg="All 2 paths already cached"4364client # copying 1 paths to 'lb:50051'...4365client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4366worker2 # [ 50.521415] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500304367worker2 # [ 50.537529] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4368worker2 # [ 50.696961] nix-daemon[839]: accepted connection from pid 984, user nix-grpc-daemon (trusted)4369worker2 # [ 50.722472] nix-daemon[839]: reaped child process 987, status = succeeded4370worker2 # [ 50.828283] nix-daemon[839]: accepted connection from pid 989, user nix-grpc-daemon (trusted)4371worker2 # [ 50.850652] nix-daemon[839]: reaped child process 992, status = succeeded4372lb # [ 50.822256] niks3-server[1007]: 2026/09/22 14:51:55 INFO Received uploads request method=POST path=/api/pending_closures4373worker2 # [ 50.933311] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.532Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4374worker2 # [ 50.936993] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.532Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4375lb # [ 50.922851] niks3-server[1007]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4376worker2 # [ 50.974541] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.574Z level=INFO msg="Uploading 1 narinfos"4377lb # [ 50.938784] niks3-server[1007]: 2026/09/22 14:51:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4378lb # [ 50.949357] niks3-server[1007]: 2026/09/22 14:51:55 INFO Signed narinfos id=7 count=14379lb # [ 50.956377] niks3-server[1007]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4380lb # [ 50.967475] niks3-server[1007]: 2026/09/22 14:51:55 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4381worker2 # [ 51.010133] nix-grpc-daemon[852]: time=2026-09-22T14:51:55.610Z level=INFO msg="Upload complete. (454ms)"4382lb # [ 50.976284] niks3-server[1007]: 2026/09/22 14:51:55 INFO Completed upload id=74383lb # [ 50.980999] niks3-server[1007]: 2026/09/22 14:51:55 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4384worker2 # [ 51.020501] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=7524385worker2 # [ 51.041835] nix-grpc-daemon[770]: ts=2026-09-22T14:51:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304386worker2 # [ 51.053095] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4387worker2 # [ 51.061041] nix-daemon[839]: reaped child process 982, status = succeeded4388client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4389worker2 # [ 51.075192] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4390client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4391client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4392client # farm-top> LOG-farm-top-t1b4393worker2 # [ 51.914250] nix-daemon[839]: reaped child process 996, status = succeeded4394worker2 # [ 52.021733] nix-daemon[839]: accepted connection from pid 1012, user nix-grpc-daemon (trusted)4395worker2 # [ 52.046079] nix-daemon[839]: reaped child process 1015, status = succeeded4396lb # [ 52.014638] niks3-server[1007]: 2026/09/22 14:51:56 INFO Received uploads request method=POST path=/api/pending_closures4397worker2 # [ 52.064493] nix-grpc-daemon[852]: time=2026-09-22T14:51:56.662Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4398worker2 # [ 52.066019] nix-grpc-daemon[852]: time=2026-09-22T14:51:56.662Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4399lb # [ 52.057132] niks3-server[1007]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4400worker2 # [ 52.118676] nix-grpc-daemon[852]: time=2026-09-22T14:51:56.718Z level=INFO msg="Uploading 1 narinfos"4401lb # [ 52.076826] niks3-server[1007]: 2026/09/22 14:51:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4402lb # [ 52.088350] niks3-server[1007]: 2026/09/22 14:51:56 INFO Signed narinfos id=8 count=14403lb # [ 52.095412] niks3-server[1007]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4404lb # [ 52.102267] niks3-server[1007]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4405lb # [ 52.112132] niks3-server[1007]: 2026/09/22 14:51:56 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4406worker2 # [ 52.154701] nix-grpc-daemon[852]: time=2026-09-22T14:51:56.754Z level=INFO msg="Upload complete. (263ms)"4407lb # [ 52.118538] niks3-server[1007]: 2026/09/22 14:51:56 INFO Completed upload id=84408lb # [ 52.123908] niks3-server[1007]: 2026/09/22 14:51:56 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4409worker2 # [ 52.169302] nix-grpc-daemon[770]: ts=2026-09-22T14:51:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14410worker2 # [ 52.187873] nix-daemon[839]: reaped child process 994, status = succeeded4411client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.03 seconds)4412client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4413client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.17 seconds)4414lb: must succeed: curl -sf localhost:9901/clusters4415lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4416lb: must succeed: curl -sf localhost:9901/clusters4417lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4418worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4419worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4420(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.56 seconds)4421subtest: outputs read back through any worker carry the cache signature4422client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24423client # don't know how to build these paths:4424client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4425worker1 # [ 52.735591] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461484426worker1 # [ 52.743794] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484427worker1 # [ 52.766285] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04428worker1 # [ 52.774233] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=14429worker1 # [ 52.783878] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484430worker1 # [ 52.795941] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484431client # copying 1 paths to 'lb:50051'...4432client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4433worker1 # [ 52.812960] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461484434worker1 # [ 52.828843] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4435worker1 # [ 52.903702] nix-grpc-daemon[766]: ts=2026-09-22T14:51:57Z level=info event=niks3_push_started pid=9014436worker1 # [ 53.093711] nix-grpc-daemon[901]: time=2026-09-22T14:51:57.710Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4437worker1 # [ 53.208677] nix-daemon[849]: accepted connection from pid 914, user nix-grpc-daemon (trusted)4438worker1 # [ 53.229050] nix-daemon[849]: reaped child process 917, status = succeeded4439worker1 # [ 53.338313] nix-daemon[849]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4440worker1 # [ 53.357425] nix-daemon[849]: reaped child process 922, status = succeeded4441lb # [ 53.376760] niks3-server[1007]: 2026/09/22 14:51:57 INFO Received uploads request method=POST path=/api/pending_closures4442worker1 # [ 53.375441] nix-grpc-daemon[901]: time=2026-09-22T14:51:57.992Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4443worker1 # [ 53.377439] nix-grpc-daemon[901]: time=2026-09-22T14:51:57.992Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4444lb # [ 53.416917] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4445worker1 # [ 53.417964] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.036Z level=INFO msg="Uploading 1 narinfos"4446lb # [ 53.430381] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4447lb # [ 53.442635] niks3-server[1007]: 2026/09/22 14:51:58 INFO Signed narinfos id=9 count=14448lb # [ 53.449419] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4449lb # [ 53.456771] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4450worker1 # [ 53.452732] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.071Z level=INFO msg="Upload complete. (359ms)"4451lb # [ 53.467229] niks3-server[1007]: 2026/09/22 14:51:58 INFO Completed upload id=94452lb # [ 53.473408] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4453worker1 # [ 53.464303] nix-grpc-daemon[766]: ts=2026-09-22T14:51:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=5124454worker1 # [ 53.475453] nix-grpc-daemon[766]: ts=2026-09-22T14:51:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484455worker1 # [ 53.482304] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4456worker1 # [ 53.487306] nix-daemon[849]: reaped child process 899, status = succeeded4457worker1 # [ 53.497965] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4458client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4459client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4460client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4461client # farm-a> LOG-farm-a-sig4462worker1 # [ 54.136858] nix-daemon[849]: reaped child process 926, status = succeeded4463worker1 # [ 54.254386] nix-daemon[849]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4464worker1 # [ 54.275779] nix-daemon[849]: reaped child process 945, status = succeeded4465lb # [ 54.289754] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received uploads request method=POST path=/api/pending_closures4466worker1 # [ 54.288394] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.905Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4467worker1 # [ 54.289851] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.905Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4468lb # [ 54.328733] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4469lb # [ 54.346238] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4470lb # [ 54.358787] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4471worker1 # [ 54.346881] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.964Z level=INFO msg="Uploading 1 narinfos"4472lb # [ 54.365652] niks3-server[1007]: 2026/09/22 14:51:58 INFO Signed narinfos id=10 count=14473lb # [ 54.372162] niks3-server[1007]: 2026/09/22 14:51:58 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4474lb # [ 54.385211] niks3-server[1007]: 2026/09/22 14:51:58 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4475worker1 # [ 54.377659] nix-grpc-daemon[901]: time=2026-09-22T14:51:58.996Z level=INFO msg="Upload complete. (268ms)"4476lb # [ 54.392319] niks3-server[1007]: 2026/09/22 14:51:58 INFO Completed upload id=104477lb # [ 54.395133] niks3-server[1007]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4478worker1 # [ 54.397097] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14479worker1 # [ 54.413501] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04480worker1 # [ 54.417348] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14481worker1 # [ 54.421343] nix-daemon[849]: reaped child process 924, status = succeeded4482worker1 # [ 54.444241] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484483worker1 # [ 54.459816] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484484worker1 # [ 54.474438] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.093Z level=INFO msg="All 1 paths already cached"4485client # copying 1 paths to 'lb:50051'...4486client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4487worker1 # [ 54.485655] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461484488worker1 # [ 54.499480] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4489worker1 # [ 54.658373] nix-daemon[849]: accepted connection from pid 951, user nix-grpc-daemon (trusted)4490worker1 # [ 54.678547] nix-daemon[849]: reaped child process 954, status = succeeded4491worker1 # [ 54.792969] nix-daemon[849]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4492worker1 # [ 54.810532] nix-daemon[849]: reaped child process 959, status = succeeded4493lb # [ 54.830343] niks3-server[1007]: 2026/09/22 14:51:59 INFO Received uploads request method=POST path=/api/pending_closures4494worker1 # [ 54.827006] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.445Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4495worker1 # [ 54.828710] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.445Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4496lb # [ 54.863138] niks3-server[1007]: 2026/09/22 14:51:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4497worker1 # [ 54.857407] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.475Z level=INFO msg="Uploading 1 narinfos"4498lb # [ 54.877818] niks3-server[1007]: 2026/09/22 14:51:59 INFO Signed narinfos id=11 count=14499lb # [ 54.887120] niks3-server[1007]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4500lb # [ 54.901837] niks3-server[1007]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4501lb # [ 54.906766] niks3-server[1007]: 2026/09/22 14:51:59 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4502worker1 # [ 54.899263] nix-grpc-daemon[901]: time=2026-09-22T14:51:59.517Z level=INFO msg="Upload complete. (384ms)"4503lb # [ 54.914305] niks3-server[1007]: 2026/09/22 14:51:59 INFO Completed upload id=114504worker1 # [ 54.908272] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=6244505lb # [ 54.921850] niks3-server[1007]: 2026/09/22 14:51:59 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4506worker1 # [ 54.919369] nix-grpc-daemon[766]: ts=2026-09-22T14:51:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484507worker1 # [ 54.925637] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4508worker1 # [ 54.930619] nix-daemon[849]: reaped child process 949, status = succeeded4509worker1 # [ 54.940776] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4510client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4511client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4512client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4513client # farm-b> LOG-farm-b-sig4514worker1 # [ 55.820352] nix-daemon[849]: reaped child process 963, status = succeeded4515worker1 # [ 55.920164] nix-daemon[849]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4516worker1 # [ 55.941189] nix-daemon[849]: reaped child process 982, status = succeeded4517lb # [ 55.958206] niks3-server[1007]: 2026/09/22 14:52:00 INFO Received uploads request method=POST path=/api/pending_closures4518worker1 # [ 55.954107] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.571Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4519worker1 # [ 55.955612] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.571Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4520lb # [ 56.000797] niks3-server[1007]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4521worker1 # [ 55.999188] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.617Z level=INFO msg="Uploading 1 narinfos"4522lb # [ 56.017088] niks3-server[1007]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4523lb # [ 56.025605] niks3-server[1007]: 2026/09/22 14:52:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4524lb # [ 56.036325] niks3-server[1007]: 2026/09/22 14:52:00 INFO Signed narinfos id=12 count=14525worker1 # [ 56.023094] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.641Z level=INFO msg="Upload complete. (270ms)"4526lb # [ 56.038381] niks3-server[1007]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4527lb # [ 56.041693] niks3-server[1007]: 2026/09/22 14:52:00 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4528lb # [ 56.046039] niks3-server[1007]: 2026/09/22 14:52:00 INFO Completed upload id=124529lb # [ 56.049524] niks3-server[1007]: 2026/09/22 14:52:00 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4530worker1 # [ 56.039914] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14531worker1 # [ 56.060344] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04532worker1 # [ 56.065173] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14533worker1 # [ 56.069738] nix-daemon[849]: reaped child process 961, status = succeeded4534worker1 # [ 56.100135] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484535worker1 # [ 56.116045] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484536worker1 # [ 56.130256] nix-grpc-daemon[901]: time=2026-09-22T14:52:00.748Z level=INFO msg="All 2 paths already cached"4537client # copying 1 paths to 'lb:50051'...4538client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4539worker1 # [ 56.141878] nix-grpc-daemon[766]: ts=2026-09-22T14:52:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461484540worker1 # [ 56.156067] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4541worker1 # [ 56.291733] nix-daemon[849]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4542worker1 # [ 56.311870] nix-daemon[849]: reaped child process 991, status = succeeded4543worker1 # [ 56.420031] nix-daemon[849]: accepted connection from pid 993, user nix-grpc-daemon (trusted)4544worker1 # [ 56.438867] nix-daemon[849]: reaped child process 996, status = succeeded4545lb # [ 56.458800] niks3-server[1007]: 2026/09/22 14:52:01 INFO Received uploads request method=POST path=/api/pending_closures4546worker1 # [ 56.455846] nix-grpc-daemon[901]: time=2026-09-22T14:52:01.074Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4547worker1 # [ 56.457616] nix-grpc-daemon[901]: time=2026-09-22T14:52:01.074Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4548lb # [ 56.492707] niks3-server[1007]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4549lb # [ 56.507170] niks3-server[1007]: 2026/09/22 14:52:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4550worker1 # [ 56.498758] nix-grpc-daemon[901]: time=2026-09-22T14:52:01.117Z level=INFO msg="Uploading 1 narinfos"4551lb # [ 56.514651] niks3-server[1007]: 2026/09/22 14:52:01 INFO Signed narinfos id=13 count=14552lb # [ 56.520772] niks3-server[1007]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4553lb # [ 56.531901] niks3-server[1007]: 2026/09/22 14:52:01 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4554worker1 # [ 56.524795] nix-grpc-daemon[901]: time=2026-09-22T14:52:01.143Z level=INFO msg="Upload complete. (352ms)"4555lb # [ 56.539918] niks3-server[1007]: 2026/09/22 14:52:01 INFO Completed upload id=134556lb # [ 56.544990] niks3-server[1007]: 2026/09/22 14:52:01 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4557worker1 # [ 56.536122] nix-grpc-daemon[766]: ts=2026-09-22T14:52:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=7524558worker1 # [ 56.547560] nix-grpc-daemon[766]: ts=2026-09-22T14:52:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484559worker1 # [ 56.555005] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4560worker1 # [ 56.560754] nix-daemon[849]: reaped child process 986, status = succeeded4561client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4562client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4563worker1 # [ 56.575422] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4564client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4565client # farm-top> LOG-farm-top-sig4566worker1 # [ 58.056317] nix-daemon[849]: reaped child process 1000, status = succeeded4567lb # [ 58.102696] envoy[897]: conn peer=[2001:db8:1::1]:49842 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9506 tx=4403 ms=55774568lb # [ 58.107748] envoy[897]: conn peer=[2001:db8:1::1]:49856 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=574569lb # [ 58.112635] envoy[897]: conn peer=[2001:db8:1::1]:49872 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1920 ms=17834570worker1 # [ 58.155191] nix-daemon[849]: accepted connection from pid 1016, user nix-grpc-daemon (trusted)4571worker1 # [ 58.177067] nix-daemon[849]: reaped child process 1019, status = succeeded4572lb # [ 58.194879] niks3-server[1007]: 2026/09/22 14:52:02 INFO Received uploads request method=POST path=/api/pending_closures4573worker1 # [ 58.192496] nix-grpc-daemon[901]: time=2026-09-22T14:52:02.810Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4574worker1 # [ 58.194699] nix-grpc-daemon[901]: time=2026-09-22T14:52:02.810Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4575lb # [ 58.231422] niks3-server[1007]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4576lb # [ 58.251145] niks3-server[1007]: 2026/09/22 14:52:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4577worker1 # [ 58.244285] nix-grpc-daemon[901]: time=2026-09-22T14:52:02.862Z level=INFO msg="Uploading 1 narinfos"4578lb # [ 58.263627] niks3-server[1007]: 2026/09/22 14:52:02 INFO Signed narinfos id=14 count=14579lb # [ 58.269667] niks3-server[1007]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4580lb # [ 58.279645] niks3-server[1007]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4581worker1 # [ 58.271544] nix-grpc-daemon[901]: time=2026-09-22T14:52:02.890Z level=INFO msg="Upload complete. (284ms)"4582lb # [ 58.286857] niks3-server[1007]: 2026/09/22 14:52:02 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4583lb # [ 58.292919] niks3-server[1007]: 2026/09/22 14:52:02 INFO Completed upload id=144584lb # [ 58.295675] niks3-server[1007]: 2026/09/22 14:52:02 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4585worker1 # [ 58.285114] nix-grpc-daemon[766]: ts=2026-09-22T14:52:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14586worker1 # [ 58.306223] nix-daemon[849]: reaped child process 998, status = succeeded4587client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.84 seconds)4588client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4589client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4590client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4591worker1 # [ 58.663788] nix-grpc-daemon[766]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:340824592worker1 # [ 58.706079] nix-grpc-daemon[766]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:340824593client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.24 seconds)4594client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4595worker2 # [ 58.994728] nix-grpc-daemon[770]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:490464596client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4597client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4598client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4599client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4600worker2 # [ 59.028988] nix-grpc-daemon[770]: ts=2026-09-22T14:52:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:490464601worker2 # [ 59.038067] nix-grpc-daemon[770]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4602worker2 # [ 59.056198] nix-grpc-daemon[770]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4603worker2 # [ 59.071607] nix-grpc-daemon[770]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4604client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds)4605client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4606client # finding garbage collector roots...4607client # removing stale temporary roots file "/nix/var/nix/temproots/926"4608client # removing stale temporary roots file "/nix/var/nix/temproots/849"4609client # removing stale temporary roots file "/nix/var/nix/temproots/941"4610client # removing stale temporary roots file "/nix/var/nix/temproots/897"4611client # removing stale temporary roots file "/nix/var/nix/temproots/918"4612client # removing stale temporary roots file "/nix/var/nix/temproots/905"4613client # removing stale temporary roots file "/nix/var/nix/temproots/865"4614client # removing stale temporary roots file "/nix/var/nix/temproots/886"4615client # deleting specified paths...4616client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4617client # deleting unused links...4618client # note: hard linking is currently saving 0.0 KiB4619worker2 # [ 59.459985] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500304620worker2 # [ 59.468681] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500304621worker2 # [ 59.476457] nix-grpc-daemon[852]: time=2026-09-22T14:52:04.074Z level=INFO msg="All 1 paths already cached"4622worker1 # [ 59.430767] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461484623worker2 # [ 59.489403] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:500304624worker1 # [ 59.448954] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461484625client # copying 3 paths...4626worker2 # [ 59.512674] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:500304627client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4628worker2 # [ 59.539025] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:500304629worker2 # [ 59.547014] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=2324630worker1 # [ 59.503835] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:461484631worker1 # [ 59.511733] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_out=1764632worker2 # [ 59.569344] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:500304633worker2 # [ 59.576166] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=1284634client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4635client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4636client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.56 seconds)4637(finished: subtest: outputs read back through any worker carry the cache signature, in 7.11 seconds)4638subtest: two clients wanting the same drv build it once4639client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4640client # [ 59.757748] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4641client # Running as unit: dedup1.service; invocation ID: 4908e2eca0a44b08b68866ba6a2bdb394642client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4643client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4644worker2 # [ 59.924442] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500304645client # [ 59.958425] nix[994]: don't know how to build these paths:4646client # [ 59.960264] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4647worker1 # [ 59.892090] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04648worker1 # [ 59.897556] nix-grpc-daemon[766]: ts=2026-09-22T14:52:04Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14649worker2 # [ 59.988713] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304650worker2 # [ 60.002017] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500304651client # [ 60.040221] nix[994]: copying 1 paths to 'lb:50051'...4652client # [ 60.043727] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4653worker2 # [ 60.014406] nix-grpc-daemon[770]: ts=2026-09-22T14:52:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500304654worker2 # [ 60.030525] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4655worker2 # [ 60.199071] nix-daemon[839]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)4656worker2 # [ 60.230533] nix-daemon[839]: reaped child process 1025, status = succeeded4657worker2 # [ 60.327393] nix-daemon[839]: accepted connection from pid 1027, user nix-grpc-daemon (trusted)4658worker2 # [ 60.346765] nix-daemon[839]: reaped child process 1030, status = succeeded4659lb # [ 60.315505] niks3-server[1007]: 2026/09/22 14:52:04 INFO Received uploads request method=POST path=/api/pending_closures4660worker2 # [ 60.365766] nix-grpc-daemon[852]: time=2026-09-22T14:52:04.965Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4661worker2 # [ 60.367228] nix-grpc-daemon[852]: time=2026-09-22T14:52:04.965Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4662lb # [ 60.348813] niks3-server[1007]: 2026/09/22 14:52:04 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4663worker2 # [ 60.402080] nix-grpc-daemon[852]: time=2026-09-22T14:52:05.002Z level=INFO msg="Uploading 1 narinfos"4664lb # [ 60.365513] niks3-server[1007]: 2026/09/22 14:52:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4665lb # [ 60.377365] niks3-server[1007]: 2026/09/22 14:52:04 INFO Signed narinfos id=15 count=14666lb # [ 60.391854] niks3-server[1007]: 2026/09/22 14:52:04 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4667lb # [ 60.397870] niks3-server[1007]: 2026/09/22 14:52:05 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4668lb # [ 60.409896] niks3-server[1007]: 2026/09/22 14:52:05 INFO Completed upload id=154669worker2 # [ 60.446785] nix-grpc-daemon[852]: time=2026-09-22T14:52:05.047Z level=INFO msg="Upload complete. (399ms)"4670lb # [ 60.413047] niks3-server[1007]: 2026/09/22 14:52:05 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4671worker2 # [ 60.456383] nix-grpc-daemon[770]: ts=2026-09-22T14:52:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=4324672worker2 # [ 60.468948] nix-grpc-daemon[770]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304673worker2 # [ 60.482400] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4674worker2 # [ 60.488163] nix-daemon[839]: reaped child process 1020, status = succeeded4675client # [ 60.525887] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4676worker2 # [ 60.502206] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4677client # [ 60.850305] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4678client # Running as unit: dedup2.service; invocation ID: 4186601b977047d99081da3bbbef5b9f4679client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4680worker1 # [ 60.999170] nix-grpc-daemon[766]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461484681client # [ 61.082997] nix[1010]: don't know how to build these paths:4682client # [ 61.084983] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4683worker1 # [ 61.017283] nix-grpc-daemon[766]: ts=2026-09-22T14:52:05Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04684worker2 # [ 61.073982] nix-grpc-daemon[770]: ts=2026-09-22T14:52:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304685client # [ 61.108172] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4686client: must succeed: sleep 24687client: (finished: must succeed: sleep 2, in 2.06 seconds)4688client: waiting for success: ! systemctl is-active dedup1 dedup24689worker2 # [ 64.534543] nix-daemon[839]: reaped child process 1034, status = succeeded4690worker2 # [ 64.656511] nix-daemon[839]: accepted connection from pid 1064, user nix-grpc-daemon (trusted)4691worker2 # [ 64.679263] nix-daemon[839]: reaped child process 1067, status = succeeded4692lb # [ 64.642287] niks3-server[1007]: 2026/09/22 14:52:09 INFO Received uploads request method=POST path=/api/pending_closures4693worker2 # [ 64.691089] nix-grpc-daemon[852]: time=2026-09-22T14:52:09.289Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4694worker2 # [ 64.694627] nix-grpc-daemon[852]: time=2026-09-22T14:52:09.289Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4695lb # [ 64.684272] niks3-server[1007]: 2026/09/22 14:52:09 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4696lb # [ 64.696986] niks3-server[1007]: 2026/09/22 14:52:09 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4697lb # [ 64.710178] niks3-server[1007]: 2026/09/22 14:52:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4698worker2 # [ 64.751243] nix-grpc-daemon[852]: time=2026-09-22T14:52:09.351Z level=INFO msg="Uploading 1 narinfos"4699lb # [ 64.718431] niks3-server[1007]: 2026/09/22 14:52:09 INFO Signed narinfos id=16 count=14700lb # [ 64.723662] niks3-server[1007]: 2026/09/22 14:52:09 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4701lb # [ 64.734070] niks3-server[1007]: 2026/09/22 14:52:09 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4702lb # [ 64.740410] niks3-server[1007]: 2026/09/22 14:52:09 INFO Completed upload id=164703worker2 # [ 64.776135] nix-grpc-daemon[852]: time=2026-09-22T14:52:09.376Z level=INFO msg="Upload complete. (255ms)"4704lb # [ 64.743587] niks3-server[1007]: 2026/09/22 14:52:09 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4705worker2 # [ 64.789273] nix-grpc-daemon[770]: ts=2026-09-22T14:52:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14706worker2 # [ 64.808378] nix-grpc-daemon[770]: ts=2026-09-22T14:52:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14707worker2 # [ 64.815358] nix-daemon[839]: reaped child process 1032, status = succeeded4708client # [ 64.851898] systemd[1]: dedup1.service: Deactivated successfully.4709client # [ 64.856553] systemd[1]: dedup1.service: Consumed 179ms CPU time over 5.098s wall clock time, 41.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.4710client # [ 64.895946] systemd[1]: dedup2.service: Deactivated successfully.4711client # [ 64.899798] systemd[1]: dedup2.service: Consumed 203ms CPU time over 4.049s wall clock time, 35.3M memory peak, 6.1K incoming IP traffic, 5.6K outgoing IP traffic.4712client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4713client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4714client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4715worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4716worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4717worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4718worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.08 seconds)4719worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4720worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4721worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4722worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4723(finished: subtest: two clients wanting the same drv build it once, in 6.10 seconds)4724subtest: --timeout and --max-silent-time reach the builder4725worker1 # [ 65.961351] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461484726worker1 # [ 65.979710] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04727worker1 # [ 65.987223] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14728worker1 # [ 65.996939] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484729worker1 # [ 66.008266] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484730worker1 # [ 66.022721] nix-grpc-daemon[766]: ts=2026-09-22T14:52:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461484731worker1 # [ 66.038073] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4732worker1 # [ 66.196772] nix-daemon[849]: accepted connection from pid 1050, user nix-grpc-daemon (trusted)4733worker1 # [ 66.220603] nix-daemon[849]: reaped child process 1053, status = succeeded4734worker1 # [ 66.328619] nix-daemon[849]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4735worker1 # [ 66.349896] nix-daemon[849]: reaped child process 1058, status = succeeded4736lb # [ 66.362261] niks3-server[1007]: 2026/09/22 14:52:10 INFO Received uploads request method=POST path=/api/pending_closures4737worker1 # [ 66.359964] nix-grpc-daemon[901]: time=2026-09-22T14:52:10.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4738worker1 # [ 66.361590] nix-grpc-daemon[901]: time=2026-09-22T14:52:10.978Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4739lb # [ 66.395622] niks3-server[1007]: 2026/09/22 14:52:11 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4740worker1 # [ 66.405513] nix-grpc-daemon[901]: time=2026-09-22T14:52:11.024Z level=INFO msg="Uploading 1 narinfos"4741lb # [ 66.413125] niks3-server[1007]: 2026/09/22 14:52:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4742lb # [ 66.421164] niks3-server[1007]: 2026/09/22 14:52:11 INFO Signed narinfos id=17 count=14743lb # [ 66.427590] niks3-server[1007]: 2026/09/22 14:52:11 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4744lb # [ 66.437072] niks3-server[1007]: 2026/09/22 14:52:11 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4745worker1 # [ 66.432346] nix-grpc-daemon[901]: time=2026-09-22T14:52:11.051Z level=INFO msg="Upload complete. (378ms)"4746lb # [ 66.447213] niks3-server[1007]: 2026/09/22 14:52:11 INFO Completed upload id=174747lb # [ 66.453738] niks3-server[1007]: 2026/09/22 14:52:11 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4748worker1 # [ 66.440263] nix-grpc-daemon[766]: ts=2026-09-22T14:52:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=4244749worker1 # [ 66.452338] nix-grpc-daemon[766]: ts=2026-09-22T14:52:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484750worker1 # [ 66.464515] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4751worker1 # [ 66.466996] nix-daemon[849]: reaped child process 1048, status = succeeded4752worker1 # [ 66.477353] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4753lb # [ 68.104987] envoy[897]: conn peer=[2001:db8:1::1]:53044 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9502 tx=4369 ms=55774754lb # [ 68.109878] envoy[897]: conn peer=[2001:db8:1::1]:53058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2930 ms=1914755lb # [ 68.114765] envoy[897]: conn peer=[2001:db8:1::1]:53064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=1124756lb # [ 68.119639] envoy[897]: conn peer=[2001:db8:1::1]:53096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=844757lb # [ 68.124457] envoy[897]: conn peer=[2001:db8:1::1]:53080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=994758lb # [ 68.129094] envoy[897]: conn peer=[2001:db8:1::1]:53102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1538 ms=48784759lb # [ 68.134163] envoy[897]: conn peer=[2001:db8:1::1]:46654 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1194 ms=37994760worker1 # [ 72.296551] nix-grpc-daemon[766]: ts=2026-09-22T14:52:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04761worker1 # [ 72.342099] nix-daemon[849]: reaped child process 1060, status = succeeded4762worker1 # [ 72.346121] nix-daemon[849]: reaped child process 1062, status = succeeded4763worker2 # [ 72.625591] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500304764worker1 # [ 72.594927] nix-grpc-daemon[766]: ts=2026-09-22T14:52:17Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04765worker1 # [ 72.601375] nix-grpc-daemon[766]: ts=2026-09-22T14:52:17Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14766worker2 # [ 72.655204] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304767worker2 # [ 72.667468] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500304768worker2 # [ 72.679572] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500304769worker2 # [ 72.695523] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4770worker2 # [ 72.861006] nix-daemon[839]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)4771worker2 # [ 72.881999] nix-daemon[839]: reaped child process 1083, status = succeeded4772worker2 # [ 72.981490] nix-daemon[839]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)4773lb # [ 72.963128] niks3-server[1007]: 2026/09/22 14:52:17 INFO Received uploads request method=POST path=/api/pending_closures4774worker2 # [ 73.001562] nix-daemon[839]: reaped child process 1088, status = succeeded4775worker2 # [ 73.006527] nix-grpc-daemon[852]: time=2026-09-22T14:52:17.605Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4776worker2 # [ 73.008342] nix-grpc-daemon[852]: time=2026-09-22T14:52:17.605Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4777worker2 # [ 73.028864] nix-grpc-daemon[852]: time=2026-09-22T14:52:17.629Z level=INFO msg="Uploading 1 narinfos"4778lb # [ 72.986370] niks3-server[1007]: 2026/09/22 14:52:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4779lb # [ 72.994937] niks3-server[1007]: 2026/09/22 14:52:17 INFO Signed narinfos id=18 count=14780lb # [ 72.999544] niks3-server[1007]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4781lb # [ 73.005938] niks3-server[1007]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4782lb # [ 73.012892] niks3-server[1007]: 2026/09/22 14:52:17 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4783worker2 # [ 73.055065] nix-grpc-daemon[852]: time=2026-09-22T14:52:17.655Z level=INFO msg="Upload complete. (340ms)"4784lb # [ 73.019821] niks3-server[1007]: 2026/09/22 14:52:17 INFO Completed upload id=184785lb # [ 73.023260] niks3-server[1007]: 2026/09/22 14:52:17 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4786worker2 # [ 73.063370] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=4244787worker2 # [ 73.067077] nix-grpc-daemon[770]: ts=2026-09-22T14:52:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304788worker2 # [ 73.076321] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4789worker2 # [ 73.079884] nix-daemon[839]: reaped child process 1078, status = succeeded4790worker2 # [ 73.090872] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4791lb # [ 78.102319] envoy[897]: conn peer=[2001:db8:1::1]:46664 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3001 tx=952 ms=63974792worker2 # [ 79.509825] nix-grpc-daemon[770]: ts=2026-09-22T14:52:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04793worker2 # [ 79.545455] nix-daemon[839]: reaped child process 1090, status = succeeded4794worker2 # [ 79.547750] nix-daemon[839]: reaped child process 1092, status = succeeded4795(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.92 seconds)4796subtest: the first client leaving does not fail the second4797client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4798client # [ 79.789106] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4799client # Running as unit: share1.service; invocation ID: 2ac284a750024a9db40d8dbb7f5827c14800client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4801client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4802worker1 # [ 79.921544] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461484803client # [ 80.005775] nix[1084]: don't know how to build these paths:4804client # [ 80.006724] nix[1084]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4805worker1 # [ 79.941642] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04806worker1 # [ 79.951818] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14807worker1 # [ 79.960961] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484808worker1 # [ 79.968650] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484809client # [ 80.053438] nix[1084]: copying 1 paths to 'lb:50051'...4810client # [ 80.057064] nix[1084]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4811worker1 # [ 79.981350] nix-grpc-daemon[766]: ts=2026-09-22T14:52:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461484812worker1 # [ 79.996899] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4813worker1 # [ 80.154582] nix-daemon[849]: accepted connection from pid 1089, user nix-grpc-daemon (trusted)4814worker1 # [ 80.181032] nix-daemon[849]: reaped child process 1092, status = succeeded4815worker1 # [ 80.275739] nix-daemon[849]: accepted connection from pid 1094, user nix-grpc-daemon (trusted)4816worker1 # [ 80.296252] nix-daemon[849]: reaped child process 1097, status = succeeded4817lb # [ 80.308818] niks3-server[1007]: 2026/09/22 14:52:24 INFO Received uploads request method=POST path=/api/pending_closures4818worker1 # [ 80.306824] nix-grpc-daemon[901]: time=2026-09-22T14:52:24.924Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4819worker1 # [ 80.308378] nix-grpc-daemon[901]: time=2026-09-22T14:52:24.924Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4820lb # [ 80.344211] niks3-server[1007]: 2026/09/22 14:52:24 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4821lb # [ 80.363282] niks3-server[1007]: 2026/09/22 14:52:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4822worker1 # [ 80.357110] nix-grpc-daemon[901]: time=2026-09-22T14:52:24.975Z level=INFO msg="Uploading 1 narinfos"4823lb # [ 80.373724] niks3-server[1007]: 2026/09/22 14:52:24 INFO Signed narinfos id=19 count=14824lb # [ 80.380481] niks3-server[1007]: 2026/09/22 14:52:24 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4825lb # [ 80.392595] niks3-server[1007]: 2026/09/22 14:52:24 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4826lb # [ 80.402016] niks3-server[1007]: 2026/09/22 14:52:25 INFO Completed upload id=194827worker1 # [ 80.386654] nix-grpc-daemon[901]: time=2026-09-22T14:52:25.005Z level=INFO msg="Upload complete. (373ms)"4828lb # [ 80.406664] niks3-server[1007]: 2026/09/22 14:52:25 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4829worker1 # [ 80.396395] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=4324830worker1 # [ 80.412141] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484831worker1 # [ 80.420499] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4832worker1 # [ 80.423288] nix-daemon[849]: reaped child process 1087, status = succeeded4833worker1 # [ 80.433551] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4834client # [ 80.509164] nix[1084]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4835client # [ 80.894147] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4836client # Running as unit: share2.service; invocation ID: 1ddf70d8171b4ed7a48f55da11b84ca84837client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4838worker1 # [ 81.034523] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461484839client # [ 81.125009] nix[1100]: this derivation will be built:4840worker2 # [ 81.103259] nix-grpc-daemon[770]: ts=2026-09-22T14:52:25Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:500304841client # [ 81.156854] nix[1100]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4842worker1 # [ 81.093238] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04843client # [ 81.177946] nix[1100]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4844worker1 # [ 81.100805] nix-grpc-daemon[766]: ts=2026-09-22T14:52:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461484845worker2 # [ 81.111253] nix-grpc-daemon[770]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' from 'http://lb:5751'...4846client: must succeed: sleep 2; systemctl kill -s INT share14847client # [ 84.278533] systemd[1]: share1.service: Sent signal SIGINT to main process 1084 (nix) on client request.4848client # [ 84.284946] nix[1084]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4849client # [ 84.291741] nix[1084]: [mutex.cc : 451] RAW: Unlock 0xf55357dfc788 @ 0xf5575c4359a4 0xf5575d50d40c 0xf5577c9a57bc 0xf5577c9a7414 0xf5577c99e1d4 0xf5577b9e4f7c 0xf5577b772d804850client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4851client: waiting for success: ! systemctl is-active share14852client # [ 84.324884] systemd-coredump[1115]: Process 1084 (nix) of user 0 terminated abnormally with signal 11/SEGV, processing...4853client # [ 84.349358] systemd[1]: Created slice Slice /system/systemd-coredump.4854client # [ 84.358135] systemd[1]: Started Process Core Dump (PID 1115/UID 0).4855client # [ 86.764889] systemd-coredump[1120]: Cannot store coredump of 1084 (nix): No space left on device4856client # [ 86.889519] systemd-coredump[1120]: Process 1084 (nix) of user 0 terminated abnormally without generating a coredump.4857client # [ 86.906537] systemd[1]: systemd-coredump@0-1-1115_1116-0.service: Deactivated successfully.4858client # [ 86.908007] systemd[1]: systemd-coredump@0-1-1115_1116-0.service: Consumed 2.031s CPU time over 2.542s wall clock time, 968.8M memory peak, 926.3M written to disk.4859client # [ 86.916415] systemd[1]: share1.service: Main process exited, code=dumped, status=11/SEGV4860client # [ 86.919216] systemd[1]: share1.service: Failed with result 'core-dump'.4861client # [ 86.920707] systemd[1]: share1.service: Consumed 368ms CPU time over 7.130s wall clock time, 41.5M memory peak, 4K written to disk, 6.1K incoming IP traffic, 8.1K outgoing IP traffic.4862client: (finished: waiting for success: ! systemctl is-active share1, in 3.26 seconds)4863client: waiting for success: ! systemctl is-active share24864worker1 # [ 87.906791] nix-daemon[849]: reaped child process 1101, status = succeeded4865worker1 # [ 88.033450] nix-daemon[849]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)4866worker1 # [ 88.053829] nix-daemon[849]: reaped child process 1140, status = succeeded4867lb # [ 88.072759] niks3-server[1007]: 2026/09/22 14:52:32 INFO Received uploads request method=POST path=/api/pending_closures4868worker1 # [ 88.072424] nix-grpc-daemon[901]: time=2026-09-22T14:52:32.688Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4869worker1 # [ 88.074091] nix-grpc-daemon[901]: time=2026-09-22T14:52:32.688Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4870lb # [ 88.106497] envoy[897]: conn peer=[2001:db8:1::1]:45306 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=975 ms=69034871lb # [ 88.120549] envoy[897]: conn peer=[2001:db8:1::1]:45314 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2977 tx=787 ms=69104872lb # [ 88.135281] niks3-server[1007]: 2026/09/22 14:52:32 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4873lb # [ 88.147009] niks3-server[1007]: 2026/09/22 14:52:32 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4874worker1 # [ 88.141962] nix-grpc-daemon[901]: time=2026-09-22T14:52:32.759Z level=INFO msg="Uploading 1 narinfos"4875lb # [ 88.156429] niks3-server[1007]: 2026/09/22 14:52:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4876lb # [ 88.163364] niks3-server[1007]: 2026/09/22 14:52:32 INFO Signed narinfos id=20 count=14877lb # [ 88.167198] niks3-server[1007]: 2026/09/22 14:52:32 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4878worker1 # [ 88.164448] nix-grpc-daemon[901]: time=2026-09-22T14:52:32.783Z level=INFO msg="Upload complete. (309ms)"4879lb # [ 88.177535] niks3-server[1007]: 2026/09/22 14:52:32 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4880lb # [ 88.183019] niks3-server[1007]: 2026/09/22 14:52:32 INFO Completed upload id=204881lb # [ 88.187206] niks3-server[1007]: 2026/09/22 14:52:32 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4882worker1 # [ 88.177161] nix-grpc-daemon[766]: ts=2026-09-22T14:52:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14883worker1 # [ 88.198683] nix-grpc-daemon[766]: ts=2026-09-22T14:52:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14884worker1 # [ 88.206202] nix-daemon[849]: reaped child process 1099, status = succeeded4885client # [ 88.302871] systemd[1]: share2.service: Deactivated successfully.4886client # [ 88.308003] systemd[1]: share2.service: Consumed 200ms CPU time over 7.410s wall clock time, 35.3M memory peak, 4K written to disk, 6.5K incoming IP traffic, 5.9K outgoing IP traffic.4887client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4888client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4889client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4890(finished: subtest: the first client leaving does not fail the second, in 9.26 seconds)4891subtest: build hook: nix-daemon with builders = grpc://lb4892client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4893client # [ 89.121464] systemd[1]: Started Nix Daemon.4894client # [ 89.236768] nix-daemon[1166]: accepted connection from pid 1163, user root (trusted)4895worker2 # [ 89.359691] nix-grpc-daemon[770]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:500304896worker1 # [ 89.315463] nix-grpc-daemon[766]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461484897worker1 # [ 89.329041] nix-grpc-daemon[766]: ts=2026-09-22T14:52:33Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04898worker2 # [ 89.381715] nix-grpc-daemon[770]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304899worker1 # [ 89.335332] nix-grpc-daemon[766]: ts=2026-09-22T14:52:33Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14900worker2 # [ 89.397265] nix-grpc-daemon[770]: ts=2026-09-22T14:52:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500304901worker2 # [ 89.412851] nix-grpc-daemon[770]: ts=2026-09-22T14:52:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500304902worker2 # [ 89.427220] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4903worker2 # [ 89.590654] nix-daemon[839]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)4904worker2 # [ 89.617442] nix-daemon[839]: reaped child process 1135, status = succeeded4905worker2 # [ 89.712118] nix-daemon[839]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)4906worker2 # [ 89.729429] nix-daemon[839]: reaped child process 1140, status = succeeded4907lb # [ 89.699785] niks3-server[1007]: 2026/09/22 14:52:34 INFO Received uploads request method=POST path=/api/pending_closures4908worker2 # [ 89.747689] nix-grpc-daemon[852]: time=2026-09-22T14:52:34.347Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4909worker2 # [ 89.749389] nix-grpc-daemon[852]: time=2026-09-22T14:52:34.347Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4910lb # [ 89.735264] niks3-server[1007]: 2026/09/22 14:52:34 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4911worker2 # [ 89.785019] nix-grpc-daemon[852]: time=2026-09-22T14:52:34.385Z level=INFO msg="Uploading 1 narinfos"4912lb # [ 89.749095] niks3-server[1007]: 2026/09/22 14:52:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4913lb # [ 89.759620] niks3-server[1007]: 2026/09/22 14:52:34 INFO Signed narinfos id=21 count=14914lb # [ 89.768140] niks3-server[1007]: 2026/09/22 14:52:34 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4915worker2 # [ 89.809836] nix-grpc-daemon[852]: time=2026-09-22T14:52:34.410Z level=INFO msg="Upload complete. (364ms)"4916lb # [ 89.774384] niks3-server[1007]: 2026/09/22 14:52:34 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4917worker2 # [ 89.816237] nix-grpc-daemon[770]: ts=2026-09-22T14:52:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=5124918lb # [ 89.779880] niks3-server[1007]: 2026/09/22 14:52:34 INFO Completed upload id=214919lb # [ 89.786520] niks3-server[1007]: 2026/09/22 14:52:34 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4920worker2 # [ 89.830979] nix-grpc-daemon[770]: ts=2026-09-22T14:52:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304921worker2 # [ 89.837836] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4922worker2 # [ 89.842816] nix-daemon[839]: reaped child process 1130, status = succeeded4923worker2 # [ 89.853808] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4924worker2 # [ 91.119713] nix-daemon[839]: reaped child process 1144, status = succeeded4925worker2 # [ 91.231070] nix-daemon[839]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)4926worker2 # [ 91.247892] nix-daemon[839]: reaped child process 1163, status = succeeded4927lb # [ 91.219061] niks3-server[1007]: 2026/09/22 14:52:35 INFO Received uploads request method=POST path=/api/pending_closures4928worker2 # [ 91.265409] nix-grpc-daemon[852]: time=2026-09-22T14:52:35.865Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4929worker2 # [ 91.270288] nix-grpc-daemon[852]: time=2026-09-22T14:52:35.868Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4930lb # [ 91.248631] niks3-server[1007]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4931lb # [ 91.254386] niks3-server[1007]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4932lb # [ 91.261820] niks3-server[1007]: 2026/09/22 14:52:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4933worker2 # [ 91.302340] nix-grpc-daemon[852]: time=2026-09-22T14:52:35.902Z level=INFO msg="Uploading 1 narinfos"4934lb # [ 91.267024] niks3-server[1007]: 2026/09/22 14:52:35 INFO Signed narinfos id=22 count=14935lb # [ 91.270499] niks3-server[1007]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4936lb # [ 91.277104] niks3-server[1007]: 2026/09/22 14:52:35 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4937worker2 # [ 91.317933] nix-grpc-daemon[852]: time=2026-09-22T14:52:35.918Z level=INFO msg="Upload complete. (251ms)"4938lb # [ 91.283015] niks3-server[1007]: 2026/09/22 14:52:35 INFO Completed upload id=224939lb # [ 91.285149] niks3-server[1007]: 2026/09/22 14:52:35 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4940worker2 # [ 91.328883] nix-grpc-daemon[770]: ts=2026-09-22T14:52:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14941worker1 # [ 91.285534] nix-grpc-daemon[766]: ts=2026-09-22T14:52:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461484942worker2 # [ 91.342483] nix-daemon[839]: reaped child process 1142, status = succeeded4943worker1 # [ 91.297324] nix-grpc-daemon[766]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4944worker2 # [ 91.383973] nix-grpc-daemon[770]: ts=2026-09-22T14:52:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:500304945worker2 # [ 91.391634] nix-grpc-daemon[770]: ts=2026-09-22T14:52:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=1284946worker1 # [ 91.579982] nix-grpc-daemon[766]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:461484947worker2 # [ 91.634949] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500304948worker2 # [ 91.645417] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.245Z level=INFO msg="All 1 paths already cached"4949worker1 # [ 91.614815] nix-grpc-daemon[766]: ts=2026-09-22T14:52:36Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04950worker1 # [ 91.621946] nix-grpc-daemon[766]: ts=2026-09-22T14:52:36Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14951worker2 # [ 91.674670] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304952worker2 # [ 91.690718] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500304953worker2 # [ 91.705353] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.305Z level=INFO msg="All 1 paths already cached"4954worker2 # [ 91.717196] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500304955worker2 # [ 91.729140] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4956worker2 # [ 91.882509] nix-daemon[839]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)4957worker2 # [ 91.905177] nix-daemon[839]: reaped child process 1170, status = succeeded4958worker2 # [ 92.009761] nix-daemon[839]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)4959worker2 # [ 92.030602] nix-daemon[839]: reaped child process 1175, status = succeeded4960lb # [ 91.992429] niks3-server[1007]: 2026/09/22 14:52:36 INFO Received uploads request method=POST path=/api/pending_closures4961worker2 # [ 92.038379] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.638Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4962worker2 # [ 92.039952] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.638Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4963lb # [ 92.018980] niks3-server[1007]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4964lb # [ 92.035568] niks3-server[1007]: 2026/09/22 14:52:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4965worker2 # [ 92.078368] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.678Z level=INFO msg="Uploading 1 narinfos"4966lb # [ 92.049140] niks3-server[1007]: 2026/09/22 14:52:36 INFO Signed narinfos id=23 count=14967lb # [ 92.057124] niks3-server[1007]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4968lb # [ 92.069002] niks3-server[1007]: 2026/09/22 14:52:36 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4969worker2 # [ 92.109779] nix-grpc-daemon[852]: time=2026-09-22T14:52:36.710Z level=INFO msg="Upload complete. (366ms)"4970lb # [ 92.075837] niks3-server[1007]: 2026/09/22 14:52:36 INFO Completed upload id=234971lb # [ 92.079229] niks3-server[1007]: 2026/09/22 14:52:36 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4972worker2 # [ 92.120249] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=6244973worker2 # [ 92.129986] nix-grpc-daemon[770]: ts=2026-09-22T14:52:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500304974worker2 # [ 92.137530] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4975worker2 # [ 92.142868] nix-daemon[839]: reaped child process 1165, status = succeeded4976worker2 # [ 92.156759] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)4977worker2 # [ 93.636345] nix-daemon[839]: reaped child process 1179, status = succeeded4978worker2 # [ 93.739055] nix-daemon[839]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)4979worker2 # [ 93.758712] nix-daemon[839]: reaped child process 1198, status = succeeded4980lb # [ 93.731058] niks3-server[1007]: 2026/09/22 14:52:38 INFO Received uploads request method=POST path=/api/pending_closures4981worker2 # [ 93.776438] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.375Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4982worker2 # [ 93.778171] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.375Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4983lb # [ 93.773578] niks3-server[1007]: 2026/09/22 14:52:38 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4984lb # [ 93.787270] niks3-server[1007]: 2026/09/22 14:52:38 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4985worker2 # [ 93.837065] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.437Z level=INFO msg="Uploading 1 narinfos"4986lb # [ 93.801922] niks3-server[1007]: 2026/09/22 14:52:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4987lb # [ 93.810783] niks3-server[1007]: 2026/09/22 14:52:38 INFO Signed narinfos id=24 count=14988lb # [ 93.817087] niks3-server[1007]: 2026/09/22 14:52:38 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4989worker2 # [ 93.863631] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.463Z level=INFO msg="Upload complete. (296ms)"4990lb # [ 93.827468] niks3-server[1007]: 2026/09/22 14:52:38 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4991lb # [ 93.834449] niks3-server[1007]: 2026/09/22 14:52:38 INFO Completed upload id=244992lb # [ 93.838286] niks3-server[1007]: 2026/09/22 14:52:38 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4993worker2 # [ 93.877223] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14994worker1 # [ 93.836710] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461484995worker2 # [ 93.896246] nix-daemon[839]: reaped child process 1177, status = succeeded4996worker1 # [ 93.849556] nix-grpc-daemon[766]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4997worker1 # [ 93.890272] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:461484998worker1 # [ 93.897500] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_out=1764999worker2 # [ 94.238772] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:500305000worker1 # [ 94.194405] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461485001worker1 # [ 94.205938] nix-grpc-daemon[901]: time=2026-09-22T14:52:38.823Z level=INFO msg="All 2 paths already cached"5002worker1 # [ 94.226178] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=05003worker1 # [ 94.232674] nix-grpc-daemon[766]: ts=2026-09-22T14:52:38Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=15004worker2 # [ 94.322005] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500305005worker2 # [ 94.339692] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500305006worker2 # [ 94.354400] nix-grpc-daemon[852]: time=2026-09-22T14:52:38.954Z level=INFO msg="All 2 paths already cached"5007worker2 # [ 94.364902] nix-grpc-daemon[770]: ts=2026-09-22T14:52:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500305008worker2 # [ 94.379955] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5009worker2 # [ 94.545991] nix-daemon[839]: accepted connection from pid 1202, user nix-grpc-daemon (trusted)5010worker2 # [ 94.570881] nix-daemon[839]: reaped child process 1205, status = succeeded5011worker2 # [ 94.669446] nix-daemon[839]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)5012worker2 # [ 94.687054] nix-daemon[839]: reaped child process 1210, status = succeeded5013lb # [ 94.656636] niks3-server[1007]: 2026/09/22 14:52:39 INFO Received uploads request method=POST path=/api/pending_closures5014worker2 # [ 94.702939] nix-grpc-daemon[852]: time=2026-09-22T14:52:39.303Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5015worker2 # [ 94.704522] nix-grpc-daemon[852]: time=2026-09-22T14:52:39.303Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"5016lb # [ 94.684169] niks3-server[1007]: 2026/09/22 14:52:39 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst5017worker2 # [ 94.735537] nix-grpc-daemon[852]: time=2026-09-22T14:52:39.335Z level=INFO msg="Uploading 1 narinfos"5018lb # [ 94.698987] niks3-server[1007]: 2026/09/22 14:52:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5019lb # [ 94.712333] niks3-server[1007]: 2026/09/22 14:52:39 INFO Signed narinfos id=25 count=15020lb # [ 94.715586] niks3-server[1007]: 2026/09/22 14:52:39 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls5021worker2 # [ 94.768690] nix-grpc-daemon[852]: time=2026-09-22T14:52:39.368Z level=INFO msg="Upload complete. (370ms)"5022lb # [ 94.733849] niks3-server[1007]: 2026/09/22 14:52:39 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5023lb # [ 94.742605] niks3-server[1007]: 2026/09/22 14:52:39 INFO Completed upload id=255024worker2 # [ 94.779015] nix-grpc-daemon[770]: ts=2026-09-22T14:52:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=7525025lb # [ 94.745340] niks3-server[1007]: 2026/09/22 14:52:39 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo5026worker2 # [ 94.790227] nix-grpc-daemon[770]: ts=2026-09-22T14:52:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500305027worker2 # [ 94.795865] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5028worker2 # [ 94.800631] nix-daemon[839]: reaped child process 1200, status = succeeded5029worker2 # [ 94.813308] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5030worker2 # [ 95.636736] nix-daemon[839]: reaped child process 1214, status = succeeded5031worker2 # [ 95.741983] nix-daemon[839]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5032worker2 # [ 95.762047] nix-daemon[839]: reaped child process 1233, status = succeeded5033lb # [ 95.729921] niks3-server[1007]: 2026/09/22 14:52:40 INFO Received uploads request method=POST path=/api/pending_closures5034worker2 # [ 95.778931] nix-grpc-daemon[852]: time=2026-09-22T14:52:40.378Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5035worker2 # [ 95.780594] nix-grpc-daemon[852]: time=2026-09-22T14:52:40.378Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"5036lb # [ 95.775571] niks3-server[1007]: 2026/09/22 14:52:40 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv5037lb # [ 95.788868] niks3-server[1007]: 2026/09/22 14:52:40 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst5038lb # [ 95.800302] niks3-server[1007]: 2026/09/22 14:52:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5039worker2 # [ 95.842448] nix-grpc-daemon[852]: time=2026-09-22T14:52:40.442Z level=INFO msg="Uploading 1 narinfos"5040lb # [ 95.809969] niks3-server[1007]: 2026/09/22 14:52:40 INFO Signed narinfos id=26 count=15041lb # [ 95.815558] niks3-server[1007]: 2026/09/22 14:52:40 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls5042lb # [ 95.822752] niks3-server[1007]: 2026/09/22 14:52:40 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5043lb # [ 95.831170] niks3-server[1007]: 2026/09/22 14:52:40 INFO Completed upload id=265044worker2 # [ 95.867620] nix-grpc-daemon[852]: time=2026-09-22T14:52:40.467Z level=INFO msg="Upload complete. (245ms)"5045lb # [ 95.835692] niks3-server[1007]: 2026/09/22 14:52:40 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo5046worker2 # [ 95.881110] nix-grpc-daemon[770]: ts=2026-09-22T14:52:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=15047worker1 # [ 95.837953] nix-grpc-daemon[766]: ts=2026-09-22T14:52:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461485048worker2 # [ 95.898676] nix-daemon[839]: reaped child process 1212, status = succeeded5049worker1 # [ 95.850562] nix-grpc-daemon[766]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...5050worker2 # [ 95.940487] nix-grpc-daemon[770]: ts=2026-09-22T14:52:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:500305051worker2 # [ 95.947478] nix-grpc-daemon[770]: ts=2026-09-22T14:52:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=2325052client # [ 96.095190] nix-daemon[1166]: reaped child process 1168, status = succeeded5053client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.13 seconds)5054client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top5055client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)5056client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5057client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)5058client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5059client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5060(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.24 seconds)5061subtest: an input only one worker has still reaches the builder5062client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only5063client # this derivation will be built:5064client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv5065client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...5066client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)5067client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure5068client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)5069worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5070worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.15 seconds)5071worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only5072worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)5073client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&25074client # [ 97.003249] nix-daemon[1166]: accepted connection from pid 1248, user root (trusted)5075client # this derivation will be built:5076client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5077client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5078worker2 # [ 97.129407] nix-grpc-daemon[770]: ts=2026-09-22T14:52:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:500305079worker2 # [ 97.137491] nix-grpc-daemon[770]: ts=2026-09-22T14:52:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500305080client # copying 1 paths...5081client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...5082worker1 # [ 97.102530] nix-grpc-daemon[766]: ts=2026-09-22T14:52:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461485083worker1 # [ 97.116227] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5084worker1 # [ 97.287365] nix-daemon[849]: accepted connection from pid 1158, user nix-grpc-daemon (trusted)5085worker1 # [ 97.308298] nix-daemon[849]: reaped child process 1161, status = succeeded5086lb # [ 97.329803] niks3-server[1007]: 2026/09/22 14:52:41 INFO Received uploads request method=POST path=/api/pending_closures5087worker1 # [ 97.325018] nix-grpc-daemon[901]: time=2026-09-22T14:52:41.943Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5088worker1 # [ 97.326614] nix-grpc-daemon[901]: time=2026-09-22T14:52:41.943Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"5089lb # [ 97.355132] niks3-server[1007]: 2026/09/22 14:52:41 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5090worker1 # [ 97.359078] nix-grpc-daemon[901]: time=2026-09-22T14:52:41.977Z level=INFO msg="Uploading 1 narinfos"5091lb # [ 97.371997] niks3-server[1007]: 2026/09/22 14:52:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5092lb # [ 97.382275] niks3-server[1007]: 2026/09/22 14:52:41 INFO Signed narinfos id=27 count=15093lb # [ 97.390229] niks3-server[1007]: 2026/09/22 14:52:41 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls5094lb # [ 97.403127] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5095lb # [ 97.410060] niks3-server[1007]: 2026/09/22 14:52:42 INFO Completed upload id=275096worker1 # [ 97.397016] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.015Z level=INFO msg="Upload complete. (265ms)"5097lb # [ 97.416105] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo5098worker1 # [ 97.408388] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=1205099worker1 # [ 97.425811] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=05100worker1 # [ 97.438061] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15101worker1 # [ 97.444475] nix-daemon[849]: reaped child process 1156, status = succeeded5102worker1 # [ 97.471576] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461485103worker1 # [ 97.487095] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461485104client # copying 2 paths to 'lb:50051'...5105client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...5106worker1 # [ 97.505961] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461485107client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...5108worker1 # [ 97.520629] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5109worker1 # [ 97.672090] nix-daemon[849]: accepted connection from pid 1167, user nix-grpc-daemon (trusted)5110worker1 # [ 97.700498] nix-daemon[849]: reaped child process 1170, status = succeeded5111worker1 # [ 97.791858] nix-daemon[849]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)5112worker1 # [ 97.810778] nix-daemon[849]: reaped child process 1175, status = succeeded5113lb # [ 97.828618] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received uploads request method=POST path=/api/pending_closures5114lb # [ 97.837860] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received uploads request method=POST path=/api/pending_closures5115worker1 # [ 97.833804] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.452Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5116worker1 # [ 97.835285] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.452Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"5117worker1 # [ 97.837129] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.452Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"5118lb # [ 97.873206] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst5119lb # [ 97.888982] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst5120lb # [ 97.902062] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5121worker1 # [ 97.899072] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.516Z level=INFO msg="Uploading 3 narinfos"5122lb # [ 97.910995] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5123lb # [ 97.915768] niks3-server[1007]: 2026/09/22 14:52:42 INFO Signed narinfos id=28 count=15124lb # [ 97.920320] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5125lb # [ 97.922920] niks3-server[1007]: 2026/09/22 14:52:42 INFO Signed narinfos id=29 count=25126lb # [ 97.931581] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls5127lb # [ 97.944632] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5128lb # [ 97.950775] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5129lb # [ 97.960712] niks3-server[1007]: 2026/09/22 14:52:42 INFO Completed upload id=285130lb # [ 97.962109] niks3-server[1007]: 2026/09/22 14:52:42 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5131lb # [ 97.965332] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5132lb # [ 97.968257] niks3-server[1007]: 2026/09/22 14:52:42 INFO Completed upload id=295133worker1 # [ 97.955316] nix-grpc-daemon[901]: time=2026-09-22T14:52:42.573Z level=INFO msg="Upload complete. (416ms)"5134lb # [ 97.970693] niks3-server[1007]: 2026/09/22 14:52:42 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5135worker1 # [ 97.966395] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=2 nar_bytes_in=9845136worker1 # [ 97.984996] nix-grpc-daemon[766]: ts=2026-09-22T14:52:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461485137worker1 # [ 97.993725] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5138worker1 # [ 97.999996] nix-daemon[849]: reaped child process 1165, status = succeeded5139client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5140client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5141worker1 # [ 98.018433] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5142lb # [ 98.109572] envoy[897]: conn peer=[2001:db8:1::1]:58138 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1592 tx=1599 ms=71695143lb # [ 98.114514] envoy[897]: conn peer=[2001:db8:1::1]:58150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3414 tx=1968 ms=20505144lb # [ 98.119583] envoy[897]: conn peer=[2001:db8:1::1]:57196 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=333 ms=275145lb # [ 98.124336] envoy[897]: conn peer=[2001:db8:1::1]:57208 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3899 tx=2178 ms=23755146lb # [ 98.129090] envoy[897]: conn peer=[2001:db8:1::1]:57210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=715147lb # [ 98.133734] envoy[897]: conn peer=[2001:db8:1::1]:57214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4338 tx=2413 ms=17685148lb # [ 98.139186] envoy[897]: conn peer=[2001:db8:1::1]:57228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=685149client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5150worker1 # [ 99.248717] nix-daemon[849]: reaped child process 1179, status = succeeded5151worker1 # [ 99.349474] nix-daemon[849]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5152worker1 # [ 99.372440] nix-daemon[849]: reaped child process 1198, status = succeeded5153lb # [ 99.384717] niks3-server[1007]: 2026/09/22 14:52:43 INFO Received uploads request method=POST path=/api/pending_closures5154worker1 # [ 99.382508] nix-grpc-daemon[901]: time=2026-09-22T14:52:44.000Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5155lb # [ 99.421508] niks3-server[1007]: 2026/09/22 14:52:44 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5156worker1 # [ 99.419852] nix-grpc-daemon[901]: time=2026-09-22T14:52:44.038Z level=INFO msg="Uploading 1 narinfos"5157lb # [ 99.434432] niks3-server[1007]: 2026/09/22 14:52:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5158lb # [ 99.446721] niks3-server[1007]: 2026/09/22 14:52:44 INFO Signed narinfos id=30 count=15159lb # [ 99.456634] niks3-server[1007]: 2026/09/22 14:52:44 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5160worker1 # [ 99.453102] nix-grpc-daemon[901]: time=2026-09-22T14:52:44.071Z level=INFO msg="Upload complete. (221ms)"5161lb # [ 99.468243] niks3-server[1007]: 2026/09/22 14:52:44 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5162lb # [ 99.474294] niks3-server[1007]: 2026/09/22 14:52:44 INFO Completed upload id=305163lb # [ 99.477719] niks3-server[1007]: 2026/09/22 14:52:44 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5164worker1 # [ 99.465833] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15165client # copying 1 paths...5166worker1 # [ 99.482232] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461485167worker1 # [ 99.486930] nix-daemon[849]: reaped child process 1177, status = succeeded5168client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5169worker2 # [ 99.566931] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:500305170worker2 # [ 99.581040] nix-grpc-daemon[770]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...5171worker2 # [ 99.599417] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_out=1205172client # [ 99.739730] nix-daemon[1166]: reaped child process 1251, status = succeeded5173client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.93 seconds)5174client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25175client # [ 99.887145] nix-daemon[1166]: accepted connection from pid 1270, user root (trusted)5176client # this derivation will be built:5177client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5178worker2 # [ 100.015544] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:500305179client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5180worker1 # [ 99.970191] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461485181worker1 # [ 99.981344] nix-grpc-daemon[901]: time=2026-09-22T14:52:44.599Z level=INFO msg="All 1 paths already cached"5182worker1 # [ 100.001367] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05183worker1 # [ 100.008687] nix-grpc-daemon[766]: ts=2026-09-22T14:52:44Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15184worker2 # [ 100.061592] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500305185worker2 # [ 100.078616] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500305186client # copying 1 paths to 'lb:50051'...5187client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5188worker2 # [ 100.101922] nix-grpc-daemon[770]: ts=2026-09-22T14:52:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500305189worker2 # [ 100.117519] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5190worker2 # [ 100.141458] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 37 ms (attempt 1/5)5191worker2 # [ 100.180555] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 2/5)5192worker2 # [ 100.218824] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 213 ms (attempt 3/5)5193worker2 # [ 100.433957] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 4/5)5194worker2 # [ 100.523044] nix-grpc-daemon[770]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5195worker2 # [ 100.529501] nix-grpc-daemon[770]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5196worker2 # [ 100.537132] nix-grpc-daemon[770]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5197worker2 # [ 100.719058] nix-daemon[839]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5198worker2 # [ 100.740362] nix-daemon[839]: reaped child process 1249, status = succeeded5199worker2 # [ 100.844851] nix-daemon[839]: accepted connection from pid 1251, user nix-grpc-daemon (trusted)5200worker2 # [ 100.862535] nix-daemon[839]: reaped child process 1254, status = succeeded5201lb # [ 100.831068] niks3-server[1007]: 2026/09/22 14:52:45 INFO Received uploads request method=POST path=/api/pending_closures5202worker2 # [ 100.878086] nix-grpc-daemon[852]: time=2026-09-22T14:52:45.477Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5203worker2 # [ 100.879528] nix-grpc-daemon[852]: time=2026-09-22T14:52:45.477Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5204lb # [ 100.860476] niks3-server[1007]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5205lb # [ 100.876499] niks3-server[1007]: 2026/09/22 14:52:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5206worker2 # [ 100.919959] nix-grpc-daemon[852]: time=2026-09-22T14:52:45.520Z level=INFO msg="Uploading 1 narinfos"5207lb # [ 100.888502] niks3-server[1007]: 2026/09/22 14:52:45 INFO Signed narinfos id=31 count=15208lb # [ 100.891770] niks3-server[1007]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5209lb # [ 100.906671] niks3-server[1007]: 2026/09/22 14:52:45 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5210worker2 # [ 100.948717] nix-grpc-daemon[852]: time=2026-09-22T14:52:45.548Z level=INFO msg="Upload complete. (390ms)"5211lb # [ 100.915856] niks3-server[1007]: 2026/09/22 14:52:45 INFO Completed upload id=315212lb # [ 100.918074] niks3-server[1007]: 2026/09/22 14:52:45 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5213worker2 # [ 100.962453] nix-grpc-daemon[770]: ts=2026-09-22T14:52:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=0 paths=1 nar_bytes_in=5685214worker2 # [ 100.976856] nix-grpc-daemon[770]: ts=2026-09-22T14:52:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500305215worker2 # [ 100.983179] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5216worker2 # [ 100.990740] nix-daemon[839]: reaped child process 1239, status = succeeded5217worker2 # [ 101.006555] nix-grpc-daemon[770]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...5218client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5219worker2 # [ 101.023838] nix-daemon[839]: accepted connection from pid 770, user nix-grpc-daemon (trusted)5220client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5221client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5222worker2 # [ 102.728618] nix-daemon[839]: reaped child process 1259, status = succeeded5223worker2 # [ 102.809868] nix-daemon[839]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5224worker2 # [ 102.829983] nix-daemon[839]: reaped child process 1278, status = succeeded5225lb # [ 102.799139] niks3-server[1007]: 2026/09/22 14:52:47 INFO Received uploads request method=POST path=/api/pending_closures5226worker2 # [ 102.848679] nix-grpc-daemon[852]: time=2026-09-22T14:52:47.445Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5227lb # [ 102.836155] niks3-server[1007]: 2026/09/22 14:52:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5228worker2 # [ 102.881055] nix-grpc-daemon[852]: time=2026-09-22T14:52:47.481Z level=INFO msg="Uploading 1 narinfos"5229lb # [ 102.849623] niks3-server[1007]: 2026/09/22 14:52:47 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5230lb # [ 102.858230] niks3-server[1007]: 2026/09/22 14:52:47 INFO Signed narinfos id=32 count=15231lb # [ 102.867003] niks3-server[1007]: 2026/09/22 14:52:47 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5232worker2 # [ 102.910801] nix-grpc-daemon[852]: time=2026-09-22T14:52:47.511Z level=INFO msg="Upload complete. (272ms)"5233lb # [ 102.877026] niks3-server[1007]: 2026/09/22 14:52:47 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5234lb # [ 102.882826] niks3-server[1007]: 2026/09/22 14:52:47 INFO Completed upload id=325235lb # [ 102.886892] niks3-server[1007]: 2026/09/22 14:52:47 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5236worker2 # [ 102.921132] nix-grpc-daemon[770]: ts=2026-09-22T14:52:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50030 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15237client # copying 1 paths...5238worker2 # [ 102.943549] nix-grpc-daemon[770]: ts=2026-09-22T14:52:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:500305239worker2 # [ 102.949101] nix-daemon[839]: reaped child process 1256, status = succeeded5240client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5241worker1 # [ 102.931455] nix-grpc-daemon[766]: ts=2026-09-22T14:52:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:461485242worker1 # [ 102.945610] nix-grpc-daemon[766]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5243worker1 # [ 102.963968] nix-grpc-daemon[766]: ts=2026-09-22T14:52:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_out=1205244client # [ 103.170615] nix-daemon[1166]: reaped child process 1273, status = succeeded5245client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.43 seconds)5246client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25247client # [ 103.341311] nix-daemon[1166]: accepted connection from pid 1291, user root (trusted)5248client # this derivation will be built:5249client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5250worker1 # [ 103.415578] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:461485251client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5252worker1 # [ 103.424412] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461485253worker1 # [ 103.433041] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.049Z level=INFO msg="All 1 paths already cached"5254worker1 # [ 103.445293] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05255worker1 # [ 103.453126] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15256worker1 # [ 103.462269] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461485257worker1 # [ 103.473091] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:461485258worker1 # [ 103.487131] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.105Z level=INFO msg="All 1 paths already cached"5259client # copying 1 paths to 'lb:50051'...5260client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5261worker1 # [ 103.497875] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:461485262worker1 # [ 103.512366] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5263worker1 # [ 103.662076] nix-daemon[849]: accepted connection from pid 1207, user nix-grpc-daemon (trusted)5264worker1 # [ 103.680333] nix-daemon[849]: reaped child process 1210, status = succeeded5265worker1 # [ 103.784420] nix-daemon[849]: accepted connection from pid 1212, user nix-grpc-daemon (trusted)5266worker1 # [ 103.801257] nix-daemon[849]: reaped child process 1215, status = succeeded5267lb # [ 103.821859] niks3-server[1007]: 2026/09/22 14:52:48 INFO Received uploads request method=POST path=/api/pending_closures5268worker1 # [ 103.816630] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.435Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5269worker1 # [ 103.818125] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.435Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5270lb # [ 103.849601] niks3-server[1007]: 2026/09/22 14:52:48 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5271lb # [ 103.868516] niks3-server[1007]: 2026/09/22 14:52:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5272worker1 # [ 103.861580] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.480Z level=INFO msg="Uploading 1 narinfos"5273lb # [ 103.883675] niks3-server[1007]: 2026/09/22 14:52:48 INFO Signed narinfos id=33 count=15274lb # [ 103.891144] niks3-server[1007]: 2026/09/22 14:52:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5275lb # [ 103.902668] niks3-server[1007]: 2026/09/22 14:52:48 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5276worker1 # [ 103.897058] nix-grpc-daemon[901]: time=2026-09-22T14:52:48.515Z level=INFO msg="Upload complete. (367ms)"5277lb # [ 103.910973] niks3-server[1007]: 2026/09/22 14:52:48 INFO Completed upload id=335278lb # [ 103.914081] niks3-server[1007]: 2026/09/22 14:52:48 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5279worker1 # [ 103.908323] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_in=5685280worker1 # [ 103.924502] nix-grpc-daemon[766]: ts=2026-09-22T14:52:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:461485281worker1 # [ 103.931543] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5282worker1 # [ 103.937478] nix-daemon[849]: reaped child process 1205, status = succeeded5283client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5284worker1 # [ 103.951027] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5285client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5286client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5287worker1 # [ 104.825013] nix-daemon[849]: reaped child process 1219, status = succeeded5288worker1 # [ 104.936938] nix-daemon[849]: accepted connection from pid 1235, user nix-grpc-daemon (trusted)5289worker1 # [ 104.956932] nix-daemon[849]: reaped child process 1238, status = succeeded5290lb # [ 104.978358] niks3-server[1007]: 2026/09/22 14:52:49 INFO Received uploads request method=POST path=/api/pending_closures5291worker1 # [ 104.974919] nix-grpc-daemon[901]: time=2026-09-22T14:52:49.592Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5292lb # [ 105.016199] niks3-server[1007]: 2026/09/22 14:52:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5293worker1 # [ 105.011014] nix-grpc-daemon[901]: time=2026-09-22T14:52:49.629Z level=INFO msg="Uploading 1 narinfos"5294lb # [ 105.028382] niks3-server[1007]: 2026/09/22 14:52:49 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5295lb # [ 105.038011] niks3-server[1007]: 2026/09/22 14:52:49 INFO Signed narinfos id=34 count=15296worker1 # [ 105.034939] nix-grpc-daemon[901]: time=2026-09-22T14:52:49.653Z level=INFO msg="Upload complete. (270ms)"5297lb # [ 105.048474] niks3-server[1007]: 2026/09/22 14:52:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5298lb # [ 105.058110] niks3-server[1007]: 2026/09/22 14:52:49 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5299lb # [ 105.063374] niks3-server[1007]: 2026/09/22 14:52:49 INFO Completed upload id=345300client # copying 1 paths...5301worker1 # [ 105.049217] nix-grpc-daemon[766]: ts=2026-09-22T14:52:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15302lb # [ 105.065960] niks3-server[1007]: 2026/09/22 14:52:49 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5303worker2 # [ 105.109592] nix-grpc-daemon[770]: ts=2026-09-22T14:52:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:500305304worker1 # [ 105.067625] nix-daemon[849]: reaped child process 1217, status = succeeded5305worker2 # [ 105.126549] nix-grpc-daemon[770]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5306client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5307worker1 # [ 105.124001] nix-grpc-daemon[766]: ts=2026-09-22T14:52:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:461485308worker1 # [ 105.130846] nix-grpc-daemon[766]: ts=2026-09-22T14:52:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:46148 duration_s=0 paths=1 nar_bytes_out=1205309client # [ 105.334863] nix-daemon[1166]: reaped child process 1294, status = succeeded5310client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.16 seconds)5311(finished: subtest: an input only one worker has still reaches the builder, in 9.12 seconds)5312subtest: upload whose reference the worker lacks is completed from the cache5313client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5314client # this derivation will be built:5315client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5316client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5317client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5318client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5319client # this derivation will be built:5320client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5321client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5322client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)5323client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5324client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5325worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5326worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5327worker2 # [ 106.314553] systemd[1]: Stopping Nix worker-protocol over gRPC...5328worker2 # [ 106.507686] nix-grpc-daemon[770]: ts=2026-09-22T14:52:51Z level=info event=signal_exit5329worker2 # [ 107.983099] nix-grpc-daemon[770]: ts=2026-09-22T14:52:52Z level=info event=scheduler_restarting5330worker1 # [ 107.936577] nix-grpc-daemon[766]: ts=2026-09-22T14:52:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:545345331worker1 # [ 107.941744] nix-grpc-daemon[766]: ts=2026-09-22T14:52:52Z level=info event=worker_gone addr=192.168.1.4:500515332worker2 # [ 108.023820] nix-grpc-daemon[770]: ts=2026-09-22T14:52:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5333worker2 # [ 108.057364] nix-daemon[839]: reaped child process 841, status = succeeded5334lb # [ 108.114502] envoy[897]: conn peer=[2001:db8:1::1]:57234 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4254 tx=2069 ms=25285335lb # [ 108.119974] envoy[897]: conn peer=[2001:db8:1::1]:57250 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=965336lb # [ 108.126616] envoy[897]: conn peer=[2001:db8:1::1]:57264 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2127 ms=30585337lb # [ 108.131662] envoy[897]: conn peer=[2001:db8:1::1]:34342 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=1015338lb # [ 108.137311] envoy[897]: conn peer=[2001:db8:1::1]:34354 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2124 ms=17725339lb # [ 108.139404] envoy[897]: conn peer=[2001:db8:1::1]:34366 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=675340lb # [ 108.141555] envoy[897]: conn peer=[2001:db8:1::4]:33140 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=76789 tx=4223 ms=894735341worker2 # [ 108.709701] nix-grpc-daemon[770]: E0000 00:00:1790088773.309486 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5342worker2 # [ 108.716614] nix-grpc-daemon[770]: E0000 00:00:1790088773.309886 815 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5343worker2 # [ 109.861533] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5344worker2 # [ 109.871730] systemd[1]: Stopped Nix worker-protocol over gRPC.5345worker2 # [ 109.877237] systemd[1]: nix-grpc-daemon.service: Consumed 4.535s CPU time over 1min 31.597s wall clock time, 65.2M memory peak, 104.3K incoming IP traffic, 250.9K outgoing IP traffic.5346worker2 # [ 109.894688] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5347worker2 # [ 109.897966] systemd[1]: Closed Nix worker-protocol over gRPC.5348worker2 # [ 109.899753] systemd[1]: nix-grpc-daemon.socket: Consumed 628us CPU time over 1min 39.792s wall clock time, 256K memory peak, 105K incoming IP traffic, 87.8K outgoing IP traffic.5349lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5350lb: must succeed: curl -sf localhost:9901/clusters5351lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5352lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5353client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25354worker1 # [ 110.157342] nix-grpc-daemon[766]: ts=2026-09-22T14:52:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:461485355worker1 # [ 110.166263] nix-grpc-daemon[766]: ts=2026-09-22T14:52:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:461485356client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5357client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5358worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5359worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5360worker2 # [ 110.342855] systemd[1]: Listening on Nix worker-protocol over gRPC.5361worker1 # [ 110.367669] systemd[1]: Stopping Nix worker-protocol over gRPC...5362worker2 # [ 110.858257] systemd[1]: Starting scheduler-token.service...5363worker2 # [ 110.950962] systemd[1]: scheduler-token.service: Deactivated successfully.5364worker2 # [ 110.961326] systemd[1]: Finished scheduler-token.service.5365worker2 # [ 110.966481] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 96ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5366worker2 # [ 110.974844] systemd[1]: Starting Nix worker-protocol over gRPC...5367worker2 # [ 111.110450] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5368worker2 # [ 111.115782] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5369worker2 # [ 111.138174] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15370worker2 # [ 111.164317] nix-grpc-daemon[1303]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5371worker2 # [ 111.172805] nix-grpc-daemon[1303]: E0000 00:00:1790088775.720049 1303 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5372worker2 # [ 111.175656] nix-grpc-daemon[1303]: E0000 00:00:1790088775.722323 1303 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5373worker2 # [ 111.178263] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5374worker2 # [ 111.180051] nix-grpc-daemon[1303]: E0000 00:00:1790088775.733665 1344 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5375worker2 # [ 111.185362] nix-grpc-daemon[1303]: E0000 00:00:1790088775.754647 1344 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5376worker2 # [ 111.189686] nix-grpc-daemon[1303]: E0000 00:00:1790088775.754806 1344 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5377worker2 # [ 111.192289] systemd[1]: Started Nix worker-protocol over gRPC.5378worker2 # [ 111.193094] nix-grpc-daemon[1303]: E0000 00:00:1790088775.772148 1344 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5379worker2 # [ 111.195332] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:55Z level=info event=scheduler_connected addr=lb:500515380worker1 # [ 111.142762] nix-grpc-daemon[766]: ts=2026-09-22T14:52:55Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:545345381worker1 # [ 111.143225] nix-grpc-daemon[766]: ts=2026-09-22T14:52:55Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05382worker1 # [ 111.361714] nix-grpc-daemon[766]: ts=2026-09-22T14:52:55Z level=info event=signal_exit5383lb # [ 111.942853] niks3-server[1007]: 2026/09/22 14:52:56 INFO lead: released remote=192.168.1.3:483985384worker1 # [ 112.921703] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=scheduler_restarting5385worker2 # [ 112.973902] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_restarting addr=lb:500515386worker1 # [ 112.934032] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:545345387worker1 # [ 112.939105] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=worker_gone addr=192.168.1.4:500515388worker1 # [ 112.943592] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=scheduler_restarting addr=lb:500515389worker1 # [ 112.948690] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:545345390worker1 # [ 112.953480] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=worker_gone addr=192.168.1.3:500515391worker1 # [ 112.981305] nix-daemon[849]: reaped child process 851, status = succeeded5392worker2 # [ 113.173718] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_connected addr=lb:500515393worker2 # [ 113.179133] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5394worker1 # [ 113.155067] nix-grpc-daemon[766]: ts=2026-09-22T14:52:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5395worker2 # [ 113.380264] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_connected addr=lb:500515396worker2 # [ 113.385368] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:57Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5397worker2 # [ 113.586464] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_connected addr=lb:500515398worker2 # [ 113.591624] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5399worker1 # [ 113.551621] nix-grpc-daemon[766]: E0000 00:00:1790088778.170219 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5400worker1 # [ 113.561358] nix-grpc-daemon[766]: E0000 00:00:1790088778.179939 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5401worker2 # [ 113.792791] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_connected addr=lb:500515402worker2 # [ 113.798187] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5403worker1 # [ 113.752470] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5404worker1 # [ 113.759552] systemd[1]: Stopped Nix worker-protocol over gRPC.5405worker1 # [ 113.765811] systemd[1]: nix-grpc-daemon.service: Consumed 3.833s CPU time over 1min 35.655s wall clock time, 64.1M memory peak, 97.7K incoming IP traffic, 193.6K outgoing IP traffic.5406worker1 # [ 113.781354] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5407worker1 # [ 113.783449] systemd[1]: Closed Nix worker-protocol over gRPC.5408worker1 # [ 113.788895] systemd[1]: nix-grpc-daemon.socket: Consumed 581us CPU time over 1min 43.616s wall clock time, 256K memory peak, 122.6K incoming IP traffic, 105.5K outgoing IP traffic.5409lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5410lb: must succeed: curl -sf localhost:9901/clusters5411lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5412lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5413client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25414worker2 # [ 114.000603] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_connected addr=lb:500515415worker2 # [ 114.005225] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker2 # [ 114.142593] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625417worker2 # [ 114.160782] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5418worker2 # [ 114.189610] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 1/5)5419worker2 # [ 114.206317] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_connected addr=lb:500515420worker2 # [ 114.211526] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:58Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5421worker2 # [ 114.218750] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 2/5)5422worker2 # [ 114.267098] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 329 ms (attempt 3/5)5423worker2 # [ 114.412889] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:500515424worker2 # [ 114.418045] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425worker2 # [ 114.598289] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 341 ms (attempt 4/5)5426worker2 # [ 114.619143] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:500515427worker2 # [ 114.624430] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5428worker2 # [ 114.825606] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:500515429worker2 # [ 114.830793] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5430worker2 # [ 114.941994] nix-grpc-daemon[1303]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5431worker2 # [ 114.948531] nix-grpc-daemon[1303]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5432client # copying 2 paths...5433client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5434worker2 # [ 114.981720] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537625435client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5436worker2 # [ 114.999799] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5437worker2 # [ 115.038874] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=niks3_push_started pid=13635438worker2 # [ 115.058301] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:500515439worker2 # [ 115.062851] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5440worker2 # [ 115.196338] nix-grpc-daemon[1363]: time=2026-09-22T14:52:59.794Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5441worker2 # [ 115.246809] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_connected addr=lb:500515442worker2 # [ 115.248953] nix-grpc-daemon[1303]: ts=2026-09-22T14:52:59Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5443worker2 # [ 115.307356] nix-daemon[839]: accepted connection from pid 1376, user nix-grpc-daemon (trusted)5444worker2 # [ 115.326345] nix-daemon[839]: reaped child process 1379, status = succeeded5445lb # [ 115.296665] niks3-server[1007]: 2026/09/22 14:52:59 INFO Received uploads request method=POST path=/api/pending_closures5446lb # [ 115.307256] niks3-server[1007]: 2026/09/22 14:52:59 INFO Received uploads request method=POST path=/api/pending_closures5447worker2 # [ 115.355177] nix-grpc-daemon[1363]: time=2026-09-22T14:52:59.953Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5448worker2 # [ 115.358262] nix-grpc-daemon[1363]: time=2026-09-22T14:52:59.953Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5449worker2 # [ 115.360614] nix-grpc-daemon[1363]: time=2026-09-22T14:52:59.953Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5450lb # [ 115.356932] niks3-server[1007]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5451lb # [ 115.371442] niks3-server[1007]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5452lb # [ 115.382239] niks3-server[1007]: 2026/09/22 14:52:59 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5453lb # [ 115.389829] niks3-server[1007]: 2026/09/22 14:52:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5454lb # [ 115.399443] niks3-server[1007]: 2026/09/22 14:53:00 INFO Signed narinfos id=35 count=15455worker2 # [ 115.436136] nix-grpc-daemon[1363]: time=2026-09-22T14:53:00.035Z level=INFO msg="Uploading 3 narinfos"5456lb # [ 115.405860] niks3-server[1007]: 2026/09/22 14:53:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5457worker2 # [ 115.450348] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_connected addr=lb:500515458lb # [ 115.416237] niks3-server[1007]: 2026/09/22 14:53:00 INFO Signed narinfos id=36 count=25459lb # [ 115.418691] niks3-server[1007]: 2026/09/22 14:53:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5460worker2 # [ 115.455786] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5461lb # [ 115.440553] niks3-server[1007]: 2026/09/22 14:53:00 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5462lb # [ 115.449103] niks3-server[1007]: 2026/09/22 14:53:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5463lb # [ 115.452234] niks3-server[1007]: 2026/09/22 14:53:00 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5464lb # [ 115.455506] niks3-server[1007]: 2026/09/22 14:53:00 INFO Completed upload id=355465lb # [ 115.458812] niks3-server[1007]: 2026/09/22 14:53:00 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5466worker2 # [ 115.495761] nix-grpc-daemon[1363]: time=2026-09-22T14:53:00.095Z level=INFO msg="Upload complete. (300ms)"5467lb # [ 115.461980] niks3-server[1007]: 2026/09/22 14:53:00 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5468lb # [ 115.464537] niks3-server[1007]: 2026/09/22 14:53:00 INFO Completed upload id=365469worker2 # [ 115.508338] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=2 nar_bytes_in=3125470worker2 # [ 115.524870] nix-daemon[839]: reaped child process 1361, status = succeeded5471client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.63 seconds)5472worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5473worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5474worker1 # [ 115.603162] systemd[1]: Listening on Nix worker-protocol over gRPC.5475worker2 # [ 115.656937] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_connected addr=lb:500515476worker1 # [ 115.614004] systemd[1]: Starting Nix worker-protocol over gRPC...5477worker2 # [ 115.662195] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5478worker1 # [ 115.762932] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5479worker1 # [ 115.768337] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5480worker1 # [ 115.770238] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15481lb # [ 115.804651] niks3-server[1007]: 2026/09/22 14:53:00 INFO lead: acquired remote=192.168.1.3:391805482lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5483worker1 # [ 115.790798] nix-grpc-daemon[1261]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5484lb: must succeed: curl -sf localhost:9901/clusters5485worker1 # [ 115.811914] nix-grpc-daemon[1261]: E0000 00:00:1790088780.393006 1261 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5486worker2 # [ 115.863307] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_connected addr=lb:500515487worker1 # [ 115.835879] nix-grpc-daemon[1261]: E0000 00:00:1790088780.395688 1261 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5488worker1 # [ 115.838878] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5489worker1 # [ 115.840842] systemd[1]: Started Nix worker-protocol over gRPC.5490worker1 # [ 115.841657] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=scheduler_take_over5491worker1 # [ 115.842762] nix-grpc-daemon[1261]: E0000 00:00:1790088780.411581 1303 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5492worker1 # [ 115.845160] nix-grpc-daemon[1261]: E0000 00:00:1790088780.435052 1303 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5493worker1 # [ 115.847457] nix-grpc-daemon[1261]: E0000 00:00:1790088780.435337 1303 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5494worker1 # [ 115.852811] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=oidc_jwks provider=mock keys=15495worker1 # [ 115.856448] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:589705496worker1 # [ 115.858407] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05497worker1 # [ 115.860132] nix-grpc-daemon[1261]: E0000 00:00:1790088780.453822 1303 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5498worker1 # [ 115.862414] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=scheduler_connected addr=lb:500515499worker1 # [ 115.863682] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:589705500worker1 # [ 115.865308] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05501lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5502lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5503(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.61 seconds)5504subtest: a niks3 restart does not cost the leader its role5505lb: must succeed: curl -sf localhost:9901/clusters5506lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5507lb: must succeed: curl -sf localhost:9901/clusters5508lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5509lb # [ 115.999372] niks3-server[1007]: 2026/09/22 14:53:00 INFO Shutdown signal received, draining in-flight requests timeout=10s5510worker2 # [ 116.036613] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:00Z level=info event=scheduler_lock_lost status=2005511worker1 # [ 115.986979] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:00Z level=info event=scheduler_lock_lost status=2005512lb # [ 116.018790] niks3-server[1007]: 2026/09/22 14:53:00 INFO lead: released remote=192.168.1.3:391805513lb # [ 116.024903] systemd[1]: Stopping niks3 server...5514lb # [ 116.027802] systemd[1]: niks3.service: Deactivated successfully.5515lb # [ 116.029975] systemd[1]: Stopped niks3 server.5516lb # [ 116.030680] systemd[1]: niks3.service: Consumed 650ms CPU time over 1min 28.340s wall clock time, 27.7M memory peak, 100.6K incoming IP traffic, 142.9K outgoing IP traffic.5517lb # [ 116.039680] systemd[1]: Starting niks3 server...5518lb # [ 116.104813] niks3-server[1141]: 2026/09/22 14:53:00 goose: no migrations to run. current version: 202609200000005519lb # [ 116.113156] niks3-server[1141]: 2026/09/22 14:53:00 OK 1_commit_pending_closure.sql (6.34ms)5520lb # [ 116.116900] niks3-server[1141]: 2026/09/22 14:53:00 OK 2_object_stats_trigger.sql (3.31ms)5521lb # [ 116.118889] niks3-server[1141]: 2026/09/22 14:53:00 goose: up to current file version: 25522lb # [ 116.120679] niks3-server[1141]: 2026/09/22 14:53:00 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5523lb # [ 116.132002] niks3-server[1141]: 2026/09/22 14:53:00 INFO Read proxy enabled — serving cache objects from S35524lb # [ 116.133434] niks3-server[1141]: 2026/09/22 14:53:00 INFO Using socket-activated listener address=0.0.0.0:57515525lb # [ 116.138722] niks3-server[1141]: 2026/09/22 14:53:00 INFO systemd watchdog enabled interval=15s5526lb # [ 116.141991] niks3-server[1141]: 2026/09/22 14:53:00 INFO Starting HTTP server address=0.0.0.0:57515527lb # [ 116.143496] systemd[1]: Started niks3 server.5528lb: waiting for unit niks3.service5529lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5530lb: must succeed: curl -sf localhost:9901/clusters5531lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5532worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5533worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5534lb: must succeed: curl -sf localhost:9901/clusters5535lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5536lb: must succeed: curl -sf localhost:9901/clusters5537lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5538worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5539worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5540client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25541worker2 # [ 116.657565] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537625542client # don't know how to build these paths:5543client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5544worker2 # [ 116.667836] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625545worker1 # [ 116.649546] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:01Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05546worker1 # [ 116.649981] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:01Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15547worker2 # [ 116.745754] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625548worker2 # [ 116.761573] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625549client # copying 1 paths to 'lb:50051'...5550client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5551worker2 # [ 116.777899] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537625552worker2 # [ 116.793064] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5553worker2 # [ 116.965034] nix-daemon[839]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)5554worker2 # [ 116.984539] nix-daemon[839]: reaped child process 1389, status = succeeded5555lb # [ 117.005985] niks3-server[1141]: 2026/09/22 14:53:01 INFO lead: acquired remote=192.168.1.3:391965556worker2 # [ 117.090231] nix-daemon[839]: accepted connection from pid 1391, user nix-grpc-daemon (trusted)5557worker2 # [ 117.106194] nix-daemon[839]: reaped child process 1396, status = succeeded5558lb # [ 117.076889] niks3-server[1141]: 2026/09/22 14:53:01 INFO Received uploads request method=POST path=/api/pending_closures5559worker2 # [ 117.144214] nix-grpc-daemon[1363]: time=2026-09-22T14:53:01.743Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5560worker2 # [ 117.146228] nix-grpc-daemon[1363]: time=2026-09-22T14:53:01.743Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5561lb # [ 117.143952] niks3-server[1141]: 2026/09/22 14:53:01 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5562lb # [ 117.155430] niks3-server[1141]: 2026/09/22 14:53:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5563worker2 # [ 117.198364] nix-grpc-daemon[1363]: time=2026-09-22T14:53:01.798Z level=INFO msg="Uploading 1 narinfos"5564lb # [ 117.163882] niks3-server[1141]: 2026/09/22 14:53:01 INFO Signed narinfos id=37 count=15565lb # [ 117.170170] niks3-server[1141]: 2026/09/22 14:53:01 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5566lb # [ 117.175104] niks3-server[1141]: 2026/09/22 14:53:01 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5567lb # [ 117.186368] niks3-server[1141]: 2026/09/22 14:53:01 INFO Completed upload id=375568worker2 # [ 117.222686] nix-grpc-daemon[1363]: time=2026-09-22T14:53:01.822Z level=INFO msg="Upload complete. (412ms)"5569worker2 # [ 117.232340] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=5445570worker2 # [ 117.252473] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625571worker2 # [ 117.264281] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5572worker2 # [ 117.267455] nix-daemon[839]: reaped child process 1384, status = succeeded5573client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5574client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5575worker2 # [ 117.281644] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5576client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5577client # farm-a> LOG-farm-a-after-niks3-restart5578lb # [ 118.115531] envoy[897]: conn peer=[2001:db8:1::1]:34376 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=275579lb # [ 118.120690] envoy[897]: conn peer=[2001:db8:1::3]:44134 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=14117 tx=3806 ms=938365580lb # [ 118.125875] envoy[897]: conn peer=[2001:db8:1::1]:50548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=281 ms=13815581worker2 # [ 118.186212] nix-daemon[839]: reaped child process 1400, status = succeeded5582worker2 # [ 118.254864] nix-daemon[839]: accepted connection from pid 1416, user nix-grpc-daemon (trusted)5583worker2 # [ 118.278013] nix-daemon[839]: reaped child process 1419, status = succeeded5584lb # [ 118.240652] niks3-server[1141]: 2026/09/22 14:53:02 INFO Received uploads request method=POST path=/api/pending_closures5585worker2 # [ 118.291395] nix-grpc-daemon[1363]: time=2026-09-22T14:53:02.891Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5586worker2 # [ 118.295616] nix-grpc-daemon[1363]: time=2026-09-22T14:53:02.893Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5587lb # [ 118.295645] niks3-server[1141]: 2026/09/22 14:53:02 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5588lb # [ 118.310281] niks3-server[1141]: 2026/09/22 14:53:02 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5589worker2 # [ 118.357069] nix-grpc-daemon[1363]: time=2026-09-22T14:53:02.956Z level=INFO msg="Uploading 1 narinfos"5590lb # [ 118.319193] niks3-server[1141]: 2026/09/22 14:53:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5591lb # [ 118.324564] niks3-server[1141]: 2026/09/22 14:53:02 INFO Signed narinfos id=38 count=15592lb # [ 118.328619] niks3-server[1141]: 2026/09/22 14:53:02 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5593lb # [ 118.337647] niks3-server[1141]: 2026/09/22 14:53:02 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5594lb # [ 118.348288] niks3-server[1141]: 2026/09/22 14:53:02 INFO Completed upload id=385595worker2 # [ 118.385199] nix-grpc-daemon[1363]: time=2026-09-22T14:53:02.985Z level=INFO msg="Upload complete. (293ms)"5596lb # [ 118.352415] niks3-server[1141]: 2026/09/22 14:53:02 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5597worker2 # [ 118.397166] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15598worker1 # [ 118.357409] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:02Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05599worker1 # [ 118.357709] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:02Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15600worker2 # [ 118.419880] nix-daemon[839]: reaped child process 1398, status = succeeded5601worker2 # [ 118.453215] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625602worker2 # [ 118.470173] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625603worker2 # [ 118.485571] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.085Z level=INFO msg="All 1 paths already cached"5604client # copying 1 paths to 'lb:50051'...5605client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5606worker2 # [ 118.496727] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537625607worker2 # [ 118.510929] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5608worker2 # [ 118.663671] nix-daemon[839]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5609worker2 # [ 118.682680] nix-daemon[839]: reaped child process 1426, status = succeeded5610worker2 # [ 118.789371] nix-daemon[839]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)5611worker2 # [ 118.807787] nix-daemon[839]: reaped child process 1431, status = succeeded5612lb # [ 118.774650] niks3-server[1141]: 2026/09/22 14:53:03 INFO Received uploads request method=POST path=/api/pending_closures5613worker2 # [ 118.822033] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.422Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5614worker2 # [ 118.823540] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.422Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5615lb # [ 118.806211] niks3-server[1141]: 2026/09/22 14:53:03 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5616lb # [ 118.822741] niks3-server[1141]: 2026/09/22 14:53:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5617worker2 # [ 118.866650] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.466Z level=INFO msg="Uploading 1 narinfos"5618lb # [ 118.834287] niks3-server[1141]: 2026/09/22 14:53:03 INFO Signed narinfos id=39 count=15619lb # [ 118.843295] niks3-server[1141]: 2026/09/22 14:53:03 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5620lb # [ 118.858326] niks3-server[1141]: 2026/09/22 14:53:03 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5621lb # [ 118.866974] niks3-server[1141]: 2026/09/22 14:53:03 INFO Completed upload id=395622worker2 # [ 118.903920] nix-grpc-daemon[1363]: time=2026-09-22T14:53:03.504Z level=INFO msg="Upload complete. (375ms)"5623lb # [ 118.872181] niks3-server[1141]: 2026/09/22 14:53:03 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5624worker2 # [ 118.916159] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=6565625worker2 # [ 118.932680] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625626worker2 # [ 118.938787] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5627worker2 # [ 118.943613] nix-daemon[839]: reaped child process 1421, status = succeeded5628client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5629worker2 # [ 118.957053] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5630client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5631client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5632client # farm-b> LOG-farm-b-after-niks3-restart5633worker2 # [ 119.648826] nix-daemon[839]: reaped child process 1435, status = succeeded5634worker2 # [ 119.743818] nix-daemon[839]: accepted connection from pid 1451, user nix-grpc-daemon (trusted)5635worker2 # [ 119.762531] nix-daemon[839]: reaped child process 1454, status = succeeded5636lb # [ 119.731389] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received uploads request method=POST path=/api/pending_closures5637worker2 # [ 119.779036] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.378Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5638worker2 # [ 119.781078] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.378Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5639lb # [ 119.770552] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5640lb # [ 119.782574] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5641lb # [ 119.792391] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5642worker2 # [ 119.835698] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.435Z level=INFO msg="Uploading 1 narinfos"5643lb # [ 119.801176] niks3-server[1141]: 2026/09/22 14:53:04 INFO Signed narinfos id=40 count=15644lb # [ 119.803691] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5645lb # [ 119.817676] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5646lb # [ 119.825661] niks3-server[1141]: 2026/09/22 14:53:04 INFO Completed upload id=405647worker2 # [ 119.861898] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.462Z level=INFO msg="Upload complete. (294ms)"5648lb # [ 119.828902] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5649worker1 # [ 119.831127] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:04Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05650worker1 # [ 119.837585] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:04Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15651worker2 # [ 119.873185] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15652worker2 # [ 119.895615] nix-daemon[839]: reaped child process 1433, status = succeeded5653worker2 # [ 119.933290] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625654worker2 # [ 119.950057] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625655worker2 # [ 119.964991] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.565Z level=INFO msg="All 2 paths already cached"5656client # copying 1 paths to 'lb:50051'...5657client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5658worker2 # [ 119.976413] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537625659worker2 # [ 119.992239] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5660worker2 # [ 120.148117] nix-daemon[839]: accepted connection from pid 1458, user nix-grpc-daemon (trusted)5661worker2 # [ 120.166274] nix-daemon[839]: reaped child process 1461, status = succeeded5662worker2 # [ 120.272166] nix-daemon[839]: accepted connection from pid 1463, user nix-grpc-daemon (trusted)5663worker2 # [ 120.289128] nix-daemon[839]: reaped child process 1466, status = succeeded5664lb # [ 120.257766] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received uploads request method=POST path=/api/pending_closures5665worker2 # [ 120.303643] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.903Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5666worker2 # [ 120.305423] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.903Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5667lb # [ 120.286397] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5668worker2 # [ 120.345028] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.945Z level=INFO msg="Uploading 1 narinfos"5669lb # [ 120.301498] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5670lb # [ 120.314949] niks3-server[1141]: 2026/09/22 14:53:04 INFO Signed narinfos id=41 count=15671lb # [ 120.320966] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5672lb # [ 120.331396] niks3-server[1141]: 2026/09/22 14:53:04 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5673worker2 # [ 120.374363] nix-grpc-daemon[1363]: time=2026-09-22T14:53:04.974Z level=INFO msg="Upload complete. (367ms)"5674lb # [ 120.338255] niks3-server[1141]: 2026/09/22 14:53:04 INFO Completed upload id=415675lb # [ 120.342248] niks3-server[1141]: 2026/09/22 14:53:04 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5676worker2 # [ 120.384322] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=7845677worker2 # [ 120.398923] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625678worker2 # [ 120.405522] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5679worker2 # [ 120.410762] nix-daemon[839]: reaped child process 1456, status = succeeded5680client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5681worker2 # [ 120.421784] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5682client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5683client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5684client # farm-top> LOG-farm-top-after-niks3-restart5685worker2 # [ 121.508645] nix-daemon[839]: reaped child process 1470, status = succeeded5686worker2 # [ 121.607047] nix-daemon[839]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)5687worker2 # [ 121.625939] nix-daemon[839]: reaped child process 1489, status = succeeded5688lb # [ 121.596892] niks3-server[1141]: 2026/09/22 14:53:06 INFO Received uploads request method=POST path=/api/pending_closures5689worker2 # [ 121.644448] nix-grpc-daemon[1363]: time=2026-09-22T14:53:06.243Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5690worker2 # [ 121.646165] nix-grpc-daemon[1363]: time=2026-09-22T14:53:06.243Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5691lb # [ 121.633577] niks3-server[1141]: 2026/09/22 14:53:06 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5692worker2 # [ 121.687832] nix-grpc-daemon[1363]: time=2026-09-22T14:53:06.287Z level=INFO msg="Uploading 1 narinfos"5693lb # [ 121.650500] niks3-server[1141]: 2026/09/22 14:53:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5694lb # [ 121.662662] niks3-server[1141]: 2026/09/22 14:53:06 INFO Signed narinfos id=42 count=15695lb # [ 121.669619] niks3-server[1141]: 2026/09/22 14:53:06 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5696worker2 # [ 121.723275] nix-grpc-daemon[1363]: time=2026-09-22T14:53:06.323Z level=INFO msg="Upload complete. (289ms)"5697lb # [ 121.682242] niks3-server[1141]: 2026/09/22 14:53:06 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5698lb # [ 121.691462] niks3-server[1141]: 2026/09/22 14:53:06 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5699lb # [ 121.696238] niks3-server[1141]: 2026/09/22 14:53:06 INFO Completed upload id=425700lb # [ 121.699275] niks3-server[1141]: 2026/09/22 14:53:06 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5701worker2 # [ 121.737155] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15702worker2 # [ 121.755566] nix-daemon[839]: reaped child process 1468, status = succeeded5703client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.35 seconds)5704client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5705client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds)5706(finished: subtest: a niks3 restart does not cost the leader its role, in 6.04 seconds)5707subtest: scheduler down: the other node takes the lock and keeps it5708lb: must succeed: curl -sf localhost:9901/clusters5709lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5710lb: must succeed: curl -sf localhost:9901/clusters5711lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5712lb: must succeed: curl -sf localhost:9901/clusters5713lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5714lb: must succeed: curl -sf localhost:9901/clusters5715lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5716worker1 # [ 122.155680] systemd[1]: Stopping Nix worker-protocol over gRPC...5717worker1 # [ 122.785560] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:07Z level=info event=signal_exit5718lb # [ 123.016952] niks3-server[1141]: 2026/09/22 14:53:07 INFO lead: released remote=192.168.1.3:391965719worker2 # [ 124.048878] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:08Z level=info event=scheduler_restarting addr=lb:500515720worker1 # [ 124.002574] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=scheduler_restarting5721worker1 # [ 124.003046] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:589705722worker1 # [ 124.003178] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=worker_gone addr=192.168.1.3:500515723worker1 # [ 124.003291] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:589705724worker1 # [ 124.003398] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=worker_gone addr=192.168.1.4:500515725worker1 # [ 124.003506] nix-grpc-daemon[1261]: ts=2026-09-22T14:53:08Z level=info event=scheduler_restarting addr=lb:500515726worker2 # [ 124.248630] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:08Z level=info event=scheduler_connected addr=lb:500515727worker2 # [ 124.254010] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5728worker1 # [ 124.275335] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5729worker1 # [ 124.279889] systemd[1]: Stopped Nix worker-protocol over gRPC.5730worker1 # [ 124.283013] systemd[1]: nix-grpc-daemon.service: Consumed 249ms CPU time over 8.664s wall clock time, 15M memory peak, 10K incoming IP traffic, 12.4K outgoing IP traffic.5731worker1 # [ 124.291781] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5732worker1 # [ 124.295745] systemd[1]: Closed Nix worker-protocol over gRPC.5733worker1 # [ 124.298855] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 8.699s wall clock time, 256K memory peak, 17.8K incoming IP traffic, 16K outgoing IP traffic.5734worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5735worker2 # [ 124.455331] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:500515736worker2 # [ 124.460801] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5737worker2 # [ 124.661925] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:500515738worker2 # [ 124.667189] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5739worker2 # [ 124.868381] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:500515740worker2 # [ 124.873610] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5741worker2 # [ 125.076477] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:500515742worker2 # [ 125.080563] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5743worker2 # [ 125.277437] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_connected addr=lb:500515744worker2 # [ 125.282583] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5745worker2 # [ 125.483708] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:500515746worker2 # [ 125.495478] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5747worker2 # [ 125.692516] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:500515748worker2 # [ 125.697809] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5749worker2 # [ 125.898909] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:500515750worker2 # [ 125.904365] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5751worker2 # [ 126.107376] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:500515752worker2 # [ 126.111742] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5753worker2 # [ 126.308482] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_connected addr=lb:500515754worker2 # [ 126.313801] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5755worker2 # [ 126.515040] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:500515756worker2 # [ 126.520825] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5757worker2 # [ 126.721906] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:500515758worker2 # [ 126.727122] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5759worker2 # [ 126.928348] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:500515760worker2 # [ 126.933399] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5761worker2 # [ 127.043780] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_take_over5762lb # [ 127.007784] niks3-server[1141]: 2026/09/22 14:53:11 INFO lead: acquired remote=192.168.1.4:573465763worker2 # [ 127.137051] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:500515764worker2 # [ 127.141319] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5765worker2 # [ 127.338059] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=scheduler_connected addr=lb:500515766worker2 # [ 127.364147] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=oidc_jwks provider=mock keys=15767worker2 # [ 127.369282] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:382305768worker2 # [ 127.374611] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:11Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05769worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 3.33 seconds)5770worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5771worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5772client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25773client # don't know how to build these paths:5774client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5775worker2 # [ 127.954680] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:537625776worker2 # [ 127.964805] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625777worker2 # [ 127.986644] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05778worker2 # [ 127.993636] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15779worker2 # [ 128.032957] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625780worker2 # [ 128.049686] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625781client # copying 1 paths to 'lb:50051'...5782client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5783worker2 # [ 128.065998] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537625784worker2 # [ 128.081103] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5785lb # [ 128.115579] envoy[897]: conn peer=[2001:db8:1::1]:50552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9684 tx=4448 ms=50985786lb # [ 128.120896] envoy[897]: conn peer=[2001:db8:1::3]:35636 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=488 tx=177 ms=83635787worker2 # [ 128.242466] nix-daemon[839]: accepted connection from pid 1522, user nix-grpc-daemon (trusted)5788worker2 # [ 128.269887] nix-daemon[839]: reaped child process 1525, status = succeeded5789worker2 # [ 128.367145] nix-daemon[839]: accepted connection from pid 1527, user nix-grpc-daemon (trusted)5790worker2 # [ 128.386102] nix-daemon[839]: reaped child process 1530, status = succeeded5791lb # [ 128.354779] niks3-server[1141]: 2026/09/22 14:53:12 INFO Received uploads request method=POST path=/api/pending_closures5792worker2 # [ 128.402148] nix-grpc-daemon[1363]: time=2026-09-22T14:53:13.001Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5793worker2 # [ 128.403640] nix-grpc-daemon[1363]: time=2026-09-22T14:53:13.001Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5794lb # [ 128.384597] niks3-server[1141]: 2026/09/22 14:53:12 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5795lb # [ 128.399832] niks3-server[1141]: 2026/09/22 14:53:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5796lb # [ 128.420495] niks3-server[1141]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5797lb # [ 128.442356] niks3-server[1141]: 2026/09/22 14:53:13 INFO Signed narinfos id=43 count=15798worker2 # [ 128.479502] nix-grpc-daemon[1363]: time=2026-09-22T14:53:13.079Z level=INFO msg="Uploading 1 narinfos"5799lb # [ 128.462424] niks3-server[1141]: 2026/09/22 14:53:13 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5800lb # [ 128.472976] niks3-server[1141]: 2026/09/22 14:53:13 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5801lb # [ 128.488806] niks3-server[1141]: 2026/09/22 14:53:13 INFO Completed upload id=435802worker2 # [ 128.525821] nix-grpc-daemon[1363]: time=2026-09-22T14:53:13.126Z level=INFO msg="Upload complete. (428ms)"5803worker2 # [ 128.537626] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=5205804worker2 # [ 128.552329] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625805worker2 # [ 128.566052] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5806worker2 # [ 128.572584] nix-daemon[839]: reaped child process 1520, status = succeeded5807client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5808client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5809worker2 # [ 128.590117] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5810client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5811client # farm-a> LOG-farm-a-on-standby5812worker2 # [ 129.862800] nix-daemon[839]: reaped child process 1534, status = succeeded5813worker2 # [ 129.982615] nix-daemon[839]: accepted connection from pid 1550, user nix-grpc-daemon (trusted)5814worker2 # [ 130.004744] nix-daemon[839]: reaped child process 1553, status = succeeded5815lb # [ 129.967337] niks3-server[1141]: 2026/09/22 14:53:14 INFO Received uploads request method=POST path=/api/pending_closures5816worker2 # [ 130.024505] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.621Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5817worker2 # [ 130.026367] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.621Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5818lb # [ 130.016357] niks3-server[1141]: 2026/09/22 14:53:14 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5819lb # [ 130.029329] niks3-server[1141]: 2026/09/22 14:53:14 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5820worker2 # [ 130.079128] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.679Z level=INFO msg="Uploading 1 narinfos"5821lb # [ 130.042242] niks3-server[1141]: 2026/09/22 14:53:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5822lb # [ 130.052491] niks3-server[1141]: 2026/09/22 14:53:14 INFO Signed narinfos id=44 count=15823lb # [ 130.055358] niks3-server[1141]: 2026/09/22 14:53:14 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5824lb # [ 130.068421] niks3-server[1141]: 2026/09/22 14:53:14 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5825lb # [ 130.078218] niks3-server[1141]: 2026/09/22 14:53:14 INFO Completed upload id=445826worker2 # [ 130.114707] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.714Z level=INFO msg="Upload complete. (310ms)"5827lb # [ 130.081266] niks3-server[1141]: 2026/09/22 14:53:14 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5828worker2 # [ 130.137323] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15829worker2 # [ 130.156379] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05830worker2 # [ 130.163108] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15831worker2 # [ 130.168584] nix-daemon[839]: reaped child process 1532, status = succeeded5832worker2 # [ 130.178257] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625833worker2 # [ 130.195086] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625834client # copying 1 paths to 'lb:50051'...5835worker2 # [ 130.209801] nix-grpc-daemon[1363]: time=2026-09-22T14:53:14.810Z level=INFO msg="All 1 paths already cached"5836client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5837worker2 # [ 130.217976] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537625838worker2 # [ 130.232285] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5839worker2 # [ 130.384181] nix-daemon[839]: accepted connection from pid 1559, user nix-grpc-daemon (trusted)5840worker2 # [ 130.411224] nix-daemon[839]: reaped child process 1562, status = succeeded5841worker2 # [ 130.509461] nix-daemon[839]: accepted connection from pid 1564, user nix-grpc-daemon (trusted)5842worker2 # [ 130.527031] nix-daemon[839]: reaped child process 1567, status = succeeded5843lb # [ 130.498349] niks3-server[1141]: 2026/09/22 14:53:15 INFO Received uploads request method=POST path=/api/pending_closures5844worker2 # [ 130.545461] nix-grpc-daemon[1363]: time=2026-09-22T14:53:15.145Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5845worker2 # [ 130.546980] nix-grpc-daemon[1363]: time=2026-09-22T14:53:15.145Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5846lb # [ 130.530142] niks3-server[1141]: 2026/09/22 14:53:15 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5847worker2 # [ 130.583325] nix-grpc-daemon[1363]: time=2026-09-22T14:53:15.183Z level=INFO msg="Uploading 1 narinfos"5848lb # [ 130.546507] niks3-server[1141]: 2026/09/22 14:53:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5849lb # [ 130.558265] niks3-server[1141]: 2026/09/22 14:53:15 INFO Signed narinfos id=45 count=15850lb # [ 130.565485] niks3-server[1141]: 2026/09/22 14:53:15 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5851worker2 # [ 130.613301] nix-grpc-daemon[1363]: time=2026-09-22T14:53:15.213Z level=INFO msg="Upload complete. (366ms)"5852lb # [ 130.578304] niks3-server[1141]: 2026/09/22 14:53:15 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5853lb # [ 130.586330] niks3-server[1141]: 2026/09/22 14:53:15 INFO Completed upload id=455854worker2 # [ 130.623575] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=6405855lb # [ 130.590201] niks3-server[1141]: 2026/09/22 14:53:15 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5856worker2 # [ 130.632882] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625857worker2 # [ 130.643676] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5858worker2 # [ 130.648221] nix-daemon[839]: reaped child process 1557, status = succeeded5859client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5860worker2 # [ 130.660232] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5861client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5862client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5863client # farm-b> LOG-farm-b-on-standby5864worker2 # [ 131.489141] nix-daemon[839]: reaped child process 1571, status = succeeded5865worker2 # [ 131.594532] nix-daemon[839]: accepted connection from pid 1587, user nix-grpc-daemon (trusted)5866worker2 # [ 131.615692] nix-daemon[839]: reaped child process 1590, status = succeeded5867lb # [ 131.586093] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received uploads request method=POST path=/api/pending_closures5868worker2 # [ 131.632394] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.231Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5869worker2 # [ 131.633878] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.231Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5870lb # [ 131.637359] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5871worker2 # [ 131.683579] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.283Z level=INFO msg="Uploading 1 narinfos"5872lb # [ 131.654271] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5873lb # [ 131.665500] niks3-server[1141]: 2026/09/22 14:53:16 INFO Signed narinfos id=46 count=15874lb # [ 131.670391] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5875lb # [ 131.681635] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5876lb # [ 131.689497] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5877lb # [ 131.694675] niks3-server[1141]: 2026/09/22 14:53:16 INFO Completed upload id=465878worker2 # [ 131.725188] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.325Z level=INFO msg="Upload complete. (254ms)"5879lb # [ 131.699479] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5880worker2 # [ 131.744872] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15881worker2 # [ 131.764381] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05882worker2 # [ 131.770061] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15883worker2 # [ 131.775137] nix-daemon[839]: reaped child process 1569, status = succeeded5884worker2 # [ 131.793383] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625885worker2 # [ 131.809633] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625886worker2 # [ 131.826192] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.426Z level=INFO msg="All 2 paths already cached"5887client # copying 1 paths to 'lb:50051'...5888client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5889worker2 # [ 131.838387] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537625890worker2 # [ 131.852366] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5891worker2 # [ 132.008896] nix-daemon[839]: accepted connection from pid 1596, user nix-grpc-daemon (trusted)5892worker2 # [ 132.028373] nix-daemon[839]: reaped child process 1599, status = succeeded5893worker2 # [ 132.138287] nix-daemon[839]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)5894worker2 # [ 132.159265] nix-daemon[839]: reaped child process 1604, status = succeeded5895lb # [ 132.120665] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received uploads request method=POST path=/api/pending_closures5896worker2 # [ 132.166669] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.766Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5897worker2 # [ 132.168316] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.766Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5898lb # [ 132.149892] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5899worker2 # [ 132.201494] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.801Z level=INFO msg="Uploading 1 narinfos"5900lb # [ 132.165982] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5901lb # [ 132.177523] niks3-server[1141]: 2026/09/22 14:53:16 INFO Signed narinfos id=47 count=15902lb # [ 132.184584] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5903lb # [ 132.195991] niks3-server[1141]: 2026/09/22 14:53:16 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5904worker2 # [ 132.239371] nix-grpc-daemon[1363]: time=2026-09-22T14:53:16.839Z level=INFO msg="Upload complete. (371ms)"5905lb # [ 132.203839] niks3-server[1141]: 2026/09/22 14:53:16 INFO Completed upload id=475906lb # [ 132.207179] niks3-server[1141]: 2026/09/22 14:53:16 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5907worker2 # [ 132.248259] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=7685908worker2 # [ 132.256287] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625909worker2 # [ 132.267713] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5910worker2 # [ 132.273125] nix-daemon[839]: reaped child process 1594, status = succeeded5911client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5912worker2 # [ 132.287140] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5913client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5914client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5915client # farm-top> LOG-farm-top-on-standby5916worker2 # [ 133.194183] nix-daemon[839]: reaped child process 1608, status = succeeded5917worker2 # [ 133.266958] nix-daemon[839]: accepted connection from pid 1624, user nix-grpc-daemon (trusted)5918worker2 # [ 133.284947] nix-daemon[839]: reaped child process 1627, status = succeeded5919lb # [ 133.257866] niks3-server[1141]: 2026/09/22 14:53:17 INFO Received uploads request method=POST path=/api/pending_closures5920worker2 # [ 133.303966] nix-grpc-daemon[1363]: time=2026-09-22T14:53:17.903Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5921worker2 # [ 133.305611] nix-grpc-daemon[1363]: time=2026-09-22T14:53:17.903Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5922lb # [ 133.300190] niks3-server[1141]: 2026/09/22 14:53:17 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5923lb # [ 133.312471] niks3-server[1141]: 2026/09/22 14:53:17 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5924lb # [ 133.324464] niks3-server[1141]: 2026/09/22 14:53:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5925worker2 # [ 133.367125] nix-grpc-daemon[1363]: time=2026-09-22T14:53:17.967Z level=INFO msg="Uploading 1 narinfos"5926lb # [ 133.336288] niks3-server[1141]: 2026/09/22 14:53:17 INFO Signed narinfos id=48 count=15927lb # [ 133.338811] niks3-server[1141]: 2026/09/22 14:53:17 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5928lb # [ 133.350426] niks3-server[1141]: 2026/09/22 14:53:17 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5929lb # [ 133.355448] niks3-server[1141]: 2026/09/22 14:53:17 INFO Completed upload id=485930worker2 # [ 133.392328] nix-grpc-daemon[1363]: time=2026-09-22T14:53:17.992Z level=INFO msg="Upload complete. (294ms)"5931lb # [ 133.358669] niks3-server[1141]: 2026/09/22 14:53:17 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5932worker2 # [ 133.405053] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15933worker2 # [ 133.425720] nix-daemon[839]: reaped child process 1606, status = succeeded5934client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.73 seconds)5935client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5936client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5937worker1 # [ 133.663032] systemd[1]: Listening on Nix worker-protocol over gRPC.5938worker1 # [ 133.672715] systemd[1]: Starting Nix worker-protocol over gRPC...5939worker1 # [ 133.820495] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5940worker1 # [ 133.822405] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5941worker1 # [ 133.826213] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15942worker1 # [ 133.832076] nix-grpc-daemon[1344]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5943worker1 # [ 133.833524] nix-grpc-daemon[1344]: E0000 00:00:1790088798.449262 1344 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5944lb: must succeed: curl -sf localhost:9901/clusters5945worker1 # [ 133.876085] nix-grpc-daemon[1344]: E0000 00:00:1790088798.455652 1344 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5946worker1 # [ 133.878440] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5947worker1 # [ 133.889585] nix-grpc-daemon[1344]: E0000 00:00:1790088798.469728 1386 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5948worker1 # [ 133.892443] nix-grpc-daemon[1344]: E0000 00:00:1790088798.488989 1386 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5949worker1 # [ 133.894864] nix-grpc-daemon[1344]: E0000 00:00:1790088798.489126 1386 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5950worker1 # [ 133.900151] systemd[1]: Started Nix worker-protocol over gRPC.5951worker1 # [ 133.902774] nix-grpc-daemon[1344]: E0000 00:00:1790088798.507608 1386 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5952worker1 # [ 133.905482] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:18Z level=info event=scheduler_connected addr=lb:500515953worker2 # [ 133.953392] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:18Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:382305954worker2 # [ 133.959409] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:18Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05955lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5956worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5957worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5958lb: must succeed: curl -sf localhost:9901/clusters5959lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5960lb: must succeed: curl -sf localhost:9901/clusters5961lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5962worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5963worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5964(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.24 seconds)5965subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5966client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5967client # [ 134.310829] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5968client # Running as unit: rs.service; invocation ID: 004dacd07350486192513362c1a3877c5969client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5970worker1 # [ 134.447736] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414545971worker1 # [ 134.462808] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)5972client # [ 134.559237] nix[1417]: don't know how to build these paths:5973client # [ 134.561406] nix[1417]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5974worker2 # [ 134.543173] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05975worker2 # [ 134.548644] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15976worker2 # [ 134.589660] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537625977worker2 # [ 134.605796] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:537625978client # [ 134.647920] nix[1417]: copying 1 paths to 'lb:50051'...5979worker2 # [ 134.621054] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:537625980client # [ 134.650402] nix[1417]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5981worker2 # [ 134.635786] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)5982worker2 # [ 134.794741] nix-daemon[839]: accepted connection from pid 1640, user nix-grpc-daemon (trusted)5983worker2 # [ 134.821692] nix-daemon[839]: reaped child process 1643, status = succeeded5984worker2 # [ 134.923199] nix-daemon[839]: accepted connection from pid 1645, user nix-grpc-daemon (trusted)5985worker2 # [ 134.940530] nix-daemon[839]: reaped child process 1648, status = succeeded5986lb # [ 134.912809] niks3-server[1141]: 2026/09/22 14:53:19 INFO Received uploads request method=POST path=/api/pending_closures5987worker2 # [ 134.965059] nix-grpc-daemon[1363]: time=2026-09-22T14:53:19.564Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5988worker2 # [ 134.966608] nix-grpc-daemon[1363]: time=2026-09-22T14:53:19.564Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5989lb # [ 134.951050] niks3-server[1141]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5990worker2 # [ 134.998816] nix-grpc-daemon[1363]: time=2026-09-22T14:53:19.599Z level=INFO msg="Uploading 1 narinfos"5991lb # [ 134.963280] niks3-server[1141]: 2026/09/22 14:53:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5992lb # [ 134.970445] niks3-server[1141]: 2026/09/22 14:53:19 INFO Signed narinfos id=49 count=15993lb # [ 134.976444] niks3-server[1141]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5994lb # [ 134.984527] niks3-server[1141]: 2026/09/22 14:53:19 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5995lb # [ 134.990605] niks3-server[1141]: 2026/09/22 14:53:19 INFO Completed upload id=495996worker2 # [ 135.025877] nix-grpc-daemon[1363]: time=2026-09-22T14:53:19.626Z level=INFO msg="Upload complete. (372ms)"5997lb # [ 134.992468] niks3-server[1141]: 2026/09/22 14:53:19 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5998worker2 # [ 135.035849] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=0 paths=1 nar_bytes_in=4165999worker2 # [ 135.044467] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:537626000worker2 # [ 135.060329] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)6001worker2 # [ 135.063219] nix-daemon[839]: reaped child process 1638, status = succeeded6002client # [ 135.100360] nix[1417]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...6003worker2 # [ 135.076452] nix-daemon[839]: accepted connection from pid 1303, user nix-grpc-daemon (trusted)6004lb: must succeed: curl -sf localhost:9901/clusters6005lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6006lb: must succeed: curl -sf localhost:9901/clusters6007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6008lb: must succeed: curl -sf localhost:9901/clusters6009lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6010lb: must succeed: curl -sf localhost:9901/clusters6011lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6012worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6013worker1 # [ 137.039749] systemd-journald[269]: Received client request to rotate journal, rotating.6014worker1 # [ 137.064934] systemd-journald[269]: Vacuuming done, freed 0B of archived journals from /var/log/journal/117159a78fbd43749c28f85305e28709.6015worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.08 seconds)6016worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6017worker2 # [ 137.151081] systemd[1]: Stopping Nix worker-protocol over gRPC...6018worker2 # [ 137.153131] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:21Z level=info event=draining6019worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)6020client: must succeed: sleep 5.06021lb # [ 138.115433] envoy[897]: conn peer=[2001:db8:1::1]:46672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9619 tx=4371 ms=54636022client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6023worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6024worker2 # [ 142.508789] nix-daemon[839]: reaped child process 1652, status = succeeded6025worker2 # [ 142.602838] nix-daemon[839]: accepted connection from pid 1693, user nix-grpc-daemon (trusted)6026worker2 # [ 142.621772] nix-daemon[839]: reaped child process 1696, status = succeeded6027lb # [ 142.592907] niks3-server[1141]: 2026/09/22 14:53:27 INFO Received uploads request method=POST path=/api/pending_closures6028worker2 # [ 142.644073] nix-grpc-daemon[1363]: time=2026-09-22T14:53:27.241Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6029worker2 # [ 142.645587] nix-grpc-daemon[1363]: time=2026-09-22T14:53:27.241Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"6030lb # [ 142.630144] niks3-server[1141]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv6031lb # [ 142.648473] niks3-server[1141]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6032worker2 # [ 142.695209] nix-grpc-daemon[1363]: time=2026-09-22T14:53:27.295Z level=INFO msg="Uploading 1 narinfos"6033lb # [ 142.659744] niks3-server[1141]: 2026/09/22 14:53:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6034lb # [ 142.668505] niks3-server[1141]: 2026/09/22 14:53:27 INFO Signed narinfos id=50 count=16035lb # [ 142.673127] niks3-server[1141]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls6036lb # [ 142.681973] niks3-server[1141]: 2026/09/22 14:53:27 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6037worker2 # [ 142.723102] nix-grpc-daemon[1363]: time=2026-09-22T14:53:27.323Z level=INFO msg="Upload complete. (231ms)"6038lb # [ 142.686282] niks3-server[1141]: 2026/09/22 14:53:27 INFO Completed upload id=506039lb # [ 142.691262] niks3-server[1141]: 2026/09/22 14:53:27 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo6040worker2 # [ 142.740127] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53762 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=16041worker2 # [ 142.753012] nix-daemon[839]: reaped child process 1650, status = succeeded6042client # [ 142.798401] systemd[1]: rs.service: Deactivated successfully.6043client # [ 142.803715] systemd[1]: rs.service: Consumed 198ms CPU time over 8.488s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6044worker2 # [ 143.155618] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:27Z level=info event=signal_exit6045lb # [ 144.017200] niks3-server[1141]: 2026/09/22 14:53:28 INFO lead: released remote=192.168.1.4:573466046worker1 # [ 144.999631] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:29Z level=info event=scheduler_restarting addr=lb:500516047worker2 # [ 145.052945] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=scheduler_restarting6048worker2 # [ 145.058288] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=scheduler_restarting addr=lb:500516049worker2 # [ 145.063468] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:382306050worker2 # [ 145.068766] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=worker_gone addr=192.168.1.3:500516051worker2 # [ 145.073681] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:382306052worker2 # [ 145.078412] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=worker_gone addr=192.168.1.4:500516053worker2 # [ 145.105321] nix-daemon[839]: reaped child process 1355, status = succeeded6054worker2 # [ 145.223332] nix-grpc-daemon[1303]: E0000 00:00:1790088809.823293 1348 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 12, started true, shutdown true6055worker2 # [ 145.230053] nix-grpc-daemon[1303]: E0000 00:00:1790088809.823606 1348 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6056worker1 # [ 145.199303] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:29Z level=info event=scheduler_connected addr=lb:500516057worker1 # [ 145.204776] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6058worker2 # [ 145.274067] nix-grpc-daemon[1303]: ts=2026-09-22T14:53:29Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6059worker1 # [ 145.405465] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:500516060worker1 # [ 145.410642] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6061worker1 # [ 145.611480] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:500516062worker1 # [ 145.616700] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6063worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.26 seconds)6064client: waiting for success: ! systemctl is-active rs6065client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)6066client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6067client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)6068client: must fail: journalctl -u rs | grep -q 'reconnecting'6069worker1 # [ 145.817817] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:500516070worker1 # [ 145.823106] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6071client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6072lb: must succeed: curl -sf localhost:9901/clusters6073lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6074worker1 # [ 146.025577] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:500516075worker1 # [ 146.029688] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6076worker2 # [ 146.089665] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6077worker2 # [ 146.095931] systemd[1]: Stopped Nix worker-protocol over gRPC.6078worker2 # [ 146.100918] systemd[1]: nix-grpc-daemon.service: Consumed 3.256s CPU time over 35.116s wall clock time, 46.9M memory peak, 84.3K incoming IP traffic, 179.4K outgoing IP traffic.6079worker2 # [ 146.120431] systemd[1]: Starting scheduler-token.service...6080worker2 # [ 146.196354] systemd[1]: scheduler-token.service: Deactivated successfully.6081worker2 # [ 146.201259] systemd[1]: Finished scheduler-token.service.6082worker2 # [ 146.203113] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 78ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6083worker2 # [ 146.225061] systemd[1]: Starting Nix worker-protocol over gRPC...6084worker1 # [ 146.226457] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_connected addr=lb:500516085worker1 # [ 146.231723] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6086worker2 # [ 146.339786] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6087worker2 # [ 146.342153] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6088worker2 # [ 146.345895] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16089worker2 # [ 146.351153] nix-grpc-daemon[1706]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6090worker2 # [ 146.359759] nix-grpc-daemon[1706]: E0000 00:00:1790088810.950732 1706 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6091lb # [ 146.325708] niks3-server[1141]: 2026/09/22 14:53:30 INFO lead: acquired remote=192.168.1.4:550286092worker2 # [ 146.367916] nix-grpc-daemon[1706]: E0000 00:00:1790088810.954307 1706 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6093worker2 # [ 146.374332] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6094worker2 # [ 146.375945] nix-grpc-daemon[1706]: E0000 00:00:1790088810.963209 1748 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6095worker2 # [ 146.399986] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:30Z level=info event=scheduler_take_over6096worker2 # [ 146.401779] systemd[1]: Started Nix worker-protocol over gRPC.6097worker2 # [ 146.402603] nix-grpc-daemon[1706]: E0000 00:00:1790088810.984981 1748 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6098worker2 # [ 146.404924] nix-grpc-daemon[1706]: E0000 00:00:1790088810.985130 1748 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6099worker2 # [ 146.410320] nix-grpc-daemon[1706]: E0000 00:00:1790088810.999399 1748 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6100worker2 # [ 146.414826] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516101worker2 # [ 146.416167] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6102worker1 # [ 146.433056] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516103worker1 # [ 146.438351] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6104worker2 # [ 146.616330] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516105worker2 # [ 146.621301] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6106worker1 # [ 146.639318] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516107worker1 # [ 146.644752] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6108worker2 # [ 146.822328] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516109worker2 # [ 146.827527] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6110worker1 # [ 146.845511] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516111worker1 # [ 146.850803] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6112lb: must succeed: curl -sf localhost:9901/clusters6113lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6114worker2 # [ 147.028448] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516115worker2 # [ 147.033472] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6116worker1 # [ 147.052836] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516117worker2 # [ 147.106108] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:394346118worker2 # [ 147.111333] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06119worker2 # [ 147.234528] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=scheduler_connected addr=lb:500516120worker2 # [ 147.248160] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=oidc_jwks provider=mock keys=16121worker2 # [ 147.252886] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:394346122worker2 # [ 147.258196] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06123lb: must succeed: curl -sf localhost:9901/clusters6124lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6125worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6126lb # [ 148.113922] envoy[897]: conn peer=[2001:db8:1::1]:39064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1538 ms=82596127lb # [ 148.120894] envoy[897]: conn peer=[2001:db8:1::4]:38284 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=25409 tx=1859 ms=340796128worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6129(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.95 seconds)6130subtest: scheduler restart mid-build: build finishes, no double build6131client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid6132client # [ 148.265033] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.6133client # Running as unit: mid.service; invocation ID: a29aee303cdc402a916a44383c8e40ef6134client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)6135worker2 # [ 148.441666] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:394446136worker2 # [ 148.456240] nix-daemon[839]: accepted connection from pid 1706, user nix-grpc-daemon (trusted)6137client # [ 148.501056] nix[1453]: don't know how to build these paths:6138client # [ 148.502809] nix[1453]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6139worker2 # [ 148.489505] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:33Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06140worker2 # [ 148.495902] nix-grpc-daemon[1706]: ts=2026-09-22T14:53:33Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16141worker1 # [ 148.489659] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414546142worker1 # [ 148.507192] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 29 ms (attempt 1/5)6143worker1 # [ 148.545759] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 122 ms (attempt 2/5)6144worker1 # [ 148.669919] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 3/5)6145worker1 # [ 148.843598] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 197 ms (attempt 4/5)6146worker1 # [ 149.044275] nix-grpc-daemon[1344]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6147worker1 # [ 149.050916] nix-grpc-daemon[1344]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6148worker1 # [ 149.071562] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414546149client # [ 149.164224] nix[1453]: copying 1 paths to 'lb:50051'...6150worker1 # [ 149.087758] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414546151client # [ 149.164559] nix[1453]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...6152worker1 # [ 149.105484] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)6153worker1 # [ 149.131624] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:33Z level=info event=niks3_push_started pid=14646154worker1 # [ 149.282975] nix-grpc-daemon[1464]: time=2026-09-22T14:53:33.901Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6155worker1 # [ 149.401432] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6156worker1 # [ 149.432521] nix-daemon[849]: reaped child process 1483, status = succeeded6157worker1 # [ 149.553532] nix-daemon[849]: accepted connection from pid 1486, user nix-grpc-daemon (trusted)6158worker1 # [ 149.572358] nix-daemon[849]: reaped child process 1489, status = succeeded6159lb # [ 149.594096] niks3-server[1141]: 2026/09/22 14:53:34 INFO Received uploads request method=POST path=/api/pending_closures6160worker1 # [ 149.592073] nix-grpc-daemon[1464]: time=2026-09-22T14:53:34.210Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6161worker1 # [ 149.593574] nix-grpc-daemon[1464]: time=2026-09-22T14:53:34.210Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"6162lb # [ 149.645257] niks3-server[1141]: 2026/09/22 14:53:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6163lb # [ 149.663718] niks3-server[1141]: 2026/09/22 14:53:34 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst6164lb # [ 149.677321] niks3-server[1141]: 2026/09/22 14:53:34 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls6165lb # [ 149.687658] niks3-server[1141]: 2026/09/22 14:53:34 INFO Signed narinfos id=51 count=16166worker1 # [ 149.675167] nix-grpc-daemon[1464]: time=2026-09-22T14:53:34.293Z level=INFO msg="Uploading 1 narinfos"6167lb # [ 149.700253] niks3-server[1141]: 2026/09/22 14:53:34 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6168lb # [ 149.713880] niks3-server[1141]: 2026/09/22 14:53:34 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo6169lb # [ 149.728211] niks3-server[1141]: 2026/09/22 14:53:34 INFO Completed upload id=516170worker1 # [ 149.715753] nix-grpc-daemon[1464]: time=2026-09-22T14:53:34.334Z level=INFO msg="Upload complete. (428ms)"6171worker1 # [ 149.724544] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41454 duration_s=0 paths=1 nar_bytes_in=4246172worker1 # [ 149.727386] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414546173worker1 # [ 149.739068] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)6174worker1 # [ 149.743360] nix-daemon[849]: reaped child process 1462, status = succeeded6175worker1 # [ 149.752726] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)6176client # [ 149.829857] nix[1453]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6177lb: must succeed: curl -sf localhost:9901/clusters6178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6179lb: must succeed: curl -sf localhost:9901/clusters6180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6181worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6182worker2 # [ 152.213062] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1706 (nix-grpc-daemon) on client request.6183worker2 # [ 152.232468] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6184lb # [ 152.205885] niks3-server[1141]: 2026/09/22 14:53:36 INFO lead: released remote=192.168.1.4:550286185worker1 # [ 152.199291] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:36Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6186client # [ 152.277106] nix[1453]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6187worker2 # [ 152.262266] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6188worker2 # [ 152.278897] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6189worker2 # [ 152.291469] systemd[1]: nix-grpc-daemon.service: Consumed 227ms CPU time over 6.040s wall clock time, 14.4M memory peak, 9.3K incoming IP traffic, 14.8K outgoing IP traffic.6190worker2 # [ 152.293994] systemd[1]: Starting scheduler-token.service...6191worker2 # [ 152.295545] nix-daemon[839]: reaped child process 1765, status = succeeded6192worker2 # [ 152.339321] systemd[1]: scheduler-token.service: Deactivated successfully.6193worker2 # [ 152.344874] systemd[1]: Finished scheduler-token.service.6194worker2 # [ 152.346009] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 64ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6195worker2 # [ 152.349344] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6196worker2 # [ 152.353037] systemd[1]: Starting scheduler-token.service...6197worker2 # [ 152.405365] systemd[1]: scheduler-token.service: Deactivated successfully.6198worker2 # [ 152.410808] systemd[1]: Finished scheduler-token.service.6199worker2 # [ 152.411905] systemd[1]: scheduler-token.service: Consumed 28ms CPU time over 54ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6200worker2 # [ 152.416538] systemd[1]: Starting Nix worker-protocol over gRPC...6201worker1 # [ 152.399501] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:37Z level=info event=scheduler_connected addr=lb:500516202worker2 # [ 152.524594] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6203worker2 # [ 152.527187] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6204worker2 # [ 152.530984] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16205worker2 # [ 152.536063] nix-grpc-daemon[1799]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6206worker2 # [ 152.537506] nix-grpc-daemon[1799]: E0000 00:00:1790088817.135892 1799 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6207lb # [ 152.516393] niks3-server[1141]: 2026/09/22 14:53:37 INFO lead: acquired remote=192.168.1.4:345306208worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.40 seconds)6209client: must succeed: sleep 5.06210worker2 # [ 152.574391] nix-grpc-daemon[1799]: E0000 00:00:1790088817.142111 1799 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6211worker2 # [ 152.589656] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6212worker2 # [ 152.591853] nix-grpc-daemon[1799]: E0000 00:00:1790088817.152933 1838 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6213worker2 # [ 152.594737] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=scheduler_take_over6214worker2 # [ 152.595902] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:482766215worker2 # [ 152.600513] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=16216worker2 # [ 152.604663] nix-grpc-daemon[1799]: E0000 00:00:1790088817.172097 1838 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6217worker2 # [ 152.607914] nix-grpc-daemon[1799]: E0000 00:00:1790088817.172255 1838 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6218worker2 # [ 152.610845] systemd[1]: Started Nix worker-protocol over gRPC.6219worker2 # [ 152.612301] nix-grpc-daemon[1799]: E0000 00:00:1790088817.189229 1838 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6220worker2 # [ 152.614949] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=scheduler_connected addr=lb:500516221worker2 # [ 152.618197] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=oidc_jwks provider=mock keys=16222worker2 # [ 152.619476] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:482766223worker2 # [ 152.621293] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06224client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6225worker1 # [ 157.787924] nix-daemon[849]: reaped child process 1493, status = succeeded6226client: waiting for success: ! systemctl is-active mid6227worker1 # [ 157.887687] nix-daemon[849]: accepted connection from pid 1530, user nix-grpc-daemon (trusted)6228worker1 # [ 157.909799] nix-daemon[849]: reaped child process 1533, status = succeeded6229lb # [ 157.927171] niks3-server[1141]: 2026/09/22 14:53:42 INFO Received uploads request method=POST path=/api/pending_closures6230worker1 # [ 157.931472] nix-grpc-daemon[1464]: time=2026-09-22T14:53:42.547Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6231worker1 # [ 157.933865] nix-grpc-daemon[1464]: time=2026-09-22T14:53:42.547Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6232lb # [ 157.981183] niks3-server[1141]: 2026/09/22 14:53:42 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6233lb # [ 157.999767] niks3-server[1141]: 2026/09/22 14:53:42 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6234lb # [ 158.011089] niks3-server[1141]: 2026/09/22 14:53:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6235lb # [ 158.020524] niks3-server[1141]: 2026/09/22 14:53:42 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6236lb # [ 158.032532] niks3-server[1141]: 2026/09/22 14:53:42 INFO Signed narinfos id=52 count=16237worker1 # [ 158.020084] nix-grpc-daemon[1464]: time=2026-09-22T14:53:42.637Z level=INFO msg="Uploading 1 narinfos"6238lb # [ 158.040764] niks3-server[1141]: 2026/09/22 14:53:42 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6239lb # [ 158.051138] niks3-server[1141]: 2026/09/22 14:53:42 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6240lb # [ 158.058457] niks3-server[1141]: 2026/09/22 14:53:42 INFO Completed upload id=526241worker1 # [ 158.045897] nix-grpc-daemon[1464]: time=2026-09-22T14:53:42.664Z level=INFO msg="Upload complete. (276ms)"6242worker1 # [ 158.056794] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41454 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16243worker1 # [ 158.067052] nix-daemon[849]: reaped child process 1491, status = succeeded6244client # [ 158.170564] systemd[1]: mid.service: Deactivated successfully.6245client # [ 158.176286] systemd[1]: mid.service: Consumed 203ms CPU time over 9.907s wall clock time, 39.2M memory peak, 7.8K incoming IP traffic, 8.8K outgoing IP traffic.6246lb # [ 158.115895] envoy[897]: conn peer=[2001:db8:1::4]:36860 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=4102 tx=385 ms=58466247lb # [ 158.120791] envoy[897]: conn peer=[2001:db8:1::1]:42928 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3078 tx=1630 ms=96856248client: (finished: waiting for success: ! systemctl is-active mid, in 1.13 seconds)6249client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6250client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)6251(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.85 seconds)6252subtest: interrupting the client stops the build on the worker6253client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6254client # [ 159.083327] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6255client # Running as unit: intr.service; invocation ID: ae507e2abbec46519e96d7fe1381abd46256client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6257worker2 # [ 159.240767] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:482906258worker2 # [ 159.250584] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6259client # [ 159.299117] nix[1489]: don't know how to build these paths:6260client # [ 159.301129] nix[1489]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6261worker2 # [ 159.281797] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:43Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06262worker2 # [ 159.284802] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:43Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16263worker2 # [ 159.334739] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482906264worker2 # [ 159.350486] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 71 ms (attempt 1/5)6265worker2 # [ 159.429243] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 179 ms (attempt 2/5)6266worker2 # [ 159.610810] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 333 ms (attempt 3/5)6267worker2 # [ 159.946531] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 575 ms (attempt 4/5)6268worker2 # [ 160.525858] nix-grpc-daemon[1799]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6269worker2 # [ 160.532852] nix-grpc-daemon[1799]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6270worker2 # [ 160.554877] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482906271client # [ 160.598304] nix[1489]: copying 1 paths to 'lb:50051'...6272worker2 # [ 160.575361] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482906273client # [ 160.603542] nix[1489]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6274worker2 # [ 160.589927] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6275worker2 # [ 160.623337] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=info event=niks3_push_started pid=18776276worker2 # [ 160.769220] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.369Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6277worker2 # [ 160.882335] nix-daemon[839]: accepted connection from pid 1889, user nix-grpc-daemon (trusted)6278worker2 # [ 160.904660] nix-daemon[839]: reaped child process 1892, status = succeeded6279worker2 # [ 161.012857] nix-daemon[839]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)6280lb # [ 160.997074] niks3-server[1141]: 2026/09/22 14:53:45 INFO Received uploads request method=POST path=/api/pending_closures6281worker2 # [ 161.035528] nix-daemon[839]: reaped child process 1897, status = succeeded6282worker2 # [ 161.044195] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.643Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6283worker2 # [ 161.045699] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.643Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6284lb # [ 161.045910] niks3-server[1141]: 2026/09/22 14:53:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6285worker2 # [ 161.095478] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.695Z level=INFO msg="Uploading 1 narinfos"6286lb # [ 161.064681] niks3-server[1141]: 2026/09/22 14:53:45 INFO Signed narinfos id=53 count=16287lb # [ 161.072152] niks3-server[1141]: 2026/09/22 14:53:45 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6288lb # [ 161.087738] niks3-server[1141]: 2026/09/22 14:53:45 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6289lb # [ 161.096516] niks3-server[1141]: 2026/09/22 14:53:45 INFO Completed upload id=536290worker2 # [ 161.131919] nix-grpc-daemon[1877]: time=2026-09-22T14:53:45.732Z level=INFO msg="Upload complete. (359ms)"6291lb # [ 161.099980] niks3-server[1141]: 2026/09/22 14:53:45 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6292lb # [ 161.107598] niks3-server[1141]: 2026/09/22 14:53:45 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6293worker2 # [ 161.144580] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=4246294worker2 # [ 161.155887] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482906295worker2 # [ 161.169720] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6296worker2 # [ 161.175448] nix-daemon[839]: reaped child process 1875, status = succeeded6297worker2 # [ 161.186792] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6298client # [ 161.213077] nix[1489]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6299client: must succeed: systemctl kill -s INT intr6300worker2 # [ 162.720122] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:47Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6301client # [ 162.744649] systemd[1]: intr.service: Sent signal SIGINT to main process 1489 (nix) on client request.6302client # [ 162.754680] nix[1489]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6303client # [ 162.770084] nix[1489]: error: interrupted by the user6304client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6305client # [ 162.800332] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6306client # [ 162.805375] systemd[1]: intr.service: Failed with result 'exit-code'.6307client # [ 162.807819] systemd[1]: intr.service: Consumed 176ms CPU time over 3.720s wall clock time, 39.4M memory peak, 6.4K incoming IP traffic, 8.2K outgoing IP traffic.6308worker2 # [ 162.821308] nix-daemon[839]: reaped child process 1899, status = succeeded6309worker2 # [ 162.822648] nix-daemon[1901]: unexpected Nix daemon error: error: interrupted by the user6310worker2 # [ 162.838264] nix-daemon[839]: reaped child process 1901, status = failed with exit code 16311(finished: subtest: interrupting the client stops the build on the worker, in 3.84 seconds)6312subtest: a deploy mid-build drains: the build finishes, the new generation takes over6313client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6314client # [ 162.927442] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6315client # Running as unit: sw.service; invocation ID: 0beaecfb8a0e4143a101faa8c1d90f2e6316client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.05 seconds)6317client # [ 163.119602] nix[1510]: don't know how to build these paths:6318client # [ 163.121715] nix[1510]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6319worker1 # [ 163.042491] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:414546320worker2 # [ 163.108402] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:47Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06321worker2 # [ 163.115365] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:47Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16322worker1 # [ 163.113441] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414546323worker1 # [ 163.129859] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:414546324client # [ 163.220215] nix[1510]: copying 1 paths to 'lb:50051'...6325client # [ 163.222161] nix[1510]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6326worker1 # [ 163.145892] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:414546327worker1 # [ 163.162471] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)6328worker1 # [ 163.345929] nix-daemon[849]: accepted connection from pid 1561, user nix-grpc-daemon (trusted)6329worker1 # [ 163.369993] nix-daemon[849]: reaped child process 1564, status = succeeded6330worker1 # [ 163.480844] nix-daemon[849]: accepted connection from pid 1566, user nix-grpc-daemon (trusted)6331worker1 # [ 163.504314] nix-daemon[849]: reaped child process 1569, status = succeeded6332lb # [ 163.517128] niks3-server[1141]: 2026/09/22 14:53:48 INFO Received uploads request method=POST path=/api/pending_closures6333worker1 # [ 163.521462] nix-grpc-daemon[1464]: time=2026-09-22T14:53:48.139Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6334worker1 # [ 163.523007] nix-grpc-daemon[1464]: time=2026-09-22T14:53:48.139Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6335lb # [ 163.559561] niks3-server[1141]: 2026/09/22 14:53:48 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6336lb # [ 163.572523] niks3-server[1141]: 2026/09/22 14:53:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6337worker1 # [ 163.565664] nix-grpc-daemon[1464]: time=2026-09-22T14:53:48.184Z level=INFO msg="Uploading 1 narinfos"6338lb # [ 163.582198] niks3-server[1141]: 2026/09/22 14:53:48 INFO Signed narinfos id=54 count=16339lb # [ 163.588291] niks3-server[1141]: 2026/09/22 14:53:48 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6340lb # [ 163.601570] niks3-server[1141]: 2026/09/22 14:53:48 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6341lb # [ 163.608135] niks3-server[1141]: 2026/09/22 14:53:48 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6342lb # [ 163.613787] niks3-server[1141]: 2026/09/22 14:53:48 INFO Completed upload id=546343worker1 # [ 163.601035] nix-grpc-daemon[1464]: time=2026-09-22T14:53:48.219Z level=INFO msg="Upload complete. (415ms)"6344worker1 # [ 163.612299] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41454 duration_s=0 paths=1 nar_bytes_in=4166345worker1 # [ 163.628885] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:414546346worker1 # [ 163.635659] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)6347worker1 # [ 163.641366] nix-daemon[849]: reaped child process 1559, status = succeeded6348worker1 # [ 163.652542] nix-daemon[849]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)6349client # [ 163.728093] nix[1510]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6350worker1: must succeed: systemctl reload nix-grpc-daemon.service6351worker1 # [ 165.490403] systemd[1]: Reloading Nix worker-protocol over gRPC...6352worker1 # [ 165.536675] systemd[1]: Reloaded Nix worker-protocol over gRPC.6353worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.10 seconds)6354worker1: must succeed: pgrep -f 'read -t [9]0 x'6355worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6356client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26357client # don't know how to build these paths:6358client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6359worker2 # [ 165.875713] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:482906360worker2 # [ 165.884494] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482906361worker1 # [ 165.850377] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:50Z level=info event=draining6362worker2 # [ 165.913018] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06363worker2 # [ 165.919891] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16364worker2 # [ 165.961464] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482906365worker2 # [ 165.979944] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482906366client # copying 1 paths to 'lb:50051'...6367client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6368worker2 # [ 165.998300] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482906369worker2 # [ 166.014931] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6370worker2 # [ 166.177828] nix-daemon[839]: accepted connection from pid 1952, user nix-grpc-daemon (trusted)6371worker2 # [ 166.204672] nix-daemon[839]: reaped child process 1955, status = succeeded6372worker2 # [ 166.302620] nix-daemon[839]: accepted connection from pid 1957, user nix-grpc-daemon (trusted)6373worker2 # [ 166.324311] nix-daemon[839]: reaped child process 1960, status = succeeded6374lb # [ 166.286122] niks3-server[1141]: 2026/09/22 14:53:50 INFO Received uploads request method=POST path=/api/pending_closures6375worker2 # [ 166.333923] nix-grpc-daemon[1877]: time=2026-09-22T14:53:50.933Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6376worker2 # [ 166.335416] nix-grpc-daemon[1877]: time=2026-09-22T14:53:50.933Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6377lb # [ 166.327753] niks3-server[1141]: 2026/09/22 14:53:50 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6378lb # [ 166.342285] niks3-server[1141]: 2026/09/22 14:53:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6379worker2 # [ 166.386577] nix-grpc-daemon[1877]: time=2026-09-22T14:53:50.986Z level=INFO msg="Uploading 1 narinfos"6380lb # [ 166.352368] niks3-server[1141]: 2026/09/22 14:53:50 INFO Signed narinfos id=55 count=16381lb # [ 166.359549] niks3-server[1141]: 2026/09/22 14:53:50 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6382lb # [ 166.375367] niks3-server[1141]: 2026/09/22 14:53:50 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6383lb # [ 166.385988] niks3-server[1141]: 2026/09/22 14:53:50 INFO Completed upload id=556384worker2 # [ 166.422816] nix-grpc-daemon[1877]: time=2026-09-22T14:53:51.023Z level=INFO msg="Upload complete. (389ms)"6385lb # [ 166.388853] niks3-server[1141]: 2026/09/22 14:53:50 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6386worker2 # [ 166.433360] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=5286387worker2 # [ 166.450704] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482906388worker2 # [ 166.457689] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6389worker2 # [ 166.465406] nix-daemon[839]: reaped child process 1950, status = succeeded6390client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6391worker2 # [ 166.477530] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6392client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6393client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6394client # farm-a> LOG-farm-a-during-drain6395lb # [ 168.115601] envoy[897]: conn peer=[2001:db8:1::1]:52032 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2974 tx=853 ms=35206396worker2 # [ 168.168739] nix-daemon[839]: reaped child process 1964, status = succeeded6397worker2 # [ 168.270393] nix-daemon[839]: accepted connection from pid 1980, user nix-grpc-daemon (trusted)6398worker2 # [ 168.294193] nix-daemon[839]: reaped child process 1983, status = succeeded6399lb # [ 168.256499] niks3-server[1141]: 2026/09/22 14:53:52 INFO Received uploads request method=POST path=/api/pending_closures6400worker2 # [ 168.305241] nix-grpc-daemon[1877]: time=2026-09-22T14:53:52.904Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6401worker2 # [ 168.312935] nix-grpc-daemon[1877]: time=2026-09-22T14:53:52.908Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6402lb # [ 168.298324] niks3-server[1141]: 2026/09/22 14:53:52 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6403lb # [ 168.314995] niks3-server[1141]: 2026/09/22 14:53:52 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6404lb # [ 168.328082] niks3-server[1141]: 2026/09/22 14:53:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6405worker2 # [ 168.366227] nix-grpc-daemon[1877]: time=2026-09-22T14:53:52.965Z level=INFO msg="Uploading 1 narinfos"6406lb # [ 168.331774] niks3-server[1141]: 2026/09/22 14:53:52 INFO Signed narinfos id=56 count=16407lb # [ 168.342037] niks3-server[1141]: 2026/09/22 14:53:52 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6408lb # [ 168.352741] niks3-server[1141]: 2026/09/22 14:53:52 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6409lb # [ 168.359605] niks3-server[1141]: 2026/09/22 14:53:52 INFO Completed upload id=566410worker2 # [ 168.395542] nix-grpc-daemon[1877]: time=2026-09-22T14:53:52.995Z level=INFO msg="Upload complete. (301ms)"6411lb # [ 168.363081] niks3-server[1141]: 2026/09/22 14:53:52 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6412worker2 # [ 168.416299] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=16413worker2 # [ 168.432437] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06414worker2 # [ 168.437708] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16415worker2 # [ 168.442345] nix-daemon[839]: reaped child process 1962, status = succeeded6416worker2 # [ 168.461694] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482906417worker2 # [ 168.477236] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482906418worker2 # [ 168.496622] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.096Z level=INFO msg="All 1 paths already cached"6419client # copying 1 paths to 'lb:50051'...6420client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6421worker2 # [ 168.507893] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482906422worker2 # [ 168.526192] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6423worker2 # [ 168.686177] nix-daemon[839]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6424worker2 # [ 168.715304] nix-daemon[839]: reaped child process 1992, status = succeeded6425worker2 # [ 168.832142] nix-daemon[839]: accepted connection from pid 1994, user nix-grpc-daemon (trusted)6426worker2 # [ 168.850963] nix-daemon[839]: reaped child process 1997, status = succeeded6427lb # [ 168.821272] niks3-server[1141]: 2026/09/22 14:53:53 INFO Received uploads request method=POST path=/api/pending_closures6428worker2 # [ 168.872070] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.472Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6429worker2 # [ 168.874485] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.472Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6430lb # [ 168.859756] niks3-server[1141]: 2026/09/22 14:53:53 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6431worker2 # [ 168.914504] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.514Z level=INFO msg="Uploading 1 narinfos"6432lb # [ 168.877559] niks3-server[1141]: 2026/09/22 14:53:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6433lb # [ 168.884366] niks3-server[1141]: 2026/09/22 14:53:53 INFO Signed narinfos id=57 count=16434lb # [ 168.891667] niks3-server[1141]: 2026/09/22 14:53:53 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6435lb # [ 168.902807] niks3-server[1141]: 2026/09/22 14:53:53 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6436worker2 # [ 168.949552] nix-grpc-daemon[1877]: time=2026-09-22T14:53:53.549Z level=INFO msg="Upload complete. (411ms)"6437lb # [ 168.914048] niks3-server[1141]: 2026/09/22 14:53:53 INFO Completed upload id=576438lb # [ 168.919223] niks3-server[1141]: 2026/09/22 14:53:53 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6439worker2 # [ 168.960489] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=6406440worker2 # [ 168.969029] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482906441worker2 # [ 168.981947] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6442worker2 # [ 168.987116] nix-daemon[839]: reaped child process 1987, status = succeeded6443client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6444worker2 # [ 168.998186] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6445client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6446client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6447client # farm-b> LOG-farm-b-during-drain6448worker2 # [ 169.853322] nix-daemon[839]: reaped child process 2001, status = succeeded6449worker2 # [ 169.967917] nix-daemon[839]: accepted connection from pid 2017, user nix-grpc-daemon (trusted)6450worker2 # [ 169.986815] nix-daemon[839]: reaped child process 2020, status = succeeded6451lb # [ 169.959100] niks3-server[1141]: 2026/09/22 14:53:54 INFO Received uploads request method=POST path=/api/pending_closures6452worker2 # [ 170.007625] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.606Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6453worker2 # [ 170.009586] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.606Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6454lb # [ 170.000824] niks3-server[1141]: 2026/09/22 14:53:54 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6455lb # [ 170.013844] niks3-server[1141]: 2026/09/22 14:53:54 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6456worker2 # [ 170.059610] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.659Z level=INFO msg="Uploading 1 narinfos"6457lb # [ 170.024990] niks3-server[1141]: 2026/09/22 14:53:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6458lb # [ 170.033631] niks3-server[1141]: 2026/09/22 14:53:54 INFO Signed narinfos id=58 count=16459lb # [ 170.038666] niks3-server[1141]: 2026/09/22 14:53:54 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6460lb # [ 170.047878] niks3-server[1141]: 2026/09/22 14:53:54 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6461lb # [ 170.053563] niks3-server[1141]: 2026/09/22 14:53:54 INFO Completed upload id=586462worker2 # [ 170.090662] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.690Z level=INFO msg="Upload complete. (280ms)"6463lb # [ 170.056677] niks3-server[1141]: 2026/09/22 14:53:54 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6464worker2 # [ 170.110309] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=16465worker2 # [ 170.128397] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06466worker2 # [ 170.133443] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16467worker2 # [ 170.138312] nix-daemon[839]: reaped child process 1999, status = succeeded6468worker2 # [ 170.153796] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482906469worker2 # [ 170.170533] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482906470worker2 # [ 170.188615] nix-grpc-daemon[1877]: time=2026-09-22T14:53:54.788Z level=INFO msg="All 2 paths already cached"6471client # copying 1 paths to 'lb:50051'...6472client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6473worker2 # [ 170.201192] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482906474worker2 # [ 170.216093] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6475worker2 # [ 170.372967] nix-daemon[839]: accepted connection from pid 2026, user nix-grpc-daemon (trusted)6476worker2 # [ 170.396465] nix-daemon[839]: reaped child process 2029, status = succeeded6477worker2 # [ 170.503439] nix-daemon[839]: accepted connection from pid 2031, user nix-grpc-daemon (trusted)6478worker2 # [ 170.524504] nix-daemon[839]: reaped child process 2034, status = succeeded6479lb # [ 170.491100] niks3-server[1141]: 2026/09/22 14:53:55 INFO Received uploads request method=POST path=/api/pending_closures6480worker2 # [ 170.539583] nix-grpc-daemon[1877]: time=2026-09-22T14:53:55.139Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6481worker2 # [ 170.541246] nix-grpc-daemon[1877]: time=2026-09-22T14:53:55.139Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6482lb # [ 170.522183] niks3-server[1141]: 2026/09/22 14:53:55 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6483worker2 # [ 170.574945] nix-grpc-daemon[1877]: time=2026-09-22T14:53:55.175Z level=INFO msg="Uploading 1 narinfos"6484lb # [ 170.538951] niks3-server[1141]: 2026/09/22 14:53:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6485lb # [ 170.551362] niks3-server[1141]: 2026/09/22 14:53:55 INFO Signed narinfos id=59 count=16486lb # [ 170.558953] niks3-server[1141]: 2026/09/22 14:53:55 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6487lb # [ 170.574399] niks3-server[1141]: 2026/09/22 14:53:55 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6488lb # [ 170.581945] niks3-server[1141]: 2026/09/22 14:53:55 INFO Completed upload id=596489lb # [ 170.584698] niks3-server[1141]: 2026/09/22 14:53:55 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6490worker2 # [ 170.616675] nix-grpc-daemon[1877]: time=2026-09-22T14:53:55.216Z level=INFO msg="Upload complete. (383ms)"6491worker2 # [ 170.630003] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=7686492worker2 # [ 170.644962] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482906493worker2 # [ 170.652424] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6494worker2 # [ 170.657853] nix-daemon[839]: reaped child process 2024, status = succeeded6495client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6496worker2 # [ 170.672082] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6497client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6498client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6499client # farm-top> LOG-farm-top-during-drain6500worker2 # [ 171.539905] nix-daemon[839]: reaped child process 2038, status = succeeded6501worker2 # [ 171.657413] nix-daemon[839]: accepted connection from pid 2054, user nix-grpc-daemon (trusted)6502worker2 # [ 171.679702] nix-daemon[839]: reaped child process 2057, status = succeeded6503lb # [ 171.642427] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received uploads request method=POST path=/api/pending_closures6504worker2 # [ 171.692475] nix-grpc-daemon[1877]: time=2026-09-22T14:53:56.291Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6505worker2 # [ 171.694137] nix-grpc-daemon[1877]: time=2026-09-22T14:53:56.291Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6506lb # [ 171.677634] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6507lb # [ 171.691703] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6508lb # [ 171.707196] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6509worker2 # [ 171.749012] nix-grpc-daemon[1877]: time=2026-09-22T14:53:56.348Z level=INFO msg="Uploading 1 narinfos"6510lb # [ 171.715267] niks3-server[1141]: 2026/09/22 14:53:56 INFO Signed narinfos id=60 count=16511lb # [ 171.721624] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6512lb # [ 171.735571] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6513worker2 # [ 171.780144] nix-grpc-daemon[1877]: time=2026-09-22T14:53:56.380Z level=INFO msg="Upload complete. (298ms)"6514lb # [ 171.743372] niks3-server[1141]: 2026/09/22 14:53:56 INFO Completed upload id=606515lb # [ 171.747699] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6516worker2 # [ 171.793297] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=16517worker2 # [ 171.815345] nix-daemon[839]: reaped child process 2036, status = succeeded6518client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.19 seconds)6519client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6520client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6521worker1: must succeed: pkill -f 'read -t [9]0 x'6522worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6523client: waiting for success: ! systemctl is-active sw6524worker1 # [ 172.094709] nix-daemon[849]: reaped child process 1573, status = succeeded6525worker1 # [ 172.199797] nix-daemon[849]: accepted connection from pid 1614, user nix-grpc-daemon (trusted)6526worker1 # [ 172.221046] nix-daemon[849]: reaped child process 1617, status = succeeded6527lb # [ 172.238642] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received uploads request method=POST path=/api/pending_closures6528worker1 # [ 172.237155] nix-grpc-daemon[1464]: time=2026-09-22T14:53:56.854Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6529worker1 # [ 172.238829] nix-grpc-daemon[1464]: time=2026-09-22T14:53:56.854Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6530lb # [ 172.283513] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6531worker1 # [ 172.284422] nix-grpc-daemon[1464]: time=2026-09-22T14:53:56.902Z level=INFO msg="Uploading 1 narinfos"6532lb # [ 172.297813] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6533lb # [ 172.309115] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6534lb # [ 172.317017] niks3-server[1141]: 2026/09/22 14:53:56 INFO Signed narinfos id=61 count=16535lb # [ 172.320887] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6536worker1 # [ 172.315383] nix-grpc-daemon[1464]: time=2026-09-22T14:53:56.934Z level=INFO msg="Upload complete. (242ms)"6537lb # [ 172.331224] niks3-server[1141]: 2026/09/22 14:53:56 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6538lb # [ 172.334277] niks3-server[1141]: 2026/09/22 14:53:56 INFO Completed upload id=616539lb # [ 172.338876] niks3-server[1141]: 2026/09/22 14:53:56 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6540worker1 # [ 172.329220] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41454 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=16541worker1 # [ 172.346151] nix-daemon[849]: reaped child process 1571, status = succeeded6542client # [ 172.443972] systemd[1]: sw.service: Deactivated successfully.6543client # [ 172.449568] systemd[1]: sw.service: Consumed 190ms CPU time over 9.517s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.3K outgoing IP traffic.6544worker1 # [ 172.852366] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:57Z level=info event=signal_exit6545client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)6546client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6547client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6548worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6549worker2 # [ 173.914373] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:482766550worker2 # [ 173.919348] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:58Z level=info event=worker_gone addr=192.168.1.3:500516551worker1 # [ 173.872440] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:58Z level=info event=scheduler_restarting6552worker1 # [ 173.889824] nix-grpc-daemon[1344]: ts=2026-09-22T14:53:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6553worker1 # [ 173.915420] nix-daemon[849]: reaped child process 1403, status = succeeded6554worker1 # [ 173.949762] nix-grpc-daemon[1344]: E0000 00:00:1790088838.568357 1389 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6555worker1 # [ 173.953211] nix-grpc-daemon[1344]: E0000 00:00:1790088838.568514 1389 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6556worker1 # [ 174.240612] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6557worker1 # [ 174.247163] systemd[1]: nix-grpc-daemon.service: Consumed 1.201s CPU time over 40.572s wall clock time, 36.2M memory peak, 26.7K incoming IP traffic, 39K outgoing IP traffic.6558worker1 # [ 174.269195] systemd[1]: Starting Nix worker-protocol over gRPC...6559worker1 # [ 174.411867] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6560worker1 # [ 174.414521] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6561worker1 # [ 174.418310] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16562worker1 # [ 174.424080] nix-grpc-daemon[1631]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6563worker1 # [ 174.425646] nix-grpc-daemon[1631]: E0000 00:00:1790088839.041647 1631 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6564worker1 # [ 174.473922] nix-grpc-daemon[1631]: E0000 00:00:1790088839.048339 1631 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6565worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.28 seconds)6566lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6567lb: must succeed: curl -sf localhost:9901/clusters6568worker1 # [ 174.477320] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6569worker1 # [ 174.479111] nix-grpc-daemon[1631]: E0000 00:00:1790088839.055210 1670 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6570worker1 # [ 174.481592] nix-grpc-daemon[1631]: E0000 00:00:1790088839.073726 1670 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6571worker1 # [ 174.487625] nix-grpc-daemon[1631]: E0000 00:00:1790088839.074002 1670 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6572worker1 # [ 174.492660] nix-grpc-daemon[1631]: E0000 00:00:1790088839.091028 1670 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6573worker1 # [ 174.495222] systemd[1]: Started Nix worker-protocol over gRPC.6574worker1 # [ 174.496143] nix-grpc-daemon[1631]: ts=2026-09-22T14:53:59Z level=info event=scheduler_connected addr=lb:500516575worker2 # [ 174.542976] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:482766576worker2 # [ 174.549713] nix-grpc-daemon[1799]: ts=2026-09-22T14:53:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06577lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6578lb: must succeed: curl -sf localhost:9901/clusters6579lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6580lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)6581lb: must succeed: curl -sf localhost:9901/clusters6582lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6583worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6584worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6585(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.93 seconds)6586subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6587client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16588worker2 # [ 176.033135] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:482906589worker1 # [ 175.994346] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806590worker1 # [ 176.009934] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6591worker1 # [ 176.035141] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 1/5)6592worker1 # [ 176.110995] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 156 ms (attempt 2/5)6593worker1 # [ 176.269158] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 181 ms (attempt 3/5)6594worker1 # [ 176.460121] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 724 ms (attempt 4/5)6595worker1 # [ 177.188001] nix-grpc-daemon[1631]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6596worker1 # [ 177.190181] nix-grpc-daemon[1631]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6597worker2 # [ 177.301495] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:01Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06598worker2 # [ 177.308352] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:01Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16599worker2 # [ 177.314783] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:01Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06600worker2 # [ 177.320268] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:01Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06601worker1 # [ 177.305981] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806602worker1 # [ 177.326112] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806603worker1 # [ 177.343930] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806604worker1 # [ 177.360841] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6605worker1 # [ 177.396381] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:02Z level=info event=niks3_push_started pid=16916606worker1 # [ 177.542233] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.160Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6607worker1 # [ 177.654515] nix-daemon[849]: accepted connection from pid 1704, user nix-grpc-daemon (trusted)6608worker1 # [ 177.675422] nix-daemon[849]: reaped child process 1707, status = succeeded6609worker1 # [ 177.785873] nix-daemon[849]: accepted connection from pid 1709, user nix-grpc-daemon (trusted)6610worker1 # [ 177.807591] nix-daemon[849]: reaped child process 1712, status = succeeded6611lb # [ 177.823054] niks3-server[1141]: 2026/09/22 14:54:02 INFO Received uploads request method=POST path=/api/pending_closures6612worker1 # [ 177.822010] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.439Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6613worker1 # [ 177.823512] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.439Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6614lb # [ 177.865755] niks3-server[1141]: 2026/09/22 14:54:02 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6615worker1 # [ 177.873651] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.491Z level=INFO msg="Uploading 1 narinfos"6616lb # [ 177.885647] niks3-server[1141]: 2026/09/22 14:54:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6617lb # [ 177.894808] niks3-server[1141]: 2026/09/22 14:54:02 INFO Signed narinfos id=62 count=16618lb # [ 177.911124] niks3-server[1141]: 2026/09/22 14:54:02 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6619lb # [ 177.921496] niks3-server[1141]: 2026/09/22 14:54:02 INFO Completed upload id=626620worker1 # [ 177.906154] nix-grpc-daemon[1691]: time=2026-09-22T14:54:02.524Z level=INFO msg="Upload complete. (360ms)"6621lb # [ 177.927977] niks3-server[1141]: 2026/09/22 14:54:02 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6622worker1 # [ 177.916490] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=5126623worker1 # [ 177.928541] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806624worker1 # [ 177.939951] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6625worker1 # [ 177.945070] nix-daemon[849]: reaped child process 1689, status = succeeded6626worker1 # [ 177.958670] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6627lb # [ 178.116463] envoy[897]: conn peer=[2001:db8:1::1]:51204 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4402 ms=59346628lb # [ 178.119429] envoy[897]: conn peer=[2001:db8:1::1]:51190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=1517 ms=93056629lb # [ 178.125347] envoy[897]: conn peer=[2001:db8:1::3]:49218 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2445 tx=766 ms=399776630worker1 # [ 178.634086] nix-daemon[849]: reaped child process 1716, status = succeeded6631worker1 # [ 178.735574] nix-daemon[849]: accepted connection from pid 1732, user nix-grpc-daemon (trusted)6632worker1 # [ 178.759015] nix-daemon[849]: reaped child process 1735, status = succeeded6633lb # [ 178.775922] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received uploads request method=POST path=/api/pending_closures6634worker1 # [ 178.776561] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.394Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6635worker1 # [ 178.778565] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.394Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6636lb # [ 178.834917] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6637worker1 # [ 178.832164] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.449Z level=INFO msg="Uploading 1 narinfos"6638lb # [ 178.849705] niks3-server[1141]: 2026/09/22 14:54:03 INFO Signed narinfos id=63 count=16639lb # [ 178.856109] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6640lb # [ 178.865933] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6641lb # [ 178.875804] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6642lb # [ 178.888767] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6643worker1 # [ 178.881339] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.500Z level=INFO msg="Upload complete. (313ms)"6644lb # [ 178.897191] niks3-server[1141]: 2026/09/22 14:54:03 INFO Completed upload id=636645lb # [ 178.899344] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6646worker2 # [ 178.945658] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:03Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16647worker1 # [ 178.896748] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=6 outputs=16648worker2 # [ 178.956546] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:03Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06649worker1 # [ 178.916597] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806650worker1 # [ 178.925074] nix-daemon[849]: reaped child process 1714, status = succeeded6651worker1 # [ 178.933876] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806652worker1 # [ 178.947663] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806653worker1 # [ 178.961196] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6654worker1 # [ 179.104927] nix-daemon[849]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6655worker1 # [ 179.124496] nix-daemon[849]: reaped child process 1742, status = succeeded6656worker1 # [ 179.227847] nix-daemon[849]: accepted connection from pid 1744, user nix-grpc-daemon (trusted)6657worker1 # [ 179.246783] nix-daemon[849]: reaped child process 1747, status = succeeded6658lb # [ 179.263515] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received uploads request method=POST path=/api/pending_closures6659worker1 # [ 179.260099] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.878Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6660worker1 # [ 179.263014] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.878Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6661lb # [ 179.296234] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6662worker1 # [ 179.297414] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.915Z level=INFO msg="Uploading 1 narinfos"6663lb # [ 179.311100] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6664lb # [ 179.322827] niks3-server[1141]: 2026/09/22 14:54:03 INFO Signed narinfos id=64 count=16665lb # [ 179.329163] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6666lb # [ 179.340958] niks3-server[1141]: 2026/09/22 14:54:03 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6667worker1 # [ 179.332947] nix-grpc-daemon[1691]: time=2026-09-22T14:54:03.951Z level=INFO msg="Upload complete. (358ms)"6668lb # [ 179.348361] niks3-server[1141]: 2026/09/22 14:54:03 INFO Completed upload id=646669lb # [ 179.352508] niks3-server[1141]: 2026/09/22 14:54:03 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6670worker1 # [ 179.345054] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=5126671worker1 # [ 179.355852] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806672worker1 # [ 179.363668] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6673worker1 # [ 179.369227] nix-daemon[849]: reaped child process 1737, status = succeeded6674worker1 # [ 179.383224] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6675worker1 # [ 180.896654] nix-daemon[849]: reaped child process 1751, status = succeeded6676worker1 # [ 180.968967] nix-daemon[849]: accepted connection from pid 1767, user nix-grpc-daemon (trusted)6677worker1 # [ 180.991132] nix-daemon[849]: reaped child process 1770, status = succeeded6678lb # [ 181.013508] niks3-server[1141]: 2026/09/22 14:54:05 INFO Received uploads request method=POST path=/api/pending_closures6679worker1 # [ 181.013784] nix-grpc-daemon[1691]: time=2026-09-22T14:54:05.631Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6680worker1 # [ 181.015572] nix-grpc-daemon[1691]: time=2026-09-22T14:54:05.631Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6681lb # [ 181.056063] niks3-server[1141]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6682lb # [ 181.068734] niks3-server[1141]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6683worker1 # [ 181.065125] nix-grpc-daemon[1691]: time=2026-09-22T14:54:05.683Z level=INFO msg="Uploading 1 narinfos"6684lb # [ 181.078579] niks3-server[1141]: 2026/09/22 14:54:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6685lb # [ 181.088338] niks3-server[1141]: 2026/09/22 14:54:05 INFO Signed narinfos id=65 count=16686lb # [ 181.095686] niks3-server[1141]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6687lb # [ 181.110851] niks3-server[1141]: 2026/09/22 14:54:05 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6688lb # [ 181.116745] niks3-server[1141]: 2026/09/22 14:54:05 INFO Completed upload id=656689worker1 # [ 181.103417] nix-grpc-daemon[1691]: time=2026-09-22T14:54:05.722Z level=INFO msg="Upload complete. (308ms)"6690lb # [ 181.120572] niks3-server[1141]: 2026/09/22 14:54:05 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6691worker2 # [ 181.164716] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:05Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16692worker1 # [ 181.117237] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=7 outputs=16693worker2 # [ 181.178145] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:05Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06694worker1 # [ 181.138434] nix-daemon[849]: reaped child process 1749, status = succeeded6695worker1 # [ 181.172886] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806696worker1 # [ 181.191641] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806697worker1 # [ 181.208665] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806698worker1 # [ 181.223176] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6699worker1 # [ 181.387865] nix-daemon[849]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6700worker1 # [ 181.415697] nix-daemon[849]: reaped child process 1777, status = succeeded6701worker1 # [ 181.518392] nix-daemon[849]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6702worker1 # [ 181.538775] nix-daemon[849]: reaped child process 1782, status = succeeded6703lb # [ 181.555234] niks3-server[1141]: 2026/09/22 14:54:06 INFO Received uploads request method=POST path=/api/pending_closures6704worker1 # [ 181.551586] nix-grpc-daemon[1691]: time=2026-09-22T14:54:06.170Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6705worker1 # [ 181.553267] nix-grpc-daemon[1691]: time=2026-09-22T14:54:06.170Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6706lb # [ 181.582491] niks3-server[1141]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6707lb # [ 181.598672] niks3-server[1141]: 2026/09/22 14:54:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6708worker1 # [ 181.593348] nix-grpc-daemon[1691]: time=2026-09-22T14:54:06.211Z level=INFO msg="Uploading 1 narinfos"6709lb # [ 181.612454] niks3-server[1141]: 2026/09/22 14:54:06 INFO Signed narinfos id=66 count=16710lb # [ 181.615681] niks3-server[1141]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6711worker1 # [ 181.617358] nix-grpc-daemon[1691]: time=2026-09-22T14:54:06.236Z level=INFO msg="Upload complete. (377ms)"6712lb # [ 181.630303] niks3-server[1141]: 2026/09/22 14:54:06 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6713worker1 # [ 181.624142] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=5126714lb # [ 181.639537] niks3-server[1141]: 2026/09/22 14:54:06 INFO Completed upload id=666715lb # [ 181.642391] niks3-server[1141]: 2026/09/22 14:54:06 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6716worker1 # [ 181.639681] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806717worker1 # [ 181.646024] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6718worker1 # [ 181.650641] nix-daemon[849]: reaped child process 1772, status = succeeded6719worker1 # [ 181.663763] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6720worker1 # [ 182.943281] nix-daemon[849]: reaped child process 1786, status = succeeded6721worker1 # [ 183.046043] nix-daemon[849]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6722worker1 # [ 183.066815] nix-daemon[849]: reaped child process 1805, status = succeeded6723lb # [ 183.087107] niks3-server[1141]: 2026/09/22 14:54:07 INFO Received uploads request method=POST path=/api/pending_closures6724worker1 # [ 183.087564] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.704Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6725worker1 # [ 183.089617] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.704Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6726lb # [ 183.129121] niks3-server[1141]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6727lb # [ 183.141804] niks3-server[1141]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6728worker1 # [ 183.141770] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.760Z level=INFO msg="Uploading 1 narinfos"6729lb # [ 183.157865] niks3-server[1141]: 2026/09/22 14:54:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6730lb # [ 183.168842] niks3-server[1141]: 2026/09/22 14:54:07 INFO Signed narinfos id=67 count=16731lb # [ 183.171834] niks3-server[1141]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6732worker1 # [ 183.173356] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.792Z level=INFO msg="Upload complete. (299ms)"6733lb # [ 183.186787] niks3-server[1141]: 2026/09/22 14:54:07 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6734lb # [ 183.191431] niks3-server[1141]: 2026/09/22 14:54:07 INFO Completed upload id=676735lb # [ 183.196167] niks3-server[1141]: 2026/09/22 14:54:07 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6736worker2 # [ 183.233251] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:07Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16737worker1 # [ 183.188247] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=8 outputs=16738worker2 # [ 183.246671] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:07Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06739worker1 # [ 183.208511] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806740worker1 # [ 183.217464] nix-daemon[849]: reaped child process 1784, status = succeeded6741worker1 # [ 183.226563] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806742worker1 # [ 183.242485] nix-grpc-daemon[1691]: time=2026-09-22T14:54:07.861Z level=INFO msg="All 1 paths already cached"6743worker1 # [ 183.252915] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806744worker1 # [ 183.270080] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6745worker1 # [ 183.416296] nix-daemon[849]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6746worker1 # [ 183.439847] nix-daemon[849]: reaped child process 1812, status = succeeded6747worker1 # [ 183.546894] nix-daemon[849]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)6748worker1 # [ 183.567074] nix-daemon[849]: reaped child process 1817, status = succeeded6749lb # [ 183.584596] niks3-server[1141]: 2026/09/22 14:54:08 INFO Received uploads request method=POST path=/api/pending_closures6750worker1 # [ 183.581168] nix-grpc-daemon[1691]: time=2026-09-22T14:54:08.199Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6751worker1 # [ 183.582927] nix-grpc-daemon[1691]: time=2026-09-22T14:54:08.199Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6752lb # [ 183.615533] niks3-server[1141]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6753worker1 # [ 183.619061] nix-grpc-daemon[1691]: time=2026-09-22T14:54:08.237Z level=INFO msg="Uploading 1 narinfos"6754lb # [ 183.631546] niks3-server[1141]: 2026/09/22 14:54:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6755lb # [ 183.642482] niks3-server[1141]: 2026/09/22 14:54:08 INFO Signed narinfos id=68 count=16756lb # [ 183.648892] niks3-server[1141]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6757lb # [ 183.662329] niks3-server[1141]: 2026/09/22 14:54:08 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6758worker1 # [ 183.656912] nix-grpc-daemon[1691]: time=2026-09-22T14:54:08.275Z level=INFO msg="Upload complete. (374ms)"6759lb # [ 183.672457] niks3-server[1141]: 2026/09/22 14:54:08 INFO Completed upload id=686760lb # [ 183.676317] niks3-server[1141]: 2026/09/22 14:54:08 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6761worker1 # [ 183.668347] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=6246762worker1 # [ 183.684267] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806763worker1 # [ 183.690965] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6764worker1 # [ 183.696964] nix-daemon[849]: reaped child process 1807, status = succeeded6765worker1 # [ 183.708998] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6766worker1 # [ 185.024143] nix-daemon[849]: reaped child process 1821, status = succeeded6767worker1 # [ 185.098172] nix-daemon[849]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6768worker1 # [ 185.118725] nix-daemon[849]: reaped child process 1840, status = succeeded6769lb # [ 185.141213] niks3-server[1141]: 2026/09/22 14:54:09 INFO Received uploads request method=POST path=/api/pending_closures6770worker1 # [ 185.142789] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.758Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6771worker1 # [ 185.144661] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.758Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6772lb # [ 185.191548] niks3-server[1141]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6773worker1 # [ 185.192043] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.810Z level=INFO msg="Uploading 1 narinfos"6774lb # [ 185.207815] niks3-server[1141]: 2026/09/22 14:54:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6775lb # [ 185.217548] niks3-server[1141]: 2026/09/22 14:54:09 INFO Signed narinfos id=69 count=16776lb # [ 185.225351] niks3-server[1141]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6777worker1 # [ 185.221085] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.839Z level=INFO msg="Upload complete. (297ms)"6778lb # [ 185.236245] niks3-server[1141]: 2026/09/22 14:54:09 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6779lb # [ 185.244171] niks3-server[1141]: 2026/09/22 14:54:09 INFO Completed upload id=696780worker2 # [ 185.281453] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:09Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16781lb # [ 185.247192] niks3-server[1141]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6782worker1 # [ 185.233220] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=9 outputs=16783lb # [ 185.252202] niks3-server[1141]: 2026/09/22 14:54:09 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6784worker2 # [ 185.296075] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:09Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06785worker1 # [ 185.256499] nix-daemon[849]: reaped child process 1819, status = succeeded6786worker1 # [ 185.284467] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806787worker1 # [ 185.303370] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806788worker1 # [ 185.321123] nix-grpc-daemon[1691]: time=2026-09-22T14:54:09.939Z level=INFO msg="All 1 paths already cached"6789worker1 # [ 185.334349] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806790worker1 # [ 185.347355] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6791worker1 # [ 185.504625] nix-daemon[849]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6792worker1 # [ 185.527182] nix-daemon[849]: reaped child process 1847, status = succeeded6793worker1 # [ 185.636999] nix-daemon[849]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6794worker1 # [ 185.657101] nix-daemon[849]: reaped child process 1852, status = succeeded6795lb # [ 185.673971] niks3-server[1141]: 2026/09/22 14:54:10 INFO Received uploads request method=POST path=/api/pending_closures6796worker1 # [ 185.671567] nix-grpc-daemon[1691]: time=2026-09-22T14:54:10.289Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6797worker1 # [ 185.673243] nix-grpc-daemon[1691]: time=2026-09-22T14:54:10.290Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6798lb # [ 185.704495] niks3-server[1141]: 2026/09/22 14:54:10 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6799worker1 # [ 185.704748] nix-grpc-daemon[1691]: time=2026-09-22T14:54:10.323Z level=INFO msg="Uploading 1 narinfos"6800lb # [ 185.719373] niks3-server[1141]: 2026/09/22 14:54:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6801lb # [ 185.730275] niks3-server[1141]: 2026/09/22 14:54:10 INFO Signed narinfos id=70 count=16802lb # [ 185.736675] niks3-server[1141]: 2026/09/22 14:54:10 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6803worker1 # [ 185.733489] nix-grpc-daemon[1691]: time=2026-09-22T14:54:10.352Z level=INFO msg="Upload complete. (369ms)"6804lb # [ 185.746277] niks3-server[1141]: 2026/09/22 14:54:10 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6805worker1 # [ 185.740289] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=6246806lb # [ 185.755233] niks3-server[1141]: 2026/09/22 14:54:10 INFO Completed upload id=706807lb # [ 185.758836] niks3-server[1141]: 2026/09/22 14:54:10 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6808worker1 # [ 185.755711] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806809worker1 # [ 185.762391] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6810worker1 # [ 185.767322] nix-daemon[849]: reaped child process 1842, status = succeeded6811worker1 # [ 185.779111] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6812worker1 # [ 186.610371] nix-daemon[849]: reaped child process 1856, status = succeeded6813worker1 # [ 186.717937] nix-daemon[849]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6814worker1 # [ 186.740819] nix-daemon[849]: reaped child process 1875, status = succeeded6815lb # [ 186.757676] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received uploads request method=POST path=/api/pending_closures6816worker1 # [ 186.756580] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.372Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6817worker1 # [ 186.760641] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.372Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6818lb # [ 186.796205] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6819lb # [ 186.810924] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6820lb # [ 186.824613] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6821worker1 # [ 186.817957] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.436Z level=INFO msg="Uploading 1 narinfos"6822lb # [ 186.836535] niks3-server[1141]: 2026/09/22 14:54:11 INFO Signed narinfos id=71 count=16823lb # [ 186.839251] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6824lb # [ 186.853501] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6825lb # [ 186.860259] niks3-server[1141]: 2026/09/22 14:54:11 INFO Completed upload id=716826worker1 # [ 186.847204] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.465Z level=INFO msg="Upload complete. (255ms)"6827lb # [ 186.864365] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6828worker2 # [ 186.911488] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:11Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16829worker1 # [ 186.864526] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=10 outputs=16830worker2 # [ 186.919362] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:11Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06831worker1 # [ 186.880546] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806832worker1 # [ 186.888637] nix-daemon[849]: reaped child process 1854, status = succeeded6833worker1 # [ 186.893451] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806834worker1 # [ 186.903297] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.521Z level=INFO msg="All 1 paths already cached"6835worker1 # [ 186.912822] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806836worker1 # [ 186.925108] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6837worker1 # [ 187.070400] nix-daemon[849]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)6838worker1 # [ 187.096717] nix-daemon[849]: reaped child process 1882, status = succeeded6839worker1 # [ 187.198802] nix-daemon[849]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6840worker1 # [ 187.218021] nix-daemon[849]: reaped child process 1887, status = succeeded6841lb # [ 187.238487] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received uploads request method=POST path=/api/pending_closures6842worker1 # [ 187.235230] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.853Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6843worker1 # [ 187.236881] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.853Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6844lb # [ 187.270658] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6845worker1 # [ 187.273508] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.892Z level=INFO msg="Uploading 1 narinfos"6846lb # [ 187.287137] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6847lb # [ 187.298117] niks3-server[1141]: 2026/09/22 14:54:11 INFO Signed narinfos id=72 count=16848lb # [ 187.305074] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6849lb # [ 187.318041] niks3-server[1141]: 2026/09/22 14:54:11 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6850worker1 # [ 187.310618] nix-grpc-daemon[1691]: time=2026-09-22T14:54:11.929Z level=INFO msg="Upload complete. (371ms)"6851lb # [ 187.326629] niks3-server[1141]: 2026/09/22 14:54:11 INFO Completed upload id=726852lb # [ 187.330806] niks3-server[1141]: 2026/09/22 14:54:11 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6853worker1 # [ 187.320332] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=6246854worker1 # [ 187.336517] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806855worker1 # [ 187.343830] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6856worker1 # [ 187.350460] nix-daemon[849]: reaped child process 1877, status = succeeded6857worker1 # [ 187.365678] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6858worker1 # [ 188.456761] nix-daemon[849]: reaped child process 1891, status = succeeded6859worker1 # [ 188.553251] nix-daemon[849]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)6860worker1 # [ 188.575782] nix-daemon[849]: reaped child process 1910, status = succeeded6861lb # [ 188.592812] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received uploads request method=POST path=/api/pending_closures6862worker1 # [ 188.590656] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.208Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6863worker1 # [ 188.592406] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.208Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6864lb # [ 188.631777] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6865lb # [ 188.644147] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6866worker1 # [ 188.642345] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.260Z level=INFO msg="Uploading 1 narinfos"6867lb # [ 188.658322] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6868lb # [ 188.667767] niks3-server[1141]: 2026/09/22 14:54:13 INFO Signed narinfos id=73 count=16869lb # [ 188.675262] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6870worker1 # [ 188.662446] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.281Z level=INFO msg="Upload complete. (285ms)"6871lb # [ 188.681608] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6872lb # [ 188.687491] niks3-server[1141]: 2026/09/22 14:54:13 INFO Completed upload id=736873lb # [ 188.691113] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6874worker1 # [ 188.673426] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=11 outputs=16875worker2 # [ 188.726032] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:13Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16876worker2 # [ 188.739932] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:13Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06877worker1 # [ 188.696518] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806878worker1 # [ 188.703926] nix-daemon[849]: reaped child process 1889, status = succeeded6879worker1 # [ 188.712576] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806880worker1 # [ 188.726308] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.344Z level=INFO msg="All 2 paths already cached"6881worker1 # [ 188.736291] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806882worker1 # [ 188.748772] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6883worker1 # [ 188.893521] nix-daemon[849]: accepted connection from pid 1914, user nix-grpc-daemon (trusted)6884worker1 # [ 188.915114] nix-daemon[849]: reaped child process 1917, status = succeeded6885worker1 # [ 189.018712] nix-daemon[849]: accepted connection from pid 1919, user nix-grpc-daemon (trusted)6886worker1 # [ 189.037641] nix-daemon[849]: reaped child process 1922, status = succeeded6887lb # [ 189.054902] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received uploads request method=POST path=/api/pending_closures6888worker1 # [ 189.052172] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.670Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6889worker1 # [ 189.054149] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.672Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6890lb # [ 189.084558] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6891worker1 # [ 189.088478] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.707Z level=INFO msg="Uploading 1 narinfos"6892lb # [ 189.101117] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6893lb # [ 189.112947] niks3-server[1141]: 2026/09/22 14:54:13 INFO Signed narinfos id=74 count=16894lb # [ 189.121317] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6895worker1 # [ 189.122365] nix-grpc-daemon[1691]: time=2026-09-22T14:54:13.741Z level=INFO msg="Upload complete. (359ms)"6896lb # [ 189.137397] niks3-server[1141]: 2026/09/22 14:54:13 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6897worker1 # [ 189.132254] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=7526898lb # [ 189.148340] niks3-server[1141]: 2026/09/22 14:54:13 INFO Completed upload id=746899lb # [ 189.151647] niks3-server[1141]: 2026/09/22 14:54:13 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6900worker1 # [ 189.147965] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806901worker1 # [ 189.154323] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6902worker1 # [ 189.159316] nix-daemon[849]: reaped child process 1912, status = succeeded6903worker1 # [ 189.170883] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6904worker1 # [ 190.640685] nix-daemon[849]: reaped child process 1926, status = succeeded6905worker1 # [ 190.771894] nix-daemon[849]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6906worker1 # [ 190.791337] nix-daemon[849]: reaped child process 1945, status = succeeded6907lb # [ 190.809407] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received uploads request method=POST path=/api/pending_closures6908worker1 # [ 190.808463] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.424Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6909worker1 # [ 190.810322] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.424Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6910lb # [ 190.853419] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6911lb # [ 190.866166] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6912worker1 # [ 190.865715] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.484Z level=INFO msg="Uploading 1 narinfos"6913lb # [ 190.879032] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6914lb # [ 190.888585] niks3-server[1141]: 2026/09/22 14:54:15 INFO Signed narinfos id=75 count=16915lb # [ 190.894885] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6916lb # [ 190.906517] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6917lb # [ 190.914373] niks3-server[1141]: 2026/09/22 14:54:15 INFO Completed upload id=756918worker1 # [ 190.901254] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.519Z level=INFO msg="Upload complete. (315ms)"6919lb # [ 190.918206] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6920worker2 # [ 190.961809] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:15Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16921worker1 # [ 190.914321] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=12 outputs=16922worker1 # [ 190.936451] nix-daemon[849]: reaped child process 1924, status = succeeded6923worker1 # [ 190.959625] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806924worker1 # [ 190.976668] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806925worker1 # [ 190.991205] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.609Z level=INFO msg="All 2 paths already cached"6926worker1 # [ 191.003006] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806927worker1 # [ 191.015928] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6928worker1 # [ 191.171662] nix-daemon[849]: accepted connection from pid 1949, user nix-grpc-daemon (trusted)6929worker1 # [ 191.200475] nix-daemon[849]: reaped child process 1952, status = succeeded6930worker1 # [ 191.303179] nix-daemon[849]: accepted connection from pid 1954, user nix-grpc-daemon (trusted)6931worker1 # [ 191.320520] nix-daemon[849]: reaped child process 1957, status = succeeded6932lb # [ 191.339725] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received uploads request method=POST path=/api/pending_closures6933worker1 # [ 191.337128] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.955Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6934worker1 # [ 191.338659] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.955Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6935lb # [ 191.370210] niks3-server[1141]: 2026/09/22 14:54:15 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6936worker1 # [ 191.375147] nix-grpc-daemon[1691]: time=2026-09-22T14:54:15.993Z level=INFO msg="Uploading 1 narinfos"6937lb # [ 191.388477] niks3-server[1141]: 2026/09/22 14:54:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6938lb # [ 191.395453] niks3-server[1141]: 2026/09/22 14:54:15 INFO Signed narinfos id=76 count=16939lb # [ 191.403239] niks3-server[1141]: 2026/09/22 14:54:16 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6940lb # [ 191.413035] niks3-server[1141]: 2026/09/22 14:54:16 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6941worker1 # [ 191.406595] nix-grpc-daemon[1691]: time=2026-09-22T14:54:16.025Z level=INFO msg="Upload complete. (375ms)"6942lb # [ 191.423391] niks3-server[1141]: 2026/09/22 14:54:16 INFO Completed upload id=766943lb # [ 191.429866] niks3-server[1141]: 2026/09/22 14:54:16 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6944worker1 # [ 191.416276] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=7526945worker1 # [ 191.430385] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806946worker1 # [ 191.436587] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6947worker1 # [ 191.441796] nix-daemon[849]: reaped child process 1947, status = succeeded6948worker1 # [ 191.455827] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6949worker1 # [ 192.565195] nix-daemon[849]: reaped child process 1961, status = succeeded6950worker1 # [ 192.637111] nix-daemon[849]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)6951worker1 # [ 192.659539] nix-daemon[849]: reaped child process 1980, status = succeeded6952lb # [ 192.671574] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received uploads request method=POST path=/api/pending_closures6953worker1 # [ 192.665560] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.283Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6954worker1 # [ 192.667069] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.283Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6955lb # [ 192.693731] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6956lb # [ 192.700232] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6957worker1 # [ 192.693749] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.312Z level=INFO msg="Uploading 1 narinfos"6958lb # [ 192.706282] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6959lb # [ 192.713551] niks3-server[1141]: 2026/09/22 14:54:17 INFO Signed narinfos id=77 count=16960lb # [ 192.715206] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6961lb # [ 192.723786] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6962lb # [ 192.729408] niks3-server[1141]: 2026/09/22 14:54:17 INFO Completed upload id=776963lb # [ 192.731639] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6964worker1 # [ 192.715677] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.334Z level=INFO msg="Upload complete. (249ms)"6965worker2 # [ 192.773918] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:17Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16966worker1 # [ 192.725871] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=13 outputs=16967worker1 # [ 192.738634] nix-daemon[849]: reaped child process 1959, status = succeeded6968worker1 # [ 192.771265] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806969worker1 # [ 192.786473] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581806970worker1 # [ 192.800585] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.419Z level=INFO msg="All 2 paths already cached"6971worker1 # [ 192.810807] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:581806972worker1 # [ 192.824906] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6973worker1 # [ 192.974370] nix-daemon[849]: accepted connection from pid 1984, user nix-grpc-daemon (trusted)6974worker1 # [ 192.995709] nix-daemon[849]: reaped child process 1987, status = succeeded6975worker1 # [ 193.097293] nix-daemon[849]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)6976worker1 # [ 193.114045] nix-daemon[849]: reaped child process 1992, status = succeeded6977lb # [ 193.133970] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received uploads request method=POST path=/api/pending_closures6978worker1 # [ 193.129895] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.748Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6979worker1 # [ 193.131402] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.748Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6980lb # [ 193.161855] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6981lb # [ 193.178882] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6982worker1 # [ 193.171933] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.790Z level=INFO msg="Uploading 1 narinfos"6983lb # [ 193.191263] niks3-server[1141]: 2026/09/22 14:54:17 INFO Signed narinfos id=78 count=16984lb # [ 193.198038] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6985lb # [ 193.210120] niks3-server[1141]: 2026/09/22 14:54:17 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6986worker1 # [ 193.203861] nix-grpc-daemon[1691]: time=2026-09-22T14:54:17.822Z level=INFO msg="Upload complete. (364ms)"6987lb # [ 193.218795] niks3-server[1141]: 2026/09/22 14:54:17 INFO Completed upload id=786988lb # [ 193.223207] niks3-server[1141]: 2026/09/22 14:54:17 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6989worker1 # [ 193.212290] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=0 paths=1 nar_bytes_in=7526990worker1 # [ 193.228383] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:581806991worker1 # [ 193.234890] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6992worker1 # [ 193.240135] nix-daemon[849]: reaped child process 1982, status = succeeded6993worker1 # [ 193.253869] nix-daemon[849]: accepted connection from pid 1631, user nix-grpc-daemon (trusted)6994worker1 # [ 194.303419] nix-daemon[849]: reaped child process 1996, status = succeeded6995worker1 # [ 194.406639] nix-daemon[849]: accepted connection from pid 2012, user nix-grpc-daemon (trusted)6996worker1 # [ 194.430647] nix-daemon[849]: reaped child process 2015, status = succeeded6997lb # [ 194.443185] niks3-server[1141]: 2026/09/22 14:54:19 INFO Received uploads request method=POST path=/api/pending_closures6998worker1 # [ 194.442603] nix-grpc-daemon[1691]: time=2026-09-22T14:54:19.059Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6999worker1 # [ 194.445438] nix-grpc-daemon[1691]: time=2026-09-22T14:54:19.060Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7000lb # [ 194.485886] niks3-server[1141]: 2026/09/22 14:54:19 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7001lb # [ 194.500185] niks3-server[1141]: 2026/09/22 14:54:19 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7002lb # [ 194.511339] niks3-server[1141]: 2026/09/22 14:54:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7003worker1 # [ 194.504499] nix-grpc-daemon[1691]: time=2026-09-22T14:54:19.122Z level=INFO msg="Uploading 1 narinfos"7004lb # [ 194.521930] niks3-server[1141]: 2026/09/22 14:54:19 INFO Signed narinfos id=79 count=17005lb # [ 194.527075] niks3-server[1141]: 2026/09/22 14:54:19 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7006lb # [ 194.537961] niks3-server[1141]: 2026/09/22 14:54:19 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7007lb # [ 194.544969] niks3-server[1141]: 2026/09/22 14:54:19 INFO Completed upload id=797008worker1 # [ 194.531679] nix-grpc-daemon[1691]: time=2026-09-22T14:54:19.150Z level=INFO msg="Upload complete. (267ms)"7009lb # [ 194.547589] niks3-server[1141]: 2026/09/22 14:54:19 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7010worker1 # [ 194.545272] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58180 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=14 outputs=17011worker1 # [ 194.565209] nix-daemon[849]: reaped child process 1994, status = succeeded7012client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.87 seconds)7013worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7014worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7015client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7016client # [ 194.830520] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7017client # Running as unit: gpu.service; invocation ID: 02e125853a064f0fa72234cfcded51b07018client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)7019lb: must succeed: curl -sf localhost:9901/clusters7020lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7021lb: must succeed: curl -sf localhost:9901/clusters7022lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7023worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7024worker1 # [ 194.948759] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581807025client # [ 195.033188] nix[1590]: don't know how to build these paths:7026client # [ 195.034177] nix[1590]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7027worker1 # [ 194.961578] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581807028worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7029client # [ 195.064894] nix[1590]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7030worker2 # [ 195.038208] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:19Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07031lb: must succeed: curl -sf localhost:9901/clusters7032lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7033lb: must succeed: curl -sf localhost:9901/clusters7034lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7035worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7036worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7037client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7038lb # [ 198.116695] envoy[897]: conn peer=[2001:db8:1::1]:39718 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27101 tx=12377 ms=185797039client # [ 225.071123] nix[1590]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7040client # [ 225.084128] nix[1590]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7041client # [ 225.124848] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7042client # [ 225.131837] systemd[1]: gpu.service: Failed with result 'exit-code'.7043client # [ 225.136780] systemd[1]: gpu.service: Consumed 199ms CPU time over 30.299s wall clock time, 35.4M memory peak, 5.2K incoming IP traffic, 5K outgoing IP traffic.7044client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.09 seconds)7045client: must succeed: journalctl -u gpu -o cat7046client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7047lb: must succeed: curl -sf localhost:9901/clusters7048lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7049lb: must succeed: curl -sf localhost:9901/clusters7050lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7051worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7052worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7053(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.71 seconds)7054subtest: low disk drains a worker and builds go to the other7055worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7056worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.22 seconds)7057worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7058worker1 # [ 227.458502] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:52Z level=info event=unhealthy reason=min_free7059worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds)7060client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27061client # don't know how to build these paths:7062client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7063worker1 # [ 227.994374] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:581807064worker1 # [ 228.003945] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:581807065worker2 # [ 228.076152] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07066worker2 # [ 228.081902] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17067worker2 # [ 228.121066] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482907068worker2 # [ 228.137323] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482907069client # copying 1 paths to 'lb:50051'...7070client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7071lb # [ 228.114720] envoy[897]: conn peer=[2001:db8:1::1]:39130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=579 ms=300777072worker2 # [ 228.159289] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482907073worker2 # [ 228.174281] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7074worker2 # [ 228.339681] nix-daemon[839]: accepted connection from pid 2091, user nix-grpc-daemon (trusted)7075worker2 # [ 228.361339] nix-daemon[839]: reaped child process 2094, status = succeeded7076worker2 # [ 228.459649] nix-daemon[839]: accepted connection from pid 2096, user nix-grpc-daemon (trusted)7077worker2 # [ 228.479231] nix-daemon[839]: reaped child process 2099, status = succeeded7078lb # [ 228.442372] niks3-server[1141]: 2026/09/22 14:54:53 INFO Received uploads request method=POST path=/api/pending_closures7079worker2 # [ 228.491845] nix-grpc-daemon[1877]: time=2026-09-22T14:54:53.091Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7080worker2 # [ 228.493609] nix-grpc-daemon[1877]: time=2026-09-22T14:54:53.091Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7081lb # [ 228.481355] niks3-server[1141]: 2026/09/22 14:54:53 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7082worker2 # [ 228.540685] nix-grpc-daemon[1877]: time=2026-09-22T14:54:53.140Z level=INFO msg="Uploading 1 narinfos"7083lb # [ 228.504484] niks3-server[1141]: 2026/09/22 14:54:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7084lb # [ 228.507808] niks3-server[1141]: 2026/09/22 14:54:53 INFO Signed narinfos id=80 count=17085lb # [ 228.517018] niks3-server[1141]: 2026/09/22 14:54:53 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7086lb # [ 228.525598] niks3-server[1141]: 2026/09/22 14:54:53 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7087lb # [ 228.533127] niks3-server[1141]: 2026/09/22 14:54:53 INFO Completed upload id=807088worker2 # [ 228.569955] nix-grpc-daemon[1877]: time=2026-09-22T14:54:53.170Z level=INFO msg="Upload complete. (379ms)"7089worker2 # [ 228.580305] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=5127090lb # [ 228.547999] niks3-server[1141]: 2026/09/22 14:54:53 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7091worker2 # [ 228.595855] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482907092worker2 # [ 228.602719] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7093worker2 # [ 228.607883] nix-daemon[839]: reaped child process 2089, status = succeeded7094client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7095client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7096worker2 # [ 228.621675] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7097client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7098client # farm-a> LOG-farm-a-drain7099worker2 # [ 229.455741] nix-daemon[839]: reaped child process 2103, status = succeeded7100worker2 # [ 229.562269] nix-daemon[839]: accepted connection from pid 2119, user nix-grpc-daemon (trusted)7101worker2 # [ 229.582682] nix-daemon[839]: reaped child process 2122, status = succeeded7102lb # [ 229.546814] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received uploads request method=POST path=/api/pending_closures7103worker2 # [ 229.596405] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.194Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7104worker2 # [ 229.597885] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.194Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7105lb # [ 229.588342] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7106lb # [ 229.600739] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7107worker2 # [ 229.647046] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.247Z level=INFO msg="Uploading 1 narinfos"7108lb # [ 229.613901] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7109lb # [ 229.620863] niks3-server[1141]: 2026/09/22 14:54:54 INFO Signed narinfos id=81 count=17110lb # [ 229.626579] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7111lb # [ 229.634477] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7112lb # [ 229.641421] niks3-server[1141]: 2026/09/22 14:54:54 INFO Completed upload id=817113lb # [ 229.644833] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7114worker2 # [ 229.677521] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.277Z level=INFO msg="Upload complete. (246ms)"7115worker2 # [ 229.693904] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=15 outputs=17116worker2 # [ 229.709622] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07117worker2 # [ 229.713765] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17118worker2 # [ 229.718152] nix-daemon[839]: reaped child process 2101, status = succeeded7119worker2 # [ 229.737432] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482907120worker2 # [ 229.751655] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482907121worker2 # [ 229.765946] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.366Z level=INFO msg="All 1 paths already cached"7122client # copying 1 paths to 'lb:50051'...7123client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7124worker2 # [ 229.777141] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482907125worker2 # [ 229.793477] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7126worker2 # [ 229.935015] nix-daemon[839]: accepted connection from pid 2128, user nix-grpc-daemon (trusted)7127worker2 # [ 229.954577] nix-daemon[839]: reaped child process 2131, status = succeeded7128worker2 # [ 230.054825] nix-daemon[839]: accepted connection from pid 2133, user nix-grpc-daemon (trusted)7129worker2 # [ 230.070907] nix-daemon[839]: reaped child process 2136, status = succeeded7130lb # [ 230.041529] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received uploads request method=POST path=/api/pending_closures7131worker2 # [ 230.088747] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.688Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7132worker2 # [ 230.090219] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.688Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7133lb # [ 230.071726] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7134worker2 # [ 230.123007] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.723Z level=INFO msg="Uploading 1 narinfos"7135lb # [ 230.086838] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7136lb # [ 230.096589] niks3-server[1141]: 2026/09/22 14:54:54 INFO Signed narinfos id=82 count=17137lb # [ 230.102283] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7138lb # [ 230.112361] niks3-server[1141]: 2026/09/22 14:54:54 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7139lb # [ 230.117745] niks3-server[1141]: 2026/09/22 14:54:54 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7140lb # [ 230.121805] niks3-server[1141]: 2026/09/22 14:54:54 INFO Completed upload id=827141worker2 # [ 230.158310] nix-grpc-daemon[1877]: time=2026-09-22T14:54:54.758Z level=INFO msg="Upload complete. (353ms)"7142worker2 # [ 230.166923] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=6327143worker2 # [ 230.181056] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482907144worker2 # [ 230.187209] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7145worker2 # [ 230.191670] nix-daemon[839]: reaped child process 2126, status = succeeded7146client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7147worker2 # [ 230.204181] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7148client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7149client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7150client # farm-b> LOG-farm-b-drain7151worker2 # [ 231.113830] nix-daemon[839]: reaped child process 2140, status = succeeded7152worker2 # [ 231.185371] nix-daemon[839]: accepted connection from pid 2156, user nix-grpc-daemon (trusted)7153worker2 # [ 231.203915] nix-daemon[839]: reaped child process 2159, status = succeeded7154lb # [ 231.173884] niks3-server[1141]: 2026/09/22 14:54:55 INFO Received uploads request method=POST path=/api/pending_closures7155worker2 # [ 231.224136] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.820Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7156worker2 # [ 231.225772] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.820Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7157lb # [ 231.215339] niks3-server[1141]: 2026/09/22 14:54:55 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7158worker2 # [ 231.265986] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.866Z level=INFO msg="Uploading 1 narinfos"7159lb # [ 231.229703] niks3-server[1141]: 2026/09/22 14:54:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7160lb # [ 231.239518] niks3-server[1141]: 2026/09/22 14:54:55 INFO Signed narinfos id=83 count=17161lb # [ 231.246412] niks3-server[1141]: 2026/09/22 14:54:55 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7162lb # [ 231.256248] niks3-server[1141]: 2026/09/22 14:54:55 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7163worker2 # [ 231.299095] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.899Z level=INFO msg="Upload complete. (283ms)"7164lb # [ 231.264839] niks3-server[1141]: 2026/09/22 14:54:55 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7165lb # [ 231.268705] niks3-server[1141]: 2026/09/22 14:54:55 INFO Completed upload id=837166lb # [ 231.272465] niks3-server[1141]: 2026/09/22 14:54:55 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7167worker2 # [ 231.313122] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=16 outputs=17168worker2 # [ 231.334190] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07169worker2 # [ 231.343996] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17170worker2 # [ 231.352739] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482907171worker2 # [ 231.359148] nix-daemon[839]: reaped child process 2138, status = succeeded7172worker2 # [ 231.363237] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:482907173client # copying 1 paths to 'lb:50051'...7174worker2 # [ 231.369400] nix-grpc-daemon[1877]: time=2026-09-22T14:54:55.968Z level=INFO msg="All 2 paths already cached"7175client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7176worker2 # [ 231.374563] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:482907177worker2 # [ 231.386393] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7178worker2 # [ 231.517758] nix-daemon[839]: accepted connection from pid 2165, user nix-grpc-daemon (trusted)7179worker2 # [ 231.537078] nix-daemon[839]: reaped child process 2168, status = succeeded7180worker2 # [ 231.638210] nix-daemon[839]: accepted connection from pid 2170, user nix-grpc-daemon (trusted)7181worker2 # [ 231.653333] nix-daemon[839]: reaped child process 2173, status = succeeded7182lb # [ 231.623226] niks3-server[1141]: 2026/09/22 14:54:56 INFO Received uploads request method=POST path=/api/pending_closures7183worker2 # [ 231.669478] nix-grpc-daemon[1877]: time=2026-09-22T14:54:56.269Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7184worker2 # [ 231.670977] nix-grpc-daemon[1877]: time=2026-09-22T14:54:56.269Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7185lb # [ 231.652186] niks3-server[1141]: 2026/09/22 14:54:56 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7186worker2 # [ 231.703509] nix-grpc-daemon[1877]: time=2026-09-22T14:54:56.303Z level=INFO msg="Uploading 1 narinfos"7187lb # [ 231.668490] niks3-server[1141]: 2026/09/22 14:54:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7188lb # [ 231.678333] niks3-server[1141]: 2026/09/22 14:54:56 INFO Signed narinfos id=84 count=17189lb # [ 231.688561] niks3-server[1141]: 2026/09/22 14:54:56 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7190worker2 # [ 231.732143] nix-grpc-daemon[1877]: time=2026-09-22T14:54:56.332Z level=INFO msg="Upload complete. (335ms)"7191lb # [ 231.698727] niks3-server[1141]: 2026/09/22 14:54:56 INFO Completed upload id=847192worker2 # [ 231.739404] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=0 paths=1 nar_bytes_in=7607193lb # [ 231.705211] niks3-server[1141]: 2026/09/22 14:54:56 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7194lb # [ 231.709338] niks3-server[1141]: 2026/09/22 14:54:56 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7195worker2 # [ 231.748733] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:482907196worker2 # [ 231.759660] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7197worker2 # [ 231.764739] nix-daemon[839]: reaped child process 2163, status = succeeded7198client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7199worker2 # [ 231.777397] nix-daemon[839]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)7200client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7201client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7202client # farm-top> LOG-farm-top-drain7203worker2 # [ 232.856782] nix-daemon[839]: reaped child process 2177, status = succeeded7204worker2 # [ 232.962744] nix-daemon[839]: accepted connection from pid 2193, user nix-grpc-daemon (trusted)7205worker2 # [ 232.980808] nix-daemon[839]: reaped child process 2196, status = succeeded7206lb # [ 232.950871] niks3-server[1141]: 2026/09/22 14:54:57 INFO Received uploads request method=POST path=/api/pending_closures7207worker2 # [ 233.000552] nix-grpc-daemon[1877]: time=2026-09-22T14:54:57.599Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7208worker2 # [ 233.002882] nix-grpc-daemon[1877]: time=2026-09-22T14:54:57.599Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7209lb # [ 232.982396] niks3-server[1141]: 2026/09/22 14:54:57 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7210lb # [ 232.989865] niks3-server[1141]: 2026/09/22 14:54:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7211worker2 # [ 233.026702] nix-grpc-daemon[1877]: time=2026-09-22T14:54:57.626Z level=INFO msg="Uploading 1 narinfos"7212lb # [ 232.995666] niks3-server[1141]: 2026/09/22 14:54:57 INFO Signed narinfos id=85 count=17213lb # [ 232.999627] niks3-server[1141]: 2026/09/22 14:54:57 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7214lb # [ 233.005495] niks3-server[1141]: 2026/09/22 14:54:57 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7215lb # [ 233.013560] niks3-server[1141]: 2026/09/22 14:54:57 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7216worker2 # [ 233.049949] nix-grpc-daemon[1877]: time=2026-09-22T14:54:57.650Z level=INFO msg="Upload complete. (261ms)"7217lb # [ 233.015901] niks3-server[1141]: 2026/09/22 14:54:57 INFO Completed upload id=857218lb # [ 233.020490] niks3-server[1141]: 2026/09/22 14:54:57 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7219worker2 # [ 233.060947] nix-grpc-daemon[1799]: ts=2026-09-22T14:54:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48290 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=17 outputs=17220worker2 # [ 233.075464] nix-daemon[839]: reaped child process 2175, status = succeeded7221client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.32 seconds)7222client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7223client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7224worker1: must succeed: rm /nix/.rw-store/fill7225worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7226worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7227worker1 # [ 234.461946] nix-grpc-daemon[1631]: ts=2026-09-22T14:54:59Z level=info event=healthy reason=min_free7228worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7229(finished: subtest: low disk drains a worker and builds go to the other, in 9.15 seconds)7230(finished: run the VM test script, in 235.52 seconds)7231test script finished in 235.77s7232cleanup7233kill QemuMachine (pid 49)7234client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7235client # [2026-09-22T14:54:59Z INFO virtiofsd] Client disconnected, shutting down7236client # [2026-09-22T14:54:59Z INFO virtiofsd] Client disconnected, shutting down7237client # [2026-09-22T14:54:59Z INFO virtiofsd] Client disconnected, shutting down7238kill QemuMachine (pid 50)7239lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7240lb # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7241lb # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7242lb # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7243kill QemuMachine (pid 52)7244worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7245worker1 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7246worker1 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7247worker1 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7248kill QemuMachine (pid 51)7249worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7250worker2 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7251worker2 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7252worker2 # [2026-09-22T14:55:00Z INFO virtiofsd] Client disconnected, shutting down7253(finished: cleanup, in 1.53 seconds)7254additionally exposed symbols:7255 client, lb, worker1, worker2,7256 vlan1,7257 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh