vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #549
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12client # Disk image does not exist, creating the virtualisation disk image...13client # Formatting '/build/vm-state-client/tmp.aWOqDa8WgM', fmt=raw size=107374182414client # mke2fs 1.47.4 (6-Mar-2025)15client: QEMU running (pid 49)16lb # Disk image does not exist, creating the virtualisation disk image...17worker2 # Disk image does not exist, creating the virtualisation disk image...18lb: QEMU running (pid 50)19lb # Formatting '/build/vm-state-lb/tmp.yFzYSPruyo', fmt=raw size=107374182420client # Discarding device blocks: 0/262144 done21worker1: QEMU running (pid 51)22worker1 # Disk image does not exist, creating the virtualisation disk image...23worker2 # Formatting '/build/vm-state-worker2/tmp.yCwvtd0Bh9', fmt=raw size=107374182424worker2: QEMU running (pid 52)25worker1 # Formatting '/build/vm-state-worker1/tmp.dbd1zXNUtP', fmt=raw size=107374182426worker2 # mke2fs 1.47.4 (6-Mar-2025)27client # Creating filesystem with 262144 4k blocks and 65536 inodes28lb # mke2fs 1.47.4 (6-Mar-2025)29worker1 # mke2fs 1.47.4 (6-Mar-2025)30client # Filesystem UUID: caf4efb0-0876-4a1a-8394-42a283047fa031lb # Discarding device blocks: 0/262144 done32worker1 # Discarding device blocks: 0/262144 done33worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes34client # Superblock backups stored on blocks:35(finished: start all VMs, in 0.54 seconds)36lb # Creating filesystem with 262144 4k blocks and 65536 inodes37client # 32768, 98304, 163840, 22937638worker2 # Discarding device blocks: 0/262144 done39client: waiting for unit network-addresses-eth1.service40client # 41worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes42client: waiting for the VM to finish booting43client # Allocating group tables: 0/8 done44worker2 # Filesystem UUID: e6321213-4ecd-41c9-950f-931df772d14a45worker1 # Filesystem UUID: e0bccbcd-a3e1-4906-aadb-2712258afb3c46lb # Filesystem UUID: 6999c8d2-567d-437d-9f5f-8d938344736d47client # Writing inode tables: 0/8 done48worker2 # Superblock backups stored on blocks:49worker1 # Superblock backups stored on blocks:50lb # Superblock backups stored on blocks:51client # Creating journal (8192 blocks): done52worker2 # 32768, 98304, 163840, 22937653worker1 # 32768, 98304, 163840, 22937654lb # 32768, 98304, 163840, 22937655client # Writing superblocks and filesystem accounting information: 0/8 done56worker2 # 57worker1 # 58lb # 59client # 60worker2 # Allocating group tables: 0/8 done61worker1 # Allocating group tables: 0/8 done62lb # Allocating group tables: 0/8 done63client # Virtualisation disk image created.64worker2 # Writing inode tables: 0/8 done65worker1 # Writing inode tables: 0/8 done66lb # Writing inode tables: 0/8 done67client # Starting virtiofs daemons...68worker2 # Creating journal (8192 blocks): done69worker1 # Creating journal (8192 blocks): done70lb # Creating journal (8192 blocks): done71client # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72worker2 # Writing superblocks and filesystem accounting information: 0/8 done73worker1 # Writing superblocks and filesystem accounting information: 0/8 done74lb # Writing superblocks and filesystem accounting information: 0/8 done75client # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether76worker2 # 77worker1 # 78lb # 79client # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...80worker2 # Virtualisation disk image created.81worker1 # Virtualisation disk image created.82lb # Virtualisation disk image created.83client # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84worker2 # Starting virtiofs daemons...85worker1 # Starting virtiofs daemons...86lb # Starting virtiofs daemons...87client # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether88worker2 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)89worker1 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90lb # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91client # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...92worker2 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93worker1 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether94lb # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether95client # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)96worker2 # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...97worker1 # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...98lb # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...99client # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether100worker2 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)101worker1 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)102lb # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)103client # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...104worker2 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether105worker1 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether106lb # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether107client # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests108worker2 # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...109worker1 # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...110lb # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...111client # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests112worker2 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)113worker1 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)114lb # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)115client # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests116worker2 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether117worker1 # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether118worker2 # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...119worker1 # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...120worker2 # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests121worker1 # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests122worker2 # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests123worker1 # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests124worker2 # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests125worker1 # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests126lb # [2026-09-24T09:25:33Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-24T09:25:33Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-24T09:25:33Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135client # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140client # [ 0.000000] Zone ranges:141client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] DMA32 empty143client # [ 0.000000] Normal empty144client # [ 0.000000] Device empty145client # [ 0.000000] Movable zone start for each node146client # [ 0.000000] Early memory node ranges147client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150client # [ 0.000000] psci: probing for conduit method from DT.151client # [ 0.000000] psci: PSCIv1.3 detected in firmware.152client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153client # [ 0.000000] psci: Trusted OS migration not required154client # [ 0.000000] psci: SMC Calling Convention v1.1155client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157client # [ 0.000000] Detected PIPT I-cache on CPU0158client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159client # [ 0.000000] CPU features: detected: GICv3 CPU interface160client # [ 0.000000] CPU features: detected: Spectre-v4161client # [ 0.000000] CPU features: detected: Spectre-BHB162client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164client # [ 0.000000] alternatives: applying boot alternatives165client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/2dy98nvkjmgp44l5krcmvd0spg23v77k-nixos-system-client-test/init regInfo=/nix/store/yxx7fyc3pwhn2klcficaa4wfc9kv6xvy-closure-info/registration console=ttyAMA0,115200n8 console=tty0166client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/yxx7fyc3pwhn2klcficaa4wfc9kv6xvy-closure-info/registration", will be passed to user space.167client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)169client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)170client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB171client # [ 0.000000] software IO TLB: area num 1.172client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB173client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)174client # [ 0.000000] Fallback order for Node 0: 0175client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216176client # [ 0.000000] Policy zone: DMA177worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]178client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off179client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1180client # [ 0.000000] allocated 3145728 bytes of page_ext181worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026182worker1 # [ 0.000000] KASLR enabled183client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages184worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]185worker1 # [ 0.000000] random: crng init done186client # [ 0.000000] ftrace: allocated 294 pages with 4 groups187worker1 # [ 0.000000] Machine model: linux,dummy-virt188worker1 # [ 0.000000] efi: UEFI not found.189client # [ 0.000000] rcu: Hierarchical RCU implementation.190worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026191client # [ 0.000000] rcu: RCU event tracing is enabled.192worker2 # [ 0.000000] KASLR enabled193worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT194worker2 # [ 0.000000] random: crng init done195client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.196worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]197worker2 # [ 0.000000] Machine model: linux,dummy-virt198client # [ 0.000000] Trampoline variant of Tasks RCU enabled.199worker2 # [ 0.000000] efi: UEFI not found.200worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]201client # [ 0.000000] Rude variant of Tasks RCU enabled.202worker1 # [ 0.000000] Zone ranges:203worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT204client # [ 0.000000] Tracing variant of Tasks RCU enabled.205worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]206worker1 # [ 0.000000] DMA32 empty207worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]208client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.209worker1 # [ 0.000000] Normal empty210worker1 # [ 0.000000] Device empty211worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]212client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1213worker2 # [ 0.000000] Zone ranges:214worker1 # [ 0.000000] Movable zone start for each node215worker1 # [ 0.000000] Early memory node ranges216worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]217client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.218worker2 # [ 0.000000] DMA32 empty219worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]220worker2 # [ 0.000000] Normal empty221worker2 # [ 0.000000] Device empty222client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.223worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]224worker2 # [ 0.000000] Movable zone start for each node225worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000226worker2 # [ 0.000000] Early memory node ranges227client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.228worker1 # [ 0.000000] psci: probing for conduit method from DT.229worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]230client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0231worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.232client # [ 0.000000] GICv3: 256 SPIs implemented233worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]234worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs235client # [ 0.000000] GICv3: 0 Extended SPIs implemented236worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000237worker1 # [ 0.000000] psci: Trusted OS migration not required238client # [ 0.000000] Root IRQ handler: gic_handle_irq239worker1 # [ 0.000000] psci: SMC Calling Convention v1.1240worker2 # [ 0.000000] psci: probing for conduit method from DT.241client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI242worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.243client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0244worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)245worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs246client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000247worker2 # [ 0.000000] psci: Trusted OS migration not required248worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296249client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]250worker2 # [ 0.000000] psci: SMC Calling Convention v1.1251worker1 # [ 0.000000] Detected PIPT I-cache on CPU0252client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)253worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)254worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)255worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296256client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)257worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface258worker2 # [ 0.000000] Detected PIPT I-cache on CPU0259worker1 # [ 0.000000] CPU features: detected: Spectre-v4260client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000261worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)262worker1 # [ 0.000000] CPU features: detected: Spectre-BHB263client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000264worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface265worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38266worker2 # [ 0.000000] CPU features: detected: Spectre-v4267client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.268worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23269worker2 # [ 0.000000] CPU features: detected: Spectre-BHB270worker1 # [ 0.000000] alternatives: applying boot alternatives271client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns272worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38273client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).274worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23275worker2 # [ 0.000000] alternatives: applying boot alternatives276client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns277client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns278worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nrmg7j0090avzkvvf6wicig0d4chlkw6-nixos-system-worker1-test/init regInfo=/nix/store/scvcdjyc589gchnb6ray803wwi5j2a0i-closure-info/registration console=ttyAMA0,115200n8 console=tty0279client # [ 0.000033] arm-pv: using stolen time PV280client # [ 0.000654] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)281worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/scvcdjyc589gchnb6ray803wwi5j2a0i-closure-info/registration", will be passed to user space.282worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gk9qba5l2dqaxaj04yynm6zlyfi22lz9-nixos-system-worker2-test/init regInfo=/nix/store/3g28d5y9kzm5ayqk1z6a1nn8p9j8a1yx-closure-info/registration console=ttyAMA0,115200n8 console=tty0283client # [ 0.000814] Console: colour dummy device 80x25284client # [ 0.000821] printk: legacy console [tty0] enabled285worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes286lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]287worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)288client # [ 0.001069] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)289worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/3g28d5y9kzm5ayqk1z6a1nn8p9j8a1yx-closure-info/registration", will be passed to user space.290client # [ 0.001076] pid_max: default: 32768 minimum: 301291worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)292lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026293client # [ 0.001155] LSM: initializing lsm=capability,landlock,yama,bpf,ima294worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes295lb # [ 0.000000] KASLR enabled296worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB297lb # [ 0.000000] random: crng init done298client # [ 0.001366] landlock: Up and running.299worker1 # [ 0.000000] software IO TLB: area num 1.300client # [ 0.001368] Yama: becoming mindful.301worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)302lb # [ 0.000000] Machine model: linux,dummy-virt303lb # [ 0.000000] efi: UEFI not found.304client # [ 0.002010] LSM support for eBPF active305worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB306worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)307lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT308client # [ 0.002179] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)309worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)310worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB311worker1 # [ 0.000000] Fallback order for Node 0: 0312lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]313client # [ 0.002217] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)314worker2 # [ 0.000000] software IO TLB: area num 1.315worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216316client # [ 0.004104] rcu: Hierarchical SRCU implementation.317lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]318worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB319worker1 # [ 0.000000] Policy zone: DMA320lb # [ 0.000000] Zone ranges:321client # [ 0.004109] rcu: Max phase no-delay instances is 1000.322lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]323worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)324client # [ 0.005348] fsl-mc MSI: its@8080000 domain created325worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off326lb # [ 0.000000] DMA32 empty327worker2 # [ 0.000000] Fallback order for Node 0: 0328client # [ 0.005437] EFI services will not be available.329lb # [ 0.000000] Normal empty330worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1331lb # [ 0.000000] Device empty332client # [ 0.005502] smp: Bringing up secondary CPUs ...333worker1 # [ 0.000000] allocated 3145728 bytes of page_ext334worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216335lb # [ 0.000000] Movable zone start for each node336client # [ 0.005510] smp: Brought up 1 node, 1 CPU337worker2 # [ 0.000000] Policy zone: DMA338worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages339lb # [ 0.000000] Early memory node ranges340client # [ 0.005514] SMP: Total of 1 processors activated.341worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off342worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups343lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]344client # [ 0.005519] CPU: All CPU(s) started at EL1345worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.346worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1347client # [ 0.005529] CPU features: detected: Branch Target Identification348lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]349worker1 # [ 0.000000] rcu: RCU event tracing is enabled.350worker2 # [ 0.000000] allocated 3145728 bytes of page_ext351client # [ 0.005534] CPU features: detected: ARMv8.4 Translation Table Level352lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000353worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.354worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages355lb # [ 0.000000] psci: probing for conduit method from DT.356worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.357worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups358client # [ 0.005537] CPU features: detected: Instruction cache invalidation not required for I/D coherence359lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.360worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.361worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.362lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs363worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.364worker2 # [ 0.000000] rcu: RCU event tracing is enabled.365client # [ 0.005541] CPU features: detected: Data cache clean to the PoU not required for I/D coherence366lb # [ 0.000000] psci: Trusted OS migration not required367client # [ 0.005545] CPU features: detected: Common not Private translations368worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.369worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.370lb # [ 0.000000] psci: SMC Calling Convention v1.1371worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.372client # [ 0.005548] CPU features: detected: CRC32 instructions373worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1374worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.375lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)376client # [ 0.005551] CPU features: detected: Data cache clean to Point of Deep Persistence377worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.378lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296379worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.380client # [ 0.005554] CPU features: detected: Data cache clean to Point of Persistence381lb # [ 0.000000] Detected PIPT I-cache on CPU0382worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.383worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.384client # [ 0.005557] CPU features: detected: Data independent timing control (DIT)385lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)386worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1387client # [ 0.005560] CPU features: detected: E0PD388worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.389lb # [ 0.000000] CPU features: detected: GICv3 CPU interface390client # [ 0.005563] CPU features: detected: Enhanced Counter Virtualization391worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.392lb # [ 0.000000] CPU features: detected: Spectre-v4393worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0394worker1 # [ 0.000000] GICv3: 256 SPIs implemented395lb # [ 0.000000] CPU features: detected: Spectre-BHB396client # [ 0.005566] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)397worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.398worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented399lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38400client # [ 0.005569] CPU features: detected: Enhanced Virtualization Traps401worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq402client # [ 0.005572] CPU features: detected: Fine Grained Traps403lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23404worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.405worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI406lb # [ 0.000000] alternatives: applying boot alternatives407worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0408worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0409client # [ 0.005576] CPU features: detected: Generic authentication (architected QARMA5 algorithm)410worker2 # [ 0.000000] GICv3: 256 SPIs implemented411worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000412client # [ 0.005581] CPU features: detected: RCpc load-acquire (LDAPR)413worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented414worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]415worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq416client # [ 0.005584] CPU features: detected: LSE atomic instructions417worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI418client # [ 0.005587] CPU features: detected: Privileged Access Never419worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)420worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0421client # [ 0.005589] CPU features: detected: PMUv3422client # [ 0.005592] CPU features: detected: RAS Extension Support423worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000424worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)425lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test/init regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration console=ttyAMA0,115200n8 console=tty0426worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]427client # [ 0.005595] CPU features: detected: RASv1p1 Extension Support428worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a20000429client # [ 0.005598] CPU features: detected: Random Number Generator430worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)431worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a30000432lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration", will be passed to user space.433client # [ 0.005600] CPU features: detected: Speculation barrier (SB)434worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.435client # [ 0.005603] CPU features: detected: Stage-2 Force Write-Back436worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)437lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes438client # [ 0.005606] CPU features: detected: TLB range maintenance instructions439worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000440worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns441lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)442client # [ 0.005611] CPU features: detected: Speculative Store Bypassing Safe (SSBS)443worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000444worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).445lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)446client # [ 0.005652] alternatives: applying system-wide alternatives447worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.448client # [ 0.008597] CPU features: detected: BBM Level 2 without TLB conflict abort449worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns450lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB451worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns452worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns453lb # [ 0.000000] software IO TLB: area num 1.454worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).455worker1 # [ 0.000029] arm-pv: using stolen time PV456client # [ 0.008778] Memory: 1405668K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)457lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB458client # [ 0.009161] devtmpfs: initialized459worker1 # [ 0.000438] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)460lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)461worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns462client # [ 0.011046] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)463worker1 # [ 0.000598] Console: colour dummy device 80x25464lb # [ 0.000000] Fallback order for Node 0: 0465worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns466worker1 # [ 0.000605] printk: legacy console [tty0] enabled467client # [ 0.011068] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).468lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216469worker2 # [ 0.000030] arm-pv: using stolen time PV470lb # [ 0.000000] Policy zone: DMA471client # [ 0.011247] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL472worker1 # [ 0.000791] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)473lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off474client # [ 0.011251] 0 pages in range for non-PLT usage475worker2 # [ 0.000453] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)476worker1 # [ 0.000798] pid_max: default: 32768 minimum: 301477client # [ 0.011252] 508272 pages in range for PLT usage478worker2 # [ 0.000629] Console: colour dummy device 80x25479lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1480worker1 # [ 0.000869] LSM: initializing lsm=capability,landlock,yama,bpf,ima481client # [ 0.011340] pinctrl core: initialized pinctrl subsystem482worker2 # [ 0.000638] printk: legacy console [tty0] enabled483lb # [ 0.000000] allocated 3145728 bytes of page_ext484worker1 # [ 0.001018] landlock: Up and running.485client # [ 0.012114] DMI not present or invalid.486worker1 # [ 0.001020] Yama: becoming mindful.487lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages488client # [ 0.015926] NET: Registered PF_NETLINK/PF_ROUTE protocol family489worker1 # [ 0.001484] LSM support for eBPF active490worker2 # [ 0.000852] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)491lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups492worker2 # [ 0.000858] pid_max: default: 32768 minimum: 301493lb # [ 0.000000] rcu: Hierarchical RCU implementation.494client # [ 0.018218] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations495worker1 # [ 0.001609] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)496lb # [ 0.000000] rcu: RCU event tracing is enabled.497worker2 # [ 0.000934] LSM: initializing lsm=capability,landlock,yama,bpf,ima498client # [ 0.018399] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations499worker1 # [ 0.001642] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)500worker2 # [ 0.001074] landlock: Up and running.501lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.502worker2 # [ 0.001077] Yama: becoming mindful.503worker1 # [ 0.003442] rcu: Hierarchical SRCU implementation.504client # [ 0.018602] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations505worker2 # [ 0.001552] LSM support for eBPF active506lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.507worker1 # [ 0.003447] rcu: Max phase no-delay instances is 1000.508lb # [ 0.000000] Rude variant of Tasks RCU enabled.509client # [ 0.018626] audit: initializing netlink subsys (disabled)510worker2 # [ 0.001702] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)511worker1 # [ 0.004691] fsl-mc MSI: its@8080000 domain created512lb # [ 0.000000] Tracing variant of Tasks RCU enabled.513client # [ 0.019167] thermal_sys: Registered thermal governor 'fair_share'514worker1 # [ 0.004781] EFI services will not be available.515worker2 # [ 0.001735] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)516worker1 # [ 0.004857] smp: Bringing up secondary CPUs ...517lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.518client # [ 0.019169] thermal_sys: Registered thermal governor 'bang_bang'519worker2 # [ 0.003765] rcu: Hierarchical SRCU implementation.520worker1 # [ 0.004865] smp: Brought up 1 node, 1 CPU521client # [ 0.019173] thermal_sys: Registered thermal governor 'step_wise'522lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1523worker1 # [ 0.004868] SMP: Total of 1 processors activated.524worker2 # [ 0.003770] rcu: Max phase no-delay instances is 1000.525worker1 # [ 0.004873] CPU: All CPU(s) started at EL1526client # [ 0.019176] thermal_sys: Registered thermal governor 'user_space'527worker2 # [ 0.004996] fsl-mc MSI: its@8080000 domain created528lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.529worker1 # [ 0.004884] CPU features: detected: Branch Target Identification530worker2 # [ 0.005084] EFI services will not be available.531client # [ 0.019181] thermal_sys: Registered thermal governor 'power_allocator'532worker2 # [ 0.005146] smp: Bringing up secondary CPUs ...533worker1 # [ 0.004889] CPU features: detected: ARMv8.4 Translation Table Level534lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.535client # [ 0.019206] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1536worker2 # [ 0.005154] smp: Brought up 1 node, 1 CPU537client # [ 0.019214] cpuidle: using governor ladder538worker2 # [ 0.005157] SMP: Total of 1 processors activated.539worker1 # [ 0.004892] CPU features: detected: Instruction cache invalidation not required for I/D coherence540lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.541client # [ 0.019219] cpuidle: using governor menu542worker2 # [ 0.005162] CPU: All CPU(s) started at EL1543lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0544worker2 # [ 0.005172] CPU features: detected: Branch Target Identification545client # [ 0.019428] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.546worker1 # [ 0.004895] CPU features: detected: Data cache clean to the PoU not required for I/D coherence547lb # [ 0.000000] GICv3: 256 SPIs implemented548client # [ 0.019444] ASID allocator initialised with 65536 entries549worker2 # [ 0.005177] CPU features: detected: ARMv8.4 Translation Table Level550worker1 # [ 0.004899] CPU features: detected: Common not Private translations551lb # [ 0.000000] GICv3: 0 Extended SPIs implemented552client # [ 0.020684] Serial: AMBA PL011 UART driver553lb # [ 0.000000] Root IRQ handler: gic_handle_irq554worker1 # [ 0.004902] CPU features: detected: CRC32 instructions555worker2 # [ 0.005180] CPU features: detected: Instruction cache invalidation not required for I/D coherence556lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI557client # [ 0.026153] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1558worker1 # [ 0.004905] CPU features: detected: Data cache clean to Point of Deep Persistence559lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0560client # [ 0.026312] printk: console [ttyAMA0] enabled561worker2 # [ 0.005184] CPU features: detected: Data cache clean to the PoU not required for I/D coherence562worker1 # [ 0.004909] CPU features: detected: Data cache clean to Point of Persistence563lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000564worker2 # [ 0.005187] CPU features: detected: Common not Private translations565lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]566worker1 # [ 0.004912] CPU features: detected: Data independent timing control (DIT)567worker2 # [ 0.005190] CPU features: detected: CRC32 instructions568worker1 # [ 0.004915] CPU features: detected: E0PD569lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)570worker2 # [ 0.005193] CPU features: detected: Data cache clean to Point of Deep Persistence571worker1 # [ 0.004917] CPU features: detected: Enhanced Counter Virtualization572worker2 # [ 0.005197] CPU features: detected: Data cache clean to Point of Persistence573lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)574worker1 # [ 0.004940] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)575worker2 # [ 0.005200] CPU features: detected: Data independent timing control (DIT)576lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000577worker1 # [ 0.004944] CPU features: detected: Enhanced Virtualization Traps578worker2 # [ 0.005203] CPU features: detected: E0PD579worker1 # [ 0.004946] CPU features: detected: Fine Grained Traps580lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000581worker2 # [ 0.005205] CPU features: detected: Enhanced Counter Virtualization582worker1 # [ 0.004950] CPU features: detected: Generic authentication (architected QARMA5 algorithm)583lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.584worker2 # [ 0.005208] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)585worker1 # [ 0.004955] CPU features: detected: RCpc load-acquire (LDAPR)586lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns587worker2 # [ 0.005212] CPU features: detected: Enhanced Virtualization Traps588worker1 # [ 0.004958] CPU features: detected: LSE atomic instructions589worker2 # [ 0.005215] CPU features: detected: Fine Grained Traps590worker1 # [ 0.004961] CPU features: detected: Privileged Access Never591lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).592client # [ 0.144182] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages593worker1 # [ 0.004963] CPU features: detected: PMUv3594worker2 # [ 0.005219] CPU features: detected: Generic authentication (architected QARMA5 algorithm)595worker1 # [ 0.004966] CPU features: detected: RAS Extension Support596client # [ 0.144199] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page597lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns598worker2 # [ 0.005224] CPU features: detected: RCpc load-acquire (LDAPR)599worker1 # [ 0.004969] CPU features: detected: RASv1p1 Extension Support600client # [ 0.144205] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages601worker2 # [ 0.005227] CPU features: detected: LSE atomic instructions602lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns603worker1 # [ 0.004972] CPU features: detected: Random Number Generator604client # [ 0.144209] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page605lb # [ 0.000030] arm-pv: using stolen time PV606worker2 # [ 0.005230] CPU features: detected: Privileged Access Never607worker1 # [ 0.004974] CPU features: detected: Speculation barrier (SB)608worker2 # [ 0.005233] CPU features: detected: PMUv3609client # [ 0.144214] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages610lb # [ 0.000405] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)611worker2 # [ 0.005236] CPU features: detected: RAS Extension Support612worker1 # [ 0.004977] CPU features: detected: Stage-2 Force Write-Back613client # [ 0.144218] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page614lb # [ 0.000582] Console: colour dummy device 80x25615worker2 # [ 0.005238] CPU features: detected: RASv1p1 Extension Support616worker1 # [ 0.004979] CPU features: detected: TLB range maintenance instructions617client # [ 0.144222] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages618lb # [ 0.000589] printk: legacy console [tty0] enabled619worker2 # [ 0.005241] CPU features: detected: Random Number Generator620worker1 # [ 0.004984] CPU features: detected: Speculative Store Bypassing Safe (SSBS)621client # [ 0.144226] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page622worker2 # [ 0.005244] CPU features: detected: Speculation barrier (SB)623worker1 # [ 0.005021] alternatives: applying system-wide alternatives624lb # [ 0.000785] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)625lb # [ 0.000792] pid_max: default: 32768 minimum: 301626worker2 # [ 0.005246] CPU features: detected: Stage-2 Force Write-Back627worker1 # [ 0.007890] CPU features: detected: BBM Level 2 without TLB conflict abort628lb # [ 0.000863] LSM: initializing lsm=capability,landlock,yama,bpf,ima629worker2 # [ 0.005249] CPU features: detected: TLB range maintenance instructions630lb # [ 0.001043] landlock: Up and running.631client # [ 0.151392] fbcon: Taking over console632client # [ 0.151408] ACPI: Interpreter disabled.633lb # [ 0.001046] Yama: becoming mindful.634worker2 # [ 0.005254] CPU features: detected: Speculative Store Bypassing Safe (SSBS)635worker1 # [ 0.008128] Memory: 1405608K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)636lb # [ 0.001503] LSM support for eBPF active637worker2 # [ 0.005292] alternatives: applying system-wide alternatives638worker1 # [ 0.008477] devtmpfs: initialized639lb # [ 0.001632] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)640worker1 # [ 0.010347] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)641worker2 # [ 0.008199] CPU features: detected: BBM Level 2 without TLB conflict abort642lb # [ 0.001664] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)643worker1 # [ 0.010368] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).644lb # [ 0.003543] rcu: Hierarchical SRCU implementation.645worker2 # [ 0.008356] Memory: 1405676K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)646worker1 # [ 0.010544] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL647lb # [ 0.003548] rcu: Max phase no-delay instances is 1000.648worker2 # [ 0.008718] devtmpfs: initialized649lb # [ 0.004773] fsl-mc MSI: its@8080000 domain created650worker2 # [ 0.010521] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)651lb # [ 0.004869] EFI services will not be available.652lb # [ 0.004932] smp: Bringing up secondary CPUs ...653worker2 # [ 0.010546] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).654lb # [ 0.004939] smp: Brought up 1 node, 1 CPU655worker2 # [ 0.010736] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL656lb # [ 0.004942] SMP: Total of 1 processors activated.657lb # [ 0.004948] CPU: All CPU(s) started at EL1658lb # [ 0.004958] CPU features: detected: Branch Target Identification659lb # [ 0.004963] CPU features: detected: ARMv8.4 Translation Table Level660client # [ 0.160378] iommu: Default domain type: Translated661client # [ 0.160388] iommu: DMA domain TLB invalidation policy: strict mode662lb # [ 0.004966] CPU features: detected: Instruction cache invalidation not required for I/D coherence663client # [ 0.160727] SCSI subsystem initialized664lb # [ 0.004970] CPU features: detected: Data cache clean to the PoU not required for I/D coherence665client # [ 0.162702] usbcore: registered new interface driver usbfs666lb # [ 0.004973] CPU features: detected: Common not Private translations667client # [ 0.162732] usbcore: registered new interface driver hub668lb # [ 0.004987] CPU features: detected: CRC32 instructions669worker1 # [ 0.010548] 0 pages in range for non-PLT usage670client # [ 0.162747] usbcore: registered new device driver usb671worker1 # [ 0.010549] 508272 pages in range for PLT usage672lb # [ 0.004990] CPU features: detected: Data cache clean to Point of Deep Persistence673client # [ 0.163004] pps_core: LinuxPPS API ver. 1 registered674worker1 # [ 0.010657] pinctrl core: initialized pinctrl subsystem675worker1 # [ 0.011388] DMI not present or invalid.676lb # [ 0.004993] CPU features: detected: Data cache clean to Point of Persistence677client # [ 0.163010] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>678worker1 # [ 0.014678] NET: Registered PF_NETLINK/PF_ROUTE protocol family679client # [ 0.163019] PTP clock support registered680lb # [ 0.004996] CPU features: detected: Data independent timing control (DIT)681client # [ 0.163068] EDAC MC: Ver: 3.0.0682lb # [ 0.005000] CPU features: detected: E0PD683worker2 # [ 0.010740] 0 pages in range for non-PLT usage684worker1 # [ 0.016889] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations685worker2 # [ 0.010742] 508272 pages in range for PLT usage686lb # [ 0.005002] CPU features: detected: Enhanced Counter Virtualization687client # [ 0.167567] scmi_core: SCMI protocol bus registered688worker1 # [ 0.017079] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations689worker2 # [ 0.010896] pinctrl core: initialized pinctrl subsystem690client # [ 0.168489] FPGA manager framework691lb # [ 0.005005] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)692worker2 # [ 0.011678] DMI not present or invalid.693worker1 # [ 0.017272] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations694client # [ 0.169447] vgaarb: loaded695lb # [ 0.005009] CPU features: detected: Enhanced Virtualization Traps696worker2 # [ 0.014775] NET: Registered PF_NETLINK/PF_ROUTE protocol family697worker1 # [ 0.017292] audit: initializing netlink subsys (disabled)698lb # [ 0.005011] CPU features: detected: Fine Grained Traps699client # [ 0.170055] clocksource: Switched to clocksource arch_sys_counter700worker2 # [ 0.016966] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations701worker1 # [ 0.017807] thermal_sys: Registered thermal governor 'fair_share'702lb # [ 0.005015] CPU features: detected: Generic authentication (architected QARMA5 algorithm)703worker1 # [ 0.017809] thermal_sys: Registered thermal governor 'bang_bang'704worker2 # [ 0.017158] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations705lb # [ 0.005020] CPU features: detected: RCpc load-acquire (LDAPR)706worker1 # [ 0.017813] thermal_sys: Registered thermal governor 'step_wise'707worker2 # [ 0.017355] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations708lb # [ 0.005023] CPU features: detected: LSE atomic instructions709worker1 # [ 0.017815] thermal_sys: Registered thermal governor 'user_space'710worker2 # [ 0.017375] audit: initializing netlink subsys (disabled)711lb # [ 0.005026] CPU features: detected: Privileged Access Never712worker1 # [ 0.017820] thermal_sys: Registered thermal governor 'power_allocator'713lb # [ 0.005028] CPU features: detected: PMUv3714worker2 # [ 0.017915] thermal_sys: Registered thermal governor 'fair_share'715client # [ 0.173802] VFS: Disk quotas dquot_6.6.0716lb # [ 0.005031] CPU features: detected: RAS Extension Support717worker1 # [ 0.017855] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1718worker2 # [ 0.017917] thermal_sys: Registered thermal governor 'bang_bang'719worker1 # [ 0.017866] cpuidle: using governor ladder720lb # [ 0.005034] CPU features: detected: RASv1p1 Extension Support721client # [ 0.173837] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)722worker2 # [ 0.017920] thermal_sys: Registered thermal governor 'step_wise'723worker1 # [ 0.017872] cpuidle: using governor menu724lb # [ 0.005037] CPU features: detected: Random Number Generator725worker2 # [ 0.017923] thermal_sys: Registered thermal governor 'user_space'726worker1 # [ 0.018060] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.727lb # [ 0.005039] CPU features: detected: Speculation barrier (SB)728worker2 # [ 0.017928] thermal_sys: Registered thermal governor 'power_allocator'729worker1 # [ 0.018074] ASID allocator initialised with 65536 entries730lb # [ 0.005042] CPU features: detected: Stage-2 Force Write-Back731worker1 # [ 0.019342] Serial: AMBA PL011 UART driver732worker2 # [ 0.017951] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1733lb # [ 0.005045] CPU features: detected: TLB range maintenance instructions734client # [ 0.177420] netfs: FS-Cache loaded735worker2 # [ 0.017959] cpuidle: using governor ladder736client # [ 0.177544] pnp: PnP ACPI: disabled737worker1 # [ 0.024479] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1738worker2 # [ 0.017964] cpuidle: using governor menu739lb # [ 0.005049] CPU features: detected: Speculative Store Bypassing Safe (SSBS)740worker1 # [ 0.024598] printk: console [ttyAMA0] enabled741worker2 # [ 0.018140] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.742lb # [ 0.005085] alternatives: applying system-wide alternatives743worker2 # [ 0.018154] ASID allocator initialised with 65536 entries744worker1 # [ 0.143332] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages745lb # [ 0.007954] CPU features: detected: BBM Level 2 without TLB conflict abort746worker2 # [ 0.019375] Serial: AMBA PL011 UART driver747worker1 # [ 0.143352] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page748worker2 # [ 0.024513] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1749worker1 # [ 0.143357] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages750lb # [ 0.008191] Memory: 1405644K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)751worker2 # [ 0.024658] printk: console [ttyAMA0] enabled752lb # [ 0.008543] devtmpfs: initialized753worker1 # [ 0.143362] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page754client # [ 0.181302] NET: Registered PF_INET protocol family755lb # [ 0.010299] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)756worker1 # [ 0.143366] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages757worker2 # [ 0.144379] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages758client # [ 0.181567] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)759worker1 # [ 0.143371] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page760worker2 # [ 0.144397] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page761lb # [ 0.010321] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).762worker1 # [ 0.143375] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages763lb # [ 0.010497] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL764worker2 # [ 0.144403] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages765lb # [ 0.010501] 0 pages in range for non-PLT usage766worker1 # [ 0.143379] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page767worker2 # [ 0.144407] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page768lb # [ 0.010502] 508272 pages in range for PLT usage769worker2 # [ 0.144411] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages770lb # [ 0.010623] pinctrl core: initialized pinctrl subsystem771lb # [ 0.011341] DMI not present or invalid.772worker2 # [ 0.144416] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page773lb # [ 0.014341] NET: Registered PF_NETLINK/PF_ROUTE protocol family774worker1 # [ 0.150585] fbcon: Taking over console775worker2 # [ 0.144420] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages776worker1 # [ 0.150598] ACPI: Interpreter disabled.777worker2 # [ 0.144424] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page778worker1 # [ 0.152453] iommu: Default domain type: Translated779worker1 # [ 0.152462] iommu: DMA domain TLB invalidation policy: strict mode780worker1 # [ 0.154143] SCSI subsystem initialized781worker2 # [ 0.151744] fbcon: Taking over console782worker2 # [ 0.151756] ACPI: Interpreter disabled.783worker2 # [ 0.153600] iommu: Default domain type: Translated784worker2 # [ 0.153610] iommu: DMA domain TLB invalidation policy: strict mode785worker2 # [ 0.155281] SCSI subsystem initialized786lb # [ 0.016489] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations787lb # [ 0.016673] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations788lb # [ 0.016865] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations789worker1 # [ 0.161721] usbcore: registered new interface driver usbfs790lb # [ 0.016886] audit: initializing netlink subsys (disabled)791worker1 # [ 0.161753] usbcore: registered new interface driver hub792lb # [ 0.017413] thermal_sys: Registered thermal governor 'fair_share'793worker1 # [ 0.161777] usbcore: registered new device driver usb794lb # [ 0.017415] thermal_sys: Registered thermal governor 'bang_bang'795worker1 # [ 0.162065] pps_core: LinuxPPS API ver. 1 registered796lb # [ 0.017418] thermal_sys: Registered thermal governor 'step_wise'797worker1 # [ 0.162071] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>798lb # [ 0.017421] thermal_sys: Registered thermal governor 'user_space'799worker2 # [ 0.162930] usbcore: registered new interface driver usbfs800worker1 # [ 0.162081] PTP clock support registered801worker1 # [ 0.162122] EDAC MC: Ver: 3.0.0802worker2 # [ 0.162968] usbcore: registered new interface driver hub803lb # [ 0.017426] thermal_sys: Registered thermal governor 'power_allocator'804worker2 # [ 0.162984] usbcore: registered new device driver usb805worker1 # [ 0.166726] scmi_core: SCMI protocol bus registered806lb # [ 0.017457] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1807worker2 # [ 0.163283] pps_core: LinuxPPS API ver. 1 registered808lb # [ 0.017465] cpuidle: using governor ladder809worker1 # [ 0.167671] FPGA manager framework810lb # [ 0.017470] cpuidle: using governor menu811worker1 # [ 0.168560] vgaarb: loaded812worker2 # [ 0.163289] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>813worker2 # [ 0.163299] PTP clock support registered814lb # [ 0.017657] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.815worker2 # [ 0.163339] EDAC MC: Ver: 3.0.0816lb # [ 0.017671] ASID allocator initialised with 65536 entries817worker1 # [ 0.169185] clocksource: Switched to clocksource arch_sys_counter818lb # [ 0.018900] Serial: AMBA PL011 UART driver819worker2 # [ 0.167937] scmi_core: SCMI protocol bus registered820worker1 # [ 0.169790] VFS: Disk quotas dquot_6.6.0821lb # [ 0.024062] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1822worker2 # [ 0.168874] FPGA manager framework823worker1 # [ 0.169816] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)824lb # [ 0.024230] printk: console [ttyAMA0] enabled825worker2 # [ 0.169760] vgaarb: loaded826worker1 # [ 0.172106] netfs: FS-Cache loaded827worker1 # [ 0.172220] pnp: PnP ACPI: disabled828lb # [ 0.144653] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages829worker2 # [ 0.170393] clocksource: Switched to clocksource arch_sys_counter830lb # [ 0.144670] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page831worker2 # [ 0.171192] VFS: Disk quotas dquot_6.6.0832lb # [ 0.144675] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages833worker2 # [ 0.171217] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)834lb # [ 0.144679] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page835worker2 # [ 0.173604] netfs: FS-Cache loaded836worker2 # [ 0.173715] pnp: PnP ACPI: disabled837lb # [ 0.144683] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages838lb # [ 0.144688] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page839lb # [ 0.144692] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages840lb # [ 0.144696] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page841lb # [ 0.152135] fbcon: Taking over console842lb # [ 0.152150] ACPI: Interpreter disabled.843worker1 # [ 0.178214] NET: Registered PF_INET protocol family844lb # [ 0.154061] iommu: Default domain type: Translated845worker1 # [ 0.178481] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)846client # [ 0.210592] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)847lb # [ 0.154071] iommu: DMA domain TLB invalidation policy: strict mode848client # [ 0.210638] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)849lb # [ 0.155732] SCSI subsystem initialized850client # [ 0.210669] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)851worker2 # [ 0.179747] NET: Registered PF_INET protocol family852client # [ 0.210743] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)853worker2 # [ 0.180007] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)854client # [ 0.210885] TCP: Hash tables configured (established 16384 bind 16384)855client # [ 0.210967] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)856client # [ 0.211002] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)857client # [ 0.211037] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)858client # [ 0.211128] NET: Registered PF_UNIX/PF_LOCAL protocol family859lb # [ 0.161084] usbcore: registered new interface driver usbfs860client # [ 0.211148] NET: Registered PF_XDP protocol family861lb # [ 0.161117] usbcore: registered new interface driver hub862client # [ 0.211164] PCI: CLS 0 bytes, default 64863lb # [ 0.161141] usbcore: registered new device driver usb864client # [ 0.211401] Trying to unpack rootfs image as initramfs...865lb # [ 0.161404] pps_core: LinuxPPS API ver. 1 registered866lb # [ 0.161410] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>867lb # [ 0.161421] PTP clock support registered868lb # [ 0.161491] EDAC MC: Ver: 3.0.0869client # [ 0.227669] kvm [1]: HYP mode not available870lb # [ 0.166095] scmi_core: SCMI protocol bus registered871lb # [ 0.167070] FPGA manager framework872lb # [ 0.168023] vgaarb: loaded873lb # [ 0.168648] clocksource: Switched to clocksource arch_sys_counter874lb # [ 0.172404] VFS: Disk quotas dquot_6.6.0875lb # [ 0.172432] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)876lb # [ 0.176258] netfs: FS-Cache loaded877lb # [ 0.176386] pnp: PnP ACPI: disabled878lb # [ 0.180228] NET: Registered PF_INET protocol family879lb # [ 0.180477] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)880worker1 # [ 0.207095] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881worker1 # [ 0.207139] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882worker1 # [ 0.207164] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883worker1 # [ 0.207239] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)884worker2 # [ 0.208923] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)885worker1 # [ 0.207380] TCP: Hash tables configured (established 16384 bind 16384)886worker2 # [ 0.208967] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)887worker1 # [ 0.207462] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)888worker2 # [ 0.208992] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)889worker1 # [ 0.207497] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)890worker2 # [ 0.209064] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)891worker1 # [ 0.207532] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)892worker1 # [ 0.207628] NET: Registered PF_UNIX/PF_LOCAL protocol family893worker2 # [ 0.209206] TCP: Hash tables configured (established 16384 bind 16384)894worker1 # [ 0.207682] NET: Registered PF_XDP protocol family895worker2 # [ 0.209285] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)896worker1 # [ 0.207700] PCI: CLS 0 bytes, default 64897worker1 # [ 0.207943] Trying to unpack rootfs image as initramfs...898worker2 # [ 0.209317] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)899worker2 # [ 0.209352] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker2 # [ 0.209453] NET: Registered PF_UNIX/PF_LOCAL protocol family901worker2 # [ 0.209473] NET: Registered PF_XDP protocol family902worker2 # [ 0.209490] PCI: CLS 0 bytes, default 64903worker2 # [ 0.209742] Trying to unpack rootfs image as initramfs...904worker1 # [ 0.223039] kvm [1]: HYP mode not available905worker2 # [ 0.224299] kvm [1]: HYP mode not available906lb # [ 0.210447] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907lb # [ 0.210496] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908lb # [ 0.210525] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909lb # [ 0.210601] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910lb # [ 0.210745] TCP: Hash tables configured (established 16384 bind 16384)911lb # [ 0.210830] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912lb # [ 0.210867] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913lb # [ 0.210904] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914lb # [ 0.211005] NET: Registered PF_UNIX/PF_LOCAL protocol family915lb # [ 0.211063] NET: Registered PF_XDP protocol family916lb # [ 0.211081] PCI: CLS 0 bytes, default 64917lb # [ 0.211333] Trying to unpack rootfs image as initramfs...918lb # [ 0.226418] kvm [1]: HYP mode not available919client # [ 0.318610] Initialise system trusted keyrings920client # [ 0.319362] workingset: timestamp_bits=42 max_order=19 bucket_order=0921client # [ 0.320607] squashfs: version 4.0 (2009/01/31) Phillip Lougher922client # [ 0.321388] 9p: Installing v9fs 9p2000 file system support923worker1 # [ 0.313754] Initialise system trusted keyrings924client # [ 0.350110] Key type asymmetric registered925worker1 # [ 0.314492] workingset: timestamp_bits=42 max_order=19 bucket_order=0926client # [ 0.350143] Asymmetric key parser 'x509' registered927client # [ 0.350238] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)928worker1 # [ 0.315733] squashfs: version 4.0 (2009/01/31) Phillip Lougher929client # [ 0.352358] io scheduler mq-deadline registered930client # [ 0.352369] io scheduler kyber registered931worker2 # [ 0.314983] Initialise system trusted keyrings932worker1 # [ 0.316514] 9p: Installing v9fs 9p2000 file system support933worker2 # [ 0.315731] workingset: timestamp_bits=42 max_order=19 bucket_order=0934worker2 # [ 0.316985] squashfs: version 4.0 (2009/01/31) Phillip Lougher935worker2 # [ 0.317741] 9p: Installing v9fs 9p2000 file system support936client # [ 0.362212] pl061_gpio 9030000.pl061: PL061 GPIO chip registered937client # [ 0.363636] ledtrig-cpu: registered to indicate activity on CPUs938client # [ 0.364042] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:939client # [ 0.364060] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000940client # [ 0.364073] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000941client # [ 0.364081] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000942client # [ 0.364104] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits943client # [ 0.364129] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]944client # [ 0.364201] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00945client # [ 0.364211] pci_bus 0000:00: root bus resource [bus 00-ff]946client # [ 0.364216] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]947client # [ 0.364222] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]948client # [ 0.364227] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]949worker2 # [ 0.338359] Key type asymmetric registered950worker2 # [ 0.338386] Asymmetric key parser 'x509' registered951client # [ 0.364284] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint952client # [ 0.364716] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint953client # [ 0.364922] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]954client # [ 0.364939] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]955client # [ 0.364968] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]956client # [ 0.364984] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]957worker1 # [ 0.345288] Key type asymmetric registered958worker1 # [ 0.345313] Asymmetric key parser 'x509' registered959client # [ 0.365434] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint960client # [ 0.365615] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]961worker1 # [ 0.345381] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)962client # [ 0.365631] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]963worker1 # [ 0.347505] io scheduler mq-deadline registered964worker1 # [ 0.347516] io scheduler kyber registered965client # [ 0.365660] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]966worker2 # [ 0.346491] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)967lb # [ 0.321230] Initialise system trusted keyrings968worker2 # [ 0.347560] io scheduler mq-deadline registered969worker2 # [ 0.347571] io scheduler kyber registered970lb # [ 0.321970] workingset: timestamp_bits=42 max_order=19 bucket_order=0971client # [ 0.384897] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint972client # [ 0.385082] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]973lb # [ 0.323215] squashfs: version 4.0 (2009/01/31) Phillip Lougher974client # [ 0.385098] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]975client # [ 0.385127] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]976lb # [ 0.323992] 9p: Installing v9fs 9p2000 file system support977client # [ 0.385588] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint978client # [ 0.385771] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]979client # [ 0.385786] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]980client # [ 0.385818] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]981client # [ 0.385837] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]982worker1 # [ 0.357316] pl061_gpio 9030000.pl061: PL061 GPIO chip registered983worker1 # [ 0.358721] ledtrig-cpu: registered to indicate activity on CPUs984worker2 # [ 0.358529] pl061_gpio 9030000.pl061: PL061 GPIO chip registered985worker1 # [ 0.359118] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:986worker1 # [ 0.359135] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000987worker1 # [ 0.359147] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000988worker2 # [ 0.359962] ledtrig-cpu: registered to indicate activity on CPUs989worker1 # [ 0.359155] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000990worker2 # [ 0.360346] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:991worker1 # [ 0.359174] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits992worker2 # [ 0.360363] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000993worker1 # [ 0.359198] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]994worker2 # [ 0.360375] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000995worker1 # [ 0.359272] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00996worker2 # [ 0.360384] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000997worker1 # [ 0.359281] pci_bus 0000:00: root bus resource [bus 00-ff]998worker2 # [ 0.360403] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits999worker1 # [ 0.359287] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1000client # [ 0.398391] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1001worker1 # [ 0.359292] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1002worker2 # [ 0.360428] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1003client # [ 0.398600] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1004worker1 # [ 0.359297] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1005worker2 # [ 0.360499] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001006client # [ 0.398631] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1007worker2 # [ 0.360508] pci_bus 0000:00: root bus resource [bus 00-ff]1008lb # [ 0.344619] Key type asymmetric registered1009worker1 # [ 0.359352] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1010client # [ 0.399089] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1011lb # [ 0.344642] Asymmetric key parser 'x509' registered1012worker2 # [ 0.360514] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1013worker1 # [ 0.359794] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1014client # [ 0.399279] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1015worker2 # [ 0.360519] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1016worker1 # [ 0.359978] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1017client # [ 0.399309] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1018worker1 # [ 0.359994] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1019worker2 # [ 0.360524] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1020client # [ 0.399728] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1021worker1 # [ 0.360023] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1022worker2 # [ 0.360576] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1023client # [ 0.399912] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1024worker1 # [ 0.360038] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1025worker2 # [ 0.361012] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1026client # [ 0.400157] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1027worker1 # [ 0.360479] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1028worker2 # [ 0.361199] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1029client # [ 0.400346] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1030worker1 # [ 0.360660] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1031worker2 # [ 0.361215] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1032client # [ 0.400375] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033worker1 # [ 0.360675] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1034worker2 # [ 0.361245] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1035worker1 # [ 0.360704] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1036client # [ 0.400825] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1037worker2 # [ 0.361261] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1038client # [ 0.401025] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1039worker1 # [ 0.361151] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1040worker2 # [ 0.361723] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1041client # [ 0.401055] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042lb # [ 0.352741] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1043worker1 # [ 0.381115] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1044worker2 # [ 0.361908] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1045lb # [ 0.353779] io scheduler mq-deadline registered1046worker1 # [ 0.381134] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1047client # [ 0.401509] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1048worker2 # [ 0.361923] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1049lb # [ 0.353790] io scheduler kyber registered1050client # [ 0.401698] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1051worker1 # [ 0.381163] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052worker2 # [ 0.361953] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1053client # [ 0.401728] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054client # [ 0.402222] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1055worker2 # [ 0.381258] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1056client # [ 0.402514] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1057worker2 # [ 0.381445] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1058client # [ 0.402531] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1059worker2 # [ 0.381461] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1060client # [ 0.402561] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1061worker2 # [ 0.381491] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1062client # [ 0.403029] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1063worker2 # [ 0.381953] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1064client # [ 0.403222] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1065worker2 # [ 0.382136] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1066client # [ 0.403238] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1067worker2 # [ 0.382152] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1068client # [ 0.403267] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069worker2 # [ 0.382181] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070client # [ 0.403834] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1071worker2 # [ 0.382200] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1072client # [ 0.403845] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1073client # [ 0.403850] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1074worker2 # [ 0.389848] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1075client # [ 0.403895] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1076lb # [ 0.364796] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1077worker2 # [ 0.390039] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1078worker1 # [ 0.389721] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1079worker1 # [ 0.389925] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1080worker2 # [ 0.390068] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081client # [ 0.403941] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1082worker1 # [ 0.389942] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1083client # [ 0.403989] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1084worker1 # [ 0.389971] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085client # [ 0.404036] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1086worker1 # [ 0.389990] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1087lb # [ 0.366188] ledtrig-cpu: registered to indicate activity on CPUs1088client # [ 0.404082] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1089worker1 # [ 0.390451] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1090lb # [ 0.366609] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1091worker1 # [ 0.390639] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1092client # [ 0.404128] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1093lb # [ 0.366626] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001094worker1 # [ 0.390668] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1095client # [ 0.404174] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1096lb # [ 0.366639] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001097worker1 # [ 0.391117] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1098client # [ 0.404219] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1099worker1 # [ 0.391305] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1100lb # [ 0.366647] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001101client # [ 0.404265] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1102worker1 # [ 0.391334] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103lb # [ 0.366668] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1104client # [ 0.404337] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1105worker1 # [ 0.391736] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1106lb # [ 0.366694] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1107client # [ 0.404383] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1108worker1 # [ 0.391919] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1109lb # [ 0.366767] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001110client # [ 0.404405] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1111worker1 # [ 0.392162] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1112lb # [ 0.366776] pci_bus 0000:00: root bus resource [bus 00-ff]1113client # [ 0.404426] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1114worker1 # [ 0.392348] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1115lb # [ 0.366782] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1116client # [ 0.404448] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1117worker1 # [ 0.392378] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1118lb # [ 0.366788] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1119client # [ 0.404469] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1120worker1 # [ 0.392822] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1121lb # [ 0.366793] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1122client # [ 0.404490] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1123worker1 # [ 0.393017] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1124lb # [ 0.366848] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1125client # [ 0.404511] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1126worker1 # [ 0.393047] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127client # [ 0.404535] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1128lb # [ 0.367296] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1129worker2 # [ 0.398601] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1130client # [ 0.404557] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1131lb # [ 0.367481] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1132worker1 # [ 0.408878] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1133worker2 # [ 0.398829] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1134client # [ 0.404579] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1135lb # [ 0.367497] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1136worker1 # [ 0.409079] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1137worker2 # [ 0.398860] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1138lb # [ 0.367527] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1139client # [ 0.404601] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1140worker1 # [ 0.409109] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1141worker2 # [ 0.399295] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1142lb # [ 0.367543] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1143client # [ 0.404624] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1144worker2 # [ 0.399480] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1145client # [ 0.404644] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1146lb # [ 0.367997] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1147worker2 # [ 0.399732] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1148client # [ 0.404666] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1149lb # [ 0.368181] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1150worker1 # [ 0.412009] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1151worker2 # [ 0.399920] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1152client # [ 0.404686] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1153lb # [ 0.368196] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1154worker1 # [ 0.412285] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1155client # [ 0.404707] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1156worker2 # [ 0.399950] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1157lb # [ 0.368225] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158worker1 # [ 0.412302] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1159client # [ 0.404728] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1160worker2 # [ 0.400408] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1161worker1 # [ 0.412331] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1162client # [ 0.404749] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1163worker2 # [ 0.400597] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1164client # [ 0.404776] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1165worker1 # [ 0.412789] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1166worker2 # [ 0.400627] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167lb # [ 0.387323] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1168worker1 # [ 0.412982] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1169client # [ 0.404785] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1170lb # [ 0.387509] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1171worker2 # [ 0.401079] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1172client # [ 0.404789] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1173worker1 # [ 0.412997] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1174lb # [ 0.387525] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1175worker2 # [ 0.401268] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1176worker1 # [ 0.413027] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1177client # [ 0.405623] pci 0000:00:07.0: enabling device (0000 -> 0002)1178lb # [ 0.387554] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1179worker2 # [ 0.401298] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180lb # [ 0.388018] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1181worker2 # [ 0.401756] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1182lb # [ 0.388200] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1183worker2 # [ 0.402025] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1184lb # [ 0.388216] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1185worker2 # [ 0.402042] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1186lb # [ 0.388245] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1187worker2 # [ 0.402073] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1188lb # [ 0.388264] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1189worker2 # [ 0.402567] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1190worker2 # [ 0.402756] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1191worker2 # [ 0.402771] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1192worker2 # [ 0.402801] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1193worker2 # [ 0.403376] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1194worker2 # [ 0.403387] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1195client # [ 0.462208] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 55267 usecs1196worker2 # [ 0.403392] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1197worker2 # [ 0.403447] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1198worker1 # [ 0.425699] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1199worker2 # [ 0.403494] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1200worker1 # [ 0.425721] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1201worker2 # [ 0.403542] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1202worker1 # [ 0.425727] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1203worker1 # [ 0.425777] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1204worker2 # [ 0.403590] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1205worker2 # [ 0.403638] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1206worker1 # [ 0.425826] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1207worker2 # [ 0.403685] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1208worker1 # [ 0.425874] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1209worker2 # [ 0.403732] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1210worker1 # [ 0.425921] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1211worker2 # [ 0.403778] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1212worker1 # [ 0.425969] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1213lb # [ 0.400796] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1214worker2 # [ 0.403824] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1215worker1 # [ 0.426016] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1216lb # [ 0.401017] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1217worker1 # [ 0.426064] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1218worker2 # [ 0.403901] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1219lb # [ 0.401048] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1220worker2 # [ 0.403947] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1221worker1 # [ 0.426115] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1222lb # [ 0.401583] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1223worker2 # [ 0.403969] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1224lb # [ 0.401770] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1225worker1 # [ 0.426162] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1226worker2 # [ 0.403991] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1227lb # [ 0.401800] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1228worker1 # [ 0.426272] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1229worker2 # [ 0.404013] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1230lb # [ 0.402196] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1231worker1 # [ 0.426321] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1232worker2 # [ 0.404034] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1233lb # [ 0.402379] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1234worker1 # [ 0.426343] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1235worker2 # [ 0.404056] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1236worker1 # [ 0.426364] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1237lb # [ 0.402625] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1238worker2 # [ 0.404078] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1239worker1 # [ 0.426386] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1240lb # [ 0.402812] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1241worker2 # [ 0.404104] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1242worker1 # [ 0.426407] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1243lb # [ 0.402842] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1244worker2 # [ 0.404126] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1245worker1 # [ 0.426431] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1246lb # [ 0.403289] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1247worker2 # [ 0.404149] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1248worker1 # [ 0.426453] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1249lb # [ 0.403475] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1250worker2 # [ 0.404171] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1251worker1 # [ 0.426479] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1252lb # [ 0.403504] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1253worker2 # [ 0.404194] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1254worker1 # [ 0.426501] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1255lb # [ 0.403963] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1256worker2 # [ 0.404215] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1257worker1 # [ 0.426523] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1258lb # [ 0.404150] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1259worker2 # [ 0.404237] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1260worker1 # [ 0.426545] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1261worker2 # [ 0.404258] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1262lb # [ 0.404180] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1263client # [ 0.482780] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1264worker1 # [ 0.426567] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1265worker2 # [ 0.404280] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1266lb # [ 0.404643] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1267client # [ 0.484867] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1268worker1 # [ 0.426588] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1269worker2 # [ 0.404301] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1270lb # [ 0.404919] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1271worker1 # [ 0.426610] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1272worker2 # [ 0.404323] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1273lb # [ 0.404936] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1274worker1 # [ 0.426630] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1275worker2 # [ 0.404348] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1276lb # [ 0.404965] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1277worker1 # [ 0.426651] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1278worker2 # [ 0.404358] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1279lb # [ 0.405439] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1280worker1 # [ 0.426672] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1281worker2 # [ 0.404362] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1282lb # [ 0.405621] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1283worker1 # [ 0.426693] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1284worker2 # [ 0.405177] pci 0000:00:07.0: enabling device (0000 -> 0002)1285lb # [ 0.405637] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1286worker1 # [ 0.426719] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1287lb # [ 0.405667] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1288worker1 # [ 0.426728] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1289lb # [ 0.406248] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1290worker1 # [ 0.426733] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1291worker1 # [ 0.427540] pci 0000:00:07.0: enabling device (0000 -> 0002)1292lb # [ 0.406259] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1293lb # [ 0.406264] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1294lb # [ 0.406307] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1295lb # [ 0.406353] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1296lb # [ 0.406400] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1297client # [ 0.495992] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1298lb # [ 0.406446] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1299lb # [ 0.406492] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1300lb # [ 0.406538] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1301lb # [ 0.406583] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1302lb # [ 0.406629] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1303lb # [ 0.406675] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1304lb # [ 0.406742] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1305lb # [ 0.406788] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1306lb # [ 0.406809] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1307lb # [ 0.406830] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1308client # [ 0.502167] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1309lb # [ 0.406851] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1310client # [ 0.504186] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1311lb # [ 0.406871] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1312lb # [ 0.406893] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1313lb # [ 0.406914] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1314lb # [ 0.406938] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1315lb # [ 0.406959] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1316lb # [ 0.406981] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1317lb # [ 0.407003] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1318worker1 # [ 0.476955] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1319lb # [ 0.407025] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1320lb # [ 0.407046] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1321lb # [ 0.407066] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1322worker2 # [ 0.477284] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1323lb # [ 0.407087] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1324lb # [ 0.407107] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1325lb # [ 0.407128] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1326client # [ 0.514225] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1327lb # [ 0.407149] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1328client # [ 0.516238] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1329lb # [ 0.407175] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1330client # [ 0.518044] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1331lb # [ 0.407184] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1332lb # [ 0.407189] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1333lb # [ 0.407994] pci 0000:00:07.0: enabling device (0000 -> 0002)1334worker2 # [ 0.483815] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1335worker1 # [ 0.486534] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1336worker2 # [ 0.485774] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1337worker1 # [ 0.490818] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1338client # [ 0.528735] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1339worker1 # [ 0.493041] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1340client # [ 0.531366] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1341client # [ 0.533681] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1342worker2 # [ 0.495834] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1343worker2 # [ 0.497934] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1344worker1 # [ 0.502939] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1345worker2 # [ 0.509384] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1346lb # [ 0.484310] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1347worker1 # [ 0.509359] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1348worker1 # [ 0.511269] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1349client # [ 0.551296] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1350worker2 # [ 0.511996] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1351worker2 # [ 0.513832] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1352client # [ 0.553936] msm_serial: driver initialized1353client # [ 0.554627] SuperH (H)SCI(F) driver initialized1354client # [ 0.554689] STM32 USART driver initialized1355lb # [ 0.494082] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1356worker2 # [ 0.523946] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1357worker1 # [ 0.521264] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1358worker2 # [ 0.525963] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1359worker1 # [ 0.523168] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1360worker1 # [ 0.525097] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1361worker1 # [ 0.527431] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1362lb # [ 0.498289] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1363lb # [ 0.500317] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1364lb # [ 0.502483] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1365lb # [ 0.504569] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1366worker2 # [ 0.537595] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1367worker1 # [ 0.548350] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1368worker2 # [ 0.547627] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1369client # [ 0.587293] loop: module loaded1370lb # [ 0.522087] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1371worker1 # [ 0.550859] msm_serial: driver initialized1372client # [ 0.587464] virtio_blk virtio2: 1/0/0 default/read/poll queues1373lb # [ 0.523991] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1374worker1 # [ 0.551004] SuperH (H)SCI(F) driver initialized1375worker1 # [ 0.551057] STM32 USART driver initialized1376client # [ 0.588237] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1377worker2 # [ 0.550528] msm_serial: driver initialized1378lb # [ 0.527382] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1379worker2 # [ 0.550664] SuperH (H)SCI(F) driver initialized1380worker2 # [ 0.550717] STM32 USART driver initialized1381client # [ 0.594629] megasas: 07.734.00.00-rc11382client # [ 0.595342] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1383client # [ 0.597543] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001384lb # [ 0.534439] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1385client # [ 0.597565] Intel/Sharp Extended Query Table at 0x00311386lb # [ 0.541940] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1387client # [ 0.607070] Using buffer write method1388client # [ 0.607133] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1389client # [ 0.609956] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001390client # [ 0.609978] Intel/Sharp Extended Query Table at 0x00311391worker1 # [ 0.581893] loop: module loaded1392worker1 # [ 0.582055] virtio_blk virtio2: 1/0/0 default/read/poll queues1393worker1 # [ 0.582998] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1394lb # [ 0.557875] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1395client # [ 0.623156] Using buffer write method1396client # [ 0.623186] Concatenating MTD devices:1397client # [ 0.623190] (0): "0.flash"1398client # [ 0.623199] (1): "0.flash"1399lb # [ 0.560389] msm_serial: driver initialized1400worker2 # [ 0.585230] loop: module loaded1401client # [ 0.623203] into device "0.flash"1402lb # [ 0.560551] SuperH (H)SCI(F) driver initialized1403worker2 # [ 0.585402] virtio_blk virtio2: 1/0/0 default/read/poll queues1404lb # [ 0.560605] STM32 USART driver initialized1405worker2 # [ 0.586123] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1406worker2 # [ 0.591002] megasas: 07.734.00.00-rc11407worker2 # [ 0.591739] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1408worker2 # [ 0.593954] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001409worker2 # [ 0.593975] Intel/Sharp Extended Query Table at 0x00311410worker1 # [ 0.593802] megasas: 07.734.00.00-rc11411worker1 # [ 0.594527] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1412worker1 # [ 0.596625] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001413worker1 # [ 0.596652] Intel/Sharp Extended Query Table at 0x00311414worker1 # [ 0.598272] Using buffer write method1415worker1 # [ 0.598332] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1416worker1 # [ 0.599925] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001417worker2 # [ 0.603458] Using buffer write method1418worker1 # [ 0.599946] Intel/Sharp Extended Query Table at 0x00311419worker2 # [ 0.603528] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1420worker2 # [ 0.606382] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001421worker1 # [ 0.617225] Using buffer write method1422worker2 # [ 0.614435] Intel/Sharp Extended Query Table at 0x00311423worker1 # [ 0.617253] Concatenating MTD devices:1424worker2 # [ 0.616100] Using buffer write method1425worker1 # [ 0.617257] (0): "0.flash"1426worker1 # [ 0.617260] (1): "0.flash"1427worker2 # [ 0.616128] Concatenating MTD devices:1428worker1 # [ 0.617267] into device "0.flash"1429worker2 # [ 0.616132] (0): "0.flash"1430worker2 # [ 0.616136] (1): "0.flash"1431lb # [ 0.591125] loop: module loaded1432worker2 # [ 0.616144] into device "0.flash"1433lb # [ 0.591322] virtio_blk virtio2: 1/0/0 default/read/poll queues1434lb # [ 0.592101] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1435lb # [ 0.597274] megasas: 07.734.00.00-rc11436lb # [ 0.597951] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1437lb # [ 0.600248] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001438lb # [ 0.600269] Intel/Sharp Extended Query Table at 0x00311439lb # [ 0.609677] Using buffer write method1440lb # [ 0.609745] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1441lb # [ 0.612582] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001442lb # [ 0.612605] Intel/Sharp Extended Query Table at 0x00311443lb # [ 0.625773] Using buffer write method1444lb # [ 0.625805] Concatenating MTD devices:1445lb # [ 0.625810] (0): "0.flash"1446lb # [ 0.625814] (1): "0.flash"1447lb # [ 0.625822] into device "0.flash"1448client # [ 0.854527] Freeing initrd memory: 26972K1449client # [ 0.860430] tun: Universal TUN/TAP device driver, 1.61450client # [ 0.864108] thunder_xcv, ver 1.01451client # [ 0.864152] thunder_bgx, ver 1.01452client # [ 0.864174] nicpf, ver 1.01453client # [ 0.864732] e1000: Intel(R) PRO/1000 Network Driver1454client # [ 0.864740] e1000: Copyright (c) 1999-2006 Intel Corporation.1455client # [ 0.864764] e1000e: Intel(R) PRO/1000 Network Driver1456client # [ 0.864771] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457client # [ 0.864801] igb: Intel(R) Gigabit Ethernet Network Driver1458client # [ 0.864806] igb: Copyright (c) 2007-2014 Intel Corporation.1459client # [ 0.864828] igbvf: Intel(R) Gigabit Virtual Function Network Driver1460client # [ 0.864834] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1461client # [ 0.864981] sky2: driver version 1.301462client # [ 0.873713] usbcore: registered new interface driver usb-storage1463client # [ 0.873802] usbcore: registered new interface driver usbserial_generic1464client # [ 0.873815] usbserial: USB Serial support registered for generic1465client # [ 0.874759] ehci-pci 0000:00:07.0: EHCI Host Controller1466client # [ 0.874785] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11467client # [ 0.875046] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001468client # [ 0.880073] hv_vmbus: registering driver hyperv_keyboard1469client # [ 0.881574] rtc-pl031 9010000.pl031: registered as rtc01470client # [ 0.881601] rtc-pl031 9010000.pl031: setting system clock to 2026-09-24T09:25:35 UTC (1790241935)1471client # [ 0.881890] i2c_dev: i2c /dev entries driver1472client # [ 0.887092] sdhci: Secure Digital Host Controller Interface driver1473client # [ 0.887102] sdhci: Copyright(c) Pierre Ossman1474worker1 # [ 0.853806] Freeing initrd memory: 26976K1475client # [ 0.887365] Synopsys Designware Multimedia Card Interface Driver1476client # [ 0.887723] sdhci-pltfm: SDHCI platform and OF driver helper1477client # [ 0.889203] hid: raw HID events driver (C) Jiri Kosina1478client # [ 0.889432] usbcore: registered new interface driver usbhid1479client # [ 0.889439] usbhid: USB HID core driver1480client # [ 0.890107] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001481client # [ 0.895522] hub 1-0:1.0: USB hub found1482worker1 # [ 0.859768] tun: Universal TUN/TAP device driver, 1.61483client # [ 0.895542] hub 1-0:1.0: 6 ports detected1484worker2 # [ 0.858097] Freeing initrd memory: 26944K1485client # [ 0.897304] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1486client # [ 0.899879] drop_monitor: Initializing network drop monitor service1487client # [ 0.900020] NET: Registered PF_INET6 protocol family1488worker1 # [ 0.863397] thunder_xcv, ver 1.01489worker1 # [ 0.863435] thunder_bgx, ver 1.01490worker1 # [ 0.863459] nicpf, ver 1.01491client # [ 0.901909] Segment Routing with IPv61492worker1 # [ 0.863998] e1000: Intel(R) PRO/1000 Network Driver1493worker2 # [ 0.864192] tun: Universal TUN/TAP device driver, 1.61494client # [ 0.901941] In-situ OAM (IOAM) with IPv61495worker1 # [ 0.864005] e1000: Copyright (c) 1999-2006 Intel Corporation.1496client # [ 0.901969] NET: Registered PF_PACKET protocol family1497worker1 # [ 0.864027] e1000e: Intel(R) PRO/1000 Network Driver1498worker1 # [ 0.864036] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1499worker1 # [ 0.864067] igb: Intel(R) Gigabit Ethernet Network Driver1500worker1 # [ 0.864072] igb: Copyright (c) 2007-2014 Intel Corporation.1501client # [ 0.903625] 9pnet: Installing 9P2000 support1502worker1 # [ 0.864118] igbvf: Intel(R) Gigabit Virtual Function Network Driver1503client # [ 0.903674] Key type dns_resolver registered1504worker1 # [ 0.864124] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1505worker1 # [ 0.864258] sky2: driver version 1.301506worker1 # [ 0.872905] usbcore: registered new interface driver usb-storage1507worker2 # [ 0.867883] thunder_xcv, ver 1.01508worker2 # [ 0.867928] thunder_bgx, ver 1.01509worker2 # [ 0.867950] nicpf, ver 1.01510worker1 # [ 0.872998] usbcore: registered new interface driver usbserial_generic1511worker2 # [ 0.868513] e1000: Intel(R) PRO/1000 Network Driver1512worker1 # [ 0.873012] usbserial: USB Serial support registered for generic1513worker2 # [ 0.868520] e1000: Copyright (c) 1999-2006 Intel Corporation.1514worker2 # [ 0.868547] e1000e: Intel(R) PRO/1000 Network Driver1515worker1 # [ 0.876108] ehci-pci 0000:00:07.0: EHCI Host Controller1516worker2 # [ 0.868555] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1517worker2 # [ 0.868584] igb: Intel(R) Gigabit Ethernet Network Driver1518worker1 # [ 0.876135] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11519worker2 # [ 0.868589] igb: Copyright (c) 2007-2014 Intel Corporation.1520worker1 # [ 0.876386] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001521client # [ 0.910349] registered taskstats version 11522client # [ 0.910487] Loading compiled-in X.509 certificates1523worker2 # [ 0.868612] igbvf: Intel(R) Gigabit Virtual Function Network Driver1524worker2 # [ 0.868618] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1525worker1 # [ 0.879234] hv_vmbus: registering driver hyperv_keyboard1526worker2 # [ 0.868752] sky2: driver version 1.301527worker2 # [ 0.870353] usbcore: registered new interface driver usb-storage1528worker1 # [ 0.880713] rtc-pl031 9010000.pl031: registered as rtc01529worker2 # [ 0.871082] ehci-pci 0000:00:07.0: EHCI Host Controller1530worker1 # [ 0.880738] rtc-pl031 9010000.pl031: setting system clock to 2026-09-24T09:25:35 UTC (1790241935)1531worker2 # [ 0.871108] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11532worker1 # [ 0.881045] i2c_dev: i2c /dev entries driver1533worker2 # [ 0.871344] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001534client # [ 0.919180] Demotion targets for Node 0: null1535client # [ 0.919277] Key type .fscrypt registered1536client # [ 0.919283] Key type fscrypt-provisioning registered1537client # [ 0.919384] ima: No TPM chip found, activating TPM-bypass!1538worker2 # [ 0.881735] usbcore: registered new interface driver usbserial_generic1539client # [ 0.919404] ima: Allocated hash algorithm: sha11540worker2 # [ 0.881753] usbserial: USB Serial support registered for generic1541client # [ 0.919423] ima: No architecture policies found1542worker2 # [ 0.882379] hv_vmbus: registering driver hyperv_keyboard1543worker2 # [ 0.882432] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001544client # [ 0.923399] input: gpio-keys as /devices/platform/gpio-keys/input/input01545worker2 # [ 0.882719] hub 1-0:1.0: USB hub found1546worker2 # [ 0.882741] hub 1-0:1.0: 6 ports detected1547worker1 # [ 0.886325] sdhci: Secure Digital Host Controller Interface driver1548worker1 # [ 0.886336] sdhci: Copyright(c) Pierre Ossman1549worker1 # [ 0.886603] Synopsys Designware Multimedia Card Interface Driver1550worker2 # [ 0.887802] rtc-pl031 9010000.pl031: registered as rtc01551worker1 # [ 0.886979] sdhci-pltfm: SDHCI platform and OF driver helper1552worker1 # [ 0.888493] hid: raw HID events driver (C) Jiri Kosina1553worker2 # [ 0.887830] rtc-pl031 9010000.pl031: setting system clock to 2026-09-24T09:25:35 UTC (1790241935)1554worker1 # [ 0.888737] usbcore: registered new interface driver usbhid1555lb # [ 0.863475] Freeing initrd memory: 26948K1556worker2 # [ 0.888129] i2c_dev: i2c /dev entries driver1557worker1 # [ 0.888743] usbhid: USB HID core driver1558worker1 # [ 0.893663] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001559worker1 # [ 0.895375] hub 1-0:1.0: USB hub found1560worker1 # [ 0.895859] hub 1-0:1.0: 6 ports detected1561worker2 # [ 0.893484] sdhci: Secure Digital Host Controller Interface driver1562worker2 # [ 0.893495] sdhci: Copyright(c) Pierre Ossman1563worker1 # [ 0.896813] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1564lb # [ 0.869651] tun: Universal TUN/TAP device driver, 1.61565worker2 # [ 0.893766] Synopsys Designware Multimedia Card Interface Driver1566worker2 # [ 0.894142] sdhci-pltfm: SDHCI platform and OF driver helper1567worker1 # [ 0.899476] drop_monitor: Initializing network drop monitor service1568worker1 # [ 0.899647] NET: Registered PF_INET6 protocol family1569worker1 # [ 0.901566] Segment Routing with IPv61570worker2 # [ 0.898464] hid: raw HID events driver (C) Jiri Kosina1571worker1 # [ 0.901582] In-situ OAM (IOAM) with IPv61572worker2 # [ 0.898722] usbcore: registered new interface driver usbhid1573worker1 # [ 0.901614] NET: Registered PF_PACKET protocol family1574worker2 # [ 0.898729] usbhid: USB HID core driver1575worker2 # [ 0.901634] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1576worker1 # [ 0.903237] 9pnet: Installing 9P2000 support1577lb # [ 0.873371] thunder_xcv, ver 1.01578worker1 # [ 0.903286] Key type dns_resolver registered1579lb # [ 0.873410] thunder_bgx, ver 1.01580client # [ 0.941587] clk: Disabling unused clocks1581lb # [ 0.873431] nicpf, ver 1.01582client # [ 0.941613] PM: genpd: Disabling unused power domains1583lb # [ 0.873997] e1000: Intel(R) PRO/1000 Network Driver1584worker2 # [ 0.904336] drop_monitor: Initializing network drop monitor service1585lb # [ 0.874004] e1000: Copyright (c) 1999-2006 Intel Corporation.1586worker2 # [ 0.904501] NET: Registered PF_INET6 protocol family1587lb # [ 0.874028] e1000e: Intel(R) PRO/1000 Network Driver1588worker2 # [ 0.906598] Segment Routing with IPv61589lb # [ 0.874036] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1590worker2 # [ 0.906618] In-situ OAM (IOAM) with IPv61591lb # [ 0.874064] igb: Intel(R) Gigabit Ethernet Network Driver1592worker2 # [ 0.906652] NET: Registered PF_PACKET protocol family1593client # [ 0.945706] Freeing unused kernel memory: 4736K1594lb # [ 0.874070] igb: Copyright (c) 2007-2014 Intel Corporation.1595client # [ 0.945908] Run /init as init process1596lb # [ 0.874092] igbvf: Intel(R) Gigabit Virtual Function Network Driver1597lb # [ 0.874098] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1598lb # [ 0.874230] sky2: driver version 1.301599worker2 # [ 0.908322] 9pnet: Installing 9P2000 support1600lb # [ 0.875767] usbcore: registered new interface driver usb-storage1601worker2 # [ 0.908367] Key type dns_resolver registered1602worker1 # [ 0.909783] registered taskstats version 11603lb # [ 0.875813] usbcore: registered new interface driver usbserial_generic1604worker1 # [ 0.909929] Loading compiled-in X.509 certificates1605lb # [ 0.875827] usbserial: USB Serial support registered for generic1606lb # [ 0.876411] hv_vmbus: registering driver hyperv_keyboard1607lb # [ 0.877348] ehci-pci 0000:00:07.0: EHCI Host Controller1608lb # [ 0.877373] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11609lb # [ 0.877570] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001610lb # [ 0.889302] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001611worker1 # [ 0.918539] Demotion targets for Node 0: null1612lb # [ 0.889594] hub 1-0:1.0: USB hub found1613worker1 # [ 0.918645] Key type .fscrypt registered1614lb # [ 0.889609] hub 1-0:1.0: 6 ports detected1615worker1 # [ 0.918652] Key type fscrypt-provisioning registered1616worker2 # [ 0.915145] registered taskstats version 11617worker1 # [ 0.918747] ima: No TPM chip found, activating TPM-bypass!1618worker2 # [ 0.915312] Loading compiled-in X.509 certificates1619worker1 # [ 0.918766] ima: Allocated hash algorithm: sha11620worker1 # [ 0.918786] ima: No architecture policies found1621lb # [ 0.892919] rtc-pl031 9010000.pl031: registered as rtc01622lb # [ 0.892946] rtc-pl031 9010000.pl031: setting system clock to 2026-09-24T09:25:35 UTC (1790241935)1623worker1 # [ 0.922724] input: gpio-keys as /devices/platform/gpio-keys/input/input01624lb # [ 0.893260] i2c_dev: i2c /dev entries driver1625client # [ 0.960207] systemd[1]: Successfully made /usr/ read-only.1626worker2 # [ 0.923961] Demotion targets for Node 0: null1627worker2 # [ 0.924064] Key type .fscrypt registered1628worker2 # [ 0.924071] Key type fscrypt-provisioning registered1629worker2 # [ 0.924165] ima: No TPM chip found, activating TPM-bypass!1630lb # [ 0.898526] sdhci: Secure Digital Host Controller Interface driver1631worker2 # [ 0.924184] ima: Allocated hash algorithm: sha11632lb # [ 0.898538] sdhci: Copyright(c) Pierre Ossman1633worker2 # [ 0.924205] ima: No architecture policies found1634lb # [ 0.898799] Synopsys Designware Multimedia Card Interface Driver1635worker2 # [ 0.928281] input: gpio-keys as /devices/platform/gpio-keys/input/input01636lb # [ 0.899168] sdhci-pltfm: SDHCI platform and OF driver helper1637lb # [ 0.903403] hid: raw HID events driver (C) Jiri Kosina1638lb # [ 0.903649] usbcore: registered new interface driver usbhid1639lb # [ 0.903655] usbhid: USB HID core driver1640lb # [ 0.906567] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1641lb # [ 0.908159] drop_monitor: Initializing network drop monitor service1642lb # [ 0.908348] NET: Registered PF_INET6 protocol family1643lb # [ 0.911547] Segment Routing with IPv61644lb # [ 0.911563] In-situ OAM (IOAM) with IPv61645lb # [ 0.911590] NET: Registered PF_PACKET protocol family1646worker1 # [ 0.940265] clk: Disabling unused clocks1647worker1 # [ 0.940294] PM: genpd: Disabling unused power domains1648lb # [ 0.913450] 9pnet: Installing 9P2000 support1649worker1 # [ 0.944411] Freeing unused kernel memory: 4736K1650lb # [ 0.913495] Key type dns_resolver registered1651worker1 # [ 0.944725] Run /init as init process1652worker2 # [ 0.945919] clk: Disabling unused clocks1653lb # [ 0.919946] registered taskstats version 11654worker2 # [ 0.945949] PM: genpd: Disabling unused power domains1655lb # [ 0.920101] Loading compiled-in X.509 certificates1656worker2 # [ 0.950120] Freeing unused kernel memory: 4736K1657worker2 # [ 0.950314] Run /init as init process1658lb # [ 0.928785] Demotion targets for Node 0: null1659lb # [ 0.928890] Key type .fscrypt registered1660lb # [ 0.928896] Key type fscrypt-provisioning registered1661lb # [ 0.929006] ima: No TPM chip found, activating TPM-bypass!1662lb # [ 0.929025] ima: Allocated hash algorithm: sha11663worker1 # [ 0.959373] systemd[1]: Successfully made /usr/ read-only.1664lb # [ 0.929053] ima: No architecture policies found1665lb # [ 0.933196] input: gpio-keys as /devices/platform/gpio-keys/input/input01666worker2 # [ 0.964595] systemd[1]: Successfully made /usr/ read-only.1667lb # [ 0.950852] clk: Disabling unused clocks1668lb # [ 0.950879] PM: genpd: Disabling unused power domains1669lb # [ 0.954979] Freeing unused kernel memory: 4736K1670lb # [ 0.955188] Run /init as init process1671lb # [ 0.970792] systemd[1]: Successfully made /usr/ read-only.1672client # [ 1.142108] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker2 # [ 1.130438] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker1 # [ 1.141266] usb 1-1: new high-speed USB device number 2 using ehci-pci1675lb # [ 1.136709] usb 1-1: new high-speed USB device number 2 using ehci-pci1676client # [ 1.294806] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677client # [ 1.300727] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678client # [ 1.313095] systemd[1]: Detected virtualization qemu.1679client # [ 1.315211] systemd[1]: Detected architecture arm64.1680client # [ 1.317110] systemd[1]: Running in initrd.1681client # [ 1.319779] systemd[1]: Initializing machine ID from random generator.1682worker2 # [ 1.280677] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11683client # [ 1.322709] systemd[1]: Hostname set to <client>.1684worker1 # [ 1.293697] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685worker2 # [ 1.299224] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1686worker2 # [ 1.304065] systemd[1]: Detected virtualization qemu.1687worker2 # [ 1.304894] systemd[1]: Detected architecture arm64.1688worker2 # [ 1.305759] systemd[1]: Running in initrd.1689worker1 # [ 1.299661] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1690worker2 # [ 1.308035] systemd[1]: Initializing machine ID from random generator.1691worker2 # [ 1.311195] systemd[1]: Hostname set to <worker2>.1692worker1 # [ 1.312013] systemd[1]: Detected virtualization qemu.1693worker1 # [ 1.312105] systemd[1]: Detected architecture arm64.1694worker1 # [ 1.312129] systemd[1]: Running in initrd.1695worker1 # [ 1.313123] systemd[1]: Initializing machine ID from random generator.1696worker1 # [ 1.321563] systemd[1]: Hostname set to <worker1>.1697lb # [ 1.291240] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11698lb # [ 1.305656] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1699client # [ 1.378332] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01700lb # [ 1.317836] systemd[1]: Detected virtualization qemu.1701lb # [ 1.319926] systemd[1]: Detected architecture arm64.1702lb # [ 1.321987] systemd[1]: Running in initrd.1703lb # [ 1.324578] systemd[1]: Initializing machine ID from random generator.1704lb # [ 1.327651] systemd[1]: Hostname set to <lb>.1705worker2 # [ 1.358572] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker1 # [ 1.377479] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707lb # [ 1.372860] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708client # [ 1.502151] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.478486] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker1 # [ 1.497265] usb 1-2: new high-speed USB device number 3 using ehci-pci1711lb # [ 1.492692] usb 1-2: new high-speed USB device number 3 using ehci-pci1712worker2 # [ 1.567349] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713client # [ 1.632877] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714lb # [ 1.578353] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715worker1 # [ 1.624495] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716client # [ 1.663072] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21717client # [ 1.668251] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01718worker2 # [ 1.633449] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719worker2 # [ 1.636299] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720worker2 # [ 1.654216] systemd[1]: Queued start job for default target Initrd Default Target.1721worker1 # [ 1.654016] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21722worker1 # [ 1.659163] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723lb # [ 1.634772] systemd[1]: Queued start job for default target Initrd Default Target.1724worker2 # [ 1.664981] systemd[1]: Created slice Slice /system/modprobe.1725worker2 # [ 1.666277] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1726lb # [ 1.640494] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21727worker2 # [ 1.667947] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728worker2 # [ 1.669112] systemd[1]: Reached target Path Units.1729worker2 # [ 1.670004] systemd[1]: Reached target Slice Units.1730lb # [ 1.644581] systemd[1]: Created slice Slice /system/modprobe.1731worker2 # [ 1.670982] systemd[1]: Reached target Swaps.1732worker2 # [ 1.671802] systemd[1]: Reached target Timer Units.1733worker2 # [ 1.672895] systemd[1]: Listening on D-Bus System Message Bus Socket.1734lb # [ 1.645705] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1735lb # [ 1.645742] systemd[1]: Expecting device /dev/disk/by-label/nixos...1736worker2 # [ 1.674242] systemd[1]: Listening on Journal Socket (/dev/log).1737lb # [ 1.645768] systemd[1]: Reached target Path Units.1738lb # [ 1.645785] systemd[1]: Reached target Slice Units.1739lb # [ 1.645800] systemd[1]: Reached target Swaps.1740lb # [ 1.645816] systemd[1]: Reached target Timer Units.1741worker2 # [ 1.675594] systemd[1]: Listening on Journal Sockets.1742lb # [ 1.645951] systemd[1]: Listening on D-Bus System Message Bus Socket.1743worker2 # [ 1.675744] systemd[1]: Listening on udev Control Socket.1744lb # [ 1.646097] systemd[1]: Listening on Journal Socket (/dev/log).1745worker2 # [ 1.675866] systemd[1]: Listening on udev Kernel Socket.1746lb # [ 1.646220] systemd[1]: Listening on Journal Sockets.1747lb # [ 1.646325] systemd[1]: Listening on udev Control Socket.1748worker2 # [ 1.675891] systemd[1]: Reached target Socket Units.1749lb # [ 1.646423] systemd[1]: Listening on udev Kernel Socket.1750lb # [ 1.646442] systemd[1]: Reached target Socket Units.1751worker2 # [ 1.681274] systemd[1]: Starting Create List of Static Device Nodes...1752lb # [ 1.656299] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01753worker2 # [ 1.682507] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1754lb # [ 1.658454] systemd[1]: Starting Create List of Static Device Nodes...1755lb # [ 1.658524] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1756lb # [ 1.668029] systemd[1]: Mounting Kernel Configuration File System...1757worker2 # [ 1.693824] systemd[1]: Mounting Kernel Configuration File System...1758lb # [ 1.677059] systemd[1]: Starting Journal Service...1759client # [ 1.742778] systemd[1]: Queued start job for default target Initrd Default Target.1760client # [ 1.748238] systemd[1]: Created slice Slice /system/modprobe.1761client # [ 1.749311] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1762client # [ 1.750500] systemd[1]: Expecting device /dev/disk/by-label/nixos...1763client # [ 1.750535] systemd[1]: Reached target Path Units.1764client # [ 1.750553] systemd[1]: Reached target Slice Units.1765client # [ 1.750574] systemd[1]: Reached target Swaps.1766client # [ 1.750592] systemd[1]: Reached target Timer Units.1767lb # [ 1.692159] systemd[1]: Starting Load Kernel Modules...1768client # [ 1.750782] systemd[1]: Listening on D-Bus System Message Bus Socket.1769worker2 # [ 1.718650] systemd[1]: Starting Journal Service...1770client # [ 1.750923] systemd[1]: Listening on Journal Socket (/dev/log).1771client # [ 1.751045] systemd[1]: Listening on Journal Sockets.1772client # [ 1.751162] systemd[1]: Listening on udev Control Socket.1773client # [ 1.751262] systemd[1]: Listening on udev Kernel Socket.1774client # [ 1.751282] systemd[1]: Reached target Socket Units.1775client # [ 1.760804] systemd[1]: Starting Create List of Static Device Nodes...1776client # [ 1.761832] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1777lb # [ 1.697348] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1778worker2 # [ 1.729208] systemd[1]: Starting Load Kernel Modules...1779worker2 # [ 1.729335] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1780client # [ 1.770264] systemd[1]: Mounting Kernel Configuration File System...1781worker1 # [ 1.737499] systemd[1]: Queued start job for default target Initrd Default Target.1782worker2 # [ 1.738289] systemd[1]: Starting Coldplug All udev Devices...1783client # [ 1.778349] systemd[1]: Starting Journal Service...1784lb # [ 1.713747] systemd[1]: Starting Coldplug All udev Devices...1785worker1 # [ 1.747555] systemd[1]: Created slice Slice /system/modprobe.1786worker1 # [ 1.748726] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1787lb # [ 1.720810] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1788worker1 # [ 1.750137] systemd[1]: Expecting device /dev/disk/by-label/nixos...1789lb # [ 1.722543] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1790worker1 # [ 1.751200] systemd[1]: Reached target Path Units.1791worker1 # [ 1.752014] systemd[1]: Reached target Slice Units.1792worker1 # [ 1.752831] systemd[1]: Reached target Swaps.1793worker1 # [ 1.753627] systemd[1]: Reached target Timer Units.1794worker1 # [ 1.754630] systemd[1]: Listening on D-Bus System Message Bus Socket.1795worker1 # [ 1.755846] systemd[1]: Listening on Journal Socket (/dev/log).1796worker1 # [ 1.756977] systemd[1]: Listening on Journal Sockets.1797worker2 # [ 1.754574] systemd[1]: Finished Create List of Static Device Nodes.1798worker1 # [ 1.758056] systemd[1]: Listening on udev Control Socket.1799worker2 # [ 1.755556] systemd[1]: Mounted Kernel Configuration File System.1800worker1 # [ 1.758180] systemd[1]: Listening on udev Kernel Socket.1801worker1 # [ 1.758202] systemd[1]: Reached target Socket Units.1802worker1 # [ 1.762360] systemd[1]: Starting Create List of Static Device Nodes...1803worker1 # [ 1.763491] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1804client # [ 1.802028] systemd[1]: Starting Load Kernel Modules...1805client # [ 1.803046] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1806lb # [ 1.738753] systemd[1]: Finished Create List of Static Device Nodes.1807lb # [ 1.739423] systemd[1]: Mounted Kernel Configuration File System.1808worker2 # [ 1.767039] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1809worker1 # [ 1.773340] systemd[1]: Mounting Kernel Configuration File System...1810lb # [ 1.745089] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1811worker1 # [ 1.781491] systemd[1]: Starting Journal Service...1812client # [ 1.824269] systemd[1]: Starting Coldplug All udev Devices...1813lb # [ 1.762080] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01814lb # [ 1.762315] [drm] features: -virgl +edid -resource_blob -host_visible1815lb # [ 1.762324] [drm] features: -context_init1816lb # [ 1.763020] [drm] number of scanouts: 11817lb # [ 1.763037] [drm] number of cap sets: 01818client # [ 1.830626] systemd-journald[72]: Collecting audit messages is disabled.1819worker2 # [ 1.797599] systemd-journald[72]: Collecting audit messages is disabled.1820lb # [ 1.769032] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1821lb # [ 1.769047] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01822client # [ 1.838221] systemd[1]: Finished Create List of Static Device Nodes.1823client # [ 1.839718] systemd[1]: Mounted Kernel Configuration File System.1824worker1 # [ 1.805008] systemd[1]: Starting Load Kernel Modules...1825worker1 # [ 1.809415] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1826worker2 # [ 1.808986] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1827lb # [ 1.786076] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1828lb # [ 1.791549] systemd-journald[72]: Collecting audit messages is disabled.1829lb # [ 1.795419] systemd[1]: Starting Create Static Device Nodes in /dev...1830worker1 # [ 1.825350] systemd[1]: Starting Coldplug All udev Devices...1831client # [ 1.858474] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1832worker2 # [ 1.826971] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1833worker2 # [ 1.828495] systemd[1]: Starting Create Static Device Nodes in /dev...1834worker2 # [ 1.835959] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1835worker1 # [ 1.838479] systemd-journald[72]: Collecting audit messages is disabled.1836lb # [ 1.804955] Console: switching to colour frame buffer device 160x501837lb # [ 1.811492] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1838worker2 # [ 1.847706] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01839worker1 # [ 1.849367] systemd[1]: Finished Create List of Static Device Nodes.1840worker2 # [ 1.847960] [drm] features: -virgl +edid -resource_blob -host_visible1841worker1 # [ 1.850047] systemd[1]: Mounted Kernel Configuration File System.1842worker2 # [ 1.847969] [drm] features: -context_init1843worker2 # [ 1.848660] [drm] number of scanouts: 11844client # [ 1.888042] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1845worker2 # [ 1.848678] [drm] number of cap sets: 01846worker1 # [ 1.853640] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1847client # [ 1.898670] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1848client # [ 1.901289] systemd[1]: Starting Create Static Device Nodes in /dev...1849lb # [ 1.837304] systemd[1]: Finished Create Static Device Nodes in /dev.1850lb # [ 1.837535] systemd[1]: Reached target Preparation for Local File Systems.1851lb # [ 1.837570] systemd[1]: Reached target Local File Systems.1852lb # [ 1.845029] systemd[1]: Starting Rule-based Manager for Device Events and Files...1853worker2 # [ 1.874739] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1854worker2 # [ 1.874756] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01855lb # [ 1.852754] systemd[1]: Finished Load Kernel Modules.1856worker2 # [ 1.880832] systemd[1]: Finished Create Static Device Nodes in /dev.1857worker2 # [ 1.881023] systemd[1]: Reached target Preparation for Local File Systems.1858worker2 # [ 1.881047] systemd[1]: Reached target Local File Systems.1859worker2 # [ 1.885011] systemd[1]: Starting Rule-based Manager for Device Events and Files...1860worker1 # [ 1.886718] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1861client # [ 1.918258] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1862client # [ 1.924352] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01863client # [ 1.924609] [drm] features: -virgl +edid -resource_blob -host_visible1864client # [ 1.924619] [drm] features: -context_init1865client # [ 1.925378] [drm] number of scanouts: 11866client # [ 1.925400] [drm] number of cap sets: 01867client # [ 1.942782] systemd[1]: Finished Create Static Device Nodes in /dev.1868client # [ 1.943125] systemd[1]: Reached target Preparation for Local File Systems.1869client # [ 1.943154] systemd[1]: Reached target Local File Systems.1870worker1 # [ 1.909788] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1871worker2 # [ 1.902646] Console: switching to colour frame buffer device 160x501872worker1 # [ 1.912233] systemd[1]: Starting Create Static Device Nodes in /dev...1873lb # [ 1.888828] systemd[1]: Starting Apply Kernel Variables...1874client # [ 1.950464] systemd[1]: Starting Rule-based Manager for Device Events and Files...1875client # [ 1.955485] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1876client # [ 1.955503] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01877worker1 # [ 1.917254] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1878worker1 # [ 1.923342] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01879worker1 # [ 1.923578] [drm] features: -virgl +edid -resource_blob -host_visible1880worker1 # [ 1.923588] [drm] features: -context_init1881worker1 # [ 1.924283] [drm] number of scanouts: 11882worker1 # [ 1.924301] [drm] number of cap sets: 01883lb # [ 1.905293] systemd[1]: Finished Apply Kernel Variables.1884worker2 # [ 1.931025] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1885worker2 # [ 1.943088] systemd[1]: Finished Load Kernel Modules.1886worker1 # [ 1.941516] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1887worker1 # [ 1.941534] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01888worker2 # [ 1.950782] systemd[1]: Starting Apply Kernel Variables...1889client # [ 1.986369] Console: switching to colour frame buffer device 160x501890client # [ 1.993337] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1891worker1 # [ 1.965778] systemd[1]: Finished Create Static Device Nodes in /dev.1892worker1 # [ 1.966121] systemd[1]: Reached target Preparation for Local File Systems.1893worker1 # [ 1.966148] systemd[1]: Reached target Local File Systems.1894worker1 # [ 1.970035] systemd[1]: Starting Rule-based Manager for Device Events and Files...1895lb # [ 1.952835] systemd[1]: Started Journal Service.1896worker1 # [ 1.977593] Console: switching to colour frame buffer device 160x501897client # [ 2.006281] systemd-modules-load[74]: Inserted module 'dm_mod'1898lb # [ 1.940244] systemd-modules-load[73]: Inserted module 'dm_mod'1899lb # [ 1.941350] systemd-modules-load[73]: Module 'virtio_balloon' is built in1900client # [ 2.021627] systemd[1]: Started Journal Service.1901client # [ 2.013470] systemd-modules-load[74]: Module 'virtio_balloon' is built in1902lb # [ 1.948276] systemd-modules-load[73]: Module 'virtio_console' is built in1903lb # [ 1.952436] systemd-modules-load[73]: Inserted module 'virtio_gpu'1904worker2 # [ 1.981544] systemd-modules-load[73]: Inserted module 'dm_mod'1905lb # [ 1.953949] systemd-modules-load[73]: Module 'virtio_rng' is built in1906worker2 # [ 1.997098] systemd[1]: Started Journal Service.1907client # [ 2.024355] systemd-modules-load[74]: Module 'virtio_console' is built in1908worker1 # [ 2.001827] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1909lb # [ 1.960570] systemd-udevd[78]: Using default interface naming scheme 'v261'.1910worker2 # [ 1.989196] systemd-modules-load[73]: Module 'virtio_balloon' is built in1911client # [ 2.028412] systemd-modules-load[74]: Inserted module 'virtio_gpu'1912lb # [ 1.962867] systemd[1]: Starting Create System Files and Directories...1913client # [ 2.029466] systemd-modules-load[74]: Module 'virtio_rng' is built in1914client # [ 2.032266] systemd[1]: Finished Load Kernel Modules.1915worker1 # [ 2.010492] systemd[1]: Finished Load Kernel Modules.1916worker2 # [ 1.996309] systemd-modules-load[73]: Module 'virtio_console' is built in1917worker1 # [ 2.013544] systemd[1]: Starting Apply Kernel Variables...1918worker2 # [ 1.999473] systemd-modules-load[73]: Inserted module 'virtio_gpu'1919client # [ 2.040331] systemd[1]: Starting Apply Kernel Variables...1920worker2 # [ 2.002003] systemd-modules-load[73]: Module 'virtio_rng' is built in1921client # [ 2.044310] systemd[1]: Starting Create System Files and Directories...1922worker2 # [ 2.007209] systemd-udevd[78]: Using default interface naming scheme 'v261'.1923worker2 # [ 2.012546] systemd[1]: Starting Create System Files and Directories...1924client # [ 2.051932] systemd-udevd[78]: Using default interface naming scheme 'v261'.1925lb # [ 1.986074] systemd[1]: Finished Create System Files and Directories.1926worker1 # [ 2.029399] systemd[1]: Started Journal Service.1927worker2 # [ 2.014855] systemd[1]: Finished Apply Kernel Variables.1928worker1 # [ 2.020310] systemd-modules-load[74]: Inserted module 'dm_mod'1929worker1 # [ 2.021548] systemd-modules-load[74]: Module 'virtio_balloon' is built in1930client # [ 2.064102] systemd[1]: Finished Apply Kernel Variables.1931worker1 # [ 2.029568] systemd-modules-load[74]: Module 'virtio_console' is built in1932lb # [ 2.001389] systemd[1]: Started Rule-based Manager for Device Events and Files.1933worker1 # [ 2.036305] systemd-modules-load[74]: Inserted module 'virtio_gpu'1934worker1 # [ 2.037454] systemd-modules-load[74]: Module 'virtio_rng' is built in1935worker2 # [ 2.037057] systemd[1]: Finished Create System Files and Directories.1936worker2 # [ 2.048710] systemd[1]: Started Rule-based Manager for Device Events and Files.1937worker1 # [ 2.052347] systemd[1]: Starting Create System Files and Directories...1938client # [ 2.087517] systemd[1]: Finished Create System Files and Directories.1939client # [ 2.102535] systemd[1]: Started Rule-based Manager for Device Events and Files.1940worker1 # [ 2.072435] systemd-udevd[78]: Using default interface naming scheme 'v261'.1941worker1 # [ 2.074714] systemd[1]: Finished Apply Kernel Variables.1942worker1 # [ 2.082174] systemd[1]: Finished Create System Files and Directories.1943worker1 # [ 2.105760] systemd[1]: Started Rule-based Manager for Device Events and Files.1944lb # [ 2.075974] systemd[1]: Starting Virtual Console Setup...1945worker2 # [ 2.104106] systemd[1]: Starting Virtual Console Setup...1946client # [ 2.156107] systemd[1]: Starting Virtual Console Setup...1947worker2 # [ 2.152504] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1948worker2 # [ 2.155887] systemd[1]: Finished Virtual Console Setup.1949lb # [ 2.128626] systemd-vconsole-setup[103]: Configuration of first virtual console was skipped, ignoring remaining ones.1950worker1 # [ 2.161918] systemd[1]: Starting Virtual Console Setup...1951lb # [ 2.132084] systemd[1]: Finished Virtual Console Setup.1952client # [ 2.208502] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1953client # [ 2.211856] systemd[1]: Finished Virtual Console Setup.1954worker1 # [ 2.209002] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1955worker1 # [ 2.212399] systemd[1]: Finished Virtual Console Setup.1956worker2 # [ 2.749849] systemd[1]: Finished Coldplug All udev Devices.1957worker2 # [ 2.750822] systemd[1]: Reached target System Initialization.1958worker2 # [ 2.751678] systemd[1]: Reached target Basic System.1959lb # [ 2.728725] systemd[1]: Finished Coldplug All udev Devices.1960lb # [ 2.729673] systemd[1]: Reached target System Initialization.1961lb # [ 2.730481] systemd[1]: Reached target Basic System.1962client # [ 2.813390] systemd[1]: Finished Coldplug All udev Devices.1963client # [ 2.814374] systemd[1]: Reached target System Initialization.1964client # [ 2.816131] systemd[1]: Reached target Basic System.1965worker1 # [ 2.821898] systemd[1]: Finished Coldplug All udev Devices.1966worker1 # [ 2.822899] systemd[1]: Reached target System Initialization.1967worker1 # [ 2.823740] systemd[1]: Reached target Basic System.1968lb # [ 2.843862] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1969worker2 # [ 2.891693] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1970worker2 # [ 2.919762] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1971client # [ 2.960226] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 2.893449] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973lb # [ 2.898518] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1974worker2 # [ 2.936121] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1975client # [ 3.002800] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976client # [ 3.007914] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1977worker1 # [ 2.978846] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1978lb # [ 2.961838] systemd[1]: Found device /dev/disk/by-label/nixos.1979lb # [ 2.967283] systemd[1]: Reached target Initrd Root Device.1980lb # [ 2.972093] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1981worker1 # [ 3.004105] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1982worker1 # [ 3.008718] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1983worker2 # [ 3.018015] systemd[1]: Found device /dev/disk/by-label/nixos.1984worker2 # [ 3.021434] systemd[1]: Reached target Initrd Root Device.1985worker2 # [ 3.024359] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986client # [ 3.078948] systemd[1]: Found device /dev/disk/by-label/nixos.1987client # [ 3.087987] systemd[1]: Reached target Initrd Root Device.1988client # [ 3.091548] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1989lb # [ 3.026558] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1990lb # [ 3.033938] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1991lb # [ 3.036918] systemd[1]: Mounting /sysroot...1992worker2 # [ 3.068468] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1993worker2 # [ 3.075115] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1994worker2 # [ 3.080157] systemd[1]: Mounting /sysroot...1995worker1 # [ 3.098466] systemd[1]: Found device /dev/disk/by-label/nixos.1996worker1 # [ 3.100364] systemd[1]: Reached target Initrd Root Device.1997client # [ 3.136453] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1998worker1 # [ 3.103687] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1999client # [ 3.141736] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000client # [ 3.146378] systemd[1]: Mounting /sysroot...
loading 2,111 hidden lines…
4112lb # [ 65.832798] niks3-server[1008]: 2026/09/24 09:26:40 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4113worker1 # [ 65.872482] nix-grpc-daemon[867]: time=2026-09-24T09:26:40.504Z level=INFO msg="Upload complete. (254ms)"4114worker1 # [ 65.882238] nix-grpc-daemon[488]: ts=2026-09-24T09:26:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_in=7524115worker1 # [ 65.896526] nix-grpc-daemon[488]: ts=2026-09-24T09:26:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824116worker1 # [ 65.909536] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4117worker1 # [ 65.914734] nix-daemon[855]: reaped child process 1006, status = succeeded4118client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4119worker1 # [ 65.926868] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4120client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4121client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4122client # farm-top> LOG-farm-top-t1b4123worker1 # [ 67.385818] nix-daemon[855]: reaped child process 1015, status = succeeded4124worker1 # [ 67.516240] nix-daemon[855]: accepted connection from pid 1031, user nix-grpc-daemon (trusted)4125worker1 # [ 67.535420] nix-daemon[855]: reaped child process 1034, status = succeeded4126lb # [ 67.508972] niks3-server[1008]: 2026/09/24 09:26:42 INFO Received push request method=POST path=/api/pushes4127worker1 # [ 67.555579] nix-grpc-daemon[867]: time=2026-09-24T09:26:42.185Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4128worker1 # [ 67.557504] nix-grpc-daemon[867]: time=2026-09-24T09:26:42.185Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4129lb # [ 67.548868] niks3-server[1008]: 2026/09/24 09:26:42 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4130lb # [ 67.565126] niks3-server[1008]: 2026/09/24 09:26:42 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4131worker1 # [ 67.603808] nix-grpc-daemon[867]: time=2026-09-24T09:26:42.235Z level=INFO msg="Uploading 1 narinfos"4132lb # [ 67.576336] niks3-server[1008]: 2026/09/24 09:26:42 INFO Received sign narinfos request method=POST path=/api/pushes/8/sign4133lb # [ 67.582818] niks3-server[1008]: 2026/09/24 09:26:42 INFO Signed narinfos id=8 count=14134lb # [ 67.589372] niks3-server[1008]: 2026/09/24 09:26:42 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4135lb # [ 67.595951] niks3-server[1008]: 2026/09/24 09:26:42 INFO Received complete push request method=POST path=/api/pushes/8/complete4136lb # [ 67.599876] niks3-server[1008]: 2026/09/24 09:26:42 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4137worker1 # [ 67.630727] nix-grpc-daemon[867]: time=2026-09-24T09:26:42.262Z level=INFO msg="Upload complete. (290ms)"4138worker1 # [ 67.645143] nix-grpc-daemon[488]: ts=2026-09-24T09:26:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04139worker1 # [ 67.663269] nix-daemon[855]: reaped child process 1013, status = succeeded4140client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.44 seconds)4141client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4142client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4143lb: must succeed: curl -sf localhost:9901/clusters4144lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4145lb: must succeed: curl -sf localhost:9901/clusters4146lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4147worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4148worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4149(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.97 seconds)4150subtest: outputs read back through any worker carry the cache signature4151client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24152lb # [ 68.044401] envoy[898]: conn peer=[2001:db8:1::1]:36092 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9493 tx=4396 ms=55244153lb # [ 68.049395] envoy[898]: conn peer=[2001:db8:1::1]:38704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=684154lb # [ 68.054101] envoy[898]: conn peer=[2001:db8:1::1]:38710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1918 ms=21994155client # don't know how to build these paths:4156client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4157worker2 # [ 68.249350] nix-grpc-daemon[783]: ts=2026-09-24T09:26:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:359124158worker2 # [ 68.257746] nix-grpc-daemon[783]: ts=2026-09-24T09:26:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124159worker1 # [ 68.282033] nix-grpc-daemon[488]: ts=2026-09-24T09:26:42Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04160worker2 # [ 68.282227] nix-grpc-daemon[783]: ts=2026-09-24T09:26:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124161worker1 # [ 68.288478] nix-grpc-daemon[488]: ts=2026-09-24T09:26:42Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14162worker2 # [ 68.297739] nix-grpc-daemon[783]: ts=2026-09-24T09:26:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124163client # copying 1 paths to 'lb:50051'...4164client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4165worker2 # [ 68.313095] nix-grpc-daemon[783]: ts=2026-09-24T09:26:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359124166worker2 # [ 68.329383] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4167worker2 # [ 68.358985] nix-grpc-daemon[783]: ts=2026-09-24T09:26:42Z level=info event=niks3_push_started pid=8764168worker2 # [ 68.529932] nix-grpc-daemon[876]: time=2026-09-24T09:26:43.155Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4169worker2 # [ 68.639353] nix-daemon[842]: accepted connection from pid 889, user nix-grpc-daemon (trusted)4170worker2 # [ 68.658179] nix-daemon[842]: reaped child process 892, status = succeeded4171lb # [ 68.637491] niks3-server[1008]: 2026/09/24 09:26:43 INFO Received push request method=POST path=/api/pushes4172worker2 # [ 68.674426] nix-grpc-daemon[876]: time=2026-09-24T09:26:43.299Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4173worker2 # [ 68.675918] nix-grpc-daemon[876]: time=2026-09-24T09:26:43.299Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4174lb # [ 68.675803] niks3-server[1008]: 2026/09/24 09:26:43 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4175worker2 # [ 68.715180] nix-grpc-daemon[876]: time=2026-09-24T09:26:43.340Z level=INFO msg="Uploading 1 narinfos"4176lb # [ 68.690363] niks3-server[1008]: 2026/09/24 09:26:43 INFO Signed narinfos id=9 count=14177lb # [ 68.698137] niks3-server[1008]: 2026/09/24 09:26:43 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4178lb # [ 68.711989] niks3-server[1008]: 2026/09/24 09:26:43 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4179worker2 # [ 68.749152] nix-grpc-daemon[876]: time=2026-09-24T09:26:43.375Z level=INFO msg="Upload complete. (215ms)"4180lb # [ 68.720941] niks3-server[1008]: 2026/09/24 09:26:43 INFO Received complete push request method=POST path=/api/pushes/9/complete4181lb # [ 68.724104] niks3-server[1008]: 2026/09/24 09:26:43 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4182worker2 # [ 68.756130] nix-grpc-daemon[783]: ts=2026-09-24T09:26:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_in=5124183worker2 # [ 68.770144] nix-grpc-daemon[783]: ts=2026-09-24T09:26:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124184worker2 # [ 68.776826] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4185worker2 # [ 68.782052] nix-daemon[842]: reaped child process 874, status = succeeded4186client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4187worker2 # [ 68.795364] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4188client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4189client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4190client # farm-a> LOG-farm-a-sig4191worker2 # [ 69.629881] nix-daemon[842]: reaped child process 896, status = succeeded4192worker2 # [ 69.722047] nix-daemon[842]: accepted connection from pid 912, user nix-grpc-daemon (trusted)4193worker2 # [ 69.739784] nix-daemon[842]: reaped child process 915, status = succeeded4194lb # [ 69.718510] niks3-server[1008]: 2026/09/24 09:26:44 INFO Received push request method=POST path=/api/pushes4195worker2 # [ 69.756402] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.382Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4196worker2 # [ 69.757875] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.382Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4197lb # [ 69.764793] niks3-server[1008]: 2026/09/24 09:26:44 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4198worker2 # [ 69.800313] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.426Z level=INFO msg="Uploading 1 narinfos"4199lb # [ 69.779316] niks3-server[1008]: 2026/09/24 09:26:44 INFO Signed narinfos id=10 count=14200lb # [ 69.790306] niks3-server[1008]: 2026/09/24 09:26:44 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4201lb # [ 69.800508] niks3-server[1008]: 2026/09/24 09:26:44 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4202lb # [ 69.812597] niks3-server[1008]: 2026/09/24 09:26:44 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4203lb # [ 69.817100] niks3-server[1008]: 2026/09/24 09:26:44 INFO Received complete push request method=POST path=/api/pushes/10/complete4204worker2 # [ 69.849015] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.475Z level=INFO msg="Upload complete. (244ms)"4205lb # [ 69.822066] niks3-server[1008]: 2026/09/24 09:26:44 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4206worker2 # [ 69.861033] nix-grpc-daemon[783]: ts=2026-09-24T09:26:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04207worker1 # [ 69.869041] nix-grpc-daemon[488]: ts=2026-09-24T09:26:44Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04208worker1 # [ 69.869351] nix-grpc-daemon[488]: ts=2026-09-24T09:26:44Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14209worker2 # [ 69.880394] nix-grpc-daemon[783]: ts=2026-09-24T09:26:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124210worker2 # [ 69.886159] nix-daemon[842]: reaped child process 894, status = succeeded4211worker2 # [ 69.894033] nix-grpc-daemon[783]: ts=2026-09-24T09:26:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124212worker2 # [ 69.906164] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.532Z level=INFO msg="All 1 paths already cached"4213client # copying 1 paths to 'lb:50051'...4214client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4215worker2 # [ 69.916507] nix-grpc-daemon[783]: ts=2026-09-24T09:26:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359124216worker2 # [ 69.929496] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4217worker2 # [ 70.071700] nix-daemon[842]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4218worker2 # [ 70.090393] nix-daemon[842]: reaped child process 922, status = succeeded4219lb # [ 70.069052] niks3-server[1008]: 2026/09/24 09:26:44 INFO Received push request method=POST path=/api/pushes4220worker2 # [ 70.105277] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.731Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4221worker2 # [ 70.107294] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.733Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4222lb # [ 70.102704] niks3-server[1008]: 2026/09/24 09:26:44 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4223worker2 # [ 70.137614] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.763Z level=INFO msg="Uploading 1 narinfos"4224lb # [ 70.115885] niks3-server[1008]: 2026/09/24 09:26:44 INFO Signed narinfos id=11 count=14225lb # [ 70.124248] niks3-server[1008]: 2026/09/24 09:26:44 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4226lb # [ 70.139432] niks3-server[1008]: 2026/09/24 09:26:44 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4227worker2 # [ 70.175579] nix-grpc-daemon[876]: time=2026-09-24T09:26:44.802Z level=INFO msg="Upload complete. (233ms)"4228lb # [ 70.146204] niks3-server[1008]: 2026/09/24 09:26:44 INFO Received complete push request method=POST path=/api/pushes/11/complete4229lb # [ 70.154296] niks3-server[1008]: 2026/09/24 09:26:44 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4230worker2 # [ 70.184239] nix-grpc-daemon[783]: ts=2026-09-24T09:26:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_in=6244231worker2 # [ 70.198414] nix-grpc-daemon[783]: ts=2026-09-24T09:26:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124232worker2 # [ 70.205085] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4233worker2 # [ 70.210628] nix-daemon[842]: reaped child process 917, status = succeeded4234client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4235worker2 # [ 70.224243] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4236client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4237client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4238client # farm-b> LOG-farm-b-sig4239worker2 # [ 71.688278] nix-daemon[842]: reaped child process 926, status = succeeded4240worker2 # [ 71.807017] nix-daemon[842]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4241worker2 # [ 71.825623] nix-daemon[842]: reaped child process 945, status = succeeded4242lb # [ 71.803546] niks3-server[1008]: 2026/09/24 09:26:46 INFO Received push request method=POST path=/api/pushes4243worker2 # [ 71.842620] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.467Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4244worker2 # [ 71.844839] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.467Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4245lb # [ 71.852463] niks3-server[1008]: 2026/09/24 09:26:46 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4246worker2 # [ 71.891569] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.517Z level=INFO msg="Uploading 1 narinfos"4247lb # [ 71.865277] niks3-server[1008]: 2026/09/24 09:26:46 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4248lb # [ 71.873163] niks3-server[1008]: 2026/09/24 09:26:46 INFO Signed narinfos id=12 count=14249lb # [ 71.880055] niks3-server[1008]: 2026/09/24 09:26:46 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4250lb # [ 71.891663] niks3-server[1008]: 2026/09/24 09:26:46 INFO Received complete push request method=POST path=/api/pushes/12/complete4251worker2 # [ 71.924190] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.550Z level=INFO msg="Upload complete. (287ms)"4252lb # [ 71.898743] niks3-server[1008]: 2026/09/24 09:26:46 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4253lb # [ 71.910001] niks3-server[1008]: 2026/09/24 09:26:46 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4254worker2 # [ 71.937682] nix-grpc-daemon[783]: ts=2026-09-24T09:26:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04255worker1 # [ 71.950566] nix-grpc-daemon[488]: ts=2026-09-24T09:26:46Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04256worker1 # [ 71.957070] nix-grpc-daemon[488]: ts=2026-09-24T09:26:46Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14257worker2 # [ 71.960678] nix-grpc-daemon[783]: ts=2026-09-24T09:26:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124258worker2 # [ 71.967177] nix-daemon[842]: reaped child process 924, status = succeeded4259worker2 # [ 71.975006] nix-grpc-daemon[783]: ts=2026-09-24T09:26:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124260worker2 # [ 71.988699] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.615Z level=INFO msg="All 2 paths already cached"4261client # copying 1 paths to 'lb:50051'...4262client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4263worker2 # [ 71.999078] nix-grpc-daemon[783]: ts=2026-09-24T09:26:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359124264worker2 # [ 72.014765] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4265worker2 # [ 72.157368] nix-daemon[842]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4266worker2 # [ 72.177643] nix-daemon[842]: reaped child process 952, status = succeeded4267lb # [ 72.155168] niks3-server[1008]: 2026/09/24 09:26:46 INFO Received push request method=POST path=/api/pushes4268worker2 # [ 72.194927] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.820Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4269worker2 # [ 72.196577] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.821Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4270lb # [ 72.189561] niks3-server[1008]: 2026/09/24 09:26:46 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4271lb # [ 72.201790] niks3-server[1008]: 2026/09/24 09:26:46 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4272worker2 # [ 72.238715] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.864Z level=INFO msg="Uploading 1 narinfos"4273lb # [ 72.213554] niks3-server[1008]: 2026/09/24 09:26:46 INFO Signed narinfos id=13 count=14274lb # [ 72.219007] niks3-server[1008]: 2026/09/24 09:26:46 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4275lb # [ 72.235766] niks3-server[1008]: 2026/09/24 09:26:46 INFO Received complete push request method=POST path=/api/pushes/13/complete4276worker2 # [ 72.267690] nix-grpc-daemon[876]: time=2026-09-24T09:26:46.894Z level=INFO msg="Upload complete. (241ms)"4277lb # [ 72.250907] niks3-server[1008]: 2026/09/24 09:26:46 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4278worker2 # [ 72.276584] nix-grpc-daemon[783]: ts=2026-09-24T09:26:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_in=7524279worker2 # [ 72.295613] nix-grpc-daemon[783]: ts=2026-09-24T09:26:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124280worker2 # [ 72.304341] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4281worker2 # [ 72.309085] nix-daemon[842]: reaped child process 947, status = succeeded4282client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4283client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4284worker2 # [ 72.322344] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4285client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4286client # farm-top> LOG-farm-top-sig4287worker2 # [ 73.994090] nix-daemon[842]: reaped child process 956, status = succeeded4288worker2 # [ 74.111137] nix-daemon[842]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4289worker2 # [ 74.129723] nix-daemon[842]: reaped child process 975, status = succeeded4290lb # [ 74.107841] niks3-server[1008]: 2026/09/24 09:26:48 INFO Received push request method=POST path=/api/pushes4291worker2 # [ 74.147216] nix-grpc-daemon[876]: time=2026-09-24T09:26:48.771Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4292worker2 # [ 74.149038] nix-grpc-daemon[876]: time=2026-09-24T09:26:48.771Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4293lb # [ 74.147713] niks3-server[1008]: 2026/09/24 09:26:48 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4294worker2 # [ 74.183642] nix-grpc-daemon[876]: time=2026-09-24T09:26:48.809Z level=INFO msg="Uploading 1 narinfos"4295lb # [ 74.162664] niks3-server[1008]: 2026/09/24 09:26:48 INFO Signed narinfos id=14 count=14296lb # [ 74.170964] niks3-server[1008]: 2026/09/24 09:26:48 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4297lb # [ 74.179747] niks3-server[1008]: 2026/09/24 09:26:48 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4298lb # [ 74.194911] niks3-server[1008]: 2026/09/24 09:26:48 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4299lb # [ 74.200378] niks3-server[1008]: 2026/09/24 09:26:48 INFO Received complete push request method=POST path=/api/pushes/14/complete4300lb # [ 74.204400] niks3-server[1008]: 2026/09/24 09:26:48 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4301worker2 # [ 74.232749] nix-grpc-daemon[876]: time=2026-09-24T09:26:48.859Z level=INFO msg="Upload complete. (294ms)"4302worker2 # [ 74.245407] nix-grpc-daemon[783]: ts=2026-09-24T09:26:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04303worker2 # [ 74.265025] nix-daemon[842]: reaped child process 954, status = succeeded4304client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.26 seconds)4305client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4306client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4307client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4308worker1 # [ 74.607527] nix-grpc-daemon[488]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:354924309client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4310client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4311client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4312client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4313worker1 # [ 74.626518] nix-grpc-daemon[488]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:354924314worker1 # [ 74.635426] nix-grpc-daemon[488]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4315worker1 # [ 74.651962] nix-grpc-daemon[488]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4316worker1 # [ 74.666110] nix-grpc-daemon[488]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4317client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.23 seconds)4318client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4319worker2 # [ 74.870444] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:379004320worker2 # [ 74.878712] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:379004321client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.22 seconds)4322client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4323client # finding garbage collector roots...4324client # removing stale temporary roots file "/nix/var/nix/temproots/938"4325client # removing stale temporary roots file "/nix/var/nix/temproots/916"4326client # removing stale temporary roots file "/nix/var/nix/temproots/924"4327client # removing stale temporary roots file "/nix/var/nix/temproots/883"4328client # removing stale temporary roots file "/nix/var/nix/temproots/862"4329client # removing stale temporary roots file "/nix/var/nix/temproots/903"4330client # removing stale temporary roots file "/nix/var/nix/temproots/895"4331client # removing stale temporary roots file "/nix/var/nix/temproots/848"4332client # deleting specified paths...4333client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4334client # deleting unused links...4335client # note: hard linking is currently saving 0.0 KiB4336worker1 # [ 75.223280] nix-grpc-daemon[488]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396824337worker2 # [ 75.224924] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124338worker2 # [ 75.237775] nix-grpc-daemon[876]: time=2026-09-24T09:26:49.861Z level=INFO msg="All 1 paths already cached"4339worker1 # [ 75.244333] nix-grpc-daemon[488]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396824340worker2 # [ 75.251075] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:359124341client # copying 3 paths...4342worker2 # [ 75.264558] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:359124343client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4344worker1 # [ 75.292371] nix-grpc-daemon[488]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:396824345worker1 # [ 75.299897] nix-grpc-daemon[488]: ts=2026-09-24T09:26:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_out=2324346worker2 # [ 75.303649] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:359124347worker2 # [ 75.311148] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_out=1764348worker2 # [ 75.320109] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:359124349client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4350worker2 # [ 75.325388] nix-grpc-daemon[783]: ts=2026-09-24T09:26:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_out=1284351client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4352client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.44 seconds)4353(finished: subtest: outputs read back through any worker carry the cache signature, in 7.33 seconds)4354subtest: two clients wanting the same drv build it once4355client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4356client # [ 75.435254] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4357client # Running as unit: dedup1.service; invocation ID: 78fb6206b49f4e2b850e0d05cc37b46c4358client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.05 seconds)4359client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4360worker2 # [ 75.571706] nix-grpc-daemon[783]: ts=2026-09-24T09:26:50Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:359124361client # [ 75.617255] nix[991]: don't know how to build these paths:4362client # [ 75.618843] nix[991]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4363worker1 # [ 75.592124] nix-grpc-daemon[488]: ts=2026-09-24T09:26:50Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04364worker1 # [ 75.598916] nix-grpc-daemon[488]: ts=2026-09-24T09:26:50Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14365worker1 # [ 75.609733] nix-grpc-daemon[488]: ts=2026-09-24T09:26:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824366worker1 # [ 75.621117] nix-grpc-daemon[488]: ts=2026-09-24T09:26:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396824367client # [ 75.670276] nix[991]: copying 1 paths to 'lb:50051'...4368client # [ 75.672106] nix[991]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4369worker1 # [ 75.636605] nix-grpc-daemon[488]: ts=2026-09-24T09:26:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396824370worker1 # [ 75.650521] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4371worker1 # [ 75.813469] nix-daemon[855]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)4372worker1 # [ 75.833546] nix-daemon[855]: reaped child process 1057, status = succeeded4373lb # [ 75.806620] niks3-server[1008]: 2026/09/24 09:26:50 INFO Received push request method=POST path=/api/pushes4374worker1 # [ 75.847644] nix-grpc-daemon[867]: time=2026-09-24T09:26:50.479Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4375worker1 # [ 75.849308] nix-grpc-daemon[867]: time=2026-09-24T09:26:50.479Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4376lb # [ 75.837827] niks3-server[1008]: 2026/09/24 09:26:50 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4377worker1 # [ 75.875136] nix-grpc-daemon[867]: time=2026-09-24T09:26:50.507Z level=INFO msg="Uploading 1 narinfos"4378lb # [ 75.849211] niks3-server[1008]: 2026/09/24 09:26:50 INFO Signed narinfos id=15 count=14379lb # [ 75.857258] niks3-server[1008]: 2026/09/24 09:26:50 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4380lb # [ 75.872607] niks3-server[1008]: 2026/09/24 09:26:50 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4381worker1 # [ 75.904316] nix-grpc-daemon[867]: time=2026-09-24T09:26:50.536Z level=INFO msg="Upload complete. (237ms)"4382lb # [ 75.876197] niks3-server[1008]: 2026/09/24 09:26:50 INFO Received complete push request method=POST path=/api/pushes/15/complete4383lb # [ 75.882384] niks3-server[1008]: 2026/09/24 09:26:50 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4384worker1 # [ 75.913207] nix-grpc-daemon[488]: ts=2026-09-24T09:26:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_in=4324385worker1 # [ 75.929879] nix-grpc-daemon[488]: ts=2026-09-24T09:26:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824386worker1 # [ 75.936706] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4387worker1 # [ 75.941989] nix-daemon[855]: reaped child process 1052, status = succeeded4388client # [ 75.987493] nix[991]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4389worker1 # [ 75.955863] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4390client # [ 76.517192] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4391client # Running as unit: dedup2.service; invocation ID: 1afaae839370459287778a7a1328c4d74392client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.10 seconds)4393worker1 # [ 76.707021] nix-grpc-daemon[488]: ts=2026-09-24T09:26:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396824394client # [ 76.756376] nix[1007]: this derivation will be built:4395worker1 # [ 76.722268] nix-grpc-daemon[488]: ts=2026-09-24T09:26:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396824396client # [ 76.765789] nix[1007]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4397worker1 # [ 76.738803] nix-grpc-daemon[488]: ts=2026-09-24T09:26:51Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04398client # [ 76.781487] nix[1007]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4399worker1 # [ 76.745896] nix-grpc-daemon[488]: ts=2026-09-24T09:26:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824400client: must succeed: sleep 24401lb # [ 78.045035] envoy[898]: conn peer=[2001:db8:1::1]:38720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9528 tx=4377 ms=60094402lb # [ 78.050104] envoy[898]: conn peer=[2001:db8:1::1]:52256 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1274403lb # [ 78.050313] envoy[898]: conn peer=[2001:db8:1::1]:52300 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=334404lb # [ 78.050447] envoy[898]: conn peer=[2001:db8:1::1]:52270 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=604405lb # [ 78.050548] envoy[898]: conn peer=[2001:db8:1::1]:52284 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=484406client: (finished: must succeed: sleep 2, in 2.06 seconds)4407worker1 # [ 79.983743] nix-daemon[855]: reaped child process 1061, status = succeeded4408client: waiting for success: ! systemctl is-active dedup1 dedup24409worker1 # [ 80.086281] nix-daemon[855]: accepted connection from pid 1093, user nix-grpc-daemon (trusted)4410worker1 # [ 80.106831] nix-daemon[855]: reaped child process 1096, status = succeeded4411lb # [ 80.079074] niks3-server[1008]: 2026/09/24 09:26:54 INFO Received push request method=POST path=/api/pushes4412worker1 # [ 80.120380] nix-grpc-daemon[867]: time=2026-09-24T09:26:54.751Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4413worker1 # [ 80.121852] nix-grpc-daemon[867]: time=2026-09-24T09:26:54.751Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4414lb # [ 80.113098] niks3-server[1008]: 2026/09/24 09:26:54 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4415lb # [ 80.136459] niks3-server[1008]: 2026/09/24 09:26:54 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4416worker1 # [ 80.174529] nix-grpc-daemon[867]: time=2026-09-24T09:26:54.806Z level=INFO msg="Uploading 1 narinfos"4417lb # [ 80.146792] niks3-server[1008]: 2026/09/24 09:26:54 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4418lb # [ 80.153139] niks3-server[1008]: 2026/09/24 09:26:54 INFO Signed narinfos id=16 count=14419lb # [ 80.157422] niks3-server[1008]: 2026/09/24 09:26:54 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4420lb # [ 80.168998] niks3-server[1008]: 2026/09/24 09:26:54 INFO Received complete push request method=POST path=/api/pushes/16/complete4421lb # [ 80.172419] niks3-server[1008]: 2026/09/24 09:26:54 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4422worker1 # [ 80.203624] nix-grpc-daemon[867]: time=2026-09-24T09:26:54.835Z level=INFO msg="Upload complete. (235ms)"4423worker1 # [ 80.221114] nix-grpc-daemon[488]: ts=2026-09-24T09:26:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04424worker1 # [ 80.237154] nix-grpc-daemon[488]: ts=2026-09-24T09:26:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04425worker1 # [ 80.244969] nix-daemon[855]: reaped child process 1059, status = succeeded4426client # [ 80.308642] systemd[1]: dedup2.service: Deactivated successfully.4427client # [ 80.311908] systemd[1]: dedup2.service: Consumed 205ms CPU time over 3.796s wall clock time, 33.2M memory peak, 6.3K incoming IP traffic, 5.8K outgoing IP traffic.4428client # [ 80.321063] systemd[1]: dedup1.service: Deactivated successfully.4429client # [ 80.322722] systemd[1]: dedup1.service: Consumed 164ms CPU time over 4.882s wall clock time, 41.5M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4430client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4431client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4432client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4433worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4434worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4435worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4436worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4437worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4438worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4439worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4440worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4441(finished: subtest: two clients wanting the same drv build it once, in 6.08 seconds)4442subtest: --timeout and --max-silent-time reach the builder4443worker1 # [ 81.673417] nix-grpc-daemon[488]: ts=2026-09-24T09:26:56Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396824444worker1 # [ 81.690233] nix-grpc-daemon[488]: ts=2026-09-24T09:26:56Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04445worker2 # [ 81.691974] nix-grpc-daemon[783]: ts=2026-09-24T09:26:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124446worker1 # [ 81.696537] nix-grpc-daemon[488]: ts=2026-09-24T09:26:56Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14447worker2 # [ 81.704657] nix-grpc-daemon[783]: ts=2026-09-24T09:26:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124448worker2 # [ 81.718660] nix-grpc-daemon[783]: ts=2026-09-24T09:26:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359124449worker2 # [ 81.733655] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4450worker2 # [ 81.902476] nix-daemon[842]: accepted connection from pid 1000, user nix-grpc-daemon (trusted)4451worker2 # [ 81.919583] nix-daemon[842]: reaped child process 1003, status = succeeded4452lb # [ 81.899458] niks3-server[1008]: 2026/09/24 09:26:56 INFO Received push request method=POST path=/api/pushes4453worker2 # [ 81.937223] nix-grpc-daemon[876]: time=2026-09-24T09:26:56.562Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4454worker2 # [ 81.939380] nix-grpc-daemon[876]: time=2026-09-24T09:26:56.562Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4455lb # [ 81.926455] niks3-server[1008]: 2026/09/24 09:26:56 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4456worker2 # [ 81.963651] nix-grpc-daemon[876]: time=2026-09-24T09:26:56.589Z level=INFO msg="Uploading 1 narinfos"4457lb # [ 81.938453] niks3-server[1008]: 2026/09/24 09:26:56 INFO Signed narinfos id=17 count=14458lb # [ 81.947275] niks3-server[1008]: 2026/09/24 09:26:56 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4459lb # [ 81.961555] niks3-server[1008]: 2026/09/24 09:26:56 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4460worker2 # [ 81.994928] nix-grpc-daemon[876]: time=2026-09-24T09:26:56.621Z level=INFO msg="Upload complete. (244ms)"4461lb # [ 81.967169] niks3-server[1008]: 2026/09/24 09:26:56 INFO Received complete push request method=POST path=/api/pushes/17/complete4462lb # [ 81.971459] niks3-server[1008]: 2026/09/24 09:26:56 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4463worker2 # [ 82.004321] nix-grpc-daemon[783]: ts=2026-09-24T09:26:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_in=4244464worker2 # [ 82.019564] nix-grpc-daemon[783]: ts=2026-09-24T09:26:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124465worker2 # [ 82.026485] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4466worker2 # [ 82.031707] nix-daemon[842]: reaped child process 998, status = succeeded4467worker2 # [ 82.043277] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4468lb # [ 88.040908] envoy[898]: conn peer=[2001:db8:1::1]:52318 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1598 ms=35274469worker2 # [ 88.263185] nix-grpc-daemon[783]: ts=2026-09-24T09:27:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04470worker2 # [ 88.310381] nix-daemon[842]: reaped child process 1005, status = succeeded4471worker2 # [ 88.312430] nix-daemon[842]: reaped child process 1007, status = succeeded4472worker2 # [ 88.623023] nix-grpc-daemon[783]: ts=2026-09-24T09:27:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:359124473worker1 # [ 88.665057] nix-grpc-daemon[488]: ts=2026-09-24T09:27:03Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04474worker1 # [ 88.671530] nix-grpc-daemon[488]: ts=2026-09-24T09:27:03Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14475worker1 # [ 88.681436] nix-grpc-daemon[488]: ts=2026-09-24T09:27:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824476lb # [ 88.046054] envoy[898]: conn peer=[2001:db8:1::1]:52310 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3005 tx=1563 ms=46584477worker1 # [ 88.698887] nix-grpc-daemon[488]: ts=2026-09-24T09:27:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396824478worker1 # [ 88.721730] nix-grpc-daemon[488]: ts=2026-09-24T09:27:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396824479worker1 # [ 88.748495] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4480worker1 # [ 88.994633] nix-daemon[855]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4481worker1 # [ 89.024291] nix-daemon[855]: reaped child process 1116, status = succeeded4482lb # [ 89.011520] niks3-server[1008]: 2026/09/24 09:27:03 INFO Received push request method=POST path=/api/pushes4483worker1 # [ 89.056662] nix-grpc-daemon[867]: time=2026-09-24T09:27:03.687Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4484worker1 # [ 89.058557] nix-grpc-daemon[867]: time=2026-09-24T09:27:03.687Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4485worker1 # [ 89.109984] nix-grpc-daemon[867]: time=2026-09-24T09:27:03.742Z level=INFO msg="Uploading 1 narinfos"4486lb # [ 89.069906] niks3-server[1008]: 2026/09/24 09:27:03 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4487lb # [ 89.086483] niks3-server[1008]: 2026/09/24 09:27:03 INFO Signed narinfos id=18 count=14488lb # [ 89.105379] niks3-server[1008]: 2026/09/24 09:27:03 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4489lb # [ 89.117561] niks3-server[1008]: 2026/09/24 09:27:03 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4490lb # [ 89.135265] niks3-server[1008]: 2026/09/24 09:27:03 INFO Received complete push request method=POST path=/api/pushes/18/complete4491lb # [ 89.147615] niks3-server[1008]: 2026/09/24 09:27:03 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4492worker1 # [ 89.172381] nix-grpc-daemon[867]: time=2026-09-24T09:27:03.804Z level=INFO msg="Upload complete. (392ms)"4493worker1 # [ 89.185050] nix-grpc-daemon[488]: ts=2026-09-24T09:27:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_in=4244494worker1 # [ 89.206540] nix-grpc-daemon[488]: ts=2026-09-24T09:27:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824495worker1 # [ 89.233283] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4496worker1 # [ 89.244182] nix-daemon[855]: reaped child process 1111, status = succeeded4497worker1 # [ 89.263922] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4498worker1 # [ 95.292609] nix-grpc-daemon[488]: ts=2026-09-24T09:27:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04499worker1 # [ 95.330794] nix-daemon[855]: reaped child process 1118, status = succeeded4500worker1 # [ 95.333241] nix-daemon[855]: reaped child process 1120, status = succeeded4501(finished: subtest: --timeout and --max-silent-time reach the builder, in 14.04 seconds)4502subtest: the first client leaving does not fail the second4503client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4504client # [ 95.591664] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4505client # Running as unit: share1.service; invocation ID: 7a3d22605790400198e7280d9ebea58d4506client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4507client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4508client # [ 95.801666] nix[1080]: don't know how to build these paths:4509client # [ 95.802939] nix[1080]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4510worker1 # [ 95.765194] nix-grpc-daemon[488]: ts=2026-09-24T09:27:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396824511worker1 # [ 95.779966] nix-grpc-daemon[488]: ts=2026-09-24T09:27:10Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04512worker2 # [ 95.781759] nix-grpc-daemon[783]: ts=2026-09-24T09:27:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124513worker1 # [ 95.785975] nix-grpc-daemon[488]: ts=2026-09-24T09:27:10Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14514worker2 # [ 95.799733] nix-grpc-daemon[783]: ts=2026-09-24T09:27:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124515client # [ 95.849852] nix[1080]: copying 1 paths to 'lb:50051'...4516client # [ 95.854258] nix[1080]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4517worker2 # [ 95.815390] nix-grpc-daemon[783]: ts=2026-09-24T09:27:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359124518worker2 # [ 95.832475] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4519worker2 # [ 96.004110] nix-daemon[842]: accepted connection from pid 1030, user nix-grpc-daemon (trusted)4520worker2 # [ 96.024921] nix-daemon[842]: reaped child process 1033, status = succeeded4521lb # [ 96.004602] niks3-server[1008]: 2026/09/24 09:27:10 INFO Received push request method=POST path=/api/pushes4522worker2 # [ 96.042934] nix-grpc-daemon[876]: time=2026-09-24T09:27:10.667Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4523worker2 # [ 96.044680] nix-grpc-daemon[876]: time=2026-09-24T09:27:10.668Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4524lb # [ 96.042948] niks3-server[1008]: 2026/09/24 09:27:10 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4525worker2 # [ 96.081003] nix-grpc-daemon[876]: time=2026-09-24T09:27:10.706Z level=INFO msg="Uploading 1 narinfos"4526lb # [ 96.060290] niks3-server[1008]: 2026/09/24 09:27:10 INFO Signed narinfos id=19 count=14527lb # [ 96.062796] niks3-server[1008]: 2026/09/24 09:27:10 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4528lb # [ 96.073142] niks3-server[1008]: 2026/09/24 09:27:10 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4529worker2 # [ 96.113744] nix-grpc-daemon[876]: time=2026-09-24T09:27:10.740Z level=INFO msg="Upload complete. (259ms)"4530lb # [ 96.086044] niks3-server[1008]: 2026/09/24 09:27:10 INFO Received complete push request method=POST path=/api/pushes/19/complete4531lb # [ 96.089339] niks3-server[1008]: 2026/09/24 09:27:10 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4532worker2 # [ 96.125020] nix-grpc-daemon[783]: ts=2026-09-24T09:27:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_in=4324533worker2 # [ 96.136339] nix-grpc-daemon[783]: ts=2026-09-24T09:27:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124534worker2 # [ 96.151505] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4535worker2 # [ 96.154957] nix-daemon[842]: reaped child process 1028, status = succeeded4536worker2 # [ 96.167630] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4537client # [ 96.204910] nix[1080]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4538client # [ 96.697264] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4539client # Running as unit: share2.service; invocation ID: b679799f381e400186c7e8e38451e00f4540client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4541worker1 # [ 96.871487] nix-grpc-daemon[488]: ts=2026-09-24T09:27:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396824542client # [ 96.915182] nix[1096]: don't know how to build these paths:4543client # [ 96.916395] nix[1096]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4544worker1 # [ 96.888113] nix-grpc-daemon[488]: ts=2026-09-24T09:27:11Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04545worker2 # [ 96.895718] nix-grpc-daemon[783]: ts=2026-09-24T09:27:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124546client # [ 96.941152] nix[1096]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4547client: must succeed: sleep 2; systemctl kill -s INT share14548lb # [ 98.041295] envoy[898]: conn peer=[2001:db8:1::1]:40248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3014 tx=978 ms=66304549lb # [ 98.046376] envoy[898]: conn peer=[2001:db8:1::1]:40262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2989 tx=977 ms=66954550client # [ 100.068985] systemd[1]: share1.service: Sent signal SIGINT to main process 1080 (nix) on client request.4551client # [ 100.077808] nix[1080]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4552client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.08 seconds)4553client: waiting for success: ! systemctl is-active share14554client # [ 100.087010] nix[1080]: [mutex.cc : 2506] RAW: thread should hold at least a read lock on Mutex 0xe16393ffc6784555client # [ 100.112669] systemd-coredump[1111]: Process 1080 (nix) of user 0 terminated abnormally with signal 6/ABRT, processing...4556client # [ 100.135298] systemd[1]: Created slice Slice /system/systemd-coredump.4557client # [ 100.143645] systemd[1]: Started Process Core Dump (PID 1111/UID 0).4558client # [ 102.477271] systemd-coredump[1116]: Cannot store coredump of 1080 (nix): No space left on device4559client # [ 102.601864] systemd-coredump[1116]: Process 1080 (nix) of user 0 terminated abnormally without generating a coredump.4560client # [ 102.615818] systemd[1]: systemd-coredump@0-1-1111_1112-0.service: Deactivated successfully.4561client # [ 102.622609] systemd[1]: systemd-coredump@0-1-1111_1112-0.service: Consumed 1.977s CPU time over 2.474s wall clock time, 968.8M memory peak, 926.3M written to disk.4562client # [ 102.630446] systemd[1]: share1.service: Main process exited, code=dumped, status=6/ABRT4563client # [ 102.633258] systemd[1]: share1.service: Failed with result 'core-dump'.4564client # [ 102.634593] systemd[1]: share1.service: Consumed 373ms CPU time over 7.041s wall clock time, 41.5M memory peak, 4K written to disk, 6.2K incoming IP traffic, 8.1K outgoing IP traffic.4565client: (finished: waiting for success: ! systemctl is-active share1, in 3.24 seconds)4566client: waiting for success: ! systemctl is-active share24567worker2 # [ 103.611671] nix-daemon[842]: reaped child process 1037, status = succeeded4568worker2 # [ 103.712498] nix-daemon[842]: accepted connection from pid 1071, user nix-grpc-daemon (trusted)4569worker2 # [ 103.729341] nix-daemon[842]: reaped child process 1074, status = succeeded4570lb # [ 103.707153] niks3-server[1008]: 2026/09/24 09:27:18 INFO Received push request method=POST path=/api/pushes4571worker2 # [ 103.748222] nix-grpc-daemon[876]: time=2026-09-24T09:27:18.372Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4572worker2 # [ 103.749985] nix-grpc-daemon[876]: time=2026-09-24T09:27:18.372Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4573lb # [ 103.755581] niks3-server[1008]: 2026/09/24 09:27:18 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign4574worker2 # [ 103.793221] nix-grpc-daemon[876]: time=2026-09-24T09:27:18.419Z level=INFO msg="Uploading 1 narinfos"4575lb # [ 103.767637] niks3-server[1008]: 2026/09/24 09:27:18 INFO Signed narinfos id=20 count=14576lb # [ 103.775296] niks3-server[1008]: 2026/09/24 09:27:18 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4577lb # [ 103.784344] niks3-server[1008]: 2026/09/24 09:27:18 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4578lb # [ 103.795380] niks3-server[1008]: 2026/09/24 09:27:18 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4579worker2 # [ 103.830306] nix-grpc-daemon[876]: time=2026-09-24T09:27:18.456Z level=INFO msg="Upload complete. (240ms)"4580lb # [ 103.803691] niks3-server[1008]: 2026/09/24 09:27:18 INFO Received complete push request method=POST path=/api/pushes/20/complete4581lb # [ 103.807710] niks3-server[1008]: 2026/09/24 09:27:18 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4582worker2 # [ 103.841215] nix-grpc-daemon[783]: ts=2026-09-24T09:27:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=7 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04583worker2 # [ 103.860412] nix-grpc-daemon[783]: ts=2026-09-24T09:27:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=6 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04584worker2 # [ 103.868610] nix-daemon[842]: reaped child process 1035, status = succeeded4585client # [ 103.918453] systemd[1]: share2.service: Deactivated successfully.4586client # [ 103.923498] systemd[1]: share2.service: Consumed 189ms CPU time over 7.223s wall clock time, 35.3M memory peak, 4K written to disk, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4587client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4588client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4589client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4590(finished: subtest: the first client leaving does not fail the second, in 9.23 seconds)4591subtest: build hook: nix-daemon with builders = grpc://lb4592client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4593client # [ 104.879768] systemd[1]: Started Nix Daemon.4594client # [ 104.995919] nix-daemon[1162]: accepted connection from pid 1159, user root (trusted)4595worker1 # [ 105.108926] nix-grpc-daemon[488]: ts=2026-09-24T09:27:19Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:396824596worker2 # [ 105.108629] nix-grpc-daemon[783]: ts=2026-09-24T09:27:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124597worker1 # [ 105.129948] nix-grpc-daemon[488]: ts=2026-09-24T09:27:19Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04598worker1 # [ 105.137172] nix-grpc-daemon[488]: ts=2026-09-24T09:27:19Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14599worker1 # [ 105.146343] nix-grpc-daemon[488]: ts=2026-09-24T09:27:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824600worker1 # [ 105.157551] nix-grpc-daemon[488]: ts=2026-09-24T09:27:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396824601worker1 # [ 105.172667] nix-grpc-daemon[488]: ts=2026-09-24T09:27:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396824602worker1 # [ 105.187769] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4603worker1 # [ 105.343256] nix-daemon[855]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)4604worker1 # [ 105.363539] nix-daemon[855]: reaped child process 1168, status = succeeded4605lb # [ 105.337191] niks3-server[1008]: 2026/09/24 09:27:19 INFO Received push request method=POST path=/api/pushes4606worker1 # [ 105.378682] nix-grpc-daemon[867]: time=2026-09-24T09:27:20.010Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4607worker1 # [ 105.380293] nix-grpc-daemon[867]: time=2026-09-24T09:27:20.010Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4608lb # [ 105.369155] niks3-server[1008]: 2026/09/24 09:27:19 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4609worker1 # [ 105.415072] nix-grpc-daemon[867]: time=2026-09-24T09:27:20.047Z level=INFO msg="Uploading 1 narinfos"4610lb # [ 105.384926] niks3-server[1008]: 2026/09/24 09:27:20 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign4611lb # [ 105.394853] niks3-server[1008]: 2026/09/24 09:27:20 INFO Signed narinfos id=21 count=14612lb # [ 105.401336] niks3-server[1008]: 2026/09/24 09:27:20 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4613lb # [ 105.414248] niks3-server[1008]: 2026/09/24 09:27:20 INFO Received complete push request method=POST path=/api/pushes/21/complete4614lb # [ 105.421569] niks3-server[1008]: 2026/09/24 09:27:20 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4615worker1 # [ 105.452561] nix-grpc-daemon[867]: time=2026-09-24T09:27:20.084Z level=INFO msg="Upload complete. (249ms)"4616worker1 # [ 105.464148] nix-grpc-daemon[488]: ts=2026-09-24T09:27:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_in=5124617worker1 # [ 105.480297] nix-grpc-daemon[488]: ts=2026-09-24T09:27:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824618worker1 # [ 105.487724] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4619worker1 # [ 105.493412] nix-daemon[855]: reaped child process 1163, status = succeeded4620worker1 # [ 105.506509] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4621worker1 # [ 106.615379] nix-daemon[855]: reaped child process 1172, status = succeeded4622worker1 # [ 106.686755] nix-daemon[855]: accepted connection from pid 1188, user nix-grpc-daemon (trusted)4623worker1 # [ 106.704291] nix-daemon[855]: reaped child process 1191, status = succeeded4624lb # [ 106.678607] niks3-server[1008]: 2026/09/24 09:27:21 INFO Received push request method=POST path=/api/pushes4625worker1 # [ 106.720396] nix-grpc-daemon[867]: time=2026-09-24T09:27:21.351Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4626worker1 # [ 106.721991] nix-grpc-daemon[867]: time=2026-09-24T09:27:21.351Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4627lb # [ 106.724053] niks3-server[1008]: 2026/09/24 09:27:21 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4628lb # [ 106.740259] niks3-server[1008]: 2026/09/24 09:27:21 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign4629lb # [ 106.743689] niks3-server[1008]: 2026/09/24 09:27:21 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4630worker1 # [ 106.780834] nix-grpc-daemon[867]: time=2026-09-24T09:27:21.412Z level=INFO msg="Uploading 1 narinfos"4631lb # [ 106.756425] niks3-server[1008]: 2026/09/24 09:27:21 INFO Signed narinfos id=22 count=14632lb # [ 106.758883] niks3-server[1008]: 2026/09/24 09:27:21 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4633lb # [ 106.774230] niks3-server[1008]: 2026/09/24 09:27:21 INFO Received complete push request method=POST path=/api/pushes/22/complete4634lb # [ 106.779707] niks3-server[1008]: 2026/09/24 09:27:21 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4635worker1 # [ 106.811321] nix-grpc-daemon[867]: time=2026-09-24T09:27:21.443Z level=INFO msg="Upload complete. (293ms)"4636worker1 # [ 106.826022] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04637worker1 # [ 106.844463] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396824638worker1 # [ 106.850788] nix-daemon[855]: reaped child process 1170, status = succeeded4639worker1 # [ 106.873587] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:396824640worker1 # [ 106.880669] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_out=1284641worker1 # [ 107.111350] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:396824642worker2 # [ 107.115133] nix-grpc-daemon[783]: ts=2026-09-24T09:27:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124643worker1 # [ 107.144824] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04644worker1 # [ 107.151843] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14645worker1 # [ 107.161540] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824646worker1 # [ 107.172853] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396824647worker1 # [ 107.186722] nix-grpc-daemon[867]: time=2026-09-24T09:27:21.819Z level=INFO msg="All 1 paths already cached"4648worker1 # [ 107.196604] nix-grpc-daemon[488]: ts=2026-09-24T09:27:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396824649worker1 # [ 107.211053] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4650worker1 # [ 107.367282] nix-daemon[855]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)4651worker1 # [ 107.385753] nix-daemon[855]: reaped child process 1200, status = succeeded4652lb # [ 107.358479] niks3-server[1008]: 2026/09/24 09:27:21 INFO Received push request method=POST path=/api/pushes4653worker1 # [ 107.398969] nix-grpc-daemon[867]: time=2026-09-24T09:27:22.030Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4654worker1 # [ 107.400544] nix-grpc-daemon[867]: time=2026-09-24T09:27:22.030Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4655lb # [ 107.387172] niks3-server[1008]: 2026/09/24 09:27:22 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4656worker1 # [ 107.427509] nix-grpc-daemon[867]: time=2026-09-24T09:27:22.059Z level=INFO msg="Uploading 1 narinfos"4657lb # [ 107.399340] niks3-server[1008]: 2026/09/24 09:27:22 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign4658lb # [ 107.411503] niks3-server[1008]: 2026/09/24 09:27:22 INFO Signed narinfos id=23 count=14659lb # [ 107.418347] niks3-server[1008]: 2026/09/24 09:27:22 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4660worker1 # [ 107.463734] nix-grpc-daemon[867]: time=2026-09-24T09:27:22.096Z level=INFO msg="Upload complete. (236ms)"4661lb # [ 107.433137] niks3-server[1008]: 2026/09/24 09:27:22 INFO Received complete push request method=POST path=/api/pushes/23/complete4662lb # [ 107.437697] niks3-server[1008]: 2026/09/24 09:27:22 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4663worker1 # [ 107.475294] nix-grpc-daemon[488]: ts=2026-09-24T09:27:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_in=6244664worker1 # [ 107.491612] nix-grpc-daemon[488]: ts=2026-09-24T09:27:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824665worker1 # [ 107.500825] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4666worker1 # [ 107.504791] nix-daemon[855]: reaped child process 1195, status = succeeded4667worker1 # [ 107.516201] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4668lb # [ 108.039533] envoy[898]: conn peer=[2001:db8:1::1]:52920 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2977 tx=790 ms=68234669lb # [ 108.044494] envoy[898]: conn peer=[2001:db8:1::1]:52930 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1194 ms=69904670lb # [ 108.049343] envoy[898]: conn peer=[2001:db8:1::1]:47846 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3419 tx=1942 ms=17904671lb # [ 108.054317] envoy[898]: conn peer=[2001:db8:1::1]:47862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=284672worker1 # [ 108.796672] nix-daemon[855]: reaped child process 1204, status = succeeded4673worker1 # [ 108.904718] nix-daemon[855]: accepted connection from pid 1220, user nix-grpc-daemon (trusted)4674worker1 # [ 108.923198] nix-daemon[855]: reaped child process 1223, status = succeeded4675lb # [ 108.898435] niks3-server[1008]: 2026/09/24 09:27:23 INFO Received push request method=POST path=/api/pushes4676worker1 # [ 108.944464] nix-grpc-daemon[867]: time=2026-09-24T09:27:23.572Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4677worker1 # [ 108.947165] nix-grpc-daemon[867]: time=2026-09-24T09:27:23.572Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4678lb # [ 108.938843] niks3-server[1008]: 2026/09/24 09:27:23 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign4679worker1 # [ 108.981412] nix-grpc-daemon[867]: time=2026-09-24T09:27:23.613Z level=INFO msg="Uploading 1 narinfos"4680lb # [ 108.952617] niks3-server[1008]: 2026/09/24 09:27:23 INFO Signed narinfos id=24 count=14681lb # [ 108.961781] niks3-server[1008]: 2026/09/24 09:27:23 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4682lb # [ 108.970355] niks3-server[1008]: 2026/09/24 09:27:23 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4683lb # [ 108.982294] niks3-server[1008]: 2026/09/24 09:27:23 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4684lb # [ 108.987088] niks3-server[1008]: 2026/09/24 09:27:23 INFO Received complete push request method=POST path=/api/pushes/24/complete4685worker1 # [ 109.024177] nix-grpc-daemon[867]: time=2026-09-24T09:27:23.656Z level=INFO msg="Upload complete. (294ms)"4686lb # [ 108.993302] niks3-server[1008]: 2026/09/24 09:27:23 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4687worker2 # [ 109.037053] nix-grpc-daemon[783]: ts=2026-09-24T09:27:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:359124688worker1 # [ 109.037304] nix-grpc-daemon[488]: ts=2026-09-24T09:27:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04689worker2 # [ 109.051138] nix-grpc-daemon[783]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4690worker1 # [ 109.058131] nix-daemon[855]: reaped child process 1202, status = succeeded4691worker2 # [ 109.067312] nix-grpc-daemon[783]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4692worker1 # [ 109.107821] nix-grpc-daemon[488]: ts=2026-09-24T09:27:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:396824693worker1 # [ 109.114824] nix-grpc-daemon[488]: ts=2026-09-24T09:27:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_out=1764694worker1 # [ 109.355359] nix-grpc-daemon[488]: ts=2026-09-24T09:27:23Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:396824695worker2 # [ 109.356075] nix-grpc-daemon[783]: ts=2026-09-24T09:27:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124696worker2 # [ 109.367103] nix-grpc-daemon[876]: time=2026-09-24T09:27:23.992Z level=INFO msg="All 2 paths already cached"4697worker1 # [ 109.388116] nix-grpc-daemon[488]: ts=2026-09-24T09:27:24Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04698worker1 # [ 109.394447] nix-grpc-daemon[488]: ts=2026-09-24T09:27:24Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14699worker1 # [ 109.404244] nix-grpc-daemon[488]: ts=2026-09-24T09:27:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824700worker1 # [ 109.416367] nix-grpc-daemon[488]: ts=2026-09-24T09:27:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396824701worker1 # [ 109.431224] nix-grpc-daemon[867]: time=2026-09-24T09:27:24.063Z level=INFO msg="All 2 paths already cached"4702worker1 # [ 109.441445] nix-grpc-daemon[488]: ts=2026-09-24T09:27:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396824703worker1 # [ 109.457298] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4704worker1 # [ 109.613255] nix-daemon[855]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)4705worker1 # [ 109.632525] nix-daemon[855]: reaped child process 1232, status = succeeded4706lb # [ 109.607272] niks3-server[1008]: 2026/09/24 09:27:24 INFO Received push request method=POST path=/api/pushes4707worker1 # [ 109.648464] nix-grpc-daemon[867]: time=2026-09-24T09:27:24.280Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4708worker1 # [ 109.650863] nix-grpc-daemon[867]: time=2026-09-24T09:27:24.282Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4709lb # [ 109.644120] niks3-server[1008]: 2026/09/24 09:27:24 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign4710worker1 # [ 109.681738] nix-grpc-daemon[867]: time=2026-09-24T09:27:24.313Z level=INFO msg="Uploading 1 narinfos"4711lb # [ 109.657160] niks3-server[1008]: 2026/09/24 09:27:24 INFO Signed narinfos id=25 count=14712lb # [ 109.664974] niks3-server[1008]: 2026/09/24 09:27:24 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4713lb # [ 109.676951] niks3-server[1008]: 2026/09/24 09:27:24 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4714lb # [ 109.689972] niks3-server[1008]: 2026/09/24 09:27:24 INFO Received complete push request method=POST path=/api/pushes/25/complete4715lb # [ 109.696116] niks3-server[1008]: 2026/09/24 09:27:24 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4716worker1 # [ 109.732406] nix-grpc-daemon[867]: time=2026-09-24T09:27:24.364Z level=INFO msg="Upload complete. (258ms)"4717worker1 # [ 109.741968] nix-grpc-daemon[488]: ts=2026-09-24T09:27:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_in=7524718worker1 # [ 109.760843] nix-grpc-daemon[488]: ts=2026-09-24T09:27:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824719worker1 # [ 109.770075] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4720worker1 # [ 109.777440] nix-daemon[855]: reaped child process 1227, status = succeeded4721worker1 # [ 109.789842] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4722worker1 # [ 111.264781] nix-daemon[855]: reaped child process 1236, status = succeeded4723worker1 # [ 111.374822] nix-daemon[855]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)4724worker1 # [ 111.396064] nix-daemon[855]: reaped child process 1255, status = succeeded4725lb # [ 111.366172] niks3-server[1008]: 2026/09/24 09:27:25 INFO Received push request method=POST path=/api/pushes4726worker1 # [ 111.406469] nix-grpc-daemon[867]: time=2026-09-24T09:27:26.036Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4727worker1 # [ 111.407938] nix-grpc-daemon[867]: time=2026-09-24T09:27:26.036Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4728lb # [ 111.408695] niks3-server[1008]: 2026/09/24 09:27:26 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4729worker1 # [ 111.451531] nix-grpc-daemon[867]: time=2026-09-24T09:27:26.083Z level=INFO msg="Uploading 1 narinfos"4730lb # [ 111.422851] niks3-server[1008]: 2026/09/24 09:27:26 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign4731lb # [ 111.431917] niks3-server[1008]: 2026/09/24 09:27:26 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4732lb # [ 111.437295] niks3-server[1008]: 2026/09/24 09:27:26 INFO Signed narinfos id=26 count=14733lb # [ 111.443790] niks3-server[1008]: 2026/09/24 09:27:26 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4734worker1 # [ 111.481724] nix-grpc-daemon[867]: time=2026-09-24T09:27:26.114Z level=INFO msg="Upload complete. (277ms)"4735lb # [ 111.452332] niks3-server[1008]: 2026/09/24 09:27:26 INFO Received complete push request method=POST path=/api/pushes/26/complete4736lb # [ 111.455472] niks3-server[1008]: 2026/09/24 09:27:26 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4737worker1 # [ 111.496532] nix-grpc-daemon[488]: ts=2026-09-24T09:27:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04738worker1 # [ 111.516460] nix-grpc-daemon[488]: ts=2026-09-24T09:27:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396824739worker1 # [ 111.522631] nix-daemon[855]: reaped child process 1234, status = succeeded4740worker1 # [ 111.545495] nix-grpc-daemon[488]: ts=2026-09-24T09:27:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:396824741worker1 # [ 111.552380] nix-grpc-daemon[488]: ts=2026-09-24T09:27:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_out=2324742client # [ 111.671723] nix-daemon[1162]: reaped child process 1164, status = succeeded4743client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.95 seconds)4744client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4745client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4746client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4747client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4748client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4749client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4750(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.06 seconds)4751subtest: an input only one worker has still reaches the builder4752client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4753client # this derivation will be built:4754client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4755client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4756client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)4757client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4758client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4759worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4760worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)4761worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4762worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4763client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24764client # [ 112.624220] nix-daemon[1162]: accepted connection from pid 1245, user root (trusted)4765client # this derivation will be built:4766client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4767client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4768worker2 # [ 112.733876] nix-grpc-daemon[783]: ts=2026-09-24T09:27:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:359124769worker2 # [ 112.741558] nix-grpc-daemon[783]: ts=2026-09-24T09:27:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124770client # copying 1 paths...4771client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4772worker1 # [ 112.757233] nix-grpc-daemon[488]: ts=2026-09-24T09:27:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396824773worker1 # [ 112.772427] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4774worker1 # [ 112.940099] nix-daemon[855]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)4775worker1 # [ 112.960719] nix-daemon[855]: reaped child process 1267, status = succeeded4776lb # [ 112.930166] niks3-server[1008]: 2026/09/24 09:27:27 INFO Received push request method=POST path=/api/pushes4777worker1 # [ 112.970286] nix-grpc-daemon[867]: time=2026-09-24T09:27:27.602Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4778worker1 # [ 112.971735] nix-grpc-daemon[867]: time=2026-09-24T09:27:27.602Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4779lb # [ 112.959800] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4780worker1 # [ 113.001348] nix-grpc-daemon[867]: time=2026-09-24T09:27:27.633Z level=INFO msg="Uploading 1 narinfos"4781lb # [ 112.976467] niks3-server[1008]: 2026/09/24 09:27:27 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign4782lb # [ 112.985326] niks3-server[1008]: 2026/09/24 09:27:27 INFO Signed narinfos id=27 count=14783lb # [ 112.995410] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4784lb # [ 113.002245] niks3-server[1008]: 2026/09/24 09:27:27 INFO Received complete push request method=POST path=/api/pushes/27/complete4785worker1 # [ 113.034535] nix-grpc-daemon[867]: time=2026-09-24T09:27:27.666Z level=INFO msg="Upload complete. (247ms)"4786lb # [ 113.005955] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4787worker1 # [ 113.044289] nix-grpc-daemon[488]: ts=2026-09-24T09:27:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_in=1204788worker2 # [ 113.057231] nix-grpc-daemon[783]: ts=2026-09-24T09:27:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124789worker1 # [ 113.062275] nix-grpc-daemon[488]: ts=2026-09-24T09:27:27Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04790worker1 # [ 113.066978] nix-grpc-daemon[488]: ts=2026-09-24T09:27:27Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14791worker1 # [ 113.072128] nix-daemon[855]: reaped child process 1262, status = succeeded4792worker2 # [ 113.072636] nix-grpc-daemon[783]: ts=2026-09-24T09:27:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124793client # copying 2 paths to 'lb:50051'...4794client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4795worker2 # [ 113.091246] nix-grpc-daemon[783]: ts=2026-09-24T09:27:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359124796client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4797worker2 # [ 113.106042] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4798worker2 # [ 113.274667] nix-daemon[842]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)4799worker2 # [ 113.295283] nix-daemon[842]: reaped child process 1086, status = succeeded4800lb # [ 113.264701] niks3-server[1008]: 2026/09/24 09:27:27 INFO Received push request method=POST path=/api/pushes4801worker2 # [ 113.303567] nix-grpc-daemon[876]: time=2026-09-24T09:27:27.928Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4802worker2 # [ 113.306651] nix-grpc-daemon[876]: time=2026-09-24T09:27:27.928Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4803worker2 # [ 113.309073] nix-grpc-daemon[876]: time=2026-09-24T09:27:27.928Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4804lb # [ 113.309166] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4805lb # [ 113.322090] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4806worker2 # [ 113.359315] nix-grpc-daemon[876]: time=2026-09-24T09:27:27.985Z level=INFO msg="Uploading 2 narinfos"4807lb # [ 113.334169] niks3-server[1008]: 2026/09/24 09:27:27 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign4808lb # [ 113.341578] niks3-server[1008]: 2026/09/24 09:27:27 INFO Signed narinfos id=28 count=24809lb # [ 113.343601] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4810lb # [ 113.352725] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4811lb # [ 113.362140] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4812lb # [ 113.370234] niks3-server[1008]: 2026/09/24 09:27:27 INFO Received complete push request method=POST path=/api/pushes/28/complete4813worker2 # [ 113.398345] nix-grpc-daemon[876]: time=2026-09-24T09:27:28.024Z level=INFO msg="Upload complete. (273ms)"4814lb # [ 113.373908] niks3-server[1008]: 2026/09/24 09:27:27 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4815worker2 # [ 113.409510] nix-grpc-daemon[783]: ts=2026-09-24T09:27:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=2 nar_bytes_in=9844816worker2 # [ 113.427513] nix-grpc-daemon[783]: ts=2026-09-24T09:27:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124817worker2 # [ 113.436560] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4818worker2 # [ 113.444387] nix-daemon[842]: reaped child process 1081, status = succeeded4819worker2 # [ 113.455938] nix-grpc-daemon[783]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4820client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4821worker2 # [ 113.471571] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4822client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4823client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4824worker2 # [ 114.348413] nix-daemon[842]: reaped child process 1091, status = succeeded4825worker2 # [ 114.412466] nix-daemon[842]: accepted connection from pid 1107, user nix-grpc-daemon (trusted)4826worker2 # [ 114.432526] nix-daemon[842]: reaped child process 1110, status = succeeded4827lb # [ 114.403813] niks3-server[1008]: 2026/09/24 09:27:29 INFO Received push request method=POST path=/api/pushes4828worker2 # [ 114.442362] nix-grpc-daemon[876]: time=2026-09-24T09:27:29.066Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4829lb # [ 114.432539] niks3-server[1008]: 2026/09/24 09:27:29 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign4830worker2 # [ 114.469276] nix-grpc-daemon[876]: time=2026-09-24T09:27:29.095Z level=INFO msg="Uploading 1 narinfos"4831lb # [ 114.445637] niks3-server[1008]: 2026/09/24 09:27:29 INFO Signed narinfos id=29 count=14832lb # [ 114.454530] niks3-server[1008]: 2026/09/24 09:27:29 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4833worker2 # [ 114.502970] nix-grpc-daemon[876]: time=2026-09-24T09:27:29.129Z level=INFO msg="Upload complete. (216ms)"4834lb # [ 114.475964] niks3-server[1008]: 2026/09/24 09:27:29 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4835lb # [ 114.482582] niks3-server[1008]: 2026/09/24 09:27:29 INFO Received complete push request method=POST path=/api/pushes/29/complete4836worker2 # [ 114.512835] nix-grpc-daemon[783]: ts=2026-09-24T09:27:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=1 inputs=1 inputs_fetched=1 input_bytes=120 input_s=04837client # copying 1 paths...4838lb # [ 114.491616] niks3-server[1008]: 2026/09/24 09:27:29 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4839worker2 # [ 114.525934] nix-daemon[842]: reaped child process 1088, status = succeeded4840worker2 # [ 114.529607] nix-grpc-daemon[783]: ts=2026-09-24T09:27:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:359124841client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4842worker1 # [ 114.558693] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:396824843worker1 # [ 114.571840] nix-grpc-daemon[488]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4844worker1 # [ 114.588893] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=0 paths=1 nar_bytes_out=1204845client # [ 114.707673] nix-daemon[1162]: reaped child process 1248, status = succeeded4846client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.28 seconds)4847client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24848client # [ 114.873750] nix-daemon[1162]: accepted connection from pid 1267, user root (trusted)4849client # this derivation will be built:4850client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4851worker1 # [ 114.984454] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:396824852client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4853worker1 # [ 114.992053] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396824854worker1 # [ 114.999975] nix-grpc-daemon[867]: time=2026-09-24T09:27:29.630Z level=INFO msg="All 1 paths already cached"4855worker1 # [ 115.012195] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04856worker1 # [ 115.019572] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=14857worker1 # [ 115.028690] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824858worker1 # [ 115.039321] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:396824859client # copying 1 paths to 'lb:50051'...4860client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4861worker1 # [ 115.057784] nix-grpc-daemon[488]: ts=2026-09-24T09:27:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:396824862worker1 # [ 115.072990] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4863worker1 # [ 115.091984] nix-grpc-daemon[488]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 1/5)4864worker1 # [ 115.173315] nix-grpc-daemon[488]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 63 ms (attempt 2/5)4865worker1 # [ 115.245534] nix-grpc-daemon[488]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 3/5)4866worker1 # [ 115.317286] nix-grpc-daemon[488]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 599 ms (attempt 4/5)4867worker1 # [ 115.919867] nix-grpc-daemon[488]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4868worker1 # [ 115.928375] nix-grpc-daemon[488]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4869worker1 # [ 115.935487] nix-grpc-daemon[488]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4870worker1 # [ 116.098107] nix-daemon[855]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)4871worker1 # [ 116.118911] nix-daemon[855]: reaped child process 1284, status = succeeded4872lb # [ 116.094595] niks3-server[1008]: 2026/09/24 09:27:30 INFO Received push request method=POST path=/api/pushes4873worker1 # [ 116.135284] nix-grpc-daemon[867]: time=2026-09-24T09:27:30.767Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4874worker1 # [ 116.136826] nix-grpc-daemon[867]: time=2026-09-24T09:27:30.767Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4875lb # [ 116.120947] niks3-server[1008]: 2026/09/24 09:27:30 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign4876worker1 # [ 116.160573] nix-grpc-daemon[867]: time=2026-09-24T09:27:30.792Z level=INFO msg="Uploading 1 narinfos"4877lb # [ 116.132085] niks3-server[1008]: 2026/09/24 09:27:30 INFO Signed narinfos id=30 count=14878lb # [ 116.141111] niks3-server[1008]: 2026/09/24 09:27:30 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4879lb # [ 116.154025] niks3-server[1008]: 2026/09/24 09:27:30 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4880lb # [ 116.165486] niks3-server[1008]: 2026/09/24 09:27:30 INFO Received complete push request method=POST path=/api/pushes/30/complete4881worker1 # [ 116.205982] nix-grpc-daemon[867]: time=2026-09-24T09:27:30.838Z level=INFO msg="Upload complete. (257ms)"4882lb # [ 116.176875] niks3-server[1008]: 2026/09/24 09:27:30 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4883worker1 # [ 116.215439] nix-grpc-daemon[488]: ts=2026-09-24T09:27:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=1 paths=1 nar_bytes_in=5684884worker1 # [ 116.231588] nix-grpc-daemon[488]: ts=2026-09-24T09:27:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:396824885worker1 # [ 116.245387] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4886worker1 # [ 116.253072] nix-daemon[855]: reaped child process 1274, status = succeeded4887client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4888worker1 # [ 116.269134] nix-daemon[855]: accepted connection from pid 488, user nix-grpc-daemon (trusted)4889client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4890client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4891worker1 # [ 117.726368] nix-daemon[855]: reaped child process 1288, status = succeeded4892worker1 # [ 117.846210] nix-daemon[855]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)4893worker1 # [ 117.861552] nix-daemon[855]: reaped child process 1307, status = succeeded4894lb # [ 117.838120] niks3-server[1008]: 2026/09/24 09:27:32 INFO Received push request method=POST path=/api/pushes4895worker1 # [ 117.880431] nix-grpc-daemon[867]: time=2026-09-24T09:27:32.510Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4896lb # [ 117.867141] niks3-server[1008]: 2026/09/24 09:27:32 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4897worker1 # [ 117.909756] nix-grpc-daemon[867]: time=2026-09-24T09:27:32.541Z level=INFO msg="Uploading 1 narinfos"4898lb # [ 117.880244] niks3-server[1008]: 2026/09/24 09:27:32 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign4899lb # [ 117.890983] niks3-server[1008]: 2026/09/24 09:27:32 INFO Signed narinfos id=31 count=14900lb # [ 117.896666] niks3-server[1008]: 2026/09/24 09:27:32 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4901lb # [ 117.907940] niks3-server[1008]: 2026/09/24 09:27:32 INFO Received complete push request method=POST path=/api/pushes/31/complete4902worker1 # [ 117.945975] nix-grpc-daemon[867]: time=2026-09-24T09:27:32.578Z level=INFO msg="Upload complete. (264ms)"4903lb # [ 117.916201] niks3-server[1008]: 2026/09/24 09:27:32 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4904client # copying 1 paths...4905worker1 # [ 117.958825] nix-grpc-daemon[488]: ts=2026-09-24T09:27:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39682 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04906worker2 # [ 117.962292] nix-grpc-daemon[783]: ts=2026-09-24T09:27:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:359124907worker1 # [ 117.978364] nix-daemon[855]: reaped child process 1286, status = succeeded4908worker2 # [ 117.974068] nix-grpc-daemon[783]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4909client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4910worker2 # [ 118.016398] nix-grpc-daemon[783]: ts=2026-09-24T09:27:32Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:359124911worker2 # [ 118.023315] nix-grpc-daemon[783]: ts=2026-09-24T09:27:32Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_out=1204912lb # [ 118.044137] envoy[898]: conn peer=[2001:db8:1::1]:47868 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3927 tx=2156 ms=20204913lb # [ 118.049023] envoy[898]: conn peer=[2001:db8:1::1]:47880 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=284914lb # [ 118.053747] envoy[898]: conn peer=[2001:db8:1::1]:47882 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4378 tx=2414 ms=22144915lb # [ 118.059272] envoy[898]: conn peer=[2001:db8:1::1]:55860 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=284916lb # [ 118.064167] envoy[898]: conn peer=[2001:db8:1::1]:55862 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4236 tx=2055 ms=18694917lb # [ 118.069164] envoy[898]: conn peer=[2001:db8:1::1]:55878 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=514918lb # [ 118.073807] envoy[898]: conn peer=[2001:db8:1::1]:55894 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=274919client # [ 118.145638] nix-daemon[1162]: reaped child process 1270, status = succeeded4920lb # [ 118.078526] envoy[898]: conn peer=[2001:db8:1::1]:55880 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=2122 ms=30594921client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.44 seconds)4922client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24923client # [ 118.309602] nix-daemon[1162]: accepted connection from pid 1289, user root (trusted)4924client # this derivation will be built:4925client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4926worker2 # [ 118.418053] nix-grpc-daemon[783]: ts=2026-09-24T09:27:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:359124927client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4928worker2 # [ 118.425570] nix-grpc-daemon[783]: ts=2026-09-24T09:27:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124929worker2 # [ 118.433030] nix-grpc-daemon[876]: time=2026-09-24T09:27:33.057Z level=INFO msg="All 1 paths already cached"4930worker1 # [ 118.447212] nix-grpc-daemon[488]: ts=2026-09-24T09:27:33Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04931worker2 # [ 118.447099] nix-grpc-daemon[783]: ts=2026-09-24T09:27:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124932worker1 # [ 118.454063] nix-grpc-daemon[488]: ts=2026-09-24T09:27:33Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=14933worker2 # [ 118.461423] nix-grpc-daemon[783]: ts=2026-09-24T09:27:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:359124934worker2 # [ 118.475014] nix-grpc-daemon[876]: time=2026-09-24T09:27:33.101Z level=INFO msg="All 1 paths already cached"4935client # copying 1 paths to 'lb:50051'...4936client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4937worker2 # [ 118.485187] nix-grpc-daemon[783]: ts=2026-09-24T09:27:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:359124938worker2 # [ 118.501532] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4939worker2 # [ 118.655842] nix-daemon[842]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4940worker2 # [ 118.676645] nix-daemon[842]: reaped child process 1118, status = succeeded4941lb # [ 118.651659] niks3-server[1008]: 2026/09/24 09:27:33 INFO Received push request method=POST path=/api/pushes4942worker2 # [ 118.687699] nix-grpc-daemon[876]: time=2026-09-24T09:27:33.314Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4943worker2 # [ 118.689885] nix-grpc-daemon[876]: time=2026-09-24T09:27:33.315Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4944lb # [ 118.687715] niks3-server[1008]: 2026/09/24 09:27:33 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign4945worker2 # [ 118.723674] nix-grpc-daemon[876]: time=2026-09-24T09:27:33.349Z level=INFO msg="Uploading 1 narinfos"4946lb # [ 118.700244] niks3-server[1008]: 2026/09/24 09:27:33 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4947lb # [ 118.711493] niks3-server[1008]: 2026/09/24 09:27:33 INFO Signed narinfos id=32 count=14948lb # [ 118.721340] niks3-server[1008]: 2026/09/24 09:27:33 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4949lb # [ 118.727389] niks3-server[1008]: 2026/09/24 09:27:33 INFO Received complete push request method=POST path=/api/pushes/32/complete4950lb # [ 118.731683] niks3-server[1008]: 2026/09/24 09:27:33 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4951worker2 # [ 118.759756] nix-grpc-daemon[876]: time=2026-09-24T09:27:33.386Z level=INFO msg="Upload complete. (243ms)"4952worker2 # [ 118.769420] nix-grpc-daemon[783]: ts=2026-09-24T09:27:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_in=5684953worker2 # [ 118.784805] nix-grpc-daemon[783]: ts=2026-09-24T09:27:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:359124954worker2 # [ 118.797620] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4955worker2 # [ 118.801229] nix-daemon[842]: reaped child process 1113, status = succeeded4956worker2 # [ 118.812747] nix-daemon[842]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4957client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4958client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4959client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4960worker2 # [ 119.663168] nix-daemon[842]: reaped child process 1122, status = succeeded4961worker2 # [ 119.775063] nix-daemon[842]: accepted connection from pid 1138, user nix-grpc-daemon (trusted)4962worker2 # [ 119.792579] nix-daemon[842]: reaped child process 1141, status = succeeded4963lb # [ 119.771422] niks3-server[1008]: 2026/09/24 09:27:34 INFO Received push request method=POST path=/api/pushes4964worker2 # [ 119.814370] nix-grpc-daemon[876]: time=2026-09-24T09:27:34.439Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4965lb # [ 119.809345] niks3-server[1008]: 2026/09/24 09:27:34 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign4966worker2 # [ 119.846782] nix-grpc-daemon[876]: time=2026-09-24T09:27:34.473Z level=INFO msg="Uploading 1 narinfos"4967lb # [ 119.823076] niks3-server[1008]: 2026/09/24 09:27:34 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4968lb # [ 119.831479] niks3-server[1008]: 2026/09/24 09:27:34 INFO Signed narinfos id=33 count=14969lb # [ 119.839215] niks3-server[1008]: 2026/09/24 09:27:34 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4970worker2 # [ 119.873961] nix-grpc-daemon[876]: time=2026-09-24T09:27:34.500Z level=INFO msg="Upload complete. (249ms)"4971lb # [ 119.847804] niks3-server[1008]: 2026/09/24 09:27:34 INFO Received complete push request method=POST path=/api/pushes/33/complete4972lb # [ 119.851052] niks3-server[1008]: 2026/09/24 09:27:34 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4973client # copying 1 paths...4974worker2 # [ 119.885082] nix-grpc-daemon[783]: ts=2026-09-24T09:27:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04975worker2 # [ 119.904585] nix-grpc-daemon[783]: ts=2026-09-24T09:27:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:359124976worker2 # [ 119.909711] nix-daemon[842]: reaped child process 1120, status = succeeded4977client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4978worker2 # [ 119.936447] nix-grpc-daemon[783]: ts=2026-09-24T09:27:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:359124979worker2 # [ 119.943390] nix-grpc-daemon[783]: ts=2026-09-24T09:27:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:35912 duration_s=0 paths=1 nar_bytes_out=1204980client # [ 120.065272] nix-daemon[1162]: reaped child process 1292, status = succeeded4981client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.92 seconds)4982(finished: subtest: an input only one worker has still reaches the builder, in 8.27 seconds)4983subtest: upload whose reference the worker lacks is completed from the cache4984client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4985client # this derivation will be built:4986client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4987client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4988client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.31 seconds)4989client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4990client # this derivation will be built:4991client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4992client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4993client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)4994client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4995client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)4996worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4997worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)4998worker2 # [ 121.006081] systemd[1]: Stopping Nix worker-protocol over gRPC...4999worker2 # [ 121.355289] nix-grpc-daemon[783]: ts=2026-09-24T09:27:35Z level=info event=signal_exit5000worker1 # [ 123.344523] nix-grpc-daemon[488]: ts=2026-09-24T09:27:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:460225001worker2 # [ 123.348795] nix-grpc-daemon[783]: ts=2026-09-24T09:27:37Z level=info event=scheduler_restarting5002worker1 # [ 123.353398] nix-grpc-daemon[488]: ts=2026-09-24T09:27:37Z level=info event=worker_gone addr=192.168.1.4:500515003worker2 # [ 123.378531] nix-grpc-daemon[783]: ts=2026-09-24T09:27:38Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5004worker2 # [ 123.397150] nix-daemon[842]: reaped child process 844, status = succeeded5005worker2 # [ 123.511374] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5006worker2 # [ 123.519586] systemd[1]: Stopped Nix worker-protocol over gRPC.5007worker2 # [ 123.527992] systemd[1]: nix-grpc-daemon.service: Consumed 2.529s CPU time over 1min 29.384s wall clock time, 60.2M memory peak, 161.9K incoming IP traffic, 231.2K outgoing IP traffic.5008lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux5009lb: must succeed: curl -sf localhost:9901/clusters5010lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5011lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.08 seconds)5012client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25013worker1 # [ 123.969544] nix-grpc-daemon[488]: ts=2026-09-24T09:27:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:396825014worker1 # [ 123.979583] nix-grpc-daemon[488]: ts=2026-09-24T09:27:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:396825015client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5016client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.38 seconds)5017worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5018worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5019worker2 # [ 124.103361] systemd[1]: Starting scheduler-token.service...5020worker2 # [ 124.169507] systemd[1]: scheduler-token.service: Deactivated successfully.5021worker2 # [ 124.176381] systemd[1]: Finished scheduler-token.service.5022worker2 # [ 124.177791] systemd[1]: scheduler-token.service: Consumed 37ms CPU time over 69ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5023worker2 # [ 124.192672] systemd[1]: Starting Nix worker-protocol over gRPC...5024worker2 # [ 124.301572] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:38Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5025worker2 # [ 124.304258] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:38Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5026worker2 # [ 124.308410] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:38Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15027worker2 # [ 124.315229] nix-grpc-daemon[1165]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5028worker2 # [ 124.325588] nix-grpc-daemon[1165]: E0000 00:00:1790242058.939369 1165 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5029worker2 # [ 124.344096] nix-grpc-daemon[1165]: E0000 00:00:1790242058.941581 1165 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5030worker2 # [ 124.346444] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:38Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5031worker2 # [ 124.365713] nix-grpc-daemon[1165]: E0000 00:00:1790242058.957028 1205 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5032worker2 # [ 124.368759] systemd[1]: Started Nix worker-protocol over gRPC.5033worker2 # [ 124.369648] nix-grpc-daemon[1165]: E0000 00:00:1790242058.976578 1205 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5034worker2 # [ 124.371916] nix-grpc-daemon[1165]: E0000 00:00:1790242058.976776 1205 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5035worker2 # [ 124.377639] nix-grpc-daemon[1165]: E0000 00:00:1790242058.991256 1205 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5036worker2 # [ 124.382158] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:39Z level=info event=scheduler_connected addr=lb:500515037worker1 # [ 124.395480] nix-grpc-daemon[488]: ts=2026-09-24T09:27:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:460225038worker1 # [ 124.420294] nix-grpc-daemon[488]: ts=2026-09-24T09:27:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05039worker1 # [ 124.427118] nix-grpc-daemon[488]: ts=2026-09-24T09:27:39Z level=info event=signal_exit5040worker1 # [ 124.430323] systemd[1]: Stopping Nix worker-protocol over gRPC...5041lb # [ 124.433424] niks3-server[1008]: 2026/09/24 09:27:39 INFO lead: released remote=192.168.1.3:561785042worker1 # [ 125.458486] nix-grpc-daemon[488]: ts=2026-09-24T09:27:40Z level=info event=scheduler_restarting5043worker2 # [ 125.458167] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_restarting addr=lb:500515044lb # [ 125.426759] envoy[898]: [2026-09-24 09:27:40.048][898][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5045worker1 # [ 125.464602] nix-grpc-daemon[488]: ts=2026-09-24T09:27:40Z level=info event=scheduler_restarting addr=lb:500515046worker1 # [ 125.474973] nix-grpc-daemon[488]: ts=2026-09-24T09:27:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:460225047worker1 # [ 125.480968] nix-grpc-daemon[488]: ts=2026-09-24T09:27:40Z level=info event=worker_gone addr=192.168.1.3:500515048worker1 # [ 125.485818] nix-grpc-daemon[488]: ts=2026-09-24T09:27:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:460225049worker1 # [ 125.490840] nix-grpc-daemon[488]: ts=2026-09-24T09:27:40Z level=info event=worker_gone addr=192.168.1.4:500515050worker1 # [ 125.520198] nix-daemon[855]: reaped child process 857, status = succeeded5051worker2 # [ 125.658537] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_connected addr=lb:500515052worker2 # [ 125.663934] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5053worker1 # [ 125.694365] nix-grpc-daemon[488]: ts=2026-09-24T09:27:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5054worker2 # [ 125.865059] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_connected addr=lb:500515055worker2 # [ 125.870246] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5056worker2 # [ 126.071245] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_connected addr=lb:500515057worker2 # [ 126.076554] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5058worker2 # [ 126.277594] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_connected addr=lb:500515059worker2 # [ 126.282700] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5060worker2 # [ 126.485165] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_connected addr=lb:500515061worker2 # [ 126.489259] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5062worker2 # [ 126.685809] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_connected addr=lb:500515063worker2 # [ 126.691299] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5064worker2 # [ 126.892353] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_connected addr=lb:500515065worker2 # [ 126.897374] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5066worker2 # [ 127.098250] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_connected addr=lb:500515067worker2 # [ 127.103404] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5068worker2 # [ 127.304433] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_connected addr=lb:500515069worker2 # [ 127.309362] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5070worker1 # [ 127.454467] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5071worker1 # [ 127.462889] systemd[1]: Stopped Nix worker-protocol over gRPC.5072worker1 # [ 127.471460] systemd[1]: nix-grpc-daemon.service: Consumed 4.032s CPU time over 1min 57.413s wall clock time, 68.3M memory peak, 465.4K incoming IP traffic, 377.3K outgoing IP traffic.5073lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5074lb: must succeed: curl -sf localhost:9901/clusters5075worker2 # [ 127.512071] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_connected addr=lb:500515076worker2 # [ 127.516149] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5077lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5078lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.07 seconds)5079client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25080worker2 # [ 127.712937] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_connected addr=lb:500515081worker2 # [ 127.717949] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5082worker2 # [ 127.786975] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540045083worker2 # [ 127.801702] nix-daemon[842]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)5084worker2 # [ 127.827393] nix-grpc-daemon[1165]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 94 ms (attempt 1/5)5085worker2 # [ 127.918878] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_connected addr=lb:500515086worker2 # [ 127.924009] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5087worker2 # [ 127.931245] nix-grpc-daemon[1165]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 2/5)5088lb # [ 128.044640] envoy[898]: conn peer=[2001:db8:1::1]:55904 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3768 tx=2126 ms=15415089lb # [ 128.049641] envoy[898]: conn peer=[2001:db8:1::1]:55908 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=285090lb # [ 128.054358] envoy[898]: conn peer=[2001:db8:1::4]:44352 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=20841 tx=1732 ms=889685091lb # [ 128.059491] envoy[898]: conn peer=[2001:db8:1::1]:45626 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=325092lb # [ 128.064439] envoy[898]: conn peer=192.168.1.3:34374 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4306 tx=1628 ms=892605093worker2 # [ 128.099021] nix-grpc-daemon[1165]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 187 ms (attempt 3/5)5094worker2 # [ 128.125009] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_connected addr=lb:500515095worker2 # [ 128.130100] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5096worker2 # [ 128.287994] nix-grpc-daemon[1165]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 571 ms (attempt 4/5)5097worker2 # [ 128.330995] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_connected addr=lb:500515098worker2 # [ 128.336219] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:42Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5099worker2 # [ 128.539453] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_connected addr=lb:500515100worker2 # [ 128.544008] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5101worker2 # [ 128.739704] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_connected addr=lb:500515102worker2 # [ 128.744866] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5103worker2 # [ 128.861314] nix-grpc-daemon[1165]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5104worker2 # [ 128.868115] nix-grpc-daemon[1165]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5105client # copying 2 paths...5106client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5107worker2 # [ 128.890941] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540045108client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5109worker2 # [ 128.914172] nix-daemon[842]: accepted connection from pid 1165, user nix-grpc-daemon (trusted)5110worker2 # [ 128.944082] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=niks3_push_started pid=12255111worker2 # [ 128.968525] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_connected addr=lb:500515112worker2 # [ 128.973139] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5113worker2 # [ 129.098846] nix-grpc-daemon[1225]: time=2026-09-24T09:27:43.724Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5114worker2 # [ 129.156171] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_connected addr=lb:500515115worker2 # [ 129.158063] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5116worker2 # [ 129.209406] nix-daemon[842]: accepted connection from pid 1238, user nix-grpc-daemon (trusted)5117worker2 # [ 129.228442] nix-daemon[842]: reaped child process 1241, status = succeeded5118lb # [ 129.205565] niks3-server[1008]: 2026/09/24 09:27:43 INFO Received push request method=POST path=/api/pushes5119worker2 # [ 129.244077] nix-grpc-daemon[1225]: time=2026-09-24T09:27:43.869Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5120worker2 # [ 129.245557] nix-grpc-daemon[1225]: time=2026-09-24T09:27:43.869Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5121worker2 # [ 129.250774] nix-grpc-daemon[1225]: time=2026-09-24T09:27:43.869Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5122lb # [ 129.266688] niks3-server[1008]: 2026/09/24 09:27:43 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5123worker2 # [ 129.306498] nix-grpc-daemon[1225]: time=2026-09-24T09:27:43.931Z level=INFO msg="Uploading 2 narinfos"5124lb # [ 129.281891] niks3-server[1008]: 2026/09/24 09:27:43 INFO Signed narinfos id=34 count=25125lb # [ 129.288759] niks3-server[1008]: 2026/09/24 09:27:43 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5126lb # [ 129.296247] niks3-server[1008]: 2026/09/24 09:27:43 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5127worker2 # [ 129.330233] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_take_over5128lb # [ 129.302998] niks3-server[1008]: 2026/09/24 09:27:43 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5129lb # [ 129.311170] niks3-server[1008]: 2026/09/24 09:27:43 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5130lb # [ 129.323725] niks3-server[1008]: 2026/09/24 09:27:43 INFO lead: acquired remote=192.168.1.4:488985131worker2 # [ 129.359151] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_connected addr=lb:500515132lb # [ 129.330446] niks3-server[1008]: 2026/09/24 09:27:43 INFO Received complete push request method=POST path=/api/pushes/34/complete5133worker2 # [ 129.365620] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:43Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5134lb # [ 129.340324] niks3-server[1008]: 2026/09/24 09:27:43 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5135lb # [ 129.348244] niks3-server[1008]: 2026/09/24 09:27:43 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5136worker2 # [ 129.392620] nix-grpc-daemon[1225]: time=2026-09-24T09:27:44.018Z level=INFO msg="Upload complete. (290ms)"5137worker2 # [ 129.404345] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54004 duration_s=0 paths=2 nar_bytes_in=3125138worker2 # [ 129.421265] nix-daemon[842]: reaped child process 1223, status = succeeded5139client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.88 seconds)5140worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5141worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5142worker1 # [ 129.548817] systemd[1]: Starting Nix worker-protocol over gRPC...5143worker2 # [ 129.568697] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=scheduler_connected addr=lb:500515144worker2 # [ 129.572781] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5145worker1 # [ 129.680435] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5146worker1 # [ 129.682897] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5147worker1 # [ 129.686917] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15148worker1 # [ 129.692066] nix-grpc-daemon[1332]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5149worker1 # [ 129.693515] nix-grpc-daemon[1332]: E0000 00:00:1790242064.324038 1332 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5150worker1 # [ 129.698174] nix-grpc-daemon[1332]: E0000 00:00:1790242064.330417 1332 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5151lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5152lb: must succeed: curl -sf localhost:9901/clusters5153worker1 # [ 129.740118] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5154worker1 # [ 129.741818] nix-grpc-daemon[1332]: E0000 00:00:1790242064.339768 1375 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5155worker1 # [ 129.746925] nix-grpc-daemon[1332]: E0000 00:00:1790242064.357983 1375 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5156worker1 # [ 129.749904] nix-grpc-daemon[1332]: E0000 00:00:1790242064.358129 1375 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5157worker1 # [ 129.752541] systemd[1]: Started Nix worker-protocol over gRPC.5158worker1 # [ 129.753364] nix-grpc-daemon[1332]: E0000 00:00:1790242064.378170 1375 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5159worker1 # [ 129.764335] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=scheduler_connected addr=lb:500515160worker1 # [ 129.765939] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5161worker2 # [ 129.769789] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=scheduler_connected addr=lb:500515162worker2 # [ 129.775037] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5163lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5164worker1 # [ 129.967017] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=scheduler_connected addr=lb:500515165worker1 # [ 129.972314] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5166worker2 # [ 129.975863] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=scheduler_connected addr=lb:500515167worker2 # [ 129.981005] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5168worker1 # [ 130.172877] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=scheduler_connected addr=lb:500515169worker1 # [ 130.177978] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5170worker2 # [ 130.181857] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=scheduler_connected addr=lb:500515171worker2 # [ 130.186872] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5172worker1 # [ 130.378695] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:45Z level=info event=scheduler_connected addr=lb:500515173worker2 # [ 130.387749] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:45Z level=info event=scheduler_connected addr=lb:500515174worker2 # [ 130.397106] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:540085175worker2 # [ 130.404236] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05176worker2 # [ 130.415655] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:45Z level=info event=oidc_jwks provider=mock keys=15177worker2 # [ 130.419866] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540085178worker2 # [ 130.426228] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05179lb: must succeed: curl -sf localhost:9901/clusters5180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5181lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 1.15 seconds)5182(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.83 seconds)5183subtest: a niks3 restart does not cost the leader its role5184lb: must succeed: curl -sf localhost:9901/clusters5185lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5186lb: must succeed: curl -sf localhost:9901/clusters5187lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5188lb # [ 130.966236] systemd[1]: Stopping niks3 server...5189lb # [ 130.967165] niks3-server[1008]: 2026/09/24 09:27:45 INFO Shutdown signal received, draining in-flight requests timeout=10s5190lb # [ 130.972230] niks3-server[1008]: 2026/09/24 09:27:45 INFO lead: released remote=192.168.1.4:488985191worker2 # [ 131.000788] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:45Z level=info event=scheduler_lock_lost status=2005192worker1 # [ 131.004230] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:45Z level=info event=scheduler_lock_lost status=2005193lb # [ 130.992391] systemd[1]: niks3.service: Deactivated successfully.5194lb # [ 130.994045] systemd[1]: Stopped niks3 server.5195lb # [ 130.994909] systemd[1]: niks3.service: Consumed 633ms CPU time over 1min 42.830s wall clock time, 27.7M memory peak, 95.7K incoming IP traffic, 138.2K outgoing IP traffic.5196lb # [ 131.005150] systemd[1]: Starting niks3 server...5197lb # [ 131.066530] niks3-server[1190]: 2026/09/24 09:27:45 goose: no migrations to run. current version: 202609231200005198lb # [ 131.072293] niks3-server[1190]: 2026/09/24 09:27:45 OK 1_commit_pending_closure.sql (7.2ms)5199lb # [ 131.075631] niks3-server[1190]: 2026/09/24 09:27:45 OK 2_object_stats_trigger.sql (3.29ms)5200lb # [ 131.078709] niks3-server[1190]: 2026/09/24 09:27:45 OK 3_commit_push.sql (2.76ms)5201lb # [ 131.079775] niks3-server[1190]: 2026/09/24 09:27:45 goose: up to current file version: 35202lb # [ 131.081444] niks3-server[1190]: 2026/09/24 09:27:45 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5203lb # [ 131.091722] niks3-server[1190]: 2026/09/24 09:27:45 INFO Read proxy enabled — serving cache objects from S35204lb # [ 131.096586] niks3-server[1190]: 2026/09/24 09:27:45 INFO Using socket-activated listener address=0.0.0.0:57515205lb # [ 131.098946] niks3-server[1190]: 2026/09/24 09:27:45 INFO systemd watchdog enabled interval=15s5206lb # [ 131.101698] niks3-server[1190]: 2026/09/24 09:27:45 INFO Starting HTTP server address=0.0.0.0:57515207lb # [ 131.103196] systemd[1]: Started niks3 server.5208lb: waiting for unit niks3.service5209lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5210lb: must succeed: curl -sf localhost:9901/clusters5211lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5212worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5213worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5214lb: must succeed: curl -sf localhost:9901/clusters5215lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5216lb: must succeed: curl -sf localhost:9901/clusters5217lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5218worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5219worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5220client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25221client # don't know how to build these paths:5222client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5223worker2 # [ 131.604425] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:540045224worker2 # [ 131.612730] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540045225worker2 # [ 131.638718] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:46Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05226worker2 # [ 131.645043] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:46Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=15227worker1 # [ 131.658796] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465228worker1 # [ 131.674311] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5229worker1 # [ 131.704686] nix-grpc-daemon[1332]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5230worker1 # [ 131.718125] nix-grpc-daemon[1332]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 32 ms (attempt 2/5)5231worker1 # [ 131.751935] nix-grpc-daemon[1332]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 260 ms (attempt 3/5)5232lb # [ 131.977831] niks3-server[1190]: 2026/09/24 09:27:46 INFO lead: acquired remote=192.168.1.4:349285233worker1 # [ 132.015319] nix-grpc-daemon[1332]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 639 ms (attempt 4/5)5234worker1 # [ 132.657219] nix-grpc-daemon[1332]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5235worker1 # [ 132.661767] nix-grpc-daemon[1332]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5236worker1 # [ 132.675834] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403465237client # copying 1 paths to 'lb:50051'...5238worker1 # [ 132.697252] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403465239client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5240worker1 # [ 132.717355] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5241worker1 # [ 132.746096] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:47Z level=info event=niks3_push_started pid=13945242worker1 # [ 132.896652] nix-grpc-daemon[1394]: time=2026-09-24T09:27:47.528Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5243worker1 # [ 133.023549] nix-daemon[855]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5244worker1 # [ 133.041344] nix-daemon[855]: reaped child process 1410, status = succeeded5245lb # [ 133.016569] niks3-server[1190]: 2026/09/24 09:27:47 INFO Received push request method=POST path=/api/pushes5246worker1 # [ 133.063213] nix-grpc-daemon[1394]: time=2026-09-24T09:27:47.694Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5247worker1 # [ 133.067132] nix-grpc-daemon[1394]: time=2026-09-24T09:27:47.694Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5248lb # [ 133.064088] niks3-server[1190]: 2026/09/24 09:27:47 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5249lb # [ 133.088659] niks3-server[1190]: 2026/09/24 09:27:47 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5250lb # [ 133.112409] niks3-server[1190]: 2026/09/24 09:27:47 INFO Signed narinfos id=35 count=15251worker1 # [ 133.147868] nix-grpc-daemon[1394]: time=2026-09-24T09:27:47.778Z level=INFO msg="Uploading 1 narinfos"5252lb # [ 133.124549] niks3-server[1190]: 2026/09/24 09:27:47 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5253lb # [ 133.136371] niks3-server[1190]: 2026/09/24 09:27:47 INFO Received complete push request method=POST path=/api/pushes/35/complete5254lb # [ 133.138541] niks3-server[1190]: 2026/09/24 09:27:47 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5255worker1 # [ 133.175729] nix-grpc-daemon[1394]: time=2026-09-24T09:27:47.807Z level=INFO msg="Upload complete. (275ms)"5256worker1 # [ 133.189171] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=0 paths=1 nar_bytes_in=5445257worker1 # [ 133.204670] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465258worker1 # [ 133.213923] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5259worker1 # [ 133.221476] nix-daemon[855]: reaped child process 1392, status = succeeded5260client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5261client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5262worker1 # [ 133.236231] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5263client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5264client # farm-a> LOG-farm-a-after-niks3-restart5265worker1 # [ 134.745920] nix-daemon[855]: reaped child process 1414, status = succeeded5266worker1 # [ 134.819316] nix-daemon[855]: accepted connection from pid 1430, user nix-grpc-daemon (trusted)5267worker1 # [ 134.835245] nix-daemon[855]: reaped child process 1433, status = succeeded5268lb # [ 134.811630] niks3-server[1190]: 2026/09/24 09:27:49 INFO Received push request method=POST path=/api/pushes5269worker1 # [ 134.856419] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.487Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5270worker1 # [ 134.858123] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.487Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5271lb # [ 134.850212] niks3-server[1190]: 2026/09/24 09:27:49 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5272lb # [ 134.865740] niks3-server[1190]: 2026/09/24 09:27:49 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5273lb # [ 134.877583] niks3-server[1190]: 2026/09/24 09:27:49 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5274worker1 # [ 134.913162] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.544Z level=INFO msg="Uploading 1 narinfos"5275lb # [ 134.885770] niks3-server[1190]: 2026/09/24 09:27:49 INFO Signed narinfos id=36 count=15276lb # [ 134.890536] niks3-server[1190]: 2026/09/24 09:27:49 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5277lb # [ 134.900390] niks3-server[1190]: 2026/09/24 09:27:49 INFO Received complete push request method=POST path=/api/pushes/36/complete5278worker1 # [ 134.936895] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.569Z level=INFO msg="Upload complete. (286ms)"5279lb # [ 134.907245] niks3-server[1190]: 2026/09/24 09:27:49 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5280worker2 # [ 134.953359] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:49Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05281worker1 # [ 134.952295] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05282worker2 # [ 134.959577] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:49Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=15283worker1 # [ 134.971997] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465284worker1 # [ 134.978578] nix-daemon[855]: reaped child process 1412, status = succeeded5285worker1 # [ 134.989045] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403465286worker1 # [ 135.002115] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.634Z level=INFO msg="All 1 paths already cached"5287client # copying 1 paths to 'lb:50051'...5288client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5289worker1 # [ 135.012895] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403465290worker1 # [ 135.029024] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5291worker1 # [ 135.183479] nix-daemon[855]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)5292worker1 # [ 135.200855] nix-daemon[855]: reaped child process 1440, status = succeeded5293lb # [ 135.174106] niks3-server[1190]: 2026/09/24 09:27:49 INFO Received push request method=POST path=/api/pushes5294worker1 # [ 135.214402] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.846Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5295worker1 # [ 135.215929] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.846Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5296lb # [ 135.204863] niks3-server[1190]: 2026/09/24 09:27:49 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5297worker1 # [ 135.248043] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.879Z level=INFO msg="Uploading 1 narinfos"5298lb # [ 135.220125] niks3-server[1190]: 2026/09/24 09:27:49 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5299lb # [ 135.229725] niks3-server[1190]: 2026/09/24 09:27:49 INFO Signed narinfos id=37 count=15300lb # [ 135.235324] niks3-server[1190]: 2026/09/24 09:27:49 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5301lb # [ 135.247348] niks3-server[1190]: 2026/09/24 09:27:49 INFO Received complete push request method=POST path=/api/pushes/37/complete5302lb # [ 135.252316] niks3-server[1190]: 2026/09/24 09:27:49 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5303worker1 # [ 135.284209] nix-grpc-daemon[1394]: time=2026-09-24T09:27:49.916Z level=INFO msg="Upload complete. (239ms)"5304worker1 # [ 135.296276] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=0 paths=1 nar_bytes_in=6565305worker1 # [ 135.308468] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465306worker1 # [ 135.316957] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5307worker1 # [ 135.324341] nix-daemon[855]: reaped child process 1435, status = succeeded5308client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5309client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5310worker1 # [ 135.337062] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5311client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5312client # farm-b> LOG-farm-b-after-niks3-restart5313worker1 # [ 136.362025] nix-daemon[855]: reaped child process 1444, status = succeeded5314worker1 # [ 136.456058] nix-daemon[855]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)5315worker1 # [ 136.476321] nix-daemon[855]: reaped child process 1463, status = succeeded5316lb # [ 136.443969] niks3-server[1190]: 2026/09/24 09:27:51 INFO Received push request method=POST path=/api/pushes5317worker1 # [ 136.487309] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.118Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5318worker1 # [ 136.489001] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.118Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5319worker1 # [ 136.527150] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.159Z level=INFO msg="Uploading 1 narinfos"5320lb # [ 136.484260] niks3-server[1190]: 2026/09/24 09:27:51 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5321lb # [ 136.502821] niks3-server[1190]: 2026/09/24 09:27:51 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5322lb # [ 136.513850] niks3-server[1190]: 2026/09/24 09:27:51 INFO Signed narinfos id=38 count=15323lb # [ 136.524253] niks3-server[1190]: 2026/09/24 09:27:51 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5324lb # [ 136.527582] niks3-server[1190]: 2026/09/24 09:27:51 INFO Received complete push request method=POST path=/api/pushes/38/complete5325worker1 # [ 136.566256] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.198Z level=INFO msg="Upload complete. (219ms)"5326lb # [ 136.536952] niks3-server[1190]: 2026/09/24 09:27:51 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5327lb # [ 136.541562] niks3-server[1190]: 2026/09/24 09:27:51 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5328worker1 # [ 136.577012] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05329worker2 # [ 136.582306] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:51Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05330worker2 # [ 136.588568] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:51Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=15331worker1 # [ 136.597293] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465332worker1 # [ 136.603364] nix-daemon[855]: reaped child process 1442, status = succeeded5333worker1 # [ 136.610462] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403465334worker1 # [ 136.622656] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.254Z level=INFO msg="All 2 paths already cached"5335client # copying 1 paths to 'lb:50051'...5336client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5337worker1 # [ 136.632332] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403465338worker1 # [ 136.648337] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5339worker1 # [ 136.794608] nix-daemon[855]: accepted connection from pid 1467, user nix-grpc-daemon (trusted)5340worker1 # [ 136.810973] nix-daemon[855]: reaped child process 1470, status = succeeded5341lb # [ 136.785682] niks3-server[1190]: 2026/09/24 09:27:51 INFO Received push request method=POST path=/api/pushes5342worker1 # [ 136.828615] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.460Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5343worker1 # [ 136.830160] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.460Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5344lb # [ 136.817518] niks3-server[1190]: 2026/09/24 09:27:51 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5345worker1 # [ 136.854738] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.486Z level=INFO msg="Uploading 1 narinfos"5346lb # [ 136.830022] niks3-server[1190]: 2026/09/24 09:27:51 INFO Signed narinfos id=39 count=15347lb # [ 136.840115] niks3-server[1190]: 2026/09/24 09:27:51 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5348lb # [ 136.849857] niks3-server[1190]: 2026/09/24 09:27:51 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5349lb # [ 136.864083] niks3-server[1190]: 2026/09/24 09:27:51 INFO Received complete push request method=POST path=/api/pushes/39/complete5350lb # [ 136.872563] niks3-server[1190]: 2026/09/24 09:27:51 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5351worker1 # [ 136.904475] nix-grpc-daemon[1394]: time=2026-09-24T09:27:51.536Z level=INFO msg="Upload complete. (241ms)"5352worker1 # [ 136.916383] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=0 paths=1 nar_bytes_in=7845353worker1 # [ 136.932731] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465354worker1 # [ 136.939083] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5355worker1 # [ 136.945372] nix-daemon[855]: reaped child process 1465, status = succeeded5356client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5357client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5358worker1 # [ 136.957908] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5359client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5360client # farm-top> LOG-farm-top-after-niks3-restart5361lb # [ 138.041986] envoy[898]: conn peer=[2001:db8:1::1]:45634 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1009 tx=282 ms=16325362worker2 # [ 138.090994] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:52Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:540385363worker1 # [ 138.225574] nix-daemon[855]: reaped child process 1474, status = succeeded5364worker1 # [ 138.344968] nix-daemon[855]: accepted connection from pid 1490, user nix-grpc-daemon (trusted)5365worker1 # [ 138.366191] nix-daemon[855]: reaped child process 1493, status = succeeded5366lb # [ 138.339330] niks3-server[1190]: 2026/09/24 09:27:52 INFO Received push request method=POST path=/api/pushes5367worker1 # [ 138.383404] nix-grpc-daemon[1394]: time=2026-09-24T09:27:53.013Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5368worker1 # [ 138.385432] nix-grpc-daemon[1394]: time=2026-09-24T09:27:53.013Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5369lb # [ 138.377057] niks3-server[1190]: 2026/09/24 09:27:53 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5370worker1 # [ 138.424717] nix-grpc-daemon[1394]: time=2026-09-24T09:27:53.056Z level=INFO msg="Uploading 1 narinfos"5371lb # [ 138.393317] niks3-server[1190]: 2026/09/24 09:27:53 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5372lb # [ 138.405336] niks3-server[1190]: 2026/09/24 09:27:53 INFO Signed narinfos id=40 count=15373lb # [ 138.407571] niks3-server[1190]: 2026/09/24 09:27:53 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5374lb # [ 138.421586] niks3-server[1190]: 2026/09/24 09:27:53 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5375worker1 # [ 138.453131] nix-grpc-daemon[1394]: time=2026-09-24T09:27:53.085Z level=INFO msg="Upload complete. (284ms)"5376lb # [ 138.426686] niks3-server[1190]: 2026/09/24 09:27:53 INFO Received complete push request method=POST path=/api/pushes/40/complete5377lb # [ 138.430172] niks3-server[1190]: 2026/09/24 09:27:53 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5378worker1 # [ 138.465295] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05379worker1 # [ 138.487196] nix-daemon[855]: reaped child process 1472, status = succeeded5380client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 7.12 seconds)5381client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5382client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds)5383(finished: subtest: a niks3 restart does not cost the leader its role, in 7.83 seconds)5384subtest: scheduler down: the other node takes the lock and keeps it5385lb: must succeed: curl -sf localhost:9901/clusters5386lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5387lb: must succeed: curl -sf localhost:9901/clusters5388lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5389lb: must succeed: curl -sf localhost:9901/clusters5390lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5391lb: must succeed: curl -sf localhost:9901/clusters5392lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5393worker2 # [ 138.958266] systemd[1]: Stopping Nix worker-protocol over gRPC...5394worker2 # [ 139.325188] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:53Z level=info event=signal_exit5395lb # [ 139.989762] niks3-server[1190]: 2026/09/24 09:27:54 INFO lead: released remote=192.168.1.4:349285396worker2 # [ 141.016510] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:55Z level=info event=scheduler_restarting5397worker1 # [ 141.019372] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:55Z level=info event=scheduler_restarting addr=lb:500515398lb # [ 140.985787] envoy[898]: [2026-09-24 09:27:55.606][898][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5399worker2 # [ 141.020432] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:55Z level=info event=scheduler_restarting addr=lb:500515400worker2 # [ 141.029748] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540085401worker2 # [ 141.036680] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:55Z level=info event=worker_gone addr=192.168.1.4:500515402worker2 # [ 141.041215] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:55Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540085403worker2 # [ 141.046428] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:55Z level=info event=worker_gone addr=192.168.1.3:500515404worker2 # [ 141.072407] nix-daemon[842]: reaped child process 1217, status = succeeded5405worker1 # [ 141.218999] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:55Z level=info event=scheduler_connected addr=lb:500515406worker1 # [ 141.224715] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5407worker2 # [ 141.234405] nix-grpc-daemon[1165]: ts=2026-09-24T09:27:55Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5408worker2 # [ 141.288758] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5409worker2 # [ 141.295924] systemd[1]: Stopped Nix worker-protocol over gRPC.5410worker2 # [ 141.303097] systemd[1]: nix-grpc-daemon.service: Consumed 592ms CPU time over 17.108s wall clock time, 33.5M memory peak, 167.7K incoming IP traffic, 84.1K outgoing IP traffic.5411worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5412worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5413worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5414worker1 # [ 141.425124] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_connected addr=lb:500515415worker1 # [ 141.430859] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5416worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5417worker1 # [ 141.629560] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_connected addr=lb:500515418worker1 # [ 141.634815] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5419worker1 # [ 141.835482] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_connected addr=lb:500515420worker1 # [ 141.840925] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5421lb # [ 141.979519] niks3-server[1190]: 2026/09/24 09:27:56 INFO lead: acquired remote=192.168.1.3:499545422worker1 # [ 142.015529] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_take_over5423worker1 # [ 142.043649] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_connected addr=lb:500515424worker1 # [ 142.047871] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5425worker1 # [ 142.245263] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_connected addr=lb:500515426worker1 # [ 142.249958] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5427worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5428worker1 # [ 142.450586] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=info event=scheduler_connected addr=lb:500515429worker1 # [ 142.473171] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:386445430worker1 # [ 142.483025] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05431worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5432client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25433worker1 # [ 142.620121] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:386505434worker1 # [ 142.754122] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:403465435client # don't know how to build these paths:5436client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5437worker1 # [ 142.763381] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403465438worker1 # [ 142.789717] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05439worker1 # [ 142.796946] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15440worker1 # [ 142.805883] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465441worker1 # [ 142.817087] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403465442client # copying 1 paths to 'lb:50051'...5443client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5444worker1 # [ 142.832344] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403465445worker1 # [ 142.847335] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5446worker1 # [ 142.996596] nix-daemon[855]: accepted connection from pid 1511, user nix-grpc-daemon (trusted)5447worker1 # [ 143.018355] nix-daemon[855]: reaped child process 1514, status = succeeded5448lb # [ 142.988813] niks3-server[1190]: 2026/09/24 09:27:57 INFO Received push request method=POST path=/api/pushes5449worker1 # [ 143.030237] nix-grpc-daemon[1394]: time=2026-09-24T09:27:57.661Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5450worker1 # [ 143.031784] nix-grpc-daemon[1394]: time=2026-09-24T09:27:57.661Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5451lb # [ 143.018176] niks3-server[1190]: 2026/09/24 09:27:57 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5452worker1 # [ 143.057734] nix-grpc-daemon[1394]: time=2026-09-24T09:27:57.689Z level=INFO msg="Uploading 1 narinfos"5453lb # [ 143.031171] niks3-server[1190]: 2026/09/24 09:27:57 INFO Signed narinfos id=41 count=15454lb # [ 143.040611] niks3-server[1190]: 2026/09/24 09:27:57 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5455lb # [ 143.051189] niks3-server[1190]: 2026/09/24 09:27:57 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5456lb # [ 143.064581] niks3-server[1190]: 2026/09/24 09:27:57 INFO Received complete push request method=POST path=/api/pushes/41/complete5457worker1 # [ 143.105171] nix-grpc-daemon[1394]: time=2026-09-24T09:27:57.737Z level=INFO msg="Upload complete. (242ms)"5458lb # [ 143.074284] niks3-server[1190]: 2026/09/24 09:27:57 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5459worker1 # [ 143.117067] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=0 paths=1 nar_bytes_in=5205460worker1 # [ 143.132555] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465461worker1 # [ 143.143486] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5462worker1 # [ 143.146891] nix-daemon[855]: reaped child process 1509, status = succeeded5463client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5464worker1 # [ 143.157057] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5465client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5466client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5467client # farm-a> LOG-farm-a-on-standby5468worker1 # [ 144.625066] nix-daemon[855]: reaped child process 1518, status = succeeded5469worker1 # [ 144.741387] nix-daemon[855]: accepted connection from pid 1534, user nix-grpc-daemon (trusted)5470worker1 # [ 144.760658] nix-daemon[855]: reaped child process 1537, status = succeeded5471lb # [ 144.729753] niks3-server[1190]: 2026/09/24 09:27:59 INFO Received push request method=POST path=/api/pushes5472worker1 # [ 144.770412] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.401Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5473worker1 # [ 144.771904] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.401Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5474worker1 # [ 144.809750] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.441Z level=INFO msg="Uploading 1 narinfos"5475lb # [ 144.771348] niks3-server[1190]: 2026/09/24 09:27:59 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5476lb # [ 144.791354] niks3-server[1190]: 2026/09/24 09:27:59 INFO Signed narinfos id=42 count=15477lb # [ 144.799526] niks3-server[1190]: 2026/09/24 09:27:59 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5478lb # [ 144.812799] niks3-server[1190]: 2026/09/24 09:27:59 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5479worker1 # [ 144.852812] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.485Z level=INFO msg="Upload complete. (283ms)"5480lb # [ 144.824241] niks3-server[1190]: 2026/09/24 09:27:59 INFO Received complete push request method=POST path=/api/pushes/42/complete5481lb # [ 144.826944] niks3-server[1190]: 2026/09/24 09:27:59 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5482lb # [ 144.846609] niks3-server[1190]: 2026/09/24 09:27:59 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5483worker1 # [ 144.871938] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05484worker1 # [ 144.894454] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:59Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05485worker1 # [ 144.900884] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:59Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15486worker1 # [ 144.907400] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465487worker1 # [ 144.913410] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403465488worker1 # [ 144.919003] nix-daemon[855]: reaped child process 1516, status = succeeded5489worker1 # [ 144.925121] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.557Z level=INFO msg="All 1 paths already cached"5490client # copying 1 paths to 'lb:50051'...5491client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5492worker1 # [ 144.934228] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403465493worker1 # [ 144.946137] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5494worker1 # [ 145.081398] nix-daemon[855]: accepted connection from pid 1543, user nix-grpc-daemon (trusted)5495worker1 # [ 145.098377] nix-daemon[855]: reaped child process 1546, status = succeeded5496lb # [ 145.073413] niks3-server[1190]: 2026/09/24 09:27:59 INFO Received push request method=POST path=/api/pushes5497worker1 # [ 145.116979] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.749Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5498worker1 # [ 145.118517] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.749Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5499lb # [ 145.110360] niks3-server[1190]: 2026/09/24 09:27:59 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5500worker1 # [ 145.151529] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.783Z level=INFO msg="Uploading 1 narinfos"5501lb # [ 145.124085] niks3-server[1190]: 2026/09/24 09:27:59 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5502lb # [ 145.133363] niks3-server[1190]: 2026/09/24 09:27:59 INFO Signed narinfos id=43 count=15503lb # [ 145.135933] niks3-server[1190]: 2026/09/24 09:27:59 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5504lb # [ 145.150348] niks3-server[1190]: 2026/09/24 09:27:59 INFO Received complete push request method=POST path=/api/pushes/43/complete5505worker1 # [ 145.182008] nix-grpc-daemon[1394]: time=2026-09-24T09:27:59.814Z level=INFO msg="Upload complete. (224ms)"5506lb # [ 145.153875] niks3-server[1190]: 2026/09/24 09:27:59 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5507worker1 # [ 145.191669] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=0 paths=1 nar_bytes_in=6405508worker1 # [ 145.208487] nix-grpc-daemon[1332]: ts=2026-09-24T09:27:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465509worker1 # [ 145.218335] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5510worker1 # [ 145.221611] nix-daemon[855]: reaped child process 1541, status = succeeded5511client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5512client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5513worker1 # [ 145.236826] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5514client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5515client # farm-b> LOG-farm-b-on-standby5516worker1 # [ 146.682375] nix-daemon[855]: reaped child process 1550, status = succeeded5517worker1 # [ 146.798690] nix-daemon[855]: accepted connection from pid 1566, user nix-grpc-daemon (trusted)5518worker1 # [ 146.819853] nix-daemon[855]: reaped child process 1569, status = succeeded5519lb # [ 146.793449] niks3-server[1190]: 2026/09/24 09:28:01 INFO Received push request method=POST path=/api/pushes5520worker1 # [ 146.836412] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.466Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5521worker1 # [ 146.838055] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.466Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5522lb # [ 146.839787] niks3-server[1190]: 2026/09/24 09:28:01 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5523worker1 # [ 146.879561] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.511Z level=INFO msg="Uploading 1 narinfos"5524lb # [ 146.852188] niks3-server[1190]: 2026/09/24 09:28:01 INFO Signed narinfos id=44 count=15525lb # [ 146.860436] niks3-server[1190]: 2026/09/24 09:28:01 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5526lb # [ 146.873003] niks3-server[1190]: 2026/09/24 09:28:01 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5527lb # [ 146.884746] niks3-server[1190]: 2026/09/24 09:28:01 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5528lb # [ 146.891322] niks3-server[1190]: 2026/09/24 09:28:01 INFO Received complete push request method=POST path=/api/pushes/44/complete5529worker1 # [ 146.923139] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.555Z level=INFO msg="Upload complete. (274ms)"5530lb # [ 146.894977] niks3-server[1190]: 2026/09/24 09:28:01 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5531worker1 # [ 146.941683] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05532worker1 # [ 146.964432] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:01Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05533worker1 # [ 146.970708] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:01Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15534worker1 # [ 146.977471] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465535worker1 # [ 146.983314] nix-daemon[855]: reaped child process 1548, status = succeeded5536worker1 # [ 146.985901] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403465537worker1 # [ 146.994585] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.626Z level=INFO msg="All 2 paths already cached"5538client # copying 1 paths to 'lb:50051'...5539client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5540worker1 # [ 147.004100] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403465541worker1 # [ 147.017168] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5542worker1 # [ 147.152977] nix-daemon[855]: accepted connection from pid 1575, user nix-grpc-daemon (trusted)5543worker1 # [ 147.170411] nix-daemon[855]: reaped child process 1578, status = succeeded5544lb # [ 147.145921] niks3-server[1190]: 2026/09/24 09:28:01 INFO Received push request method=POST path=/api/pushes5545worker1 # [ 147.186403] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.818Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5546worker1 # [ 147.187894] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.818Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5547lb # [ 147.178395] niks3-server[1190]: 2026/09/24 09:28:01 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5548lb # [ 147.190021] niks3-server[1190]: 2026/09/24 09:28:01 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5549worker1 # [ 147.228116] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.860Z level=INFO msg="Uploading 1 narinfos"5550lb # [ 147.202920] niks3-server[1190]: 2026/09/24 09:28:01 INFO Signed narinfos id=45 count=15551lb # [ 147.207620] niks3-server[1190]: 2026/09/24 09:28:01 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5552lb # [ 147.220863] niks3-server[1190]: 2026/09/24 09:28:01 INFO Received complete push request method=POST path=/api/pushes/45/complete5553worker1 # [ 147.258848] nix-grpc-daemon[1394]: time=2026-09-24T09:28:01.891Z level=INFO msg="Upload complete. (229ms)"5554lb # [ 147.228949] niks3-server[1190]: 2026/09/24 09:28:01 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5555worker1 # [ 147.268629] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=0 paths=1 nar_bytes_in=7685556worker1 # [ 147.284499] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465557worker1 # [ 147.293543] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5558worker1 # [ 147.300377] nix-daemon[855]: reaped child process 1573, status = succeeded5559client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5560worker1 # [ 147.309278] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5561client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5562lb # [ 148.048184] envoy[898]: conn peer=[2001:db8:1::1]:49700 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9710 tx=4470 ms=68785563lb # [ 148.048481] envoy[898]: conn peer=[2001:db8:1::4]:36254 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=19464 tx=1062 ms=168475564client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5565client # farm-top> LOG-farm-top-on-standby5566worker1 # [ 148.776678] nix-daemon[855]: reaped child process 1582, status = succeeded5567worker1 # [ 148.886355] nix-daemon[855]: accepted connection from pid 1598, user nix-grpc-daemon (trusted)5568worker1 # [ 148.904284] nix-daemon[855]: reaped child process 1601, status = succeeded5569lb # [ 148.881273] niks3-server[1190]: 2026/09/24 09:28:03 INFO Received push request method=POST path=/api/pushes5570worker1 # [ 148.923699] nix-grpc-daemon[1394]: time=2026-09-24T09:28:03.554Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5571worker1 # [ 148.925576] nix-grpc-daemon[1394]: time=2026-09-24T09:28:03.554Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5572lb # [ 148.916408] niks3-server[1190]: 2026/09/24 09:28:03 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5573lb # [ 148.931478] niks3-server[1190]: 2026/09/24 09:28:03 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5574worker1 # [ 148.971797] nix-grpc-daemon[1394]: time=2026-09-24T09:28:03.603Z level=INFO msg="Uploading 1 narinfos"5575lb # [ 148.943651] niks3-server[1190]: 2026/09/24 09:28:03 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign5576lb # [ 148.953069] niks3-server[1190]: 2026/09/24 09:28:03 INFO Signed narinfos id=46 count=15577lb # [ 148.959096] niks3-server[1190]: 2026/09/24 09:28:03 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5578lb # [ 148.971744] niks3-server[1190]: 2026/09/24 09:28:03 INFO Received complete push request method=POST path=/api/pushes/46/complete5579lb # [ 148.980123] niks3-server[1190]: 2026/09/24 09:28:03 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5580worker1 # [ 149.012115] nix-grpc-daemon[1394]: time=2026-09-24T09:28:03.644Z level=INFO msg="Upload complete. (292ms)"5581worker1 # [ 149.025091] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05582worker1 # [ 149.045364] nix-daemon[855]: reaped child process 1580, status = succeeded5583client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 6.53 seconds)5584client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5585client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5586worker2 # [ 149.317224] systemd[1]: Starting scheduler-token.service...5587worker2 # [ 149.398238] systemd[1]: scheduler-token.service: Deactivated successfully.5588worker2 # [ 149.403213] systemd[1]: Finished scheduler-token.service.5589worker2 # [ 149.409043] systemd[1]: scheduler-token.service: Consumed 43ms CPU time over 84ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5590worker2 # [ 149.415562] systemd[1]: Starting Nix worker-protocol over gRPC...5591worker2 # [ 149.535154] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5592worker2 # [ 149.537455] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5593worker2 # [ 149.541243] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15594worker2 # [ 149.546634] nix-grpc-daemon[1281]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5595worker2 # [ 149.549447] nix-grpc-daemon[1281]: E0000 00:00:1790242084.172233 1281 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5596worker2 # [ 149.551810] nix-grpc-daemon[1281]: E0000 00:00:1790242084.175737 1281 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5597lb: must succeed: curl -sf localhost:9901/clusters5598worker2 # [ 149.562513] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:04Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5599worker2 # [ 149.576300] systemd[1]: Started Nix worker-protocol over gRPC.5600worker2 # [ 149.577128] nix-grpc-daemon[1281]: E0000 00:00:1790242084.191676 1322 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5601worker2 # [ 149.600543] nix-grpc-daemon[1281]: E0000 00:00:1790242084.211397 1322 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5602worker2 # [ 149.603348] nix-grpc-daemon[1281]: E0000 00:00:1790242084.211558 1322 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5603worker2 # [ 149.606046] nix-grpc-daemon[1281]: E0000 00:00:1790242084.226124 1322 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5604worker2 # [ 149.615451] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:04Z level=info event=scheduler_connected addr=lb:500515605worker1 # [ 149.627969] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=info event=oidc_jwks provider=mock keys=15606worker1 # [ 149.635951] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:386445607worker1 # [ 149.642462] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05608lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5609worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5610worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5611lb: must succeed: curl -sf localhost:9901/clusters5612lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5613lb: must succeed: curl -sf localhost:9901/clusters5614lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5615worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5616worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5617(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.19 seconds)5618subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5619client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5620client # [ 149.996312] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5621client # Running as unit: rs.service; invocation ID: 915284c12a3d4b64a3f68794d9dfe2135622client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.08 seconds)5623worker1 # [ 150.161507] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:403465624client # [ 150.204124] nix[1415]: don't know how to build these paths:5625client # [ 150.205231] nix[1415]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5626worker1 # [ 150.180235] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05627worker1 # [ 150.187207] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15628worker1 # [ 150.196161] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465629worker1 # [ 150.207041] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403465630client # [ 150.252963] nix[1415]: copying 1 paths to 'lb:50051'...5631client # [ 150.257106] nix[1415]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5632worker1 # [ 150.222159] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403465633worker1 # [ 150.237209] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5634worker1 # [ 150.386850] nix-daemon[855]: accepted connection from pid 1614, user nix-grpc-daemon (trusted)5635worker1 # [ 150.404162] nix-daemon[855]: reaped child process 1617, status = succeeded5636lb # [ 150.378579] niks3-server[1190]: 2026/09/24 09:28:05 INFO Received push request method=POST path=/api/pushes5637worker1 # [ 150.419415] nix-grpc-daemon[1394]: time=2026-09-24T09:28:05.051Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5638worker1 # [ 150.421069] nix-grpc-daemon[1394]: time=2026-09-24T09:28:05.051Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5639lb # [ 150.415506] niks3-server[1190]: 2026/09/24 09:28:05 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign5640worker1 # [ 150.456152] nix-grpc-daemon[1394]: time=2026-09-24T09:28:05.088Z level=INFO msg="Uploading 1 narinfos"5641lb # [ 150.429029] niks3-server[1190]: 2026/09/24 09:28:05 INFO Signed narinfos id=47 count=15642lb # [ 150.431745] niks3-server[1190]: 2026/09/24 09:28:05 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5643lb # [ 150.444925] niks3-server[1190]: 2026/09/24 09:28:05 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5644lb # [ 150.456607] niks3-server[1190]: 2026/09/24 09:28:05 INFO Received complete push request method=POST path=/api/pushes/47/complete5645lb # [ 150.466449] niks3-server[1190]: 2026/09/24 09:28:05 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5646worker1 # [ 150.497697] nix-grpc-daemon[1394]: time=2026-09-24T09:28:05.129Z level=INFO msg="Upload complete. (245ms)"5647worker1 # [ 150.508633] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=0 paths=1 nar_bytes_in=4165648worker1 # [ 150.525940] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403465649worker1 # [ 150.536097] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5650worker1 # [ 150.546181] nix-daemon[855]: reaped child process 1612, status = succeeded5651client # [ 150.586564] nix[1415]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5652worker1 # [ 150.556169] nix-daemon[855]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5653lb: must succeed: curl -sf localhost:9901/clusters5654lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5655lb: must succeed: curl -sf localhost:9901/clusters5656lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5657lb: must succeed: curl -sf localhost:9901/clusters5658lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5659lb: must succeed: curl -sf localhost:9901/clusters5660lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5661worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5662worker2 # [ 152.766952] systemd-journald[270]: Received client request to rotate journal, rotating.5663worker2 # [ 152.785751] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/d43bc1fc91984435805700edcfb8e9e0.5664worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5665worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5666worker1 # [ 152.841413] systemd[1]: Stopping Nix worker-protocol over gRPC...5667worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5668client: must succeed: sleep 5.05669worker1 # [ 153.715887] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:08Z level=info event=draining5670client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5671lb # [ 158.047692] envoy[898]: conn peer=[2001:db8:1::1]:55560 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=4403 ms=62815672worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5673worker1 # [ 158.190571] nix-daemon[855]: reaped child process 1621, status = succeeded5674worker1 # [ 158.294028] nix-daemon[855]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)5675worker1 # [ 158.310970] nix-daemon[855]: reaped child process 1665, status = succeeded5676lb # [ 158.287694] niks3-server[1190]: 2026/09/24 09:28:12 INFO Received push request method=POST path=/api/pushes5677worker1 # [ 158.332382] nix-grpc-daemon[1394]: time=2026-09-24T09:28:12.962Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5678worker1 # [ 158.333846] nix-grpc-daemon[1394]: time=2026-09-24T09:28:12.962Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5679lb # [ 158.350650] niks3-server[1190]: 2026/09/24 09:28:12 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5680lb # [ 158.362721] niks3-server[1190]: 2026/09/24 09:28:12 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign5681worker1 # [ 158.401693] nix-grpc-daemon[1394]: time=2026-09-24T09:28:13.033Z level=INFO msg="Uploading 1 narinfos"5682lb # [ 158.373170] niks3-server[1190]: 2026/09/24 09:28:12 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5683lb # [ 158.383165] niks3-server[1190]: 2026/09/24 09:28:12 INFO Signed narinfos id=48 count=15684lb # [ 158.388504] niks3-server[1190]: 2026/09/24 09:28:12 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5685lb # [ 158.402748] niks3-server[1190]: 2026/09/24 09:28:13 INFO Received complete push request method=POST path=/api/pushes/48/complete5686worker1 # [ 158.440531] nix-grpc-daemon[1394]: time=2026-09-24T09:28:13.072Z level=INFO msg="Upload complete. (267ms)"5687lb # [ 158.410428] niks3-server[1190]: 2026/09/24 09:28:13 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5688worker1 # [ 158.453200] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40346 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05689worker1 # [ 158.476176] nix-daemon[855]: reaped child process 1619, status = succeeded5690client # [ 158.520336] systemd[1]: rs.service: Deactivated successfully.5691client # [ 158.525402] systemd[1]: rs.service: Consumed 190ms CPU time over 8.525s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.2K outgoing IP traffic.5692worker1 # [ 158.716653] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:13Z level=info event=signal_exit5693lb # [ 158.984214] niks3-server[1190]: 2026/09/24 09:28:13 INFO lead: released remote=192.168.1.3:499545694lb # [ 159.537105] niks3-server[1190]: 2026/09/24 09:28:14 INFO lead: acquired remote=192.168.1.4:478745695worker2 # [ 159.565688] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:14Z level=info event=scheduler_take_over5696worker1 # [ 160.018745] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:14Z level=info event=scheduler_restarting5697lb # [ 159.985782] envoy[898]: [2026-09-24 09:28:14.607][898][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5698worker2 # [ 160.024653] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:14Z level=info event=scheduler_restarting addr=lb:500515699worker1 # [ 160.029384] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:14Z level=info event=scheduler_restarting addr=lb:500515700worker1 # [ 160.039115] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386445701worker1 # [ 160.044348] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:14Z level=info event=worker_gone addr=192.168.1.4:500515702worker1 # [ 160.049071] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:386445703worker1 # [ 160.054608] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:14Z level=info event=worker_gone addr=192.168.1.3:500515704worker1 # [ 160.080162] nix-daemon[855]: reaped child process 1384, status = succeeded5705worker2 # [ 160.225847] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:14Z level=info event=scheduler_connected addr=lb:500515706worker2 # [ 160.231455] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:14Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5707worker1 # [ 160.257337] nix-grpc-daemon[1332]: ts=2026-09-24T09:28:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5708worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.18 seconds)5709client: waiting for success: ! systemctl is-active rs5710client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5711client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5712worker2 # [ 160.432433] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:15Z level=info event=scheduler_connected addr=lb:500515713client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5714client: must fail: journalctl -u rs | grep -q 'reconnecting'5715worker2 # [ 160.450111] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:15Z level=info event=oidc_jwks provider=mock keys=15716worker2 # [ 160.454622] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:15Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:501985717worker2 # [ 160.460070] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:15Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05718client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5719lb: must succeed: curl -sf localhost:9901/clusters5720worker2 # [ 160.472134] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:15Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:502025721lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5722worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5723worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5724lb: must succeed: curl -sf localhost:9901/clusters5725lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5726worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5727worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5728worker1 # [ 161.759866] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5729worker1 # [ 161.766114] systemd[1]: Stopped Nix worker-protocol over gRPC.5730worker1 # [ 161.771699] systemd[1]: nix-grpc-daemon.service: Consumed 2.355s CPU time over 32.218s wall clock time, 46.8M memory peak, 365.4K incoming IP traffic, 210.8K outgoing IP traffic.5731worker1 # [ 161.790213] systemd[1]: Starting Nix worker-protocol over gRPC...5732worker1 # [ 161.928410] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5733worker1 # [ 161.930716] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5734worker1 # [ 161.934436] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15735worker1 # [ 161.940070] nix-grpc-daemon[1673]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5736worker1 # [ 161.941534] nix-grpc-daemon[1673]: E0000 00:00:1790242096.571299 1673 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5737worker1 # [ 161.945267] nix-grpc-daemon[1673]: E0000 00:00:1790242096.577523 1673 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5738worker1 # [ 161.949833] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:16Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5739worker1 # [ 161.951567] systemd[1]: Started Nix worker-protocol over gRPC.5740worker1 # [ 161.959203] nix-grpc-daemon[1673]: E0000 00:00:1790242096.588975 1713 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5741worker1 # [ 161.984201] nix-grpc-daemon[1673]: E0000 00:00:1790242096.606716 1713 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5742worker1 # [ 161.986431] nix-grpc-daemon[1673]: E0000 00:00:1790242096.606878 1713 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5743worker1 # [ 161.994644] nix-grpc-daemon[1673]: E0000 00:00:1790242096.626085 1713 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5744worker1 # [ 162.007228] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:16Z level=info event=scheduler_connected addr=lb:500515745worker2 # [ 162.006012] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:501985746worker2 # [ 162.012388] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05747lb: must succeed: curl -sf localhost:9901/clusters5748lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5749worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5750worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5751(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.97 seconds)5752subtest: scheduler restart mid-build: build finishes, no double build5753client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5754client # [ 162.965985] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5755client # Running as unit: mid.service; invocation ID: 3e927c468bbe49faaee00724a344b9095756client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5757worker1 # [ 163.131340] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:560505758worker1 # [ 163.146017] nix-daemon[855]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5759client # [ 163.199080] nix[1451]: don't know how to build these paths:5760client # [ 163.200357] nix[1451]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5761worker2 # [ 163.176254] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:17Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05762worker2 # [ 163.181987] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:17Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15763worker2 # [ 163.203288] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543965764worker2 # [ 163.218364] nix-daemon[842]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5765worker2 # [ 163.241261] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 8 ms (attempt 1/5)5766worker2 # [ 163.256657] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 2/5)5767worker2 # [ 163.293454] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 95 ms (attempt 3/5)5768worker2 # [ 163.390313] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 766 ms (attempt 4/5)5769worker2 # [ 164.166281] nix-grpc-daemon[1281]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5770worker2 # [ 164.166591] nix-grpc-daemon[1281]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5771worker2 # [ 164.198053] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:543965772client # [ 164.252446] nix[1451]: copying 1 paths to 'lb:50051'...5773worker2 # [ 164.212892] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:543965774client # [ 164.252687] nix[1451]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5775worker2 # [ 164.231926] nix-daemon[842]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5776worker2 # [ 164.276290] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:18Z level=info event=niks3_push_started pid=14105777worker2 # [ 164.406906] nix-grpc-daemon[1410]: time=2026-09-24T09:28:19.032Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5778worker2 # [ 164.514819] nix-daemon[842]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5779worker2 # [ 164.534316] nix-daemon[842]: reaped child process 1426, status = succeeded5780lb # [ 164.505785] niks3-server[1190]: 2026/09/24 09:28:19 INFO Received push request method=POST path=/api/pushes5781worker2 # [ 164.546667] nix-grpc-daemon[1410]: time=2026-09-24T09:28:19.172Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5782worker2 # [ 164.550456] nix-grpc-daemon[1410]: time=2026-09-24T09:28:19.172Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5783lb # [ 164.547384] niks3-server[1190]: 2026/09/24 09:28:19 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5784worker2 # [ 164.585526] nix-grpc-daemon[1410]: time=2026-09-24T09:28:19.211Z level=INFO msg="Uploading 1 narinfos"5785lb # [ 164.558606] niks3-server[1190]: 2026/09/24 09:28:19 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign5786lb # [ 164.567517] niks3-server[1190]: 2026/09/24 09:28:19 INFO Signed narinfos id=49 count=15787lb # [ 164.572626] niks3-server[1190]: 2026/09/24 09:28:19 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5788lb # [ 164.585052] niks3-server[1190]: 2026/09/24 09:28:19 INFO Received complete push request method=POST path=/api/pushes/49/complete5789worker2 # [ 164.622668] nix-grpc-daemon[1410]: time=2026-09-24T09:28:19.249Z level=INFO msg="Upload complete. (212ms)"5790lb # [ 164.594293] niks3-server[1190]: 2026/09/24 09:28:19 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5791worker2 # [ 164.631027] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54396 duration_s=0 paths=1 nar_bytes_in=4245792worker2 # [ 164.643038] nix-grpc-daemon[1281]: ts=2026-09-24T09:28:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:543965793worker2 # [ 164.649294] nix-daemon[842]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5794worker2 # [ 164.653959] nix-daemon[842]: reaped child process 1408, status = succeeded5795worker2 # [ 164.664310] nix-daemon[842]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5796client # [ 164.701013] nix[1451]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5797lb: must succeed: curl -sf localhost:9901/clusters5798lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5799lb: must succeed: curl -sf localhost:9901/clusters5800lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5801worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5802worker2 # [ 166.852450] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1281 (nix-grpc-daemon) on client request.5803worker2 # [ 166.854022] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5804lb # [ 166.837764] niks3-server[1190]: 2026/09/24 09:28:21 INFO lead: released remote=192.168.1.4:478745805worker2 # [ 166.875959] systemd[1]: Starting scheduler-token.service...5806client # [ 166.911853] nix[1451]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5807worker1 # [ 166.876803] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:21Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5808lb # [ 166.845209] envoy[898]: [2026-09-24 09:28:21.467][898][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}5809worker2 # [ 166.880551] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5810worker2 # [ 166.882054] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5811client # [ 166.918091] nix[1451]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5812worker2 # [ 166.883296] systemd[1]: nix-grpc-daemon.service: Consumed 485ms CPU time over 17.462s wall clock time, 35M memory peak, 217.4K incoming IP traffic, 57.5K outgoing IP traffic.5813worker2 # [ 166.926842] nix-daemon[842]: reaped child process 1398, status = succeeded5814worker2 # [ 166.928327] nix-daemon[842]: reaped child process 1428, status = succeeded5815worker2 # [ 166.936586] nix-daemon[1430]: unexpected Nix daemon error: error: interrupted by the user5816worker2 # [ 166.944297] nix-daemon[842]: reaped child process 1430, status = failed with exit code 15817lb # [ 166.929032] niks3-server[1190]: 2026/09/24 09:28:21 INFO lead: acquired remote=192.168.1.3:480005818worker1 # [ 166.960960] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:21Z level=info event=scheduler_take_over5819worker2 # [ 166.969939] systemd[1]: scheduler-token.service: Deactivated successfully.5820worker2 # [ 166.970905] systemd[1]: Finished scheduler-token.service.5821worker2 # [ 166.974618] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 94ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5822worker2 # [ 166.980373] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5823worker2 # [ 166.981840] systemd[1]: Starting scheduler-token.service...5824worker2 # [ 167.025188] systemd[1]: scheduler-token.service: Deactivated successfully.5825worker2 # [ 167.029241] systemd[1]: Finished scheduler-token.service.5826worker2 # [ 167.030031] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 49ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5827worker2 # [ 167.033160] systemd[1]: Starting Nix worker-protocol over gRPC...5828worker1 # [ 167.076771] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:21Z level=info event=scheduler_connected addr=lb:500515829worker2 # [ 167.129161] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:21Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5830worker2 # [ 167.131342] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:21Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5831worker2 # [ 167.135118] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:21Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15832worker2 # [ 167.140179] nix-grpc-daemon[1473]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5833worker1 # [ 167.145183] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:21Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. retried and the latest reset reason: remote connection failure"5834worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.34 seconds)5835client: must succeed: sleep 5.05836worker2 # [ 167.152484] nix-grpc-daemon[1473]: E0000 00:00:1790242101.766111 1473 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5837worker2 # [ 167.180144] nix-grpc-daemon[1473]: E0000 00:00:1790242101.771052 1473 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5838worker2 # [ 167.182404] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:21Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5839worker2 # [ 167.195753] nix-grpc-daemon[1473]: E0000 00:00:1790242101.780160 1514 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5840worker2 # [ 167.198699] nix-grpc-daemon[1473]: E0000 00:00:1790242101.803003 1514 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5841worker2 # [ 167.201190] nix-grpc-daemon[1473]: E0000 00:00:1790242101.803137 1514 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5842worker2 # [ 167.206193] systemd[1]: Started Nix worker-protocol over gRPC.5843worker2 # [ 167.208800] nix-grpc-daemon[1473]: E0000 00:00:1790242101.821554 1514 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5844worker2 # [ 167.211866] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:21Z level=info event=scheduler_connected addr=lb:500515845worker2 # [ 167.214939] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5846worker1 # [ 167.346047] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:21Z level=info event=scheduler_connected addr=lb:500515847worker1 # [ 167.351661] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:21Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5848client # [ 167.416453] nix[1451]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting5849worker2 # [ 167.416235] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:22Z level=info event=scheduler_connected addr=lb:500515850worker2 # [ 167.425263] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5851worker1 # [ 167.552320] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:22Z level=info event=scheduler_connected addr=lb:500515852worker1 # [ 167.569786] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:560565853worker1 # [ 167.575730] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05854worker2 # [ 167.626315] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:22Z level=info event=scheduler_connected addr=lb:500515855worker2 # [ 167.632822] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:22Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5856worker2 # [ 167.833853] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:22Z level=info event=scheduler_connected addr=lb:500515857worker1 # [ 167.846851] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:22Z level=info event=oidc_jwks provider=mock keys=15858worker1 # [ 167.851353] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:22Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:560565859worker1 # [ 167.856764] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:22Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05860lb # [ 168.047109] envoy[898]: conn peer=[2001:db8:1::1]:55568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2949 tx=1498 ms=83075861lb # [ 168.052202] envoy[898]: conn peer=[2001:db8:1::3]:55990 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3294 tx=1220 ms=304805862lb # [ 168.057108] envoy[898]: conn peer=[2001:db8:1::4]:48256 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2722 tx=393 ms=172675863client # [ 168.422840] nix[1451]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting5864worker1 # [ 168.429287] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:23Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:560625865worker1 # [ 170.400504] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:25Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05866worker1 # [ 170.410563] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:25Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15867worker1 # [ 170.421825] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560505868worker1 # [ 170.429714] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 18 ms (attempt 1/5)5869worker1 # [ 170.456871] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 121 ms (attempt 2/5)5870worker1 # [ 170.579665] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 3/5)5871worker1 # [ 170.591309] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 372 ms (attempt 4/5)5872worker1 # [ 170.973212] nix-grpc-daemon[1673]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5873worker1 # [ 170.979262] nix-grpc-daemon[1673]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5874worker1 # [ 170.995244] nix-grpc-daemon[1673]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...5875worker1 # [ 171.024792] nix-daemon[855]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5876client # [ 171.078851] nix[1451]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5877worker1 # [ 171.047024] nix-daemon[855]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5878client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5879client: waiting for success: ! systemctl is-active mid5880worker1 # [ 172.470884] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:27Z level=info event=niks3_push_started pid=17815881worker1 # [ 172.511427] nix-daemon[855]: reaped child process 1756, status = succeeded5882worker1 # [ 172.616718] nix-grpc-daemon[1781]: time=2026-09-24T09:28:27.248Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5883worker1 # [ 172.726597] nix-daemon[855]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)5884worker1 # [ 172.745314] nix-daemon[855]: reaped child process 1796, status = succeeded5885lb # [ 172.721850] niks3-server[1190]: 2026/09/24 09:28:27 INFO Received push request method=POST path=/api/pushes5886worker1 # [ 172.764083] nix-grpc-daemon[1781]: time=2026-09-24T09:28:27.394Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5887worker1 # [ 172.765596] nix-grpc-daemon[1781]: time=2026-09-24T09:28:27.394Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5888lb # [ 172.773307] niks3-server[1190]: 2026/09/24 09:28:27 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign5889lb # [ 172.792756] niks3-server[1190]: 2026/09/24 09:28:27 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5890lb # [ 172.805254] niks3-server[1190]: 2026/09/24 09:28:27 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5891worker1 # [ 172.849273] nix-grpc-daemon[1781]: time=2026-09-24T09:28:27.480Z level=INFO msg="Uploading 1 narinfos"5892lb # [ 172.819191] niks3-server[1190]: 2026/09/24 09:28:27 INFO Signed narinfos id=50 count=15893lb # [ 172.833388] niks3-server[1190]: 2026/09/24 09:28:27 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5894lb # [ 172.842994] niks3-server[1190]: 2026/09/24 09:28:27 INFO Received complete push request method=POST path=/api/pushes/50/complete5895worker1 # [ 172.890005] nix-grpc-daemon[1781]: time=2026-09-24T09:28:27.522Z level=INFO msg="Upload complete. (269ms)"5896lb # [ 172.862559] niks3-server[1190]: 2026/09/24 09:28:27 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5897worker1 # [ 172.908374] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56050 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05898worker1 # [ 172.926795] nix-daemon[855]: reaped child process 1754, status = succeeded5899client # [ 172.973008] systemd[1]: mid.service: Deactivated successfully.5900client # [ 172.977711] systemd[1]: mid.service: Consumed 205ms CPU time over 10.011s wall clock time, 41.4M memory peak, 9.3K incoming IP traffic, 11.2K outgoing IP traffic.5901client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)5902client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5903client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)5904(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.77 seconds)5905subtest: interrupting the client stops the build on the worker5906client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5907client # [ 173.708130] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5908client # Running as unit: intr.service; invocation ID: 82d5a58f8aff4ee5949d3b882d57015d5909client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5910worker2 # [ 173.846476] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362565911worker2 # [ 173.855611] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5912client # [ 173.908668] nix[1487]: don't know how to build these paths:5913client # [ 173.909950] nix[1487]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5914worker1 # [ 173.885263] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:28Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05915worker1 # [ 173.891400] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:28Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15916worker2 # [ 173.891264] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362565917worker2 # [ 173.906506] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)5918worker2 # [ 173.922755] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 2/5)5919worker2 # [ 173.976853] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 3/5)5920worker2 # [ 174.149230] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 642 ms (attempt 4/5)5921worker2 # [ 174.801032] nix-grpc-daemon[1473]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5922worker2 # [ 174.801329] nix-grpc-daemon[1473]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5923worker2 # [ 174.824910] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362565924client # [ 174.876372] nix[1487]: copying 1 paths to 'lb:50051'...5925client # [ 174.880845] nix[1487]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5926worker2 # [ 174.844367] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362565927worker2 # [ 174.859437] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5928worker2 # [ 174.891405] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:29Z level=info event=niks3_push_started pid=15455929worker2 # [ 175.057473] nix-grpc-daemon[1545]: time=2026-09-24T09:28:29.681Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5930worker2 # [ 175.165434] nix-daemon[842]: accepted connection from pid 1562, user nix-grpc-daemon (trusted)5931worker2 # [ 175.182062] nix-daemon[842]: reaped child process 1565, status = succeeded5932lb # [ 175.162925] niks3-server[1190]: 2026/09/24 09:28:29 INFO Received push request method=POST path=/api/pushes5933worker2 # [ 175.204064] nix-grpc-daemon[1545]: time=2026-09-24T09:28:29.829Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5934worker2 # [ 175.205502] nix-grpc-daemon[1545]: time=2026-09-24T09:28:29.830Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5935lb # [ 175.205926] niks3-server[1190]: 2026/09/24 09:28:29 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5936worker2 # [ 175.245195] nix-grpc-daemon[1545]: time=2026-09-24T09:28:29.870Z level=INFO msg="Uploading 1 narinfos"5937lb # [ 175.216998] niks3-server[1190]: 2026/09/24 09:28:29 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign5938lb # [ 175.223594] niks3-server[1190]: 2026/09/24 09:28:29 INFO Signed narinfos id=51 count=15939lb # [ 175.232374] niks3-server[1190]: 2026/09/24 09:28:29 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5940lb # [ 175.248445] niks3-server[1190]: 2026/09/24 09:28:29 INFO Received complete push request method=POST path=/api/pushes/51/complete5941lb # [ 175.258492] niks3-server[1190]: 2026/09/24 09:28:29 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5942worker2 # [ 175.292822] nix-grpc-daemon[1545]: time=2026-09-24T09:28:29.919Z level=INFO msg="Upload complete. (236ms)"5943worker2 # [ 175.303211] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=4245944worker2 # [ 175.316889] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362565945worker2 # [ 175.328243] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5946worker2 # [ 175.333390] nix-daemon[842]: reaped child process 1543, status = succeeded5947client # [ 175.382580] nix[1487]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5948worker2 # [ 175.346337] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)5949client: must succeed: systemctl kill -s INT intr5950client # [ 177.450982] systemd[1]: intr.service: Sent signal SIGINT to main process 1487 (nix) on client request.5951worker2 # [ 177.415078] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:32Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5952client # [ 177.457494] nix[1487]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5953client # [ 177.470778] nix[1487]: error: interrupted by the user5954client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)5955client # [ 177.504122] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5956client # [ 177.510664] systemd[1]: intr.service: Failed with result 'exit-code'.5957client # [ 177.513949] systemd[1]: intr.service: Consumed 168ms CPU time over 3.800s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.5958worker2 # [ 177.571561] nix-daemon[1569]: unexpected Nix daemon error: error: interrupted by the user5959worker2 # [ 177.579114] nix-daemon[842]: reaped child process 1567, status = succeeded5960worker2 # [ 177.583674] nix-daemon[842]: reaped child process 1569, status = failed with exit code 15961lb # [ 178.051467] envoy[898]: conn peer=[2001:db8:1::1]:55238 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4301 tx=2100 ms=97895962lb # [ 178.056596] envoy[898]: conn peer=[2001:db8:1::1]:60496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=829 ms=35995963(finished: subtest: interrupting the client stops the build on the worker, in 5.07 seconds)5964subtest: a deploy mid-build drains: the build finishes, the new generation takes over5965client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5966client # [ 178.807321] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5967client # Running as unit: sw.service; invocation ID: 70303ce0b3e94a02afb6baabe595d04e5968client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)5969client # [ 179.015221] nix[1508]: don't know how to build these paths:5970client # [ 179.016302] nix[1508]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5971worker2 # [ 178.976604] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362565972worker1 # [ 178.993385] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:33Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05973worker1 # [ 178.999576] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:33Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15974worker1 # [ 179.009672] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560505975worker1 # [ 179.023776] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560505976client # [ 179.070975] nix[1508]: copying 1 paths to 'lb:50051'...5977client # [ 179.075296] nix[1508]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5978worker1 # [ 179.040371] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:560505979worker1 # [ 179.055421] nix-daemon[855]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5980worker1 # [ 179.212644] nix-daemon[855]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)5981worker1 # [ 179.230749] nix-daemon[855]: reaped child process 1835, status = succeeded5982lb # [ 179.207194] niks3-server[1190]: 2026/09/24 09:28:33 INFO Received push request method=POST path=/api/pushes5983worker1 # [ 179.250022] nix-grpc-daemon[1781]: time=2026-09-24T09:28:33.881Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5984worker1 # [ 179.251629] nix-grpc-daemon[1781]: time=2026-09-24T09:28:33.881Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5985lb # [ 179.240983] niks3-server[1190]: 2026/09/24 09:28:33 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5986worker1 # [ 179.285383] nix-grpc-daemon[1781]: time=2026-09-24T09:28:33.916Z level=INFO msg="Uploading 1 narinfos"5987lb # [ 179.254740] niks3-server[1190]: 2026/09/24 09:28:33 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign5988lb # [ 179.266602] niks3-server[1190]: 2026/09/24 09:28:33 INFO Signed narinfos id=52 count=15989lb # [ 179.272341] niks3-server[1190]: 2026/09/24 09:28:33 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5990lb # [ 179.288228] niks3-server[1190]: 2026/09/24 09:28:33 INFO Received complete push request method=POST path=/api/pushes/52/complete5991lb # [ 179.294982] niks3-server[1190]: 2026/09/24 09:28:33 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5992worker1 # [ 179.325913] nix-grpc-daemon[1781]: time=2026-09-24T09:28:33.958Z level=INFO msg="Upload complete. (252ms)"5993worker1 # [ 179.336322] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56050 duration_s=0 paths=1 nar_bytes_in=4165994worker1 # [ 179.352721] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:560505995worker1 # [ 179.361912] nix-daemon[855]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5996worker1 # [ 179.367545] nix-daemon[855]: reaped child process 1830, status = succeeded5997worker1 # [ 179.381419] nix-daemon[855]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)5998client # [ 179.415090] nix[1508]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5999worker1: must succeed: systemctl reload nix-grpc-daemon.service6000worker1 # [ 181.384155] systemd[1]: Reloading Nix worker-protocol over gRPC...6001worker1 # [ 181.420336] systemd[1]: Reloaded Nix worker-protocol over gRPC.6002worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.09 seconds)6003worker1: must succeed: pgrep -f 'read -t [9]0 x'6004worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6005client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26006worker2 # [ 181.693113] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362566007client # don't know how to build these paths:6008client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6009worker1 # [ 181.708588] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:560506010worker1 # [ 181.726164] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:36Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06011worker1 # [ 181.728289] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:36Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16012worker2 # [ 181.727695] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566013worker2 # [ 181.742679] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566014client # copying 1 paths to 'lb:50051'...6015client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6016worker2 # [ 181.758474] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362566017worker2 # [ 181.774676] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6018worker2 # [ 181.935605] nix-daemon[842]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6019worker2 # [ 181.953718] nix-daemon[842]: reaped child process 1623, status = succeeded6020worker1 # [ 181.957217] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:36Z level=info event=draining6021lb # [ 181.933166] niks3-server[1190]: 2026/09/24 09:28:36 INFO Received push request method=POST path=/api/pushes6022worker2 # [ 181.973460] nix-grpc-daemon[1545]: time=2026-09-24T09:28:36.599Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6023worker2 # [ 181.975109] nix-grpc-daemon[1545]: time=2026-09-24T09:28:36.599Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6024lb # [ 181.968620] niks3-server[1190]: 2026/09/24 09:28:36 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6025worker2 # [ 182.005751] nix-grpc-daemon[1545]: time=2026-09-24T09:28:36.631Z level=INFO msg="Uploading 1 narinfos"6026lb # [ 181.982011] niks3-server[1190]: 2026/09/24 09:28:36 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6027lb # [ 181.993769] niks3-server[1190]: 2026/09/24 09:28:36 INFO Signed narinfos id=53 count=16028lb # [ 182.000820] niks3-server[1190]: 2026/09/24 09:28:36 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6029lb # [ 182.015947] niks3-server[1190]: 2026/09/24 09:28:36 INFO Received complete push request method=POST path=/api/pushes/53/complete6030worker2 # [ 182.050967] nix-grpc-daemon[1545]: time=2026-09-24T09:28:36.677Z level=INFO msg="Upload complete. (259ms)"6031lb # [ 182.023890] niks3-server[1190]: 2026/09/24 09:28:36 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6032worker2 # [ 182.064287] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=5286033worker2 # [ 182.078627] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566034worker2 # [ 182.089307] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6035worker2 # [ 182.097814] nix-daemon[842]: reaped child process 1618, status = succeeded6036client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6037worker2 # [ 182.106341] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6038client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6039client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6040client # farm-a> LOG-farm-a-during-drain6041worker2 # [ 182.952756] nix-daemon[842]: reaped child process 1627, status = succeeded6042worker2 # [ 183.071688] nix-daemon[842]: accepted connection from pid 1643, user nix-grpc-daemon (trusted)6043worker2 # [ 183.090130] nix-daemon[842]: reaped child process 1646, status = succeeded6044lb # [ 183.067317] niks3-server[1190]: 2026/09/24 09:28:37 INFO Received push request method=POST path=/api/pushes6045worker2 # [ 183.108390] nix-grpc-daemon[1545]: time=2026-09-24T09:28:37.732Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6046worker2 # [ 183.109907] nix-grpc-daemon[1545]: time=2026-09-24T09:28:37.733Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6047lb # [ 183.113036] niks3-server[1190]: 2026/09/24 09:28:37 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6048worker2 # [ 183.154268] nix-grpc-daemon[1545]: time=2026-09-24T09:28:37.780Z level=INFO msg="Uploading 1 narinfos"6049lb # [ 183.129262] niks3-server[1190]: 2026/09/24 09:28:37 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6050lb # [ 183.135702] niks3-server[1190]: 2026/09/24 09:28:37 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6051lb # [ 183.142417] niks3-server[1190]: 2026/09/24 09:28:37 INFO Signed narinfos id=54 count=16052lb # [ 183.145943] niks3-server[1190]: 2026/09/24 09:28:37 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6053lb # [ 183.154992] niks3-server[1190]: 2026/09/24 09:28:37 INFO Received complete push request method=POST path=/api/pushes/54/complete6054lb # [ 183.157775] niks3-server[1190]: 2026/09/24 09:28:37 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6055worker2 # [ 183.191935] nix-grpc-daemon[1545]: time=2026-09-24T09:28:37.818Z level=INFO msg="Upload complete. (271ms)"6056worker1 # [ 183.207528] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:37Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06057worker1 # [ 183.209683] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:37Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16058worker2 # [ 183.207460] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06059worker2 # [ 183.224590] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566060worker2 # [ 183.231861] nix-daemon[842]: reaped child process 1625, status = succeeded6061worker2 # [ 183.240354] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566062worker2 # [ 183.253551] nix-grpc-daemon[1545]: time=2026-09-24T09:28:37.879Z level=INFO msg="All 1 paths already cached"6063client # copying 1 paths to 'lb:50051'...6064client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6065worker2 # [ 183.263724] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362566066worker2 # [ 183.280448] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6067worker2 # [ 183.431160] nix-daemon[842]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)6068worker2 # [ 183.449278] nix-daemon[842]: reaped child process 1653, status = succeeded6069lb # [ 183.426191] niks3-server[1190]: 2026/09/24 09:28:38 INFO Received push request method=POST path=/api/pushes6070worker2 # [ 183.463040] nix-grpc-daemon[1545]: time=2026-09-24T09:28:38.089Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6071worker2 # [ 183.464600] nix-grpc-daemon[1545]: time=2026-09-24T09:28:38.089Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6072worker2 # [ 183.487799] nix-grpc-daemon[1545]: time=2026-09-24T09:28:38.114Z level=INFO msg="Uploading 1 narinfos"6073lb # [ 183.459531] niks3-server[1190]: 2026/09/24 09:28:38 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6074lb # [ 183.472536] niks3-server[1190]: 2026/09/24 09:28:38 INFO Signed narinfos id=55 count=16075lb # [ 183.475821] niks3-server[1190]: 2026/09/24 09:28:38 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6076lb # [ 183.492920] niks3-server[1190]: 2026/09/24 09:28:38 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6077worker2 # [ 183.527661] nix-grpc-daemon[1545]: time=2026-09-24T09:28:38.154Z level=INFO msg="Upload complete. (232ms)"6078lb # [ 183.500425] niks3-server[1190]: 2026/09/24 09:28:38 INFO Received complete push request method=POST path=/api/pushes/55/complete6079lb # [ 183.503981] niks3-server[1190]: 2026/09/24 09:28:38 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6080worker2 # [ 183.536254] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=6406081worker2 # [ 183.550590] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566082worker2 # [ 183.559968] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6083worker2 # [ 183.562303] nix-daemon[842]: reaped child process 1648, status = succeeded6084client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6085worker2 # [ 183.573215] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6086client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6087client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6088client # farm-b> LOG-farm-b-during-drain6089worker2 # [ 184.477755] nix-daemon[842]: reaped child process 1657, status = succeeded6090worker2 # [ 184.548599] nix-daemon[842]: accepted connection from pid 1673, user nix-grpc-daemon (trusted)6091worker2 # [ 184.569427] nix-daemon[842]: reaped child process 1676, status = succeeded6092lb # [ 184.541570] niks3-server[1190]: 2026/09/24 09:28:39 INFO Received push request method=POST path=/api/pushes6093worker2 # [ 184.579637] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.205Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6094worker2 # [ 184.581316] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.205Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6095worker2 # [ 184.616407] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.242Z level=INFO msg="Uploading 1 narinfos"6096lb # [ 184.581529] niks3-server[1190]: 2026/09/24 09:28:39 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6097lb # [ 184.599077] niks3-server[1190]: 2026/09/24 09:28:39 INFO Signed narinfos id=56 count=16098lb # [ 184.607793] niks3-server[1190]: 2026/09/24 09:28:39 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6099lb # [ 184.616285] niks3-server[1190]: 2026/09/24 09:28:39 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6100lb # [ 184.622490] niks3-server[1190]: 2026/09/24 09:28:39 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6101lb # [ 184.632976] niks3-server[1190]: 2026/09/24 09:28:39 INFO Received complete push request method=POST path=/api/pushes/56/complete6102worker2 # [ 184.665320] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.291Z level=INFO msg="Upload complete. (280ms)"6103lb # [ 184.638336] niks3-server[1190]: 2026/09/24 09:28:39 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6104worker1 # [ 184.681989] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:39Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06105worker1 # [ 184.683904] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:39Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16106worker2 # [ 184.677156] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06107worker2 # [ 184.701954] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566108worker2 # [ 184.707942] nix-daemon[842]: reaped child process 1655, status = succeeded6109worker2 # [ 184.716173] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566110worker2 # [ 184.730784] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.357Z level=INFO msg="All 2 paths already cached"6111client # copying 1 paths to 'lb:50051'...6112worker2 # [ 184.742138] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362566113client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6114worker2 # [ 184.761595] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6115worker2 # [ 184.911961] nix-daemon[842]: accepted connection from pid 1680, user nix-grpc-daemon (trusted)6116worker2 # [ 184.932436] nix-daemon[842]: reaped child process 1683, status = succeeded6117lb # [ 184.902392] niks3-server[1190]: 2026/09/24 09:28:39 INFO Received push request method=POST path=/api/pushes6118worker2 # [ 184.939031] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.565Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6119worker2 # [ 184.940718] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.565Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6120lb # [ 184.932608] niks3-server[1190]: 2026/09/24 09:28:39 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6121worker2 # [ 184.969587] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.595Z level=INFO msg="Uploading 1 narinfos"6122lb # [ 184.945224] niks3-server[1190]: 2026/09/24 09:28:39 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6123lb # [ 184.956432] niks3-server[1190]: 2026/09/24 09:28:39 INFO Signed narinfos id=57 count=16124lb # [ 184.959163] niks3-server[1190]: 2026/09/24 09:28:39 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6125lb # [ 184.973250] niks3-server[1190]: 2026/09/24 09:28:39 INFO Received complete push request method=POST path=/api/pushes/57/complete6126worker2 # [ 185.005269] nix-grpc-daemon[1545]: time=2026-09-24T09:28:39.631Z level=INFO msg="Upload complete. (226ms)"6127lb # [ 184.978206] niks3-server[1190]: 2026/09/24 09:28:39 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6128worker2 # [ 185.016292] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=7686129worker2 # [ 185.031191] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566130worker2 # [ 185.035730] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6131worker2 # [ 185.041341] nix-daemon[842]: reaped child process 1678, status = succeeded6132client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6133worker2 # [ 185.053304] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6134client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6135client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6136client # farm-top> LOG-farm-top-during-drain6137worker2 # [ 186.316158] nix-daemon[842]: reaped child process 1687, status = succeeded6138worker2 # [ 186.446448] nix-daemon[842]: accepted connection from pid 1703, user nix-grpc-daemon (trusted)6139worker2 # [ 186.464724] nix-daemon[842]: reaped child process 1706, status = succeeded6140lb # [ 186.441078] niks3-server[1190]: 2026/09/24 09:28:41 INFO Received push request method=POST path=/api/pushes6141worker2 # [ 186.480323] nix-grpc-daemon[1545]: time=2026-09-24T09:28:41.104Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6142worker2 # [ 186.482044] nix-grpc-daemon[1545]: time=2026-09-24T09:28:41.104Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6143lb # [ 186.478958] niks3-server[1190]: 2026/09/24 09:28:41 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6144worker2 # [ 186.513909] nix-grpc-daemon[1545]: time=2026-09-24T09:28:41.140Z level=INFO msg="Uploading 1 narinfos"6145lb # [ 186.492586] niks3-server[1190]: 2026/09/24 09:28:41 INFO Signed narinfos id=58 count=16146lb # [ 186.495489] niks3-server[1190]: 2026/09/24 09:28:41 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6147lb # [ 186.515571] niks3-server[1190]: 2026/09/24 09:28:41 INFO Received complete push request method=POST path=/api/pushes/58/complete6148lb # [ 186.522311] niks3-server[1190]: 2026/09/24 09:28:41 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6149worker2 # [ 186.550242] nix-grpc-daemon[1545]: time=2026-09-24T09:28:41.176Z level=INFO msg="Upload complete. (285ms)"6150lb # [ 186.529722] niks3-server[1190]: 2026/09/24 09:28:41 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6151lb # [ 186.535417] niks3-server[1190]: 2026/09/24 09:28:41 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6152worker2 # [ 186.564545] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06153worker2 # [ 186.582448] nix-daemon[842]: reaped child process 1685, status = succeeded6154client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.14 seconds)6155client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6156client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6157worker1: must succeed: pkill -f 'read -t [9]0 x'6158worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6159client: waiting for success: ! systemctl is-active sw6160worker1 # [ 187.052861] nix-daemon[855]: reaped child process 1839, status = succeeded6161worker1 # [ 187.181804] nix-daemon[855]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6162worker1 # [ 187.197631] nix-daemon[855]: reaped child process 1890, status = succeeded6163lb # [ 187.173171] niks3-server[1190]: 2026/09/24 09:28:41 INFO Received push request method=POST path=/api/pushes6164worker1 # [ 187.214321] nix-grpc-daemon[1781]: time=2026-09-24T09:28:41.845Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6165worker1 # [ 187.215946] nix-grpc-daemon[1781]: time=2026-09-24T09:28:41.845Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6166worker1 # [ 187.252875] nix-grpc-daemon[1781]: time=2026-09-24T09:28:41.884Z level=INFO msg="Uploading 1 narinfos"6167lb # [ 187.213681] niks3-server[1190]: 2026/09/24 09:28:41 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6168lb # [ 187.227156] niks3-server[1190]: 2026/09/24 09:28:41 INFO Signed narinfos id=59 count=16169lb # [ 187.235802] niks3-server[1190]: 2026/09/24 09:28:41 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6170lb # [ 187.250338] niks3-server[1190]: 2026/09/24 09:28:41 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6171worker1 # [ 187.289858] nix-grpc-daemon[1781]: time=2026-09-24T09:28:41.922Z level=INFO msg="Upload complete. (289ms)"6172lb # [ 187.260113] niks3-server[1190]: 2026/09/24 09:28:41 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6173lb # [ 187.264000] niks3-server[1190]: 2026/09/24 09:28:41 INFO Received complete push request method=POST path=/api/pushes/59/complete6174lb # [ 187.269398] niks3-server[1190]: 2026/09/24 09:28:41 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6175worker1 # [ 187.305317] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56050 duration_s=7 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06176worker1 # [ 187.325203] nix-daemon[855]: reaped child process 1837, status = succeeded6177client # [ 187.397285] systemd[1]: sw.service: Deactivated successfully.6178client # [ 187.401410] systemd[1]: sw.service: Consumed 199ms CPU time over 8.589s wall clock time, 39.2M memory peak, 7.2K incoming IP traffic, 8.3K outgoing IP traffic.6179client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)6180client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6181worker1 # [ 187.960652] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:42Z level=info event=signal_exit6182client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6183worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6184lb # [ 188.056928] envoy[898]: conn peer=[2001:db8:1::1]:49970 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9604 tx=4412 ms=48886185lb # [ 188.061880] envoy[898]: conn peer=[2001:db8:1::1]:60498 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2978 tx=1516 ms=83616186lb # [ 188.933336] niks3-server[1190]: 2026/09/24 09:28:43 INFO lead: released remote=192.168.1.3:480006187worker2 # [ 189.964263] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:44Z level=info event=scheduler_restarting addr=lb:500516188worker1 # [ 189.969271] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:44Z level=info event=scheduler_restarting6189lb # [ 189.934776] envoy[898]: [2026-09-24 09:28:44.554][898][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6190worker1 # [ 189.975483] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:44Z level=info event=scheduler_restarting addr=lb:500516191worker1 # [ 189.980648] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:560566192worker1 # [ 189.985556] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:44Z level=info event=worker_gone addr=192.168.1.3:500516193worker1 # [ 189.990607] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:560566194worker1 # [ 189.996499] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:44Z level=info event=worker_gone addr=192.168.1.4:500516195worker1 # [ 190.017174] nix-daemon[855]: reaped child process 1729, status = succeeded6196worker2 # [ 190.163614] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:44Z level=info event=scheduler_connected addr=lb:500516197worker2 # [ 190.169224] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6198worker1 # [ 190.186927] nix-grpc-daemon[1673]: ts=2026-09-24T09:28:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6199worker2 # [ 190.370112] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:44Z level=info event=scheduler_connected addr=lb:500516200worker2 # [ 190.375246] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6201worker2 # [ 190.576162] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_connected addr=lb:500516202worker2 # [ 190.581175] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6203worker2 # [ 190.782380] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_connected addr=lb:500516204worker2 # [ 190.787621] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6205worker2 # [ 190.990303] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_connected addr=lb:500516206worker2 # [ 190.994420] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6207worker2 # [ 191.190986] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_connected addr=lb:500516208worker2 # [ 191.196186] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6209worker2 # [ 191.397037] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_connected addr=lb:500516210worker2 # [ 191.402084] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6211worker2 # [ 191.602995] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_connected addr=lb:500516212worker2 # [ 191.608118] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6213worker2 # [ 191.808898] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_connected addr=lb:500516214worker2 # [ 191.813873] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6215worker1 # [ 191.993171] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6216worker1 # [ 191.998809] systemd[1]: nix-grpc-daemon.service: Consumed 914ms CPU time over 30.204s wall clock time, 37.8M memory peak, 242.4K incoming IP traffic, 76.1K outgoing IP traffic.6217worker1 # [ 192.016157] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6218worker2 # [ 192.016852] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_connected addr=lb:500516219worker2 # [ 192.020911] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6220worker1 # [ 192.027762] systemd[1]: Starting Nix worker-protocol over gRPC...6221worker2 # [ 192.151910] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_take_over6222lb # [ 192.124612] niks3-server[1190]: 2026/09/24 09:28:46 INFO lead: acquired remote=192.168.1.4:598006223worker1 # [ 192.176646] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6224worker1 # [ 192.178561] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6225worker1 # [ 192.182370] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16226worker1 # [ 192.188066] nix-grpc-daemon[1905]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6227worker1 # [ 192.189519] nix-grpc-daemon[1905]: E0000 00:00:1790242126.819162 1905 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6228worker1 # [ 192.193144] nix-grpc-daemon[1905]: E0000 00:00:1790242126.825390 1905 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6229worker1 # [ 192.196050] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:46Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6230worker1 # [ 192.203740] systemd[1]: Started Nix worker-protocol over gRPC.6231worker2 # [ 192.217602] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_connected addr=lb:500516232worker2 # [ 192.222643] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6233worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 4.23 seconds)6234worker1 # [ 192.206444] nix-grpc-daemon[1905]: E0000 00:00:1790242126.838829 1946 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6235lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6236lb: must succeed: curl -sf localhost:9901/clusters6237worker1 # [ 192.236285] nix-grpc-daemon[1905]: E0000 00:00:1790242126.858100 1946 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6238worker1 # [ 192.238401] nix-grpc-daemon[1905]: E0000 00:00:1790242126.858247 1946 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6239worker1 # [ 192.247066] nix-grpc-daemon[1905]: E0000 00:00:1790242126.878579 1946 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6240worker1 # [ 192.264105] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6241lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6242worker2 # [ 192.423551] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:47Z level=info event=scheduler_connected addr=lb:500516243worker2 # [ 192.428927] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6244worker1 # [ 192.465109] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:47Z level=info event=scheduler_connected addr=lb:500516245worker2 # [ 192.463552] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:544766246worker2 # [ 192.468931] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06247worker2 # [ 192.629711] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:47Z level=info event=scheduler_connected addr=lb:500516248worker2 # [ 192.642720] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:47Z level=info event=oidc_jwks provider=mock keys=16249worker2 # [ 192.647233] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:544766250worker2 # [ 192.652589] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06251lb: must succeed: curl -sf localhost:9901/clusters6252lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6253lb: must succeed: curl -sf localhost:9901/clusters6254lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6255lb: must succeed: curl -sf localhost:9901/clusters6256lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6257worker2 # [ 195.836788] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:50Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:393106258lb: must succeed: curl -sf localhost:9901/clusters6259lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6260lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 4.35 seconds)6261lb: must succeed: curl -sf localhost:9901/clusters6262lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6263worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6264worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6265(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 17.99 seconds)6266subtest: queue gauges show a burst while every build blocks6267lb: must succeed: curl -sf localhost:9901/clusters6268lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6269lb: must succeed: curl -sf localhost:9901/clusters6270lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6271worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6272worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6273client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6274client # [ 196.958587] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6275client # Running as unit: burst.service; invocation ID: d2422c4f6ba44488bd1fb632bbc4672e6276client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6277lb: must succeed: curl -sf localhost:9901/clusters6278lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6279lb: must succeed: curl -sf localhost:9901/clusters6280lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6281worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6282worker1 # [ 197.135346] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:413486283worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6284worker1 # [ 197.149780] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6285client # [ 197.202753] nix[1564]: don't know how to build these paths:6286client # [ 197.204049] nix[1564]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6287client # [ 197.205482] nix[1564]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6288client # [ 197.208132] nix[1564]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6289client # [ 197.209592] nix[1564]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6290worker2 # [ 197.182246] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=want client=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06291worker2 # [ 197.188464] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16292worker1 # [ 197.195510] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486293worker2 # [ 197.198193] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=want client=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06294worker2 # [ 197.204427] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16295worker1 # [ 197.211987] nix-grpc-daemon[1905]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 64 ms (attempt 1/5)6296worker2 # [ 197.212829] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=want client=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06297worker2 # [ 197.220390] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=want client=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06298worker2 # [ 197.226742] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566299client # [ 197.268145] nix[1564]: copying 1 paths to 'lb:50051'...6300client # [ 197.270498] nix[1564]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6301worker2 # [ 197.232817] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566302worker2 # [ 197.238765] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362566303worker2 # [ 197.250640] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6304worker1 # [ 197.285595] nix-grpc-daemon[1905]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 2/5)6305worker2 # [ 197.385942] nix-daemon[842]: accepted connection from pid 1721, user nix-grpc-daemon (trusted)6306worker2 # [ 197.404712] nix-daemon[842]: reaped child process 1724, status = succeeded6307lb # [ 197.383167] niks3-server[1190]: 2026/09/24 09:28:52 INFO Received push request method=POST path=/api/pushes6308worker2 # [ 197.420920] nix-grpc-daemon[1545]: time=2026-09-24T09:28:52.046Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6309worker2 # [ 197.422373] nix-grpc-daemon[1545]: time=2026-09-24T09:28:52.046Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6310lb # [ 197.420905] niks3-server[1190]: 2026/09/24 09:28:52 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6311worker2 # [ 197.458931] nix-grpc-daemon[1545]: time=2026-09-24T09:28:52.085Z level=INFO msg="Uploading 1 narinfos"6312lb # [ 197.433786] niks3-server[1190]: 2026/09/24 09:28:52 INFO Signed narinfos id=60 count=16313worker1 # [ 197.471653] nix-grpc-daemon[1905]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 3/5)6314lb # [ 197.440589] niks3-server[1190]: 2026/09/24 09:28:52 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6315lb # [ 197.449690] niks3-server[1190]: 2026/09/24 09:28:52 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6316lb # [ 197.467915] niks3-server[1190]: 2026/09/24 09:28:52 INFO Received complete push request method=POST path=/api/pushes/60/complete6317lb # [ 197.476281] niks3-server[1190]: 2026/09/24 09:28:52 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6318worker2 # [ 197.510929] nix-grpc-daemon[1545]: time=2026-09-24T09:28:52.137Z level=INFO msg="Upload complete. (248ms)"6319worker2 # [ 197.520611] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=4166320worker2 # [ 197.536791] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566321worker1 # [ 197.539368] nix-grpc-daemon[1905]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 517 ms (attempt 4/5)6322worker2 # [ 197.546670] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6323worker2 # [ 197.555308] nix-daemon[842]: reaped child process 1719, status = succeeded6324client # [ 197.602243] nix[1564]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6325worker2 # [ 197.565239] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6326worker1 # [ 198.058792] nix-grpc-daemon[1905]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6327worker1 # [ 198.065944] nix-grpc-daemon[1905]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6328worker1 # [ 198.080349] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486329lb # [ 198.062218] envoy[898]: conn peer=[2001:db8:1::3]:33906 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1333 tx=710 ms=281706330client # [ 198.130198] nix[1564]: copying 1 paths to 'lb:50051'...6331client # [ 198.134571] nix[1564]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6332worker1 # [ 198.102594] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486333worker1 # [ 198.116496] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6334lb: must succeed: curl -sf localhost:9901/clusters6335worker1 # [ 198.153576] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:52Z level=info event=niks3_push_started pid=19656336lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6337lb: must succeed: curl -sf localhost:9901/clusters6338lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6339worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6340worker1 # [ 198.294962] nix-grpc-daemon[1965]: time=2026-09-24T09:28:52.926Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6341worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.10 seconds)6342worker1 # [ 198.406320] nix-daemon[855]: accepted connection from pid 1977, user nix-grpc-daemon (trusted)6343worker1 # [ 198.444235] nix-daemon[855]: reaped child process 1983, status = succeeded6344lb # [ 198.407951] niks3-server[1190]: 2026/09/24 09:28:53 INFO Received push request method=POST path=/api/pushes6345worker1 # [ 198.452986] nix-grpc-daemon[1965]: time=2026-09-24T09:28:53.081Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6346worker1 # [ 198.456128] nix-grpc-daemon[1965]: time=2026-09-24T09:28:53.081Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6347lb # [ 198.448265] niks3-server[1190]: 2026/09/24 09:28:53 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6348worker1 # [ 198.489675] nix-grpc-daemon[1965]: time=2026-09-24T09:28:53.121Z level=INFO msg="Uploading 1 narinfos"6349lb # [ 198.460859] niks3-server[1190]: 2026/09/24 09:28:53 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6350lb # [ 198.469010] niks3-server[1190]: 2026/09/24 09:28:53 INFO Signed narinfos id=61 count=16351lb # [ 198.474345] niks3-server[1190]: 2026/09/24 09:28:53 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6352lb # [ 198.486165] niks3-server[1190]: 2026/09/24 09:28:53 INFO Received complete push request method=POST path=/api/pushes/61/complete6353worker1 # [ 198.525784] nix-grpc-daemon[1965]: time=2026-09-24T09:28:53.158Z level=INFO msg="Upload complete. (227ms)"6354lb # [ 198.495611] niks3-server[1190]: 2026/09/24 09:28:53 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6355worker1 # [ 198.532209] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=4166356worker1 # [ 198.543153] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486357worker1 # [ 198.549249] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6358worker1 # [ 198.553655] nix-daemon[855]: reaped child process 1963, status = succeeded6359worker1 # [ 198.565191] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6360client # [ 198.598785] nix[1564]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6361worker1 # [ 199.827627] nix-daemon[855]: reaped child process 1988, status = succeeded6362worker2 # [ 199.826851] nix-daemon[842]: reaped child process 1728, status = succeeded6363worker2 # [ 199.924357] nix-daemon[842]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6364worker2 # [ 199.942244] nix-daemon[842]: reaped child process 1759, status = succeeded6365worker1 # [ 199.953346] nix-daemon[855]: accepted connection from pid 2009, user nix-grpc-daemon (trusted)6366lb # [ 199.922514] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received push request method=POST path=/api/pushes6367worker2 # [ 199.963081] nix-grpc-daemon[1545]: time=2026-09-24T09:28:54.587Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6368worker2 # [ 199.964654] nix-grpc-daemon[1545]: time=2026-09-24T09:28:54.587Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6369worker1 # [ 199.973642] nix-daemon[855]: reaped child process 2012, status = succeeded6370lb # [ 199.940628] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received push request method=POST path=/api/pushes6371worker1 # [ 199.984360] nix-grpc-daemon[1965]: time=2026-09-24T09:28:54.614Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6372worker1 # [ 199.985811] nix-grpc-daemon[1965]: time=2026-09-24T09:28:54.614Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6373worker2 # [ 200.025182] nix-grpc-daemon[1545]: time=2026-09-24T09:28:54.651Z level=INFO msg="Uploading 1 narinfos"6374lb # [ 199.991626] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6375lb # [ 200.002490] niks3-server[1190]: 2026/09/24 09:28:54 INFO Signed narinfos id=62 count=16376lb # [ 200.010148] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6377lb # [ 200.018452] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6378lb # [ 200.033255] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6379worker2 # [ 200.069933] nix-grpc-daemon[1545]: time=2026-09-24T09:28:54.696Z level=INFO msg="Upload complete. (289ms)"6380lb # [ 200.046734] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received complete push request method=POST path=/api/pushes/62/complete6381worker1 # [ 200.084230] nix-grpc-daemon[1965]: time=2026-09-24T09:28:54.716Z level=INFO msg="Uploading 1 narinfos"6382lb # [ 200.062952] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6383worker2 # [ 200.090869] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=2 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06384lb # [ 200.072397] niks3-server[1190]: 2026/09/24 09:28:54 INFO Signed narinfos id=63 count=16385lb # [ 200.080517] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6386lb # [ 200.082894] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6387worker2 # [ 200.110922] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:54Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16388lb # [ 200.089071] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6389worker2 # [ 200.118936] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566390worker2 # [ 200.125986] nix-daemon[842]: reaped child process 1726, status = succeeded6391lb # [ 200.098427] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received complete push request method=POST path=/api/pushes/63/complete6392lb # [ 200.103620] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6393worker1 # [ 200.135298] nix-grpc-daemon[1965]: time=2026-09-24T09:28:54.767Z level=INFO msg="Upload complete. (361ms)"6394worker2 # [ 200.131619] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566395lb # [ 200.106534] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6396worker2 # [ 200.139941] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:54Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16397client # [ 200.189232] nix[1564]: copying 1 paths to 'lb:50051'...6398worker2 # [ 200.150151] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362566399worker1 # [ 200.149183] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06400client # [ 200.193254] nix[1564]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6401worker2 # [ 200.162617] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6402client # [ 200.207884] nix[1564]: copying 1 paths to 'lb:50051'...6403worker1 # [ 200.173773] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486404client # [ 200.210313] nix[1564]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6405worker1 # [ 200.184390] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486406worker1 # [ 200.190843] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486407worker1 # [ 200.204276] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6408worker1 # [ 200.212291] nix-daemon[855]: reaped child process 1986, status = succeeded6409worker2 # [ 200.302767] nix-daemon[842]: accepted connection from pid 1763, user nix-grpc-daemon (trusted)6410worker2 # [ 200.319793] nix-daemon[842]: reaped child process 1766, status = succeeded6411lb # [ 200.297807] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received push request method=POST path=/api/pushes6412worker2 # [ 200.334134] nix-grpc-daemon[1545]: time=2026-09-24T09:28:54.960Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6413worker2 # [ 200.335716] nix-grpc-daemon[1545]: time=2026-09-24T09:28:54.960Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6414worker1 # [ 200.346177] nix-daemon[855]: accepted connection from pid 2016, user nix-grpc-daemon (trusted)6415worker1 # [ 200.364245] nix-daemon[855]: reaped child process 2019, status = succeeded6416worker2 # [ 200.366172] nix-grpc-daemon[1545]: time=2026-09-24T09:28:54.992Z level=INFO msg="Uploading 1 narinfos"6417lb # [ 200.329633] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6418lb # [ 200.342538] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6419worker1 # [ 200.387844] nix-grpc-daemon[1965]: time=2026-09-24T09:28:55.019Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6420lb # [ 200.356100] niks3-server[1190]: 2026/09/24 09:28:54 INFO Signed narinfos id=64 count=16421worker1 # [ 200.390187] nix-grpc-daemon[1965]: time=2026-09-24T09:28:55.019Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6422lb # [ 200.363454] niks3-server[1190]: 2026/09/24 09:28:54 INFO Received push request method=POST path=/api/pushes6423lb # [ 200.373506] niks3-server[1190]: 2026/09/24 09:28:54 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6424lb # [ 200.388135] niks3-server[1190]: 2026/09/24 09:28:55 INFO Received complete push request method=POST path=/api/pushes/64/complete6425worker2 # [ 200.421996] nix-grpc-daemon[1545]: time=2026-09-24T09:28:55.048Z level=INFO msg="Upload complete. (244ms)"6426worker1 # [ 200.432670] nix-grpc-daemon[1965]: time=2026-09-24T09:28:55.064Z level=INFO msg="Uploading 1 narinfos"6427lb # [ 200.400900] niks3-server[1190]: 2026/09/24 09:28:55 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6428lb # [ 200.403985] niks3-server[1190]: 2026/09/24 09:28:55 INFO Signed narinfos id=65 count=16429worker2 # [ 200.432306] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=4166430lb # [ 200.413132] niks3-server[1190]: 2026/09/24 09:28:55 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6431lb # [ 200.421989] niks3-server[1190]: 2026/09/24 09:28:55 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6432worker2 # [ 200.456188] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566433lb # [ 200.431201] niks3-server[1190]: 2026/09/24 09:28:55 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6434worker2 # [ 200.460758] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6435lb # [ 200.436346] niks3-server[1190]: 2026/09/24 09:28:55 INFO Received complete push request method=POST path=/api/pushes/65/complete6436lb # [ 200.440853] niks3-server[1190]: 2026/09/24 09:28:55 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6437worker1 # [ 200.471575] nix-grpc-daemon[1965]: time=2026-09-24T09:28:55.103Z level=INFO msg="Upload complete. (253ms)"6438worker2 # [ 200.472257] nix-daemon[842]: reaped child process 1761, status = succeeded6439client # [ 200.515053] nix[1564]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6440worker2 # [ 200.480392] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6441worker1 # [ 200.484561] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=4166442worker1 # [ 200.499007] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486443worker1 # [ 200.510856] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6444client # [ 200.558846] nix[1564]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6445worker1 # [ 200.528154] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6446worker1 # [ 200.531500] nix-daemon[855]: reaped child process 2014, status = succeeded6447worker2 # [ 202.104826] nix-daemon[842]: reaped child process 1770, status = succeeded6448worker2 # [ 202.203550] nix-daemon[842]: accepted connection from pid 1795, user nix-grpc-daemon (trusted)6449worker2 # [ 202.222165] nix-daemon[842]: reaped child process 1798, status = succeeded6450lb # [ 202.199137] niks3-server[1190]: 2026/09/24 09:28:56 INFO Received push request method=POST path=/api/pushes6451worker1 # [ 202.239045] nix-daemon[855]: reaped child process 2023, status = succeeded6452worker2 # [ 202.237448] nix-grpc-daemon[1545]: time=2026-09-24T09:28:56.863Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6453worker2 # [ 202.242189] nix-grpc-daemon[1545]: time=2026-09-24T09:28:56.865Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6454lb # [ 202.240252] niks3-server[1190]: 2026/09/24 09:28:56 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6455worker2 # [ 202.275283] nix-grpc-daemon[1545]: time=2026-09-24T09:28:56.901Z level=INFO msg="Uploading 1 narinfos"6456lb # [ 202.253974] niks3-server[1190]: 2026/09/24 09:28:56 INFO Signed narinfos id=66 count=16457lb # [ 202.261095] niks3-server[1190]: 2026/09/24 09:28:56 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6458lb # [ 202.270215] niks3-server[1190]: 2026/09/24 09:28:56 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6459worker1 # [ 202.307336] nix-daemon[855]: accepted connection from pid 2048, user nix-grpc-daemon (trusted)6460lb # [ 202.280228] niks3-server[1190]: 2026/09/24 09:28:56 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6461lb # [ 202.283310] niks3-server[1190]: 2026/09/24 09:28:56 INFO Received complete push request method=POST path=/api/pushes/66/complete6462worker2 # [ 202.316784] nix-grpc-daemon[1545]: time=2026-09-24T09:28:56.943Z level=INFO msg="Upload complete. (226ms)"6463lb # [ 202.289384] niks3-server[1190]: 2026/09/24 09:28:56 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6464worker1 # [ 202.328700] nix-daemon[855]: reaped child process 2051, status = succeeded6465lb # [ 202.299696] niks3-server[1190]: 2026/09/24 09:28:56 INFO Received push request method=POST path=/api/pushes6466worker2 # [ 202.329006] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06467worker1 # [ 202.338551] nix-grpc-daemon[1965]: time=2026-09-24T09:28:56.970Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6468worker1 # [ 202.340385] nix-grpc-daemon[1965]: time=2026-09-24T09:28:56.970Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6469worker2 # [ 202.343798] nix-daemon[842]: reaped child process 1768, status = succeeded6470worker1 # [ 202.358777] nix-grpc-daemon[1965]: time=2026-09-24T09:28:56.990Z level=INFO msg="Uploading 1 narinfos"6471lb # [ 202.326760] niks3-server[1190]: 2026/09/24 09:28:56 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6472lb # [ 202.334865] niks3-server[1190]: 2026/09/24 09:28:56 INFO Signed narinfos id=67 count=16473lb # [ 202.339341] niks3-server[1190]: 2026/09/24 09:28:56 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6474worker1 # [ 202.380429] nix-grpc-daemon[1965]: time=2026-09-24T09:28:57.012Z level=INFO msg="Upload complete. (238ms)"6475lb # [ 202.351602] niks3-server[1190]: 2026/09/24 09:28:56 INFO Received complete push request method=POST path=/api/pushes/67/complete6476lb # [ 202.355553] niks3-server[1190]: 2026/09/24 09:28:56 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6477worker1 # [ 202.388597] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06478lb # [ 202.361272] niks3-server[1190]: 2026/09/24 09:28:56 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6479lb # [ 202.365783] niks3-server[1190]: 2026/09/24 09:28:56 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6480worker1 # [ 202.401237] nix-daemon[855]: reaped child process 2021, status = succeeded6481client # [ 202.462794] systemd[1]: burst.service: Deactivated successfully.6482client # [ 202.467863] systemd[1]: burst.service: Consumed 229ms CPU time over 5.505s wall clock time, 49.7M memory peak, 14.2K incoming IP traffic, 18.9K outgoing IP traffic.6483client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6484client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6485lb: must succeed: curl -sf localhost:9901/clusters6486lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6487lb: must succeed: curl -sf localhost:9901/clusters6488lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6489worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6490worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6491lb: must succeed: curl -sf localhost:9901/clusters6492lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6493lb: must succeed: curl -sf localhost:9901/clusters6494lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6495worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6496worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6497(finished: subtest: queue gauges show a burst while every build blocks, in 7.03 seconds)6498subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6499client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16500worker2 # [ 203.950382] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362566501worker2 # [ 203.958292] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566502worker2 # [ 203.997141] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:58Z level=debug event=want client=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06503worker2 # [ 204.003219] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:58Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16504worker1 # [ 204.011787] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486505worker2 # [ 204.009699] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:58Z level=debug event=want client=2 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06506worker2 # [ 204.014934] nix-grpc-daemon[1473]: ts=2026-09-24T09:28:58Z level=debug event=want client=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06507worker1 # [ 204.027613] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486508worker1 # [ 204.043049] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486509worker1 # [ 204.058609] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6510worker1 # [ 204.221388] nix-daemon[855]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)6511worker1 # [ 204.240795] nix-daemon[855]: reaped child process 2062, status = succeeded6512lb # [ 204.215884] niks3-server[1190]: 2026/09/24 09:28:58 INFO Received push request method=POST path=/api/pushes6513worker1 # [ 204.257542] nix-grpc-daemon[1965]: time=2026-09-24T09:28:58.889Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6514worker1 # [ 204.259248] nix-grpc-daemon[1965]: time=2026-09-24T09:28:58.889Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6515lb # [ 204.249984] niks3-server[1190]: 2026/09/24 09:28:58 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6516worker1 # [ 204.291568] nix-grpc-daemon[1965]: time=2026-09-24T09:28:58.923Z level=INFO msg="Uploading 1 narinfos"6517lb # [ 204.263479] niks3-server[1190]: 2026/09/24 09:28:58 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6518lb # [ 204.275099] niks3-server[1190]: 2026/09/24 09:28:58 INFO Signed narinfos id=68 count=16519lb # [ 204.284552] niks3-server[1190]: 2026/09/24 09:28:58 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6520worker1 # [ 204.321985] nix-grpc-daemon[1965]: time=2026-09-24T09:28:58.954Z level=INFO msg="Upload complete. (246ms)"6521lb # [ 204.294200] niks3-server[1190]: 2026/09/24 09:28:58 INFO Received complete push request method=POST path=/api/pushes/68/complete6522lb # [ 204.299487] niks3-server[1190]: 2026/09/24 09:28:58 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6523worker1 # [ 204.332404] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=5126524worker1 # [ 204.351627] nix-grpc-daemon[1905]: ts=2026-09-24T09:28:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486525worker1 # [ 204.358783] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6526worker1 # [ 204.365041] nix-daemon[855]: reaped child process 2057, status = succeeded6527worker1 # [ 204.380769] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6528worker1 # [ 205.848381] nix-daemon[855]: reaped child process 2066, status = succeeded6529worker1 # [ 205.980299] nix-daemon[855]: accepted connection from pid 2082, user nix-grpc-daemon (trusted)6530worker1 # [ 206.000098] nix-daemon[855]: reaped child process 2085, status = succeeded6531lb # [ 205.974002] niks3-server[1190]: 2026/09/24 09:29:00 INFO Received push request method=POST path=/api/pushes6532worker1 # [ 206.020340] nix-grpc-daemon[1965]: time=2026-09-24T09:29:00.650Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6533worker1 # [ 206.022231] nix-grpc-daemon[1965]: time=2026-09-24T09:29:00.650Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6534lb # [ 206.017908] niks3-server[1190]: 2026/09/24 09:29:00 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6535worker1 # [ 206.058701] nix-grpc-daemon[1965]: time=2026-09-24T09:29:00.690Z level=INFO msg="Uploading 1 narinfos"6536lb # [ 206.031810] niks3-server[1190]: 2026/09/24 09:29:00 INFO Signed narinfos id=69 count=16537lb # [ 206.039181] niks3-server[1190]: 2026/09/24 09:29:00 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6538lb # [ 206.048353] niks3-server[1190]: 2026/09/24 09:29:00 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6539lb # [ 206.051801] niks3-server[1190]: 2026/09/24 09:29:00 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6540lb # [ 206.061588] niks3-server[1190]: 2026/09/24 09:29:00 INFO Received complete push request method=POST path=/api/pushes/69/complete6541lb # [ 206.066649] niks3-server[1190]: 2026/09/24 09:29:00 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6542worker1 # [ 206.097512] nix-grpc-daemon[1965]: time=2026-09-24T09:29:00.729Z level=INFO msg="Upload complete. (304ms)"6543worker2 # [ 206.105476] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:00Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16544worker1 # [ 206.111639] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06545worker2 # [ 206.117589] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:00Z level=debug event=want client=2 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06546worker1 # [ 206.135883] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486547worker1 # [ 206.142293] nix-daemon[855]: reaped child process 2064, status = succeeded6548worker1 # [ 206.145911] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486549worker1 # [ 206.158108] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486550worker1 # [ 206.168775] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6551worker1 # [ 206.310651] nix-daemon[855]: accepted connection from pid 2089, user nix-grpc-daemon (trusted)6552worker1 # [ 206.330132] nix-daemon[855]: reaped child process 2092, status = succeeded6553lb # [ 206.302639] niks3-server[1190]: 2026/09/24 09:29:00 INFO Received push request method=POST path=/api/pushes6554worker1 # [ 206.343448] nix-grpc-daemon[1965]: time=2026-09-24T09:29:00.975Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6555worker1 # [ 206.345476] nix-grpc-daemon[1965]: time=2026-09-24T09:29:00.975Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6556lb # [ 206.332135] niks3-server[1190]: 2026/09/24 09:29:00 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6557worker1 # [ 206.369655] nix-grpc-daemon[1965]: time=2026-09-24T09:29:01.001Z level=INFO msg="Uploading 1 narinfos"6558lb # [ 206.346216] niks3-server[1190]: 2026/09/24 09:29:00 INFO Signed narinfos id=70 count=16559lb # [ 206.354921] niks3-server[1190]: 2026/09/24 09:29:00 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6560worker1 # [ 206.405091] nix-grpc-daemon[1965]: time=2026-09-24T09:29:01.037Z level=INFO msg="Upload complete. (223ms)"6561lb # [ 206.366987] niks3-server[1190]: 2026/09/24 09:29:00 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6562lb # [ 206.379309] niks3-server[1190]: 2026/09/24 09:29:00 INFO Received complete push request method=POST path=/api/pushes/70/complete6563lb # [ 206.384493] niks3-server[1190]: 2026/09/24 09:29:01 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6564worker1 # [ 206.416272] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=5126565worker1 # [ 206.424534] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486566worker1 # [ 206.435856] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6567worker1 # [ 206.441660] nix-daemon[855]: reaped child process 2087, status = succeeded6568worker1 # [ 206.455651] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6569worker1 # [ 207.932719] nix-daemon[855]: reaped child process 2096, status = succeeded6570worker1 # [ 208.030234] nix-daemon[855]: accepted connection from pid 2113, user nix-grpc-daemon (trusted)6571worker1 # [ 208.047232] nix-daemon[855]: reaped child process 2116, status = succeeded6572lb # [ 208.025043] niks3-server[1190]: 2026/09/24 09:29:02 INFO Received push request method=POST path=/api/pushes6573worker1 # [ 208.068369] nix-grpc-daemon[1965]: time=2026-09-24T09:29:02.697Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6574worker1 # [ 208.070030] nix-grpc-daemon[1965]: time=2026-09-24T09:29:02.698Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6575lb # [ 208.058657] envoy[898]: conn peer=[2001:db8:1::1]:35590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9740 tx=5188 ms=52756576lb # [ 208.072840] niks3-server[1190]: 2026/09/24 09:29:02 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6577worker1 # [ 208.113797] nix-grpc-daemon[1965]: time=2026-09-24T09:29:02.745Z level=INFO msg="Uploading 1 narinfos"6578lb # [ 208.086573] niks3-server[1190]: 2026/09/24 09:29:02 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6579lb # [ 208.094792] niks3-server[1190]: 2026/09/24 09:29:02 INFO Signed narinfos id=71 count=16580lb # [ 208.099957] niks3-server[1190]: 2026/09/24 09:29:02 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6581lb # [ 208.106316] niks3-server[1190]: 2026/09/24 09:29:02 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6582lb # [ 208.117354] niks3-server[1190]: 2026/09/24 09:29:02 INFO Received complete push request method=POST path=/api/pushes/71/complete6583lb # [ 208.122962] niks3-server[1190]: 2026/09/24 09:29:02 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6584worker1 # [ 208.154672] nix-grpc-daemon[1965]: time=2026-09-24T09:29:02.786Z level=INFO msg="Upload complete. (287ms)"6585worker2 # [ 208.163845] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:02Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16586worker1 # [ 208.167811] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=6 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06587worker2 # [ 208.171862] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:02Z level=debug event=want client=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06588worker1 # [ 208.192900] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486589worker1 # [ 208.199243] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486590worker1 # [ 208.205270] nix-daemon[855]: reaped child process 2094, status = succeeded6591worker1 # [ 208.208961] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:02Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486592worker1 # [ 208.222086] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6593worker1 # [ 208.356474] nix-daemon[855]: accepted connection from pid 2120, user nix-grpc-daemon (trusted)6594worker1 # [ 208.376746] nix-daemon[855]: reaped child process 2123, status = succeeded6595lb # [ 208.343392] niks3-server[1190]: 2026/09/24 09:29:02 INFO Received push request method=POST path=/api/pushes6596worker1 # [ 208.383426] nix-grpc-daemon[1965]: time=2026-09-24T09:29:03.015Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6597worker1 # [ 208.385117] nix-grpc-daemon[1965]: time=2026-09-24T09:29:03.015Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6598lb # [ 208.369148] niks3-server[1190]: 2026/09/24 09:29:02 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6599worker1 # [ 208.408729] nix-grpc-daemon[1965]: time=2026-09-24T09:29:03.040Z level=INFO msg="Uploading 1 narinfos"6600lb # [ 208.382156] niks3-server[1190]: 2026/09/24 09:29:03 INFO Signed narinfos id=72 count=16601lb # [ 208.392484] niks3-server[1190]: 2026/09/24 09:29:03 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6602lb # [ 208.401387] niks3-server[1190]: 2026/09/24 09:29:03 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6603worker1 # [ 208.446483] nix-grpc-daemon[1965]: time=2026-09-24T09:29:03.078Z level=INFO msg="Upload complete. (214ms)"6604lb # [ 208.415125] niks3-server[1190]: 2026/09/24 09:29:03 INFO Received complete push request method=POST path=/api/pushes/72/complete6605lb # [ 208.420880] niks3-server[1190]: 2026/09/24 09:29:03 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6606worker1 # [ 208.456525] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=5126607worker1 # [ 208.468334] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486608worker1 # [ 208.477611] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6609worker1 # [ 208.481268] nix-daemon[855]: reaped child process 2118, status = succeeded6610worker1 # [ 208.492207] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6611worker1 # [ 209.190468] nix-daemon[855]: reaped child process 2127, status = succeeded6612worker1 # [ 209.268641] nix-daemon[855]: accepted connection from pid 2143, user nix-grpc-daemon (trusted)6613worker1 # [ 209.289750] nix-daemon[855]: reaped child process 2146, status = succeeded6614lb # [ 209.257009] niks3-server[1190]: 2026/09/24 09:29:03 INFO Received push request method=POST path=/api/pushes6615worker1 # [ 209.300454] nix-grpc-daemon[1965]: time=2026-09-24T09:29:03.929Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6616worker1 # [ 209.302077] nix-grpc-daemon[1965]: time=2026-09-24T09:29:03.930Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6617lb # [ 209.301923] niks3-server[1190]: 2026/09/24 09:29:03 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6618worker1 # [ 209.347911] nix-grpc-daemon[1965]: time=2026-09-24T09:29:03.980Z level=INFO msg="Uploading 1 narinfos"6619lb # [ 209.317264] niks3-server[1190]: 2026/09/24 09:29:03 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6620lb # [ 209.330435] niks3-server[1190]: 2026/09/24 09:29:03 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6621lb # [ 209.337532] niks3-server[1190]: 2026/09/24 09:29:03 INFO Signed narinfos id=73 count=16622lb # [ 209.348462] niks3-server[1190]: 2026/09/24 09:29:03 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6623lb # [ 209.351753] niks3-server[1190]: 2026/09/24 09:29:03 INFO Received complete push request method=POST path=/api/pushes/73/complete6624lb # [ 209.357809] niks3-server[1190]: 2026/09/24 09:29:03 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6625worker1 # [ 209.389214] nix-grpc-daemon[1965]: time=2026-09-24T09:29:04.021Z level=INFO msg="Upload complete. (287ms)"6626worker2 # [ 209.395155] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:04Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16627worker1 # [ 209.402331] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=7 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06628worker2 # [ 209.407855] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:04Z level=debug event=want client=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06629worker1 # [ 209.424250] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486630worker1 # [ 209.430540] nix-daemon[855]: reaped child process 2125, status = succeeded6631worker1 # [ 209.438067] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486632worker1 # [ 209.450008] nix-grpc-daemon[1965]: time=2026-09-24T09:29:04.082Z level=INFO msg="All 1 paths already cached"6633worker1 # [ 209.457933] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486634worker1 # [ 209.468165] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6635worker1 # [ 209.607224] nix-daemon[855]: accepted connection from pid 2150, user nix-grpc-daemon (trusted)6636worker1 # [ 209.622196] nix-daemon[855]: reaped child process 2153, status = succeeded6637lb # [ 209.597345] niks3-server[1190]: 2026/09/24 09:29:04 INFO Received push request method=POST path=/api/pushes6638worker1 # [ 209.637231] nix-grpc-daemon[1965]: time=2026-09-24T09:29:04.269Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6639worker1 # [ 209.638723] nix-grpc-daemon[1965]: time=2026-09-24T09:29:04.269Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6640lb # [ 209.630733] niks3-server[1190]: 2026/09/24 09:29:04 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign6641worker1 # [ 209.669386] nix-grpc-daemon[1965]: time=2026-09-24T09:29:04.301Z level=INFO msg="Uploading 1 narinfos"6642lb # [ 209.644192] niks3-server[1190]: 2026/09/24 09:29:04 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6643lb # [ 209.658758] niks3-server[1190]: 2026/09/24 09:29:04 INFO Signed narinfos id=74 count=16644worker1 # [ 209.694942] nix-grpc-daemon[1965]: time=2026-09-24T09:29:04.327Z level=INFO msg="Upload complete. (213ms)"6645lb # [ 209.664997] niks3-server[1190]: 2026/09/24 09:29:04 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6646lb # [ 209.671832] niks3-server[1190]: 2026/09/24 09:29:04 INFO Received complete push request method=POST path=/api/pushes/74/complete6647lb # [ 209.676640] niks3-server[1190]: 2026/09/24 09:29:04 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6648worker1 # [ 209.703899] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=6246649worker1 # [ 209.717506] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486650worker1 # [ 209.725465] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6651worker1 # [ 209.728313] nix-daemon[855]: reaped child process 2148, status = succeeded6652worker1 # [ 209.737782] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6653worker1 # [ 211.445696] nix-daemon[855]: reaped child process 2157, status = succeeded6654worker1 # [ 211.518788] nix-daemon[855]: accepted connection from pid 2173, user nix-grpc-daemon (trusted)6655worker1 # [ 211.537716] nix-daemon[855]: reaped child process 2176, status = succeeded6656lb # [ 211.505080] niks3-server[1190]: 2026/09/24 09:29:06 INFO Received push request method=POST path=/api/pushes6657worker1 # [ 211.547864] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.178Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6658worker1 # [ 211.550140] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.178Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6659lb # [ 211.542924] niks3-server[1190]: 2026/09/24 09:29:06 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6660worker1 # [ 211.591446] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.223Z level=INFO msg="Uploading 1 narinfos"6661lb # [ 211.559513] niks3-server[1190]: 2026/09/24 09:29:06 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign6662lb # [ 211.569727] niks3-server[1190]: 2026/09/24 09:29:06 INFO Signed narinfos id=75 count=16663lb # [ 211.576444] niks3-server[1190]: 2026/09/24 09:29:06 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6664lb # [ 211.586977] niks3-server[1190]: 2026/09/24 09:29:06 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6665worker1 # [ 211.631583] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.263Z level=INFO msg="Upload complete. (281ms)"6666lb # [ 211.599822] niks3-server[1190]: 2026/09/24 09:29:06 INFO Received complete push request method=POST path=/api/pushes/75/complete6667lb # [ 211.605508] niks3-server[1190]: 2026/09/24 09:29:06 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6668worker2 # [ 211.641119] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:06Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16669worker1 # [ 211.645853] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=8 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06670worker2 # [ 211.654857] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:06Z level=debug event=want client=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06671worker1 # [ 211.667179] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486672worker1 # [ 211.673681] nix-daemon[855]: reaped child process 2155, status = succeeded6673worker1 # [ 211.681318] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486674worker1 # [ 211.693148] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.325Z level=INFO msg="All 1 paths already cached"6675worker1 # [ 211.701329] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486676worker1 # [ 211.713457] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6677worker1 # [ 211.854144] nix-daemon[855]: accepted connection from pid 2180, user nix-grpc-daemon (trusted)6678worker1 # [ 211.871836] nix-daemon[855]: reaped child process 2183, status = succeeded6679lb # [ 211.846502] niks3-server[1190]: 2026/09/24 09:29:06 INFO Received push request method=POST path=/api/pushes6680worker1 # [ 211.886090] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.518Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6681worker1 # [ 211.887542] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.518Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6682lb # [ 211.875846] niks3-server[1190]: 2026/09/24 09:29:06 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign6683worker1 # [ 211.916162] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.548Z level=INFO msg="Uploading 1 narinfos"6684lb # [ 211.890529] niks3-server[1190]: 2026/09/24 09:29:06 INFO Signed narinfos id=76 count=16685lb # [ 211.897031] niks3-server[1190]: 2026/09/24 09:29:06 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6686lb # [ 211.911776] niks3-server[1190]: 2026/09/24 09:29:06 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6687lb # [ 211.917676] niks3-server[1190]: 2026/09/24 09:29:06 INFO Received complete push request method=POST path=/api/pushes/76/complete6688worker1 # [ 211.953896] nix-grpc-daemon[1965]: time=2026-09-24T09:29:06.586Z level=INFO msg="Upload complete. (227ms)"6689lb # [ 211.923454] niks3-server[1190]: 2026/09/24 09:29:06 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6690worker1 # [ 211.963409] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=6246691worker1 # [ 211.977943] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486692worker1 # [ 211.983950] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6693worker1 # [ 211.989303] nix-daemon[855]: reaped child process 2178, status = succeeded6694worker1 # [ 211.999829] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6695worker1 # [ 212.884685] nix-daemon[855]: reaped child process 2187, status = succeeded6696worker1 # [ 212.980882] nix-daemon[855]: accepted connection from pid 2203, user nix-grpc-daemon (trusted)6697worker1 # [ 212.997113] nix-daemon[855]: reaped child process 2206, status = succeeded6698lb # [ 212.971910] niks3-server[1190]: 2026/09/24 09:29:07 INFO Received push request method=POST path=/api/pushes6699worker1 # [ 213.016530] nix-grpc-daemon[1965]: time=2026-09-24T09:29:07.647Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6700worker1 # [ 213.018585] nix-grpc-daemon[1965]: time=2026-09-24T09:29:07.647Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6701lb # [ 213.019744] niks3-server[1190]: 2026/09/24 09:29:07 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign6702worker1 # [ 213.059587] nix-grpc-daemon[1965]: time=2026-09-24T09:29:07.691Z level=INFO msg="Uploading 1 narinfos"6703lb # [ 213.032234] niks3-server[1190]: 2026/09/24 09:29:07 INFO Signed narinfos id=77 count=16704lb # [ 213.041552] niks3-server[1190]: 2026/09/24 09:29:07 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6705lb # [ 213.050735] niks3-server[1190]: 2026/09/24 09:29:07 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6706lb # [ 213.060346] niks3-server[1190]: 2026/09/24 09:29:07 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6707lb # [ 213.063225] niks3-server[1190]: 2026/09/24 09:29:07 INFO Received complete push request method=POST path=/api/pushes/77/complete6708lb # [ 213.069149] niks3-server[1190]: 2026/09/24 09:29:07 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6709worker1 # [ 213.099570] nix-grpc-daemon[1965]: time=2026-09-24T09:29:07.731Z level=INFO msg="Upload complete. (287ms)"6710worker2 # [ 213.105258] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:07Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16711worker1 # [ 213.113176] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=9 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06712worker2 # [ 213.118494] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:07Z level=debug event=want client=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06713worker1 # [ 213.135454] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486714worker1 # [ 213.141993] nix-daemon[855]: reaped child process 2185, status = succeeded6715worker1 # [ 213.150285] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486716worker1 # [ 213.162040] nix-grpc-daemon[1965]: time=2026-09-24T09:29:07.794Z level=INFO msg="All 1 paths already cached"6717worker1 # [ 213.170122] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486718worker1 # [ 213.181215] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6719worker1 # [ 213.320808] nix-daemon[855]: accepted connection from pid 2210, user nix-grpc-daemon (trusted)6720worker1 # [ 213.335823] nix-daemon[855]: reaped child process 2213, status = succeeded6721lb # [ 213.311776] niks3-server[1190]: 2026/09/24 09:29:07 INFO Received push request method=POST path=/api/pushes6722worker1 # [ 213.351803] nix-grpc-daemon[1965]: time=2026-09-24T09:29:07.983Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6723worker1 # [ 213.353538] nix-grpc-daemon[1965]: time=2026-09-24T09:29:07.983Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6724lb # [ 213.340383] niks3-server[1190]: 2026/09/24 09:29:07 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign6725worker1 # [ 213.377231] nix-grpc-daemon[1965]: time=2026-09-24T09:29:08.009Z level=INFO msg="Uploading 1 narinfos"6726lb # [ 213.353145] niks3-server[1190]: 2026/09/24 09:29:07 INFO Signed narinfos id=78 count=16727lb # [ 213.361748] niks3-server[1190]: 2026/09/24 09:29:07 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6728worker1 # [ 213.407824] nix-grpc-daemon[1965]: time=2026-09-24T09:29:08.040Z level=INFO msg="Upload complete. (214ms)"6729lb # [ 213.376425] niks3-server[1190]: 2026/09/24 09:29:07 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6730worker1 # [ 213.416305] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=6246731lb # [ 213.387494] niks3-server[1190]: 2026/09/24 09:29:07 INFO Received complete push request method=POST path=/api/pushes/78/complete6732lb # [ 213.391848] niks3-server[1190]: 2026/09/24 09:29:08 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6733worker1 # [ 213.429533] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486734worker1 # [ 213.437507] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6735worker1 # [ 213.440270] nix-daemon[855]: reaped child process 2208, status = succeeded6736worker1 # [ 213.449961] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6737worker1 # [ 214.748743] nix-daemon[855]: reaped child process 2217, status = succeeded6738worker1 # [ 214.828763] nix-daemon[855]: accepted connection from pid 2233, user nix-grpc-daemon (trusted)6739worker1 # [ 214.845937] nix-daemon[855]: reaped child process 2236, status = succeeded6740lb # [ 214.821775] niks3-server[1190]: 2026/09/24 09:29:09 INFO Received push request method=POST path=/api/pushes6741worker1 # [ 214.865029] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.495Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6742worker1 # [ 214.866785] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.496Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6743lb # [ 214.861403] niks3-server[1190]: 2026/09/24 09:29:09 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6744worker1 # [ 214.905083] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.537Z level=INFO msg="Uploading 1 narinfos"6745lb # [ 214.876911] niks3-server[1190]: 2026/09/24 09:29:09 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign6746lb # [ 214.886969] niks3-server[1190]: 2026/09/24 09:29:09 INFO Signed narinfos id=79 count=16747lb # [ 214.892777] niks3-server[1190]: 2026/09/24 09:29:09 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6748lb # [ 214.904645] niks3-server[1190]: 2026/09/24 09:29:09 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6749worker1 # [ 214.941152] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.573Z level=INFO msg="Upload complete. (279ms)"6750lb # [ 214.910957] niks3-server[1190]: 2026/09/24 09:29:09 INFO Received complete push request method=POST path=/api/pushes/79/complete6751lb # [ 214.921691] niks3-server[1190]: 2026/09/24 09:29:09 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6752worker2 # [ 214.950734] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:09Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16753worker1 # [ 214.954093] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=10 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06754worker2 # [ 214.958228] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:09Z level=debug event=want client=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06755worker1 # [ 214.976817] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486756worker1 # [ 214.986159] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486757worker1 # [ 214.992793] nix-daemon[855]: reaped child process 2215, status = succeeded6758worker1 # [ 214.996903] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.627Z level=INFO msg="All 2 paths already cached"6759worker1 # [ 215.000998] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486760worker1 # [ 215.016288] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6761worker1 # [ 215.148040] nix-daemon[855]: accepted connection from pid 2240, user nix-grpc-daemon (trusted)6762worker1 # [ 215.166495] nix-daemon[855]: reaped child process 2243, status = succeeded6763lb # [ 215.138093] niks3-server[1190]: 2026/09/24 09:29:09 INFO Received push request method=POST path=/api/pushes6764worker1 # [ 215.178894] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.810Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6765worker1 # [ 215.180845] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.810Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6766lb # [ 215.169821] niks3-server[1190]: 2026/09/24 09:29:09 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6767worker1 # [ 215.213591] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.845Z level=INFO msg="Uploading 1 narinfos"6768lb # [ 215.185031] niks3-server[1190]: 2026/09/24 09:29:09 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign6769lb # [ 215.194128] niks3-server[1190]: 2026/09/24 09:29:09 INFO Signed narinfos id=80 count=16770lb # [ 215.199949] niks3-server[1190]: 2026/09/24 09:29:09 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6771lb # [ 215.212153] niks3-server[1190]: 2026/09/24 09:29:09 INFO Received complete push request method=POST path=/api/pushes/80/complete6772worker1 # [ 215.253577] nix-grpc-daemon[1965]: time=2026-09-24T09:29:09.885Z level=INFO msg="Upload complete. (229ms)"6773lb # [ 215.223919] niks3-server[1190]: 2026/09/24 09:29:09 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6774worker1 # [ 215.264586] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=7526775worker1 # [ 215.282091] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486776worker1 # [ 215.290372] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6777worker1 # [ 215.297540] nix-daemon[855]: reaped child process 2238, status = succeeded6778worker1 # [ 215.307265] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6779worker1 # [ 216.376801] nix-daemon[855]: reaped child process 2247, status = succeeded6780worker1 # [ 216.492463] nix-daemon[855]: accepted connection from pid 2263, user nix-grpc-daemon (trusted)6781worker1 # [ 216.511895] nix-daemon[855]: reaped child process 2266, status = succeeded6782lb # [ 216.485646] niks3-server[1190]: 2026/09/24 09:29:11 INFO Received push request method=POST path=/api/pushes6783worker1 # [ 216.532003] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.162Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6784worker1 # [ 216.533920] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.162Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6785lb # [ 216.527724] niks3-server[1190]: 2026/09/24 09:29:11 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign6786worker1 # [ 216.568279] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.200Z level=INFO msg="Uploading 1 narinfos"6787lb # [ 216.541857] niks3-server[1190]: 2026/09/24 09:29:11 INFO Signed narinfos id=81 count=16788lb # [ 216.547461] niks3-server[1190]: 2026/09/24 09:29:11 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6789lb # [ 216.558739] niks3-server[1190]: 2026/09/24 09:29:11 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6790lb # [ 216.568856] niks3-server[1190]: 2026/09/24 09:29:11 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6791lb # [ 216.574772] niks3-server[1190]: 2026/09/24 09:29:11 INFO Received complete push request method=POST path=/api/pushes/81/complete6792worker1 # [ 216.606955] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.239Z level=INFO msg="Upload complete. (288ms)"6793lb # [ 216.579315] niks3-server[1190]: 2026/09/24 09:29:11 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6794worker2 # [ 216.614421] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:11Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16795worker1 # [ 216.620458] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=11 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06796worker1 # [ 216.640355] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486797worker1 # [ 216.645804] nix-daemon[855]: reaped child process 2245, status = succeeded6798worker1 # [ 216.648966] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486799worker1 # [ 216.659977] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.292Z level=INFO msg="All 2 paths already cached"6800worker1 # [ 216.668788] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486801worker1 # [ 216.682425] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6802worker1 # [ 216.823720] nix-daemon[855]: accepted connection from pid 2270, user nix-grpc-daemon (trusted)6803worker1 # [ 216.844496] nix-daemon[855]: reaped child process 2273, status = succeeded6804lb # [ 216.811050] niks3-server[1190]: 2026/09/24 09:29:11 INFO Received push request method=POST path=/api/pushes6805worker1 # [ 216.852704] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.484Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6806worker1 # [ 216.854160] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.484Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6807lb # [ 216.839963] niks3-server[1190]: 2026/09/24 09:29:11 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6808worker1 # [ 216.884257] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.516Z level=INFO msg="Uploading 1 narinfos"6809lb # [ 216.853132] niks3-server[1190]: 2026/09/24 09:29:11 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign6810lb # [ 216.859528] niks3-server[1190]: 2026/09/24 09:29:11 INFO Signed narinfos id=82 count=16811lb # [ 216.867146] niks3-server[1190]: 2026/09/24 09:29:11 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6812lb # [ 216.881230] niks3-server[1190]: 2026/09/24 09:29:11 INFO Received complete push request method=POST path=/api/pushes/82/complete6813worker1 # [ 216.923400] nix-grpc-daemon[1965]: time=2026-09-24T09:29:11.555Z level=INFO msg="Upload complete. (230ms)"6814lb # [ 216.892278] niks3-server[1190]: 2026/09/24 09:29:11 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6815worker1 # [ 216.931809] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=7526816worker1 # [ 216.947587] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486817worker1 # [ 216.953971] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6818worker1 # [ 216.958616] nix-daemon[855]: reaped child process 2268, status = succeeded6819worker1 # [ 216.968872] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6820worker1 # [ 218.024370] nix-daemon[855]: reaped child process 2277, status = succeeded6821worker1 # [ 218.113849] nix-daemon[855]: accepted connection from pid 2293, user nix-grpc-daemon (trusted)6822worker1 # [ 218.131485] nix-daemon[855]: reaped child process 2296, status = succeeded6823lb # [ 218.104794] niks3-server[1190]: 2026/09/24 09:29:12 INFO Received push request method=POST path=/api/pushes6824worker1 # [ 218.146819] nix-grpc-daemon[1965]: time=2026-09-24T09:29:12.778Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6825worker1 # [ 218.148404] nix-grpc-daemon[1965]: time=2026-09-24T09:29:12.778Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6826lb # [ 218.140573] niks3-server[1190]: 2026/09/24 09:29:12 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6827lb # [ 218.158318] niks3-server[1190]: 2026/09/24 09:29:12 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6828worker1 # [ 218.200670] nix-grpc-daemon[1965]: time=2026-09-24T09:29:12.832Z level=INFO msg="Uploading 1 narinfos"6829lb # [ 218.170205] niks3-server[1190]: 2026/09/24 09:29:12 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign6830lb # [ 218.178051] niks3-server[1190]: 2026/09/24 09:29:12 INFO Signed narinfos id=83 count=16831lb # [ 218.182861] niks3-server[1190]: 2026/09/24 09:29:12 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6832lb # [ 218.191497] niks3-server[1190]: 2026/09/24 09:29:12 INFO Received complete push request method=POST path=/api/pushes/83/complete6833worker1 # [ 218.230334] nix-grpc-daemon[1965]: time=2026-09-24T09:29:12.862Z level=INFO msg="Upload complete. (250ms)"6834lb # [ 218.201374] niks3-server[1190]: 2026/09/24 09:29:12 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6835worker2 # [ 218.236557] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:12Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16836worker1 # [ 218.243090] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=12 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06837worker1 # [ 218.263926] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486838worker1 # [ 218.269652] nix-daemon[855]: reaped child process 2275, status = succeeded6839worker1 # [ 218.272912] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486840worker1 # [ 218.283950] nix-grpc-daemon[1965]: time=2026-09-24T09:29:12.916Z level=INFO msg="All 2 paths already cached"6841worker1 # [ 218.292781] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:413486842worker1 # [ 218.303825] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6843worker1 # [ 218.436421] nix-daemon[855]: accepted connection from pid 2300, user nix-grpc-daemon (trusted)6844worker1 # [ 218.456890] nix-daemon[855]: reaped child process 2303, status = succeeded6845lb # [ 218.423675] niks3-server[1190]: 2026/09/24 09:29:13 INFO Received push request method=POST path=/api/pushes6846worker1 # [ 218.463789] nix-grpc-daemon[1965]: time=2026-09-24T09:29:13.095Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6847worker1 # [ 218.465490] nix-grpc-daemon[1965]: time=2026-09-24T09:29:13.095Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6848lb # [ 218.451458] niks3-server[1190]: 2026/09/24 09:29:13 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign6849worker1 # [ 218.488495] nix-grpc-daemon[1965]: time=2026-09-24T09:29:13.120Z level=INFO msg="Uploading 1 narinfos"6850lb # [ 218.464770] niks3-server[1190]: 2026/09/24 09:29:13 INFO Signed narinfos id=84 count=16851lb # [ 218.474212] niks3-server[1190]: 2026/09/24 09:29:13 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6852lb # [ 218.485376] niks3-server[1190]: 2026/09/24 09:29:13 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6853worker1 # [ 218.534873] nix-grpc-daemon[1965]: time=2026-09-24T09:29:13.167Z level=INFO msg="Upload complete. (218ms)"6854lb # [ 218.500574] niks3-server[1190]: 2026/09/24 09:29:13 INFO Received complete push request method=POST path=/api/pushes/84/complete6855lb # [ 218.512278] niks3-server[1190]: 2026/09/24 09:29:13 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6856worker1 # [ 218.548330] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=0 paths=1 nar_bytes_in=7526857worker1 # [ 218.564327] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:413486858worker1 # [ 218.572154] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6859worker1 # [ 218.578223] nix-daemon[855]: reaped child process 2298, status = succeeded6860worker1 # [ 218.590804] nix-daemon[855]: accepted connection from pid 1905, user nix-grpc-daemon (trusted)6861worker1 # [ 219.425264] nix-daemon[855]: reaped child process 2307, status = succeeded6862worker1 # [ 219.535439] nix-daemon[855]: accepted connection from pid 2323, user nix-grpc-daemon (trusted)6863worker1 # [ 219.553451] nix-daemon[855]: reaped child process 2326, status = succeeded6864lb # [ 219.530199] niks3-server[1190]: 2026/09/24 09:29:14 INFO Received push request method=POST path=/api/pushes6865worker1 # [ 219.572392] nix-grpc-daemon[1965]: time=2026-09-24T09:29:14.201Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6866worker1 # [ 219.573856] nix-grpc-daemon[1965]: time=2026-09-24T09:29:14.201Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6867lb # [ 219.575325] niks3-server[1190]: 2026/09/24 09:29:14 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign6868worker1 # [ 219.613707] nix-grpc-daemon[1965]: time=2026-09-24T09:29:14.245Z level=INFO msg="Uploading 1 narinfos"6869lb # [ 219.591338] niks3-server[1190]: 2026/09/24 09:29:14 INFO Signed narinfos id=85 count=16870lb # [ 219.598305] niks3-server[1190]: 2026/09/24 09:29:14 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6871lb # [ 219.605344] niks3-server[1190]: 2026/09/24 09:29:14 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6872lb # [ 219.609855] niks3-server[1190]: 2026/09/24 09:29:14 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6873worker1 # [ 219.653433] nix-grpc-daemon[1965]: time=2026-09-24T09:29:14.285Z level=INFO msg="Upload complete. (250ms)"6874lb # [ 219.622762] niks3-server[1190]: 2026/09/24 09:29:14 INFO Received complete push request method=POST path=/api/pushes/85/complete6875lb # [ 219.630353] niks3-server[1190]: 2026/09/24 09:29:14 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6876worker1 # [ 219.664973] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:41348 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=13 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06877worker1 # [ 219.682037] nix-daemon[855]: reaped child process 2305, status = succeeded6878client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 16.01 seconds)6879worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6880worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6881client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6882client # [ 219.893582] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6883client # Running as unit: gpu.service; invocation ID: a50923ec7690448fa6a2103876074adc6884client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6885lb: must succeed: curl -sf localhost:9901/clusters6886lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6887lb: must succeed: curl -sf localhost:9901/clusters6888lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6889worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6890worker2 # [ 220.048975] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362566891client # [ 220.094394] nix[1633]: don't know how to build these paths:6892client # [ 220.095355] nix[1633]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6893worker1 # [ 220.061274] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:413486894worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6895worker2 # [ 220.082038] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:14Z level=debug event=want client=3 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06896client # [ 220.127955] nix[1633]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6897lb: must succeed: curl -sf localhost:9901/clusters6898lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6899lb: must succeed: curl -sf localhost:9901/clusters6900lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6901worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6902worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6903client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6904lb # [ 228.055649] envoy[898]: conn peer=[2001:db8:1::1]:41814 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27183 tx=12389 ms=157296905client # [ 250.129522] nix[1633]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6906client # [ 250.141629] nix[1633]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6907client # [ 250.181882] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6908client # [ 250.189917] systemd[1]: gpu.service: Failed with result 'exit-code'.6909client # [ 250.193694] systemd[1]: gpu.service: Consumed 189ms CPU time over 30.295s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6910client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.08 seconds)6911client: must succeed: journalctl -u gpu -o cat6912client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6913lb: must succeed: curl -sf localhost:9901/clusters6914lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6915lb: must succeed: curl -sf localhost:9901/clusters6916lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6917worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6918worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6919(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.81 seconds)6920subtest: low disk drains a worker and builds go to the other6921worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6922worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.14 seconds)6923worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6924worker1 # [ 252.234725] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:46Z level=info event=unhealthy reason=min_free6925worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6926client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26927client # don't know how to build these paths:6928client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6929worker2 # [ 252.998177] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:362566930worker2 # [ 253.006660] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566931worker2 # [ 253.028242] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=debug event=want client=4 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06932worker2 # [ 253.034939] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16933worker2 # [ 253.044371] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566934worker2 # [ 253.055485] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566935client # copying 1 paths to 'lb:50051'...6936client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6937worker2 # [ 253.070820] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362566938worker2 # [ 253.085543] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6939worker2 # [ 253.239855] nix-daemon[842]: accepted connection from pid 1839, user nix-grpc-daemon (trusted)6940worker2 # [ 253.260856] nix-daemon[842]: reaped child process 1842, status = succeeded6941lb # [ 253.239288] niks3-server[1190]: 2026/09/24 09:29:47 INFO Received push request method=POST path=/api/pushes6942worker2 # [ 253.275314] nix-grpc-daemon[1545]: time=2026-09-24T09:29:47.901Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6943worker2 # [ 253.277450] nix-grpc-daemon[1545]: time=2026-09-24T09:29:47.903Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6944lb # [ 253.267925] niks3-server[1190]: 2026/09/24 09:29:47 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign6945worker2 # [ 253.303179] nix-grpc-daemon[1545]: time=2026-09-24T09:29:47.929Z level=INFO msg="Uploading 1 narinfos"6946lb # [ 253.279003] niks3-server[1190]: 2026/09/24 09:29:47 INFO Signed narinfos id=86 count=16947lb # [ 253.289769] niks3-server[1190]: 2026/09/24 09:29:47 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6948lb # [ 253.303306] niks3-server[1190]: 2026/09/24 09:29:47 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6949lb # [ 253.318618] niks3-server[1190]: 2026/09/24 09:29:47 INFO Received complete push request method=POST path=/api/pushes/86/complete6950worker2 # [ 253.353083] nix-grpc-daemon[1545]: time=2026-09-24T09:29:47.979Z level=INFO msg="Upload complete. (252ms)"6951lb # [ 253.328346] niks3-server[1190]: 2026/09/24 09:29:47 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6952worker2 # [ 253.363697] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=5126953worker2 # [ 253.382439] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566954worker2 # [ 253.389868] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6955worker2 # [ 253.395743] nix-daemon[842]: reaped child process 1837, status = succeeded6956client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6957worker2 # [ 253.409267] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6958client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6959client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6960client # farm-a> LOG-farm-a-drain6961worker2 # [ 254.892760] nix-daemon[842]: reaped child process 1846, status = succeeded6962worker2 # [ 255.002938] nix-daemon[842]: accepted connection from pid 1862, user nix-grpc-daemon (trusted)6963worker2 # [ 255.022103] nix-daemon[842]: reaped child process 1865, status = succeeded6964lb # [ 255.002983] niks3-server[1190]: 2026/09/24 09:29:49 INFO Received push request method=POST path=/api/pushes6965worker2 # [ 255.038726] nix-grpc-daemon[1545]: time=2026-09-24T09:29:49.663Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6966worker2 # [ 255.040512] nix-grpc-daemon[1545]: time=2026-09-24T09:29:49.663Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6967lb # [ 255.039629] niks3-server[1190]: 2026/09/24 09:29:49 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign6968worker2 # [ 255.073433] nix-grpc-daemon[1545]: time=2026-09-24T09:29:49.699Z level=INFO msg="Uploading 1 narinfos"6969lb # [ 255.053737] niks3-server[1190]: 2026/09/24 09:29:49 INFO Signed narinfos id=87 count=16970lb # [ 255.059765] niks3-server[1190]: 2026/09/24 09:29:49 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6971lb # [ 255.067870] niks3-server[1190]: 2026/09/24 09:29:49 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6972lb # [ 255.078967] niks3-server[1190]: 2026/09/24 09:29:49 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6973worker2 # [ 255.116730] nix-grpc-daemon[1545]: time=2026-09-24T09:29:49.743Z level=INFO msg="Upload complete. (292ms)"6974lb # [ 255.088959] niks3-server[1190]: 2026/09/24 09:29:49 INFO Received complete push request method=POST path=/api/pushes/87/complete6975lb # [ 255.092970] niks3-server[1190]: 2026/09/24 09:29:49 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6976worker2 # [ 255.136537] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=14 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06977worker2 # [ 255.156881] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:49Z level=debug event=want client=4 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06978worker2 # [ 255.165454] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:49Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16979worker2 # [ 255.174979] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362566980worker2 # [ 255.181878] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362566981client # copying 1 paths to 'lb:50051'...6982worker2 # [ 255.188151] nix-daemon[842]: reaped child process 1844, status = succeeded6983worker2 # [ 255.190793] nix-grpc-daemon[1545]: time=2026-09-24T09:29:49.813Z level=INFO msg="All 1 paths already cached"6984client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6985worker2 # [ 255.195360] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362566986worker2 # [ 255.212246] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6987worker2 # [ 255.345597] nix-daemon[842]: accepted connection from pid 1871, user nix-grpc-daemon (trusted)6988worker2 # [ 255.363413] nix-daemon[842]: reaped child process 1874, status = succeeded6989lb # [ 255.341874] niks3-server[1190]: 2026/09/24 09:29:49 INFO Received push request method=POST path=/api/pushes6990worker2 # [ 255.379639] nix-grpc-daemon[1545]: time=2026-09-24T09:29:50.005Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6991worker2 # [ 255.381371] nix-grpc-daemon[1545]: time=2026-09-24T09:29:50.005Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6992lb # [ 255.374960] niks3-server[1190]: 2026/09/24 09:29:49 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign6993worker2 # [ 255.407858] nix-grpc-daemon[1545]: time=2026-09-24T09:29:50.034Z level=INFO msg="Uploading 1 narinfos"6994lb # [ 255.385425] niks3-server[1190]: 2026/09/24 09:29:50 INFO Signed narinfos id=88 count=16995lb # [ 255.387717] niks3-server[1190]: 2026/09/24 09:29:50 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6996lb # [ 255.394942] niks3-server[1190]: 2026/09/24 09:29:50 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6997lb # [ 255.404515] niks3-server[1190]: 2026/09/24 09:29:50 INFO Received complete push request method=POST path=/api/pushes/88/complete6998worker2 # [ 255.436579] nix-grpc-daemon[1545]: time=2026-09-24T09:29:50.063Z level=INFO msg="Upload complete. (217ms)"6999lb # [ 255.410366] niks3-server[1190]: 2026/09/24 09:29:50 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7000worker2 # [ 255.445075] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=6327001worker2 # [ 255.458228] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362567002worker2 # [ 255.464280] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)7003worker2 # [ 255.468875] nix-daemon[842]: reaped child process 1869, status = succeeded7004client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7005worker2 # [ 255.480628] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)7006client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7007client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7008client # farm-b> LOG-farm-b-drain7009worker2 # [ 256.784468] nix-daemon[842]: reaped child process 1878, status = succeeded7010worker2 # [ 256.855636] nix-daemon[842]: accepted connection from pid 1894, user nix-grpc-daemon (trusted)7011worker2 # [ 256.876121] nix-daemon[842]: reaped child process 1897, status = succeeded7012lb # [ 256.847129] niks3-server[1190]: 2026/09/24 09:29:51 INFO Received push request method=POST path=/api/pushes7013worker2 # [ 256.884831] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.511Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7014worker2 # [ 256.889772] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.513Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7015lb # [ 256.890052] niks3-server[1190]: 2026/09/24 09:29:51 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7016worker2 # [ 256.926679] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.552Z level=INFO msg="Uploading 1 narinfos"7017lb # [ 256.904055] niks3-server[1190]: 2026/09/24 09:29:51 INFO Signed narinfos id=89 count=17018lb # [ 256.906816] niks3-server[1190]: 2026/09/24 09:29:51 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7019lb # [ 256.924216] niks3-server[1190]: 2026/09/24 09:29:51 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7020worker2 # [ 256.957432] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.583Z level=INFO msg="Upload complete. (267ms)"7021lb # [ 256.929171] niks3-server[1190]: 2026/09/24 09:29:51 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7022lb # [ 256.936342] niks3-server[1190]: 2026/09/24 09:29:51 INFO Received complete push request method=POST path=/api/pushes/89/complete7023lb # [ 256.939937] niks3-server[1190]: 2026/09/24 09:29:51 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7024worker2 # [ 256.971171] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=15 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=07025worker2 # [ 256.992473] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:51Z level=debug event=want client=4 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07026worker2 # [ 256.997893] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:51Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17027worker2 # [ 257.003669] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362567028worker2 # [ 257.008727] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:362567029worker2 # [ 257.012743] nix-daemon[842]: reaped child process 1876, status = succeeded7030worker2 # [ 257.020795] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.646Z level=INFO msg="All 2 paths already cached"7031client # copying 1 paths to 'lb:50051'...7032client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7033worker2 # [ 257.028089] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:362567034worker2 # [ 257.038863] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)7035worker2 # [ 257.178866] nix-daemon[842]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)7036worker2 # [ 257.198420] nix-daemon[842]: reaped child process 1906, status = succeeded7037lb # [ 257.174707] niks3-server[1190]: 2026/09/24 09:29:51 INFO Received push request method=POST path=/api/pushes7038worker2 # [ 257.211234] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.837Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7039worker2 # [ 257.212975] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.837Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7040lb # [ 257.205684] niks3-server[1190]: 2026/09/24 09:29:51 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7041worker2 # [ 257.243547] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.869Z level=INFO msg="Uploading 1 narinfos"7042lb # [ 257.220432] niks3-server[1190]: 2026/09/24 09:29:51 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7043lb # [ 257.230466] niks3-server[1190]: 2026/09/24 09:29:51 INFO Signed narinfos id=90 count=17044lb # [ 257.236812] niks3-server[1190]: 2026/09/24 09:29:51 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7045lb # [ 257.249524] niks3-server[1190]: 2026/09/24 09:29:51 INFO Received complete push request method=POST path=/api/pushes/90/complete7046lb # [ 257.254715] niks3-server[1190]: 2026/09/24 09:29:51 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7047worker2 # [ 257.281672] nix-grpc-daemon[1545]: time=2026-09-24T09:29:51.908Z level=INFO msg="Upload complete. (231ms)"7048worker2 # [ 257.290492] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=0 paths=1 nar_bytes_in=7607049worker2 # [ 257.304151] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:362567050worker2 # [ 257.311037] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)7051worker2 # [ 257.316346] nix-daemon[842]: reaped child process 1901, status = succeeded7052client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7053client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7054worker2 # [ 257.330760] nix-daemon[842]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)7055client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7056lb # [ 258.058542] envoy[898]: conn peer=[2001:db8:1::1]:56554 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=614 ms=300757057client # farm-top> LOG-farm-top-drain7058worker2 # [ 258.394630] nix-daemon[842]: reaped child process 1910, status = succeeded7059worker2 # [ 258.526602] nix-daemon[842]: accepted connection from pid 1926, user nix-grpc-daemon (trusted)7060worker2 # [ 258.542477] nix-daemon[842]: reaped child process 1929, status = succeeded7061lb # [ 258.521847] niks3-server[1190]: 2026/09/24 09:29:53 INFO Received push request method=POST path=/api/pushes7062worker2 # [ 258.561097] nix-grpc-daemon[1545]: time=2026-09-24T09:29:53.186Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7063worker2 # [ 258.562999] nix-grpc-daemon[1545]: time=2026-09-24T09:29:53.186Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7064lb # [ 258.559962] niks3-server[1190]: 2026/09/24 09:29:53 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7065worker2 # [ 258.602760] nix-grpc-daemon[1545]: time=2026-09-24T09:29:53.228Z level=INFO msg="Uploading 1 narinfos"7066lb # [ 258.574405] niks3-server[1190]: 2026/09/24 09:29:53 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7067lb # [ 258.588987] niks3-server[1190]: 2026/09/24 09:29:53 INFO Signed narinfos id=91 count=17068lb # [ 258.596115] niks3-server[1190]: 2026/09/24 09:29:53 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7069lb # [ 258.604122] niks3-server[1190]: 2026/09/24 09:29:53 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7070lb # [ 258.615553] niks3-server[1190]: 2026/09/24 09:29:53 INFO Received complete push request method=POST path=/api/pushes/91/complete7071worker2 # [ 258.645828] nix-grpc-daemon[1545]: time=2026-09-24T09:29:53.270Z level=INFO msg="Upload complete. (302ms)"7072lb # [ 258.619364] niks3-server[1190]: 2026/09/24 09:29:53 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7073worker2 # [ 258.658784] nix-grpc-daemon[1473]: ts=2026-09-24T09:29:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36256 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=16 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07074worker2 # [ 258.679076] nix-daemon[842]: reaped child process 1908, status = succeeded7075client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.93 seconds)7076client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7077client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7078worker1: must succeed: rm /nix/.rw-store/fill7079worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7080worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7081worker1 # [ 259.237482] nix-grpc-daemon[1905]: ts=2026-09-24T09:29:53Z level=info event=healthy reason=min_free7082worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7083(finished: subtest: low disk drains a worker and builds go to the other, in 9.69 seconds)7084(finished: run the VM test script, in 261.15 seconds)7085test script finished in 261.38s7086cleanup7087kill QemuMachine (pid 49)7088client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7089client # [2026-09-24T09:29:54Z INFO virtiofsd] Client disconnected, shutting down7090client # [2026-09-24T09:29:54Z INFO virtiofsd] Client disconnected, shutting down7091client # [2026-09-24T09:29:54Z INFO virtiofsd] Client disconnected, shutting down7092kill QemuMachine (pid 50)7093lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7094lb # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7095lb # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7096lb # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7097kill QemuMachine (pid 51)7098worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7099worker1 # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7100worker1 # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7101worker1 # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7102kill QemuMachine (pid 52)7103worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7104worker2 # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7105worker2 # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7106worker2 # [2026-09-24T09:29:55Z INFO virtiofsd] Client disconnected, shutting down7107(finished: cleanup, in 1.33 seconds)7108additionally exposed symbols:7109 client, lb, worker1, worker2,7110 vlan1,7111 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh