vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #508
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb # Formatting '/build/vm-state-lb/tmp.nOfWybEVGv', fmt=raw size=107374182414client # Disk image does not exist, creating the virtualisation disk image...15worker1 # Disk image does not exist, creating the virtualisation disk image...16client: QEMU running (pid 49)17client # Formatting '/build/vm-state-client/tmp.FSib7LfjrR', fmt=raw size=107374182418lb # mke2fs 1.47.4 (6-Mar-2025)19lb: QEMU running (pid 50)20worker2: QEMU running (pid 52)21client # mke2fs 1.47.4 (6-Mar-2025)22lb # Discarding device blocks: 0/262144 done23worker2 # Disk image does not exist, creating the virtualisation disk image...24client # Discarding device blocks: 0/262144 done25lb # Creating filesystem with 262144 4k blocks and 65536 inodes26worker2 # Formatting '/build/vm-state-worker2/tmp.mZYubaQsHZ', fmt=raw size=107374182427client # Creating filesystem with 262144 4k blocks and 65536 inodes28lb # Filesystem UUID: ecc6f207-dc8e-431d-b120-fb330412fd4229worker2 # mke2fs 1.47.4 (6-Mar-2025)30client # Filesystem UUID: 46f38149-c1f1-4a74-96ca-737ecde3584631lb # Superblock backups stored on blocks:32worker2 # Discarding device blocks: 0/262144 done33client # Superblock backups stored on blocks:34lb # 32768, 98304, 163840, 22937635worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes36client # 32768, 98304, 163840, 22937637lb # 38worker2 # Filesystem UUID: 7967d1d0-0434-48a8-b218-64dc68f647f439client # 40lb # Allocating group tables: 0/8 done41worker2 # Superblock backups stored on blocks:42client # Allocating group tables: 0/8 done43lb # Writing inode tables: 0/8 done44worker2 # 32768, 98304, 163840, 22937645client # Writing inode tables: 0/8 done46lb # Creating journal (8192 blocks): done47worker2 # 48client # Creating journal (8192 blocks): done49lb # Writing superblocks and filesystem accounting information: 0/8 done50worker2 # Allocating group tables: 0/8 done51client # Writing superblocks and filesystem accounting information: 0/8 done52lb # 53worker2 # Writing inode tables: 0/8 done54client # 55lb # Virtualisation disk image created.56worker2 # Creating journal (8192 blocks): done57client # Virtualisation disk image created.58lb # Starting virtiofs daemons...59worker2 # Writing superblocks and filesystem accounting information: 0/8 done60client # Starting virtiofs daemons...61lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)62worker2 # 63client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)64lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether65worker2 # Virtualisation disk image created.66client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether67lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)68worker2 # Starting virtiofs daemons...69client # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...70lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)73lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether75client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether76lb # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...78client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79lb # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...80worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)81client # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82lb # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...83worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84client # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...85lb # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...86worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...87client # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...88lb # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests89worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90client # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests91lb # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests92worker2 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93client # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests94lb # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests95worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...96client # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests97worker1 # Formatting '/build/vm-state-worker1/tmp.wIlLy48BwW', fmt=raw size=107374182498worker1: QEMU running (pid 51)99worker1 # mke2fs 1.47.4 (6-Mar-2025)100worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests101worker1 # Discarding device blocks: 0/262144 done102worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests103worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes104(finished: start all VMs, in 0.51 seconds)105worker1 # Filesystem UUID: 60eb5a0c-8c20-428a-b205-a289251928cb106client: waiting for unit network-addresses-eth1.service107worker1 # Superblock backups stored on blocks:108client: waiting for the VM to finish booting109worker1 # 32768, 98304, 163840, 229376110worker2 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests111worker1 # 112worker1 # Allocating group tables: 0/8 done113worker1 # Writing inode tables: 0/8 done114worker1 # Creating journal (8192 blocks): done115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker1 # [2026-09-22T16:58:24Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-22T16:58:24Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135client # [ 0.000000] Machine model: linux,dummy-virt136client # [ 0.000000] efi: UEFI not found.137client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140client # [ 0.000000] Zone ranges:141client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142client # [ 0.000000] DMA32 empty143client # [ 0.000000] Normal empty144client # [ 0.000000] Device empty145client # [ 0.000000] Movable zone start for each node146client # [ 0.000000] Early memory node ranges147client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150client # [ 0.000000] psci: probing for conduit method from DT.151client # [ 0.000000] psci: PSCIv1.3 detected in firmware.152client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153client # [ 0.000000] psci: Trusted OS migration not required154client # [ 0.000000] psci: SMC Calling Convention v1.1155client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157client # [ 0.000000] Detected PIPT I-cache on CPU0158client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159client # [ 0.000000] CPU features: detected: GICv3 CPU interface160client # [ 0.000000] CPU features: detected: Spectre-v4161client # [ 0.000000] CPU features: detected: Spectre-BHB162client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164client # [ 0.000000] alternatives: applying boot alternatives165client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test/init regInfo=/nix/store/149kn4k15jb67kmb28r78yiz2vw4r88d-closure-info/registration console=ttyAMA0,115200n8 console=tty0166client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/149kn4k15jb67kmb28r78yiz2vw4r88d-closure-info/registration", will be passed to user space.167client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes168client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)169client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)170client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB171client # [ 0.000000] software IO TLB: area num 1.172client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB173client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)174client # [ 0.000000] Fallback order for Node 0: 0175client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216176client # [ 0.000000] Policy zone: DMA177client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off178client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1179client # [ 0.000000] allocated 3145728 bytes of page_ext180client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages181client # [ 0.000000] ftrace: allocated 294 pages with 4 groups182client # [ 0.000000] rcu: Hierarchical RCU implementation.183client # [ 0.000000] rcu: RCU event tracing is enabled.184client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.185client # [ 0.000000] Trampoline variant of Tasks RCU enabled.186client # [ 0.000000] Rude variant of Tasks RCU enabled.187client # [ 0.000000] Tracing variant of Tasks RCU enabled.188worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]189client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.190lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]191client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1192worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026193worker1 # [ 0.000000] KASLR enabled194client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.195lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026196worker1 # [ 0.000000] random: crng init done197lb # [ 0.000000] KASLR enabled198worker1 # [ 0.000000] Machine model: linux,dummy-virt199lb # [ 0.000000] random: crng init done200client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.201worker1 # [ 0.000000] efi: UEFI not found.202lb # [ 0.000000] Machine model: linux,dummy-virt203lb # [ 0.000000] efi: UEFI not found.204worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT205client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.206lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT207client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0208worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]209client # [ 0.000000] GICv3: 256 SPIs implemented210lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]211worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]212client # [ 0.000000] GICv3: 0 Extended SPIs implemented213worker1 # [ 0.000000] Zone ranges:214lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]215client # [ 0.000000] Root IRQ handler: gic_handle_irq216lb # [ 0.000000] Zone ranges:217worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]218client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI219worker1 # [ 0.000000] DMA32 empty220lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]221worker1 # [ 0.000000] Normal empty222client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0223lb # [ 0.000000] DMA32 empty224worker1 # [ 0.000000] Device empty225lb # [ 0.000000] Normal empty226worker1 # [ 0.000000] Movable zone start for each node227client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000228lb # [ 0.000000] Device empty229worker1 # [ 0.000000] Early memory node ranges230client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]231lb # [ 0.000000] Movable zone start for each node232lb # [ 0.000000] Early memory node ranges233worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]234client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)235lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]236worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]237worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000238lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]239client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)240lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000241worker1 # [ 0.000000] psci: probing for conduit method from DT.242client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000243worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.244lb # [ 0.000000] psci: probing for conduit method from DT.245lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.246worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs247client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000248lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs249worker1 # [ 0.000000] psci: Trusted OS migration not required250client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.251worker1 # [ 0.000000] psci: SMC Calling Convention v1.1252lb # [ 0.000000] psci: Trusted OS migration not required253lb # [ 0.000000] psci: SMC Calling Convention v1.1254client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns255worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)256client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).257lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)258worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296259worker1 # [ 0.000000] Detected PIPT I-cache on CPU0260lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296261client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns262lb # [ 0.000000] Detected PIPT I-cache on CPU0263worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)264client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns265worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface266lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)267client # [ 0.000034] arm-pv: using stolen time PV268worker1 # [ 0.000000] CPU features: detected: Spectre-v4269lb # [ 0.000000] CPU features: detected: GICv3 CPU interface270worker1 # [ 0.000000] CPU features: detected: Spectre-BHB271lb # [ 0.000000] CPU features: detected: Spectre-v4272client # [ 0.000645] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)273lb # [ 0.000000] CPU features: detected: Spectre-BHB274worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38275client # [ 0.000818] Console: colour dummy device 80x25276lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38277worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23278client # [ 0.000826] printk: legacy console [tty0] enabled279worker1 # [ 0.000000] alternatives: applying boot alternatives280lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23281lb # [ 0.000000] alternatives: applying boot alternatives282client # [ 0.001017] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)283client # [ 0.001024] pid_max: default: 32768 minimum: 301284client # [ 0.001099] LSM: initializing lsm=capability,landlock,yama,bpf,ima285client # [ 0.001260] landlock: Up and running.286client # [ 0.001262] Yama: becoming mindful.287client # [ 0.001918] LSM support for eBPF active288worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ry7bqhhhnwa7dlpa75fylilll15fakpb-nixos-system-worker1-test/init regInfo=/nix/store/94nix7rmj7g6brapzr4gg95i44rfy50d-closure-info/registration console=ttyAMA0,115200n8 console=tty0289client # [ 0.002093] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)290lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qvl0xxkr124pdkzdvanphvcpy7197xra-nixos-system-lb-test/init regInfo=/nix/store/4y9byqv3ifqd7wzrdp7iwk6bnwmvvvvc-closure-info/registration console=ttyAMA0,115200n8 console=tty0291worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]292client # [ 0.002127] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)293client # [ 0.004064] rcu: Hierarchical SRCU implementation.294worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026295worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/94nix7rmj7g6brapzr4gg95i44rfy50d-closure-info/registration", will be passed to user space.296worker2 # [ 0.000000] KASLR enabled297client # [ 0.004069] rcu: Max phase no-delay instances is 1000.298lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4y9byqv3ifqd7wzrdp7iwk6bnwmvvvvc-closure-info/registration", will be passed to user space.299worker2 # [ 0.000000] random: crng init done300client # [ 0.005349] fsl-mc MSI: its@8080000 domain created301worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes302worker2 # [ 0.000000] Machine model: linux,dummy-virt303lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes304client # [ 0.005441] EFI services will not be available.305worker2 # [ 0.000000] efi: UEFI not found.306worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)307client # [ 0.005530] smp: Bringing up secondary CPUs ...308worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT309lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)310client # [ 0.005538] smp: Brought up 1 node, 1 CPU311worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)312client # [ 0.005541] SMP: Total of 1 processors activated.313worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]314lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)315client # [ 0.005546] CPU: All CPU(s) started at EL1316worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB317worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]318worker1 # [ 0.000000] software IO TLB: area num 1.319lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB320worker2 # [ 0.000000] Zone ranges:321client # [ 0.005557] CPU features: detected: Branch Target Identification322lb # [ 0.000000] software IO TLB: area num 1.323worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB324client # [ 0.005562] CPU features: detected: ARMv8.4 Translation Table Level325worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]326worker2 # [ 0.000000] DMA32 empty327lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB328worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)329worker2 # [ 0.000000] Normal empty330client # [ 0.005565] CPU features: detected: Instruction cache invalidation not required for I/D coherence331worker2 # [ 0.000000] Device empty332worker1 # [ 0.000000] Fallback order for Node 0: 0333lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)334worker2 # [ 0.000000] Movable zone start for each node335lb # [ 0.000000] Fallback order for Node 0: 0336worker2 # [ 0.000000] Early memory node ranges337client # [ 0.005568] CPU features: detected: Data cache clean to the PoU not required for I/D coherence338worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216339worker1 # [ 0.000000] Policy zone: DMA340lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216341worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]342client # [ 0.005572] CPU features: detected: Common not Private translations343lb # [ 0.000000] Policy zone: DMA344worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off345client # [ 0.005575] CPU features: detected: CRC32 instructions346worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]347lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off348worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1349worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000350client # [ 0.005578] CPU features: detected: Data cache clean to Point of Deep Persistence351worker1 # [ 0.000000] allocated 3145728 bytes of page_ext352lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1353worker2 # [ 0.000000] psci: probing for conduit method from DT.354client # [ 0.005581] CPU features: detected: Data cache clean to Point of Persistence355lb # [ 0.000000] allocated 3145728 bytes of page_ext356worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages357worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.358worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups359lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages360worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs361client # [ 0.005585] CPU features: detected: Data independent timing control (DIT)362lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups363worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.364client # [ 0.005588] CPU features: detected: E0PD365worker2 # [ 0.000000] psci: Trusted OS migration not required366worker1 # [ 0.000000] rcu: RCU event tracing is enabled.367lb # [ 0.000000] rcu: Hierarchical RCU implementation.368worker2 # [ 0.000000] psci: SMC Calling Convention v1.1369client # [ 0.005590] CPU features: detected: Enhanced Counter Virtualization370lb # [ 0.000000] rcu: RCU event tracing is enabled.371worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.372client # [ 0.005593] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)373worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)374lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.375worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.376client # [ 0.005597] CPU features: detected: Enhanced Virtualization Traps377worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.378lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.379worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296380client # [ 0.005600] CPU features: detected: Fine Grained Traps381worker2 # [ 0.000000] Detected PIPT I-cache on CPU0382lb # [ 0.000000] Rude variant of Tasks RCU enabled.383worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.384lb # [ 0.000000] Tracing variant of Tasks RCU enabled.385client # [ 0.005603] CPU features: detected: Generic authentication (architected QARMA5 algorithm)386worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.387worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)388lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.389client # [ 0.005608] CPU features: detected: RCpc load-acquire (LDAPR)390worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface391worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1392client # [ 0.005611] CPU features: detected: LSE atomic instructions393worker2 # [ 0.000000] CPU features: detected: Spectre-v4394lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1395worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.396worker2 # [ 0.000000] CPU features: detected: Spectre-BHB397client # [ 0.005629] CPU features: detected: Privileged Access Never398client # [ 0.005632] CPU features: detected: PMUv3399lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.400worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38401worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.402client # [ 0.005635] CPU features: detected: RAS Extension Support403worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23404lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.405client # [ 0.005638] CPU features: detected: RASv1p1 Extension Support406worker2 # [ 0.000000] alternatives: applying boot alternatives407worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.408client # [ 0.005640] CPU features: detected: Random Number Generator409lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.410worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0411client # [ 0.005643] CPU features: detected: Speculation barrier (SB)412worker1 # [ 0.000000] GICv3: 256 SPIs implemented413lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0414worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented415client # [ 0.005646] CPU features: detected: Stage-2 Force Write-Back416lb # [ 0.000000] GICv3: 256 SPIs implemented417lb # [ 0.000000] GICv3: 0 Extended SPIs implemented418worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq419client # [ 0.005649] CPU features: detected: TLB range maintenance instructions420lb # [ 0.000000] Root IRQ handler: gic_handle_irq421worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI422lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI423worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0424client # [ 0.005654] CPU features: detected: Speculative Store Bypassing Safe (SSBS)425lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0426worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/30asfw61910plj74np6pz2ka43zplm0r-nixos-system-worker2-test/init regInfo=/nix/store/cpmzrhcw14bjzfxvmig0nm1xvcis5jif-closure-info/registration console=ttyAMA0,115200n8 console=tty0427client # [ 0.005695] alternatives: applying system-wide alternatives428worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000429lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000430worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]431client # [ 0.008615] CPU features: detected: BBM Level 2 without TLB conflict abort432lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]433worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/cpmzrhcw14bjzfxvmig0nm1xvcis5jif-closure-info/registration", will be passed to user space.434worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)435lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)436worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes437client # [ 0.008828] Memory: 1405652K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)438client # [ 0.009243] devtmpfs: initialized439worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)440worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)441lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)442worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000443client # [ 0.011094] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)444lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000445worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)446worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000447client # [ 0.011117] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).448lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000449worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB450worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.451worker2 # [ 0.000000] software IO TLB: area num 1.452client # [ 0.011322] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL453lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.454client # [ 0.011326] 0 pages in range for non-PLT usage455worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB456worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns457client # [ 0.011327] 508272 pages in range for PLT usage458lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns459worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)460client # [ 0.011451] pinctrl core: initialized pinctrl subsystem461worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).462worker2 # [ 0.000000] Fallback order for Node 0: 0463client # [ 0.012232] DMI not present or invalid.464lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).465client # [ 0.016072] NET: Registered PF_NETLINK/PF_ROUTE protocol family466worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216467worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns468worker2 # [ 0.000000] Policy zone: DMA469lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns470client # [ 0.018320] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations471worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns472worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off473worker1 # [ 0.000030] arm-pv: using stolen time PV474lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns475client # [ 0.018507] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations476worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1477lb # [ 0.000030] arm-pv: using stolen time PV478worker2 # [ 0.000000] allocated 3145728 bytes of page_ext479client # [ 0.018707] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations480worker1 # [ 0.000426] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)481worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages482client # [ 0.018730] audit: initializing netlink subsys (disabled)483lb # [ 0.000440] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)484worker1 # [ 0.000599] Console: colour dummy device 80x25485worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups486worker1 # [ 0.000606] printk: legacy console [tty0] enabled487lb # [ 0.000624] Console: colour dummy device 80x25488client # [ 0.019268] thermal_sys: Registered thermal governor 'fair_share'489worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.490lb # [ 0.000631] printk: legacy console [tty0] enabled491client # [ 0.019270] thermal_sys: Registered thermal governor 'bang_bang'492worker1 # [ 0.000792] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)493client # [ 0.019274] thermal_sys: Registered thermal governor 'step_wise'494worker2 # [ 0.000000] rcu: RCU event tracing is enabled.495lb # [ 0.000826] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)496worker1 # [ 0.000799] pid_max: default: 32768 minimum: 301497client # [ 0.019277] thermal_sys: Registered thermal governor 'user_space'498lb # [ 0.000833] pid_max: default: 32768 minimum: 301499worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.500worker1 # [ 0.000873] LSM: initializing lsm=capability,landlock,yama,bpf,ima501worker1 # [ 0.000997] landlock: Up and running.502worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.503client # [ 0.019281] thermal_sys: Registered thermal governor 'power_allocator'504lb # [ 0.000915] LSM: initializing lsm=capability,landlock,yama,bpf,ima505worker1 # [ 0.000999] Yama: becoming mindful.506lb # [ 0.001045] landlock: Up and running.507worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.508worker1 # [ 0.001465] LSM support for eBPF active509client # [ 0.019308] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1510lb # [ 0.001047] Yama: becoming mindful.511worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.512lb # [ 0.001507] LSM support for eBPF active513client # [ 0.019316] cpuidle: using governor ladder514worker1 # [ 0.001605] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)515client # [ 0.019322] cpuidle: using governor menu516worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.517lb # [ 0.001654] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)518worker1 # [ 0.001643] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)519client # [ 0.019513] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.520worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1521lb # [ 0.001686] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)522worker1 # [ 0.003506] rcu: Hierarchical SRCU implementation.523client # [ 0.019528] ASID allocator initialised with 65536 entries524lb # [ 0.003595] rcu: Hierarchical SRCU implementation.525worker1 # [ 0.003511] rcu: Max phase no-delay instances is 1000.526worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.527client # [ 0.020829] Serial: AMBA PL011 UART driver528worker1 # [ 0.004766] fsl-mc MSI: its@8080000 domain created529lb # [ 0.003600] rcu: Max phase no-delay instances is 1000.530client # [ 0.026246] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1531worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.532lb # [ 0.004844] fsl-mc MSI: its@8080000 domain created533worker1 # [ 0.004857] EFI services will not be available.534client # [ 0.026406] printk: console [ttyAMA0] enabled535lb # [ 0.004944] EFI services will not be available.536worker1 # [ 0.004921] smp: Bringing up secondary CPUs ...537worker1 # [ 0.004929] smp: Brought up 1 node, 1 CPU538worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.539lb # [ 0.005058] smp: Bringing up secondary CPUs ...540worker1 # [ 0.004932] SMP: Total of 1 processors activated.541lb # [ 0.005066] smp: Brought up 1 node, 1 CPU542worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0543worker1 # [ 0.004937] CPU: All CPU(s) started at EL1544lb # [ 0.005069] SMP: Total of 1 processors activated.545worker2 # [ 0.000000] GICv3: 256 SPIs implemented546lb # [ 0.005074] CPU: All CPU(s) started at EL1547worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented548worker1 # [ 0.004948] CPU features: detected: Branch Target Identification549worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq550lb # [ 0.005085] CPU features: detected: Branch Target Identification551worker1 # [ 0.004953] CPU features: detected: ARMv8.4 Translation Table Level552worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI553lb # [ 0.005089] CPU features: detected: ARMv8.4 Translation Table Level554worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0555worker1 # [ 0.004956] CPU features: detected: Instruction cache invalidation not required for I/D coherence556lb # [ 0.005093] CPU features: detected: Instruction cache invalidation not required for I/D coherence557worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000558worker1 # [ 0.004960] CPU features: detected: Data cache clean to the PoU not required for I/D coherence559worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]560lb # [ 0.005096] CPU features: detected: Data cache clean to the PoU not required for I/D coherence561worker1 # [ 0.004964] CPU features: detected: Common not Private translations562worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)563lb # [ 0.005100] CPU features: detected: Common not Private translations564worker1 # [ 0.004967] CPU features: detected: CRC32 instructions565lb # [ 0.005103] CPU features: detected: CRC32 instructions566worker1 # [ 0.004970] CPU features: detected: Data cache clean to Point of Deep Persistence567worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)568lb # [ 0.005106] CPU features: detected: Data cache clean to Point of Deep Persistence569client # [ 0.144405] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages570worker1 # [ 0.004973] CPU features: detected: Data cache clean to Point of Persistence571worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000572client # [ 0.144421] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page573lb # [ 0.005110] CPU features: detected: Data cache clean to Point of Persistence574worker1 # [ 0.004976] CPU features: detected: Data independent timing control (DIT)575worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000576worker1 # [ 0.004979] CPU features: detected: E0PD577client # [ 0.144427] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages578lb # [ 0.005113] CPU features: detected: Data independent timing control (DIT)579worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.580lb # [ 0.005116] CPU features: detected: E0PD581worker1 # [ 0.004982] CPU features: detected: Enhanced Counter Virtualization582client # [ 0.144432] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page583lb # [ 0.005118] CPU features: detected: Enhanced Counter Virtualization584worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns585worker1 # [ 0.004985] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)586client # [ 0.144436] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages587lb # [ 0.005121] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)588worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).589worker1 # [ 0.004989] CPU features: detected: Enhanced Virtualization Traps590client # [ 0.144440] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page591lb # [ 0.005125] CPU features: detected: Enhanced Virtualization Traps592worker1 # [ 0.004992] CPU features: detected: Fine Grained Traps593client # [ 0.144445] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages594lb # [ 0.005128] CPU features: detected: Fine Grained Traps595worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns596worker1 # [ 0.004996] CPU features: detected: Generic authentication (architected QARMA5 algorithm)597client # [ 0.144449] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page598lb # [ 0.005131] CPU features: detected: Generic authentication (architected QARMA5 algorithm)599worker1 # [ 0.005000] CPU features: detected: RCpc load-acquire (LDAPR)600worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns601worker2 # [ 0.000030] arm-pv: using stolen time PV602lb # [ 0.005136] CPU features: detected: RCpc load-acquire (LDAPR)603worker1 # [ 0.005004] CPU features: detected: LSE atomic instructions604worker1 # [ 0.005006] CPU features: detected: Privileged Access Never605lb # [ 0.005139] CPU features: detected: LSE atomic instructions606worker2 # [ 0.000400] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)607worker1 # [ 0.005009] CPU features: detected: PMUv3608client # [ 0.151631] fbcon: Taking over console609lb # [ 0.005142] CPU features: detected: Privileged Access Never610client # [ 0.151644] ACPI: Interpreter disabled.611worker2 # [ 0.000542] Console: colour dummy device 80x25612worker1 # [ 0.005012] CPU features: detected: RAS Extension Support613lb # [ 0.005145] CPU features: detected: PMUv3614worker2 # [ 0.000550] printk: legacy console [tty0] enabled615lb # [ 0.005147] CPU features: detected: RAS Extension Support616worker1 # [ 0.005015] CPU features: detected: RASv1p1 Extension Support617worker1 # [ 0.005017] CPU features: detected: Random Number Generator618lb # [ 0.005150] CPU features: detected: RASv1p1 Extension Support619worker2 # [ 0.000743] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)620worker2 # [ 0.000750] pid_max: default: 32768 minimum: 301621lb # [ 0.005153] CPU features: detected: Random Number Generator622worker2 # [ 0.000822] LSM: initializing lsm=capability,landlock,yama,bpf,ima623lb # [ 0.005155] CPU features: detected: Speculation barrier (SB)624worker2 # [ 0.000966] landlock: Up and running.625lb # [ 0.005158] CPU features: detected: Stage-2 Force Write-Back626worker2 # [ 0.000969] Yama: becoming mindful.627worker2 # [ 0.001436] LSM support for eBPF active628lb # [ 0.005161] CPU features: detected: TLB range maintenance instructions629worker2 # [ 0.001572] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)630lb # [ 0.005166] CPU features: detected: Speculative Store Bypassing Safe (SSBS)631lb # [ 0.005203] alternatives: applying system-wide alternatives632worker2 # [ 0.001609] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)633worker2 # [ 0.003466] rcu: Hierarchical SRCU implementation.634lb # [ 0.008094] CPU features: detected: BBM Level 2 without TLB conflict abort635worker2 # [ 0.003471] rcu: Max phase no-delay instances is 1000.636worker2 # [ 0.004724] fsl-mc MSI: its@8080000 domain created637client # [ 0.160654] iommu: Default domain type: Translated638worker2 # [ 0.004814] EFI services will not be available.639lb # [ 0.008256] Memory: 1405664K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)640client # [ 0.160664] iommu: DMA domain TLB invalidation policy: strict mode641lb # [ 0.008631] devtmpfs: initialized642worker2 # [ 0.004892] smp: Bringing up secondary CPUs ...643client # [ 0.161006] SCSI subsystem initialized644worker2 # [ 0.004900] smp: Brought up 1 node, 1 CPU645lb # [ 0.010412] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)646worker2 # [ 0.004903] SMP: Total of 1 processors activated.647worker1 # [ 0.005020] CPU features: detected: Speculation barrier (SB)648worker2 # [ 0.004908] CPU: All CPU(s) started at EL1649client # [ 0.162961] usbcore: registered new interface driver usbfs650worker1 # [ 0.005023] CPU features: detected: Stage-2 Force Write-Back651lb # [ 0.010440] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).652worker2 # [ 0.004919] CPU features: detected: Branch Target Identification653client # [ 0.162993] usbcore: registered new interface driver hub654worker1 # [ 0.005025] CPU features: detected: TLB range maintenance instructions655worker2 # [ 0.004923] CPU features: detected: ARMv8.4 Translation Table Level656client # [ 0.163016] usbcore: registered new device driver usb657worker1 # [ 0.005030] CPU features: detected: Speculative Store Bypassing Safe (SSBS)658client # [ 0.163295] pps_core: LinuxPPS API ver. 1 registered659worker2 # [ 0.004927] CPU features: detected: Instruction cache invalidation not required for I/D coherence660worker1 # [ 0.005067] alternatives: applying system-wide alternatives661client # [ 0.163301] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>662worker1 # [ 0.008013] CPU features: detected: BBM Level 2 without TLB conflict abort663worker2 # [ 0.004931] CPU features: detected: Data cache clean to the PoU not required for I/D coherence664client # [ 0.163311] PTP clock support registered665client # [ 0.163366] EDAC MC: Ver: 3.0.0666worker2 # [ 0.004934] CPU features: detected: Common not Private translations667client # [ 0.167986] scmi_core: SCMI protocol bus registered668worker2 # [ 0.004938] CPU features: detected: CRC32 instructions669worker1 # [ 0.008255] Memory: 1405628K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)670worker1 # [ 0.008619] devtmpfs: initialized671worker2 # [ 0.004940] CPU features: detected: Data cache clean to Point of Deep Persistence672client # [ 0.168943] FPGA manager framework673client # [ 0.169871] vgaarb: loaded674worker1 # [ 0.010401] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)675worker2 # [ 0.004944] CPU features: detected: Data cache clean to Point of Persistence676client # [ 0.170491] clocksource: Switched to clocksource arch_sys_counter677worker1 # [ 0.010430] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).678worker2 # [ 0.004947] CPU features: detected: Data independent timing control (DIT)679worker2 # [ 0.004950] CPU features: detected: E0PD680worker1 # [ 0.010677] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL681worker2 # [ 0.004953] CPU features: detected: Enhanced Counter Virtualization682worker1 # [ 0.010684] 0 pages in range for non-PLT usage683lb # [ 0.010648] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL684worker1 # [ 0.010685] 508272 pages in range for PLT usage685worker2 # [ 0.004956] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)686lb # [ 0.010652] 0 pages in range for non-PLT usage687worker1 # [ 0.010801] pinctrl core: initialized pinctrl subsystem688lb # [ 0.010653] 508272 pages in range for PLT usage689worker2 # [ 0.004959] CPU features: detected: Enhanced Virtualization Traps690worker1 # [ 0.011569] DMI not present or invalid.691lb # [ 0.010742] pinctrl core: initialized pinctrl subsystem692worker2 # [ 0.004963] CPU features: detected: Fine Grained Traps693client # [ 0.174177] VFS: Disk quotas dquot_6.6.0694worker1 # [ 0.014864] NET: Registered PF_NETLINK/PF_ROUTE protocol family695lb # [ 0.011489] DMI not present or invalid.696client # [ 0.174214] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)697worker1 # [ 0.017125] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations698worker2 # [ 0.004966] CPU features: detected: Generic authentication (architected QARMA5 algorithm)699lb # [ 0.014893] NET: Registered PF_NETLINK/PF_ROUTE protocol family700worker2 # [ 0.004971] CPU features: detected: RCpc load-acquire (LDAPR)701worker1 # [ 0.017307] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations702lb # [ 0.017093] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations703worker2 # [ 0.004974] CPU features: detected: LSE atomic instructions704worker1 # [ 0.017505] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations705worker2 # [ 0.004977] CPU features: detected: Privileged Access Never706lb # [ 0.017282] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations707worker2 # [ 0.004979] CPU features: detected: PMUv3708worker1 # [ 0.017526] audit: initializing netlink subsys (disabled)709client # [ 0.177958] netfs: FS-Cache loaded710lb # [ 0.017482] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations711worker2 # [ 0.004982] CPU features: detected: RAS Extension Support712worker1 # [ 0.018074] thermal_sys: Registered thermal governor 'fair_share'713client # [ 0.178092] pnp: PnP ACPI: disabled714lb # [ 0.017505] audit: initializing netlink subsys (disabled)715worker2 # [ 0.004985] CPU features: detected: RASv1p1 Extension Support716worker1 # [ 0.018076] thermal_sys: Registered thermal governor 'bang_bang'717lb # [ 0.018089] thermal_sys: Registered thermal governor 'fair_share'718worker2 # [ 0.004988] CPU features: detected: Random Number Generator719worker1 # [ 0.018080] thermal_sys: Registered thermal governor 'step_wise'720lb # [ 0.018091] thermal_sys: Registered thermal governor 'bang_bang'721worker2 # [ 0.004990] CPU features: detected: Speculation barrier (SB)722worker1 # [ 0.018082] thermal_sys: Registered thermal governor 'user_space'723lb # [ 0.018094] thermal_sys: Registered thermal governor 'step_wise'724worker2 # [ 0.004993] CPU features: detected: Stage-2 Force Write-Back725worker1 # [ 0.018087] thermal_sys: Registered thermal governor 'power_allocator'726lb # [ 0.018097] thermal_sys: Registered thermal governor 'user_space'727worker2 # [ 0.004996] CPU features: detected: TLB range maintenance instructions728worker1 # [ 0.018113] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1729lb # [ 0.018102] thermal_sys: Registered thermal governor 'power_allocator'730client # [ 0.181944] NET: Registered PF_INET protocol family731worker2 # [ 0.005000] CPU features: detected: Speculative Store Bypassing Safe (SSBS)732worker1 # [ 0.018121] cpuidle: using governor ladder733worker1 # [ 0.018127] cpuidle: using governor menu734lb # [ 0.018145] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1735client # [ 0.182220] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)736worker2 # [ 0.005038] alternatives: applying system-wide alternatives737lb # [ 0.018154] cpuidle: using governor ladder738worker1 # [ 0.018317] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.739lb # [ 0.018160] cpuidle: using governor menu740worker2 # [ 0.007964] CPU features: detected: BBM Level 2 without TLB conflict abort741worker1 # [ 0.018332] ASID allocator initialised with 65536 entries742worker1 # [ 0.019575] Serial: AMBA PL011 UART driver743lb # [ 0.018344] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.744lb # [ 0.018358] ASID allocator initialised with 65536 entries745worker2 # [ 0.008179] Memory: 1405708K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)746worker1 # [ 0.024808] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1747lb # [ 0.019567] Serial: AMBA PL011 UART driver748worker2 # [ 0.008577] devtmpfs: initialized749worker1 # [ 0.024971] printk: console [ttyAMA0] enabled750worker2 # [ 0.010333] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)751lb # [ 0.024943] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1752lb # [ 0.025067] printk: console [ttyAMA0] enabled753worker1 # [ 0.144264] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages754worker2 # [ 0.010355] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).755worker1 # [ 0.144285] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page756worker2 # [ 0.010524] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL757lb # [ 0.145057] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages758worker1 # [ 0.144290] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages759worker2 # [ 0.010528] 0 pages in range for non-PLT usage760worker2 # [ 0.010529] 508272 pages in range for PLT usage761lb # [ 0.145076] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page762worker1 # [ 0.144294] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page763worker2 # [ 0.010634] pinctrl core: initialized pinctrl subsystem764lb # [ 0.145081] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages765worker1 # [ 0.144299] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages766worker2 # [ 0.011386] DMI not present or invalid.767lb # [ 0.145085] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page768worker1 # [ 0.144303] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page769worker2 # [ 0.014617] NET: Registered PF_NETLINK/PF_ROUTE protocol family770lb # [ 0.145090] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages771worker2 # [ 0.016818] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations772worker1 # [ 0.144307] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages773lb # [ 0.145094] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page774worker1 # [ 0.144311] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page775lb # [ 0.145098] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages776lb # [ 0.145102] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page777worker1 # [ 0.151597] fbcon: Taking over console778worker1 # [ 0.151611] ACPI: Interpreter disabled.779worker1 # [ 0.153471] iommu: Default domain type: Translated780lb # [ 0.152594] fbcon: Taking over console781worker1 # [ 0.153481] iommu: DMA domain TLB invalidation policy: strict mode782lb # [ 0.152610] ACPI: Interpreter disabled.783lb # [ 0.154489] iommu: Default domain type: Translated784worker1 # [ 0.155162] SCSI subsystem initialized785lb # [ 0.154500] iommu: DMA domain TLB invalidation policy: strict mode786lb # [ 0.156244] SCSI subsystem initialized787worker2 # [ 0.017013] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations788worker2 # [ 0.017206] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations789worker2 # [ 0.017228] audit: initializing netlink subsys (disabled)790worker2 # [ 0.017749] thermal_sys: Registered thermal governor 'fair_share'791worker2 # [ 0.017751] thermal_sys: Registered thermal governor 'bang_bang'792worker2 # [ 0.017754] thermal_sys: Registered thermal governor 'step_wise'793worker2 # [ 0.017757] thermal_sys: Registered thermal governor 'user_space'794worker2 # [ 0.017762] thermal_sys: Registered thermal governor 'power_allocator'795worker1 # [ 0.163049] usbcore: registered new interface driver usbfs796worker1 # [ 0.163080] usbcore: registered new interface driver hub797worker2 # [ 0.017786] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1798lb # [ 0.163371] usbcore: registered new interface driver usbfs799worker1 # [ 0.163096] usbcore: registered new device driver usb800worker2 # [ 0.017794] cpuidle: using governor ladder801lb # [ 0.163404] usbcore: registered new interface driver hub802worker2 # [ 0.017800] cpuidle: using governor menu803worker1 # [ 0.163365] pps_core: LinuxPPS API ver. 1 registered804lb # [ 0.163420] usbcore: registered new device driver usb805worker2 # [ 0.017984] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.806lb # [ 0.163696] pps_core: LinuxPPS API ver. 1 registered807worker1 # [ 0.163371] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>808worker2 # [ 0.017998] ASID allocator initialised with 65536 entries809worker1 # [ 0.163381] PTP clock support registered810worker2 # [ 0.019190] Serial: AMBA PL011 UART driver811lb # [ 0.163702] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>812worker1 # [ 0.163431] EDAC MC: Ver: 3.0.0813lb # [ 0.163713] PTP clock support registered814lb # [ 0.163770] EDAC MC: Ver: 3.0.0815worker1 # [ 0.167911] scmi_core: SCMI protocol bus registered816worker2 # [ 0.024452] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1817worker2 # [ 0.024595] printk: console [ttyAMA0] enabled818lb # [ 0.168320] scmi_core: SCMI protocol bus registered819worker1 # [ 0.168862] FPGA manager framework820worker1 # [ 0.169807] vgaarb: loaded821lb # [ 0.169305] FPGA manager framework822worker2 # [ 0.144519] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages823lb # [ 0.170291] vgaarb: loaded824worker1 # [ 0.170430] clocksource: Switched to clocksource arch_sys_counter825worker2 # [ 0.144538] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page826lb # [ 0.170930] clocksource: Switched to clocksource arch_sys_counter827worker2 # [ 0.144544] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages828worker1 # [ 0.171967] VFS: Disk quotas dquot_6.6.0829worker2 # [ 0.144548] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page830worker1 # [ 0.171994] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)831worker2 # [ 0.144552] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages832lb # [ 0.172829] VFS: Disk quotas dquot_6.6.0833worker2 # [ 0.144557] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page834lb # [ 0.172859] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)835worker2 # [ 0.144561] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages836worker2 # [ 0.144565] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page837worker1 # [ 0.175650] netfs: FS-Cache loaded838client # [ 0.210641] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)839worker1 # [ 0.175747] pnp: PnP ACPI: disabled840client # [ 0.210678] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)841lb # [ 0.176553] netfs: FS-Cache loaded842lb # [ 0.176666] pnp: PnP ACPI: disabled843client # [ 0.210704] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)844worker2 # [ 0.151946] fbcon: Taking over console845worker2 # [ 0.151959] ACPI: Interpreter disabled.846client # [ 0.210776] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)847worker2 # [ 0.153830] iommu: Default domain type: Translated848client # [ 0.210917] TCP: Hash tables configured (established 16384 bind 16384)849worker2 # [ 0.153840] iommu: DMA domain TLB invalidation policy: strict mode850client # [ 0.210995] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851worker2 # [ 0.155573] SCSI subsystem initialized852client # [ 0.211027] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)853client # [ 0.211062] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)854worker1 # [ 0.179655] NET: Registered PF_INET protocol family855client # [ 0.211163] NET: Registered PF_UNIX/PF_LOCAL protocol family856client # [ 0.211188] NET: Registered PF_XDP protocol family857worker1 # [ 0.179910] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)858lb # [ 0.180700] NET: Registered PF_INET protocol family859client # [ 0.211205] PCI: CLS 0 bytes, default 64860client # [ 0.211439] Trying to unpack rootfs image as initramfs...861lb # [ 0.180954] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)862client # [ 0.227916] kvm [1]: HYP mode not available863worker2 # [ 0.163247] usbcore: registered new interface driver usbfs864worker2 # [ 0.163279] usbcore: registered new interface driver hub865worker2 # [ 0.163303] usbcore: registered new device driver usb866worker2 # [ 0.163581] pps_core: LinuxPPS API ver. 1 registered867worker2 # [ 0.163587] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>868worker2 # [ 0.163598] PTP clock support registered869worker2 # [ 0.163638] EDAC MC: Ver: 3.0.0870worker2 # [ 0.168286] scmi_core: SCMI protocol bus registered871worker2 # [ 0.169248] FPGA manager framework872worker2 # [ 0.170231] vgaarb: loaded873worker2 # [ 0.170868] clocksource: Switched to clocksource arch_sys_counter874worker2 # [ 0.172232] VFS: Disk quotas dquot_6.6.0875worker2 # [ 0.172262] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)876worker2 # [ 0.173872] netfs: FS-Cache loaded877worker2 # [ 0.173984] pnp: PnP ACPI: disabled878worker2 # [ 0.179968] NET: Registered PF_INET protocol family879worker2 # [ 0.180241] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)880lb # [ 0.210803] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)881lb # [ 0.210850] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882lb # [ 0.210878] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883worker1 # [ 0.209256] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)884worker1 # [ 0.209308] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)885worker1 # [ 0.209333] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)886worker1 # [ 0.209403] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)887worker1 # [ 0.209547] TCP: Hash tables configured (established 16384 bind 16384)888worker1 # [ 0.209633] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)889lb # [ 0.213962] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)890worker1 # [ 0.209668] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)891lb # [ 0.214118] TCP: Hash tables configured (established 16384 bind 16384)892worker1 # [ 0.209703] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)893lb # [ 0.214203] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)894worker1 # [ 0.209794] NET: Registered PF_UNIX/PF_LOCAL protocol family895lb # [ 0.214238] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)896worker1 # [ 0.209821] NET: Registered PF_XDP protocol family897worker1 # [ 0.209839] PCI: CLS 0 bytes, default 64898lb # [ 0.214274] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)899worker1 # [ 0.210083] Trying to unpack rootfs image as initramfs...900lb # [ 0.214377] NET: Registered PF_UNIX/PF_LOCAL protocol family901lb # [ 0.214428] NET: Registered PF_XDP protocol family902lb # [ 0.214446] PCI: CLS 0 bytes, default 64903lb # [ 0.214690] Trying to unpack rootfs image as initramfs...904lb # [ 0.229643] kvm [1]: HYP mode not available905worker1 # [ 0.228256] kvm [1]: HYP mode not available906worker2 # [ 0.209124] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907worker2 # [ 0.209166] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908worker2 # [ 0.209191] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909worker2 # [ 0.209267] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910worker2 # [ 0.209423] TCP: Hash tables configured (established 16384 bind 16384)911worker2 # [ 0.209504] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912worker2 # [ 0.209539] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913worker2 # [ 0.209574] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914worker2 # [ 0.209705] NET: Registered PF_UNIX/PF_LOCAL protocol family915worker2 # [ 0.209745] NET: Registered PF_XDP protocol family916worker2 # [ 0.209762] PCI: CLS 0 bytes, default 64917worker2 # [ 0.209989] Trying to unpack rootfs image as initramfs...918worker2 # [ 0.224644] kvm [1]: HYP mode not available919client # [ 0.321533] Initialise system trusted keyrings920client # [ 0.322310] workingset: timestamp_bits=42 max_order=19 bucket_order=0921client # [ 0.323602] squashfs: version 4.0 (2009/01/31) Phillip Lougher922client # [ 0.324354] 9p: Installing v9fs 9p2000 file system support923client # [ 0.348382] Key type asymmetric registered924client # [ 0.348410] Asymmetric key parser 'x509' registered925client # [ 0.348486] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)926client # [ 0.350704] io scheduler mq-deadline registered927client # [ 0.350713] io scheduler kyber registered928lb # [ 0.323576] Initialise system trusted keyrings929worker1 # [ 0.324008] Initialise system trusted keyrings930worker1 # [ 0.324741] workingset: timestamp_bits=42 max_order=19 bucket_order=0931lb # [ 0.324301] workingset: timestamp_bits=42 max_order=19 bucket_order=0932worker1 # [ 0.325977] squashfs: version 4.0 (2009/01/31) Phillip Lougher933lb # [ 0.325579] squashfs: version 4.0 (2009/01/31) Phillip Lougher934lb # [ 0.326386] 9p: Installing v9fs 9p2000 file system support935worker1 # [ 0.326797] 9p: Installing v9fs 9p2000 file system support936client # [ 0.366657] pl061_gpio 9030000.pl061: PL061 GPIO chip registered937client # [ 0.367359] ledtrig-cpu: registered to indicate activity on CPUs938client # [ 0.367763] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:939client # [ 0.367780] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000940client # [ 0.367793] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000941client # [ 0.367801] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000942client # [ 0.367823] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits943client # [ 0.367851] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]944client # [ 0.367927] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00945client # [ 0.367937] pci_bus 0000:00: root bus resource [bus 00-ff]946worker2 # [ 0.315424] Initialise system trusted keyrings947client # [ 0.367942] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]948client # [ 0.367947] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]949worker2 # [ 0.316151] workingset: timestamp_bits=42 max_order=19 bucket_order=0950client # [ 0.367952] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]951worker2 # [ 0.317434] squashfs: version 4.0 (2009/01/31) Phillip Lougher952client # [ 0.368021] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint953worker2 # [ 0.318199] 9p: Installing v9fs 9p2000 file system support954client # [ 0.368462] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint955client # [ 0.368647] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]956client # [ 0.368664] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]957client # [ 0.368694] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]958client # [ 0.368710] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]959client # [ 0.369159] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint960client # [ 0.369342] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]961client # [ 0.369357] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]962client # [ 0.369387] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]963client # [ 0.369852] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint964client # [ 0.370031] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]965client # [ 0.370046] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]966client # [ 0.370075] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]967client # [ 0.370541] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint968worker1 # [ 0.354853] Key type asymmetric registered969client # [ 0.370724] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]970worker1 # [ 0.354880] Asymmetric key parser 'x509' registered971lb # [ 0.355113] Key type asymmetric registered972client # [ 0.370739] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]973lb # [ 0.355144] Asymmetric key parser 'x509' registered974worker1 # [ 0.354953] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)975client # [ 0.370768] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]976worker1 # [ 0.357030] io scheduler mq-deadline registered977lb # [ 0.355219] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)978client # [ 0.370786] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]979worker1 # [ 0.357041] io scheduler kyber registered980lb # [ 0.357291] io scheduler mq-deadline registered981lb # [ 0.357302] io scheduler kyber registered982client # [ 0.371244] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint983client # [ 0.371425] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]984client # [ 0.371454] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]985client # [ 0.371900] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint986client # [ 0.372081] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]987client # [ 0.372110] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]988client # [ 0.372516] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint989client # [ 0.372692] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]990client # [ 0.372941] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint991client # [ 0.373125] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]992client # [ 0.373155] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]993worker1 # [ 0.366612] pl061_gpio 9030000.pl061: PL061 GPIO chip registered994client # [ 0.373596] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint995client # [ 0.373792] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]996lb # [ 0.367085] pl061_gpio 9030000.pl061: PL061 GPIO chip registered997client # [ 0.373822] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998client # [ 0.374272] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint999worker1 # [ 0.368071] ledtrig-cpu: registered to indicate activity on CPUs1000client # [ 0.374453] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1001worker1 # [ 0.368485] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1002lb # [ 0.368537] ledtrig-cpu: registered to indicate activity on CPUs1003client # [ 0.374482] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1004worker2 # [ 0.346923] Key type asymmetric registered1005lb # [ 0.368961] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1006worker1 # [ 0.368503] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001007worker2 # [ 0.346952] Asymmetric key parser 'x509' registered1008lb # [ 0.368977] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001009worker1 # [ 0.368528] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001010worker2 # [ 0.347027] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1011worker2 # [ 0.349128] io scheduler mq-deadline registered1012lb # [ 0.368990] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001013worker1 # [ 0.368537] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001014worker2 # [ 0.349139] io scheduler kyber registered1015worker1 # [ 0.368558] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1016lb # [ 0.368999] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001017lb # [ 0.369019] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1018worker1 # [ 0.368584] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1019worker1 # [ 0.368682] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001020lb # [ 0.369044] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1021worker1 # [ 0.368691] pci_bus 0000:00: root bus resource [bus 00-ff]1022lb # [ 0.369116] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001023worker1 # [ 0.368697] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1024lb # [ 0.369126] pci_bus 0000:00: root bus resource [bus 00-ff]1025worker1 # [ 0.368703] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1026lb # [ 0.369132] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1027worker1 # [ 0.368708] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1028lb # [ 0.369138] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1029worker1 # [ 0.368794] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1030lb # [ 0.369143] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1031worker1 # [ 0.369234] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1032lb # [ 0.369227] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1033worker1 # [ 0.369416] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1034lb # [ 0.369663] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1035worker1 # [ 0.369433] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1036lb # [ 0.369848] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1037worker1 # [ 0.369462] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1038worker2 # [ 0.359019] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1039lb # [ 0.369865] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1040worker1 # [ 0.369478] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1041client # [ 0.423089] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1042lb # [ 0.369895] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1043worker1 # [ 0.369927] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1044client # [ 0.423405] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1045lb # [ 0.369912] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1046worker1 # [ 0.370107] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1047client # [ 0.423424] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1048worker1 # [ 0.370123] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1049lb # [ 0.370381] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1050client # [ 0.423453] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1051worker2 # [ 0.360397] ledtrig-cpu: registered to indicate activity on CPUs1052lb # [ 0.370564] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1053worker1 # [ 0.370152] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054client # [ 0.423921] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1055worker2 # [ 0.360819] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1056lb # [ 0.370580] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1057client # [ 0.424101] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1058lb # [ 0.370609] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1059worker2 # [ 0.360836] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001060client # [ 0.424117] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1061client # [ 0.424146] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1062worker2 # [ 0.360849] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001063lb # [ 0.390125] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1064client # [ 0.424720] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1065worker2 # [ 0.360857] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001066lb # [ 0.390312] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1067client # [ 0.424731] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1068lb # [ 0.390329] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker2 # [ 0.360877] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1070client # [ 0.424737] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1071lb # [ 0.390358] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1072worker2 # [ 0.360902] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1073client # [ 0.424781] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1074lb # [ 0.390806] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1075worker2 # [ 0.360976] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001076client # [ 0.424828] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1077worker2 # [ 0.360985] pci_bus 0000:00: root bus resource [bus 00-ff]1078client # [ 0.424875] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1079worker2 # [ 0.360992] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1080worker2 # [ 0.360997] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1081client # [ 0.424922] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1082worker2 # [ 0.361002] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1083client # [ 0.424969] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1084worker2 # [ 0.361059] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1085client # [ 0.425017] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1086worker2 # [ 0.361517] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1087client # [ 0.425063] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1088worker2 # [ 0.361703] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1089client # [ 0.425110] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1090worker2 # [ 0.361720] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1091client # [ 0.425156] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1092worker2 # [ 0.361750] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093worker1 # [ 0.394706] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1094worker2 # [ 0.361766] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1095client # [ 0.425224] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1096worker1 # [ 0.394909] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1097worker2 # [ 0.362219] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1098client # [ 0.425269] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1099worker1 # [ 0.394926] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1100worker2 # [ 0.362402] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1101client # [ 0.425291] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1102worker1 # [ 0.394955] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103worker2 # [ 0.362418] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1104client # [ 0.425312] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1105worker1 # [ 0.395426] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1106worker2 # [ 0.362448] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1107client # [ 0.425333] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1108worker1 # [ 0.395610] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1109client # [ 0.425354] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1110worker1 # [ 0.395626] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1111lb # [ 0.399042] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1112client # [ 0.425375] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1113worker1 # [ 0.395655] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114lb # [ 0.399072] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1115worker2 # [ 0.381482] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1116client # [ 0.425397] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1117worker1 # [ 0.395673] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1118worker2 # [ 0.381670] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1119lb # [ 0.399103] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1120client # [ 0.425423] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1121worker1 # [ 0.396122] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1122worker2 # [ 0.381685] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1123lb # [ 0.399122] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1124client # [ 0.425444] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1125worker1 # [ 0.396308] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1126worker2 # [ 0.381715] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127lb # [ 0.399612] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1128client # [ 0.425466] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1129worker1 # [ 0.396338] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1130worker2 # [ 0.382181] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1131lb # [ 0.399805] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1132client # [ 0.425488] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1133worker2 # [ 0.382365] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1134worker1 # [ 0.396785] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1135lb # [ 0.399835] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1136client # [ 0.425510] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1137worker2 # [ 0.382380] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1138worker1 # [ 0.396971] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1139lb # [ 0.400289] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1140client # [ 0.425531] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1141worker2 # [ 0.382410] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1142worker1 # [ 0.397000] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143lb # [ 0.400477] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1144client # [ 0.425552] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1145worker2 # [ 0.382429] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1146worker1 # [ 0.397384] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1147client # [ 0.425572] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1148lb # [ 0.400507] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker1 # [ 0.397563] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1150client # [ 0.425593] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1151lb # [ 0.400894] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1152client # [ 0.425623] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1153worker1 # [ 0.397851] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1154lb # [ 0.401076] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1155worker1 # [ 0.398036] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1156client # [ 0.425644] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1157lb # [ 0.401359] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1158client # [ 0.425670] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1159worker1 # [ 0.398066] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1160lb # [ 0.401546] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1161client # [ 0.425679] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1162worker1 # [ 0.398527] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1163lb # [ 0.401576] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164client # [ 0.425684] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1165worker1 # [ 0.398718] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1166client # [ 0.426490] pci 0000:00:07.0: enabling device (0000 -> 0002)1167lb # [ 0.402024] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1168worker1 # [ 0.398748] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1169lb # [ 0.402220] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1170worker1 # [ 0.399194] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1171lb # [ 0.402250] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1172worker1 # [ 0.399378] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1173lb # [ 0.402700] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1174worker1 # [ 0.399407] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1175lb # [ 0.402885] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1176worker1 # [ 0.399866] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1177lb # [ 0.402915] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1178worker1 # [ 0.400159] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1179lb # [ 0.403394] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1180worker1 # [ 0.400177] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1181lb # [ 0.403656] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1182worker1 # [ 0.400206] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1183lb # [ 0.403674] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1184worker1 # [ 0.400660] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1185lb # [ 0.403704] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1186worker1 # [ 0.400840] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1187worker1 # [ 0.400856] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1188lb # [ 0.404170] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1189worker2 # [ 0.394951] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1190lb # [ 0.404353] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1191worker1 # [ 0.400886] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1192worker2 # [ 0.395159] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1193lb # [ 0.404369] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1194worker1 # [ 0.401442] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1195worker2 # [ 0.395190] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1196lb # [ 0.404399] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1197worker1 # [ 0.401453] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1198worker2 # [ 0.395683] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1199lb # [ 0.404958] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1200worker2 # [ 0.395870] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1201worker1 # [ 0.401458] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1202lb # [ 0.404969] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1203worker2 # [ 0.395899] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1204worker1 # [ 0.401501] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1205lb # [ 0.404975] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1206worker2 # [ 0.396287] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1207worker1 # [ 0.401548] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1208lb # [ 0.405021] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1209worker2 # [ 0.396469] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1210worker1 # [ 0.401593] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1211lb # [ 0.405067] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1212worker2 # [ 0.396720] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1213worker1 # [ 0.401637] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1214worker2 # [ 0.396907] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1215lb # [ 0.405114] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1216worker1 # [ 0.401682] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1217worker2 # [ 0.396937] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1218lb # [ 0.405161] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1219worker1 # [ 0.401727] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1220worker2 # [ 0.397402] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1221lb # [ 0.405207] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1222worker2 # [ 0.397588] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1223worker1 # [ 0.401773] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1224lb # [ 0.405252] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1225worker2 # [ 0.397618] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1226worker1 # [ 0.401818] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1227client # [ 0.481572] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1228lb # [ 0.405298] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1229worker2 # [ 0.398078] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1230worker1 # [ 0.401863] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1231lb # [ 0.405344] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1232worker2 # [ 0.398265] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1233worker1 # [ 0.401932] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1234worker2 # [ 0.398295] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1235lb # [ 0.405391] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1236worker1 # [ 0.401977] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1237worker2 # [ 0.398749] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1238lb # [ 0.405465] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1239worker1 # [ 0.401998] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1240worker2 # [ 0.399032] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1241lb # [ 0.405512] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1242worker1 # [ 0.402019] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1243worker2 # [ 0.399048] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1244lb # [ 0.405533] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1245worker1 # [ 0.402040] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1246worker2 # [ 0.399078] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1247lb # [ 0.405555] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1248worker1 # [ 0.402061] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1249worker2 # [ 0.399544] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1250lb # [ 0.405576] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1251worker1 # [ 0.402082] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1252worker2 # [ 0.399727] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1253lb # [ 0.405598] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1254worker1 # [ 0.402103] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1255worker2 # [ 0.399742] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1256lb # [ 0.405620] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1257worker1 # [ 0.402126] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1258worker2 # [ 0.399772] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1259lb # [ 0.405641] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1260worker1 # [ 0.402148] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1261worker2 # [ 0.400340] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1262lb # [ 0.405666] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1263worker1 # [ 0.402169] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1264worker2 # [ 0.400351] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1265lb # [ 0.405688] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1266worker1 # [ 0.402191] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1267worker2 # [ 0.400357] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1268lb # [ 0.405710] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1269worker1 # [ 0.402213] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1270worker2 # [ 0.400402] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1271lb # [ 0.405732] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1272worker1 # [ 0.402234] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1273client # [ 0.491848] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1274lb # [ 0.405755] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1275worker1 # [ 0.402254] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1276worker2 # [ 0.400449] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1277worker1 # [ 0.402275] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1278lb # [ 0.405777] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1279worker2 # [ 0.400496] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1280worker1 # [ 0.402295] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1281lb # [ 0.405798] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1282worker2 # [ 0.400543] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1283lb # [ 0.405819] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1284worker1 # [ 0.402316] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1285lb # [ 0.405841] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1286worker1 # [ 0.402337] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1287worker2 # [ 0.400590] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1288worker1 # [ 0.402363] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1289client # [ 0.496078] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1290worker2 # [ 0.400636] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1291lb # [ 0.405862] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1292worker1 # [ 0.402372] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1293lb # [ 0.405884] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1294client # [ 0.497972] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1295worker2 # [ 0.400682] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1296lb # [ 0.405910] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1297worker1 # [ 0.402377] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1298lb # [ 0.405919] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1299worker2 # [ 0.400728] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1300lb # [ 0.405924] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1301worker2 # [ 0.400774] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1302lb # [ 0.406728] pci 0000:00:07.0: enabling device (0000 -> 0002)1303worker2 # [ 0.400843] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1304worker2 # [ 0.400889] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1305worker2 # [ 0.400910] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1306worker2 # [ 0.400932] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1307worker2 # [ 0.400953] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1308worker2 # [ 0.400974] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1309worker2 # [ 0.400996] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1310worker1 # [ 0.463280] pci 0000:00:07.0: enabling device (0000 -> 0002)1311worker2 # [ 0.401017] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1312worker2 # [ 0.401041] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1313worker2 # [ 0.401064] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1314worker2 # [ 0.401085] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1315worker2 # [ 0.401108] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1316worker2 # [ 0.401130] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1317worker2 # [ 0.401151] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1318worker2 # [ 0.401172] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1319client # [ 0.511835] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1320worker2 # [ 0.401193] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1321client # [ 0.513802] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1322worker2 # [ 0.401214] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1323worker2 # [ 0.401235] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1324lb # [ 0.471089] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 62855 usecs1325worker2 # [ 0.401257] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1326worker2 # [ 0.401282] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1327worker2 # [ 0.401291] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1328client # [ 0.517245] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1329worker2 # [ 0.401296] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1330worker2 # [ 0.402107] pci 0000:00:07.0: enabling device (0000 -> 0002)1331client # [ 0.519934] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1332worker1 # [ 0.488785] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1333client # [ 0.530662] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1334lb # [ 0.492246] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1335client # [ 0.532547] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1336lb # [ 0.494350] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1337client # [ 0.534835] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1338worker1 # [ 0.495088] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1339worker1 # [ 0.497093] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1340worker2 # [ 0.478517] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1341client # [ 0.547797] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1342client # [ 0.550352] msm_serial: driver initialized1343lb # [ 0.505342] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1344worker1 # [ 0.507039] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1345client # [ 0.551025] SuperH (H)SCI(F) driver initialized1346client # [ 0.551086] STM32 USART driver initialized1347worker1 # [ 0.509162] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1348worker2 # [ 0.488341] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1349lb # [ 0.515804] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1350lb # [ 0.518046] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1351lb # [ 0.521737] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1352worker1 # [ 0.518614] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1353worker1 # [ 0.520797] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1354worker1 # [ 0.522625] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1355worker1 # [ 0.524391] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1356lb # [ 0.527990] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1357worker1 # [ 0.526188] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1358worker2 # [ 0.499423] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1359lb # [ 0.530610] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1360worker2 # [ 0.501280] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1361worker2 # [ 0.503205] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1362worker2 # [ 0.505240] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1363worker1 # [ 0.540005] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1364client # [ 0.584681] loop: module loaded1365lb # [ 0.541838] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1366client # [ 0.584854] virtio_blk virtio2: 1/0/0 default/read/poll queues1367worker2 # [ 0.515030] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1368client # [ 0.585654] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1369worker2 # [ 0.516794] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1370worker2 # [ 0.518591] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1371lb # [ 0.544622] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1372lb # [ 0.546925] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1373client # [ 0.591048] megasas: 07.734.00.00-rc11374client # [ 0.591722] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1375worker1 # [ 0.555690] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1376worker2 # [ 0.528952] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1377worker1 # [ 0.558247] msm_serial: driver initialized1378worker1 # [ 0.558410] SuperH (H)SCI(F) driver initialized1379lb # [ 0.564009] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1380worker1 # [ 0.559551] STM32 USART driver initialized1381lb # [ 0.566572] msm_serial: driver initialized1382lb # [ 0.566731] SuperH (H)SCI(F) driver initialized1383client # [ 0.607559] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001384lb # [ 0.566789] STM32 USART driver initialized1385client # [ 0.607610] Intel/Sharp Extended Query Table at 0x00311386client # [ 0.609752] Using buffer write method1387worker2 # [ 0.539075] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1388client # [ 0.609820] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1389worker2 # [ 0.551256] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1390worker2 # [ 0.553871] msm_serial: driver initialized1391worker2 # [ 0.554010] SuperH (H)SCI(F) driver initialized1392worker2 # [ 0.554063] STM32 USART driver initialized1393client # [ 0.623241] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001394client # [ 0.623272] Intel/Sharp Extended Query Table at 0x00311395client # [ 0.624922] Using buffer write method1396client # [ 0.624949] Concatenating MTD devices:1397client # [ 0.624953] (0): "0.flash"1398client # [ 0.624957] (1): "0.flash"1399client # [ 0.624968] into device "0.flash"1400worker1 # [ 0.589206] loop: module loaded1401worker1 # [ 0.589378] virtio_blk virtio2: 1/0/0 default/read/poll queues1402worker1 # [ 0.590124] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1403worker1 # [ 0.595054] megasas: 07.734.00.00-rc11404worker1 # [ 0.595736] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1405lb # [ 0.599957] loop: module loaded1406lb # [ 0.600147] virtio_blk virtio2: 1/0/0 default/read/poll queues1407lb # [ 0.600867] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1408worker2 # [ 0.582410] loop: module loaded1409worker2 # [ 0.582603] virtio_blk virtio2: 1/0/0 default/read/poll queues1410worker1 # [ 0.607826] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001411worker1 # [ 0.607891] Intel/Sharp Extended Query Table at 0x00311412worker1 # [ 0.609478] Using buffer write method1413worker1 # [ 0.609549] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1414worker2 # [ 0.591589] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1415lb # [ 0.611616] megasas: 07.734.00.00-rc11416lb # [ 0.612336] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1417lb # [ 0.614231] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001418lb # [ 0.614258] Intel/Sharp Extended Query Table at 0x00311419lb # [ 0.615896] Using buffer write method1420worker2 # [ 0.594737] megasas: 07.734.00.00-rc11421lb # [ 0.615971] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1422worker2 # [ 0.595478] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1423worker1 # [ 0.623226] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001424lb # [ 0.618057] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001425worker1 # [ 0.623273] Intel/Sharp Extended Query Table at 0x00311426worker1 # [ 0.624863] Using buffer write method1427lb # [ 0.618081] Intel/Sharp Extended Query Table at 0x00311428worker1 # [ 0.624890] Concatenating MTD devices:1429worker1 # [ 0.624894] (0): "0.flash"1430worker1 # [ 0.624898] (1): "0.flash"1431worker1 # [ 0.624907] into device "0.flash"1432worker2 # [ 0.606914] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001433lb # [ 0.638422] Using buffer write method1434worker2 # [ 0.606942] Intel/Sharp Extended Query Table at 0x00311435lb # [ 0.638461] Concatenating MTD devices:1436lb # [ 0.638465] (0): "0.flash"1437worker2 # [ 0.609418] Using buffer write method1438lb # [ 0.638474] (1): "0.flash"1439lb # [ 0.638478] into device "0.flash"1440worker2 # [ 0.609504] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1441worker2 # [ 0.611217] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001442worker2 # [ 0.611239] Intel/Sharp Extended Query Table at 0x00311443worker2 # [ 0.612834] Using buffer write method1444worker2 # [ 0.612860] Concatenating MTD devices:1445worker2 # [ 0.612864] (0): "0.flash"1446worker2 # [ 0.612868] (1): "0.flash"1447worker2 # [ 0.612877] into device "0.flash"1448client # [ 0.867637] Freeing initrd memory: 26972K1449client # [ 0.873657] tun: Universal TUN/TAP device driver, 1.61450client # [ 0.877375] thunder_xcv, ver 1.01451client # [ 0.877421] thunder_bgx, ver 1.01452client # [ 0.877442] nicpf, ver 1.01453client # [ 0.878066] e1000: Intel(R) PRO/1000 Network Driver1454client # [ 0.878074] e1000: Copyright (c) 1999-2006 Intel Corporation.1455client # [ 0.878100] e1000e: Intel(R) PRO/1000 Network Driver1456client # [ 0.878107] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457client # [ 0.878135] igb: Intel(R) Gigabit Ethernet Network Driver1458client # [ 0.878140] igb: Copyright (c) 2007-2014 Intel Corporation.1459client # [ 0.878163] igbvf: Intel(R) Gigabit Virtual Function Network Driver1460client # [ 0.878169] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1461client # [ 0.878299] sky2: driver version 1.301462client # [ 0.886927] usbcore: registered new interface driver usb-storage1463client # [ 0.886982] usbcore: registered new interface driver usbserial_generic1464client # [ 0.886998] usbserial: USB Serial support registered for generic1465client # [ 0.887609] hv_vmbus: registering driver hyperv_keyboard1466client # [ 0.891779] ehci-pci 0000:00:07.0: EHCI Host Controller1467client # [ 0.891806] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11468client # [ 0.892000] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001469client # [ 0.894470] rtc-pl031 9010000.pl031: registered as rtc01470client # [ 0.895151] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:58:25 UTC (1790096305)1471client # [ 0.895490] i2c_dev: i2c /dev entries driver1472client # [ 0.900181] sdhci: Secure Digital Host Controller Interface driver1473client # [ 0.900193] sdhci: Copyright(c) Pierre Ossman1474client # [ 0.900458] Synopsys Designware Multimedia Card Interface Driver1475client # [ 0.900828] sdhci-pltfm: SDHCI platform and OF driver helper1476client # [ 0.902334] hid: raw HID events driver (C) Jiri Kosina1477client # [ 0.902528] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001478client # [ 0.902835] hub 1-0:1.0: USB hub found1479client # [ 0.902855] hub 1-0:1.0: 6 ports detected1480client # [ 0.908248] usbcore: registered new interface driver usbhid1481client # [ 0.908258] usbhid: USB HID core driver1482client # [ 0.910566] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1483client # [ 0.912072] drop_monitor: Initializing network drop monitor service1484client # [ 0.912214] NET: Registered PF_INET6 protocol family1485client # [ 0.915234] Segment Routing with IPv61486client # [ 0.915251] In-situ OAM (IOAM) with IPv61487lb # [ 0.870694] Freeing initrd memory: 26948K1488client # [ 0.915293] NET: Registered PF_PACKET protocol family1489worker1 # [ 0.870675] Freeing initrd memory: 26976K1490client # [ 0.916905] 9pnet: Installing 9P2000 support1491client # [ 0.916957] Key type dns_resolver registered1492worker1 # [ 0.876542] tun: Universal TUN/TAP device driver, 1.61493lb # [ 0.876853] tun: Universal TUN/TAP device driver, 1.61494worker1 # [ 0.880183] thunder_xcv, ver 1.01495worker1 # [ 0.880227] thunder_bgx, ver 1.01496worker1 # [ 0.880250] nicpf, ver 1.01497lb # [ 0.880638] thunder_xcv, ver 1.01498worker1 # [ 0.880806] e1000: Intel(R) PRO/1000 Network Driver1499client # [ 0.923528] registered taskstats version 11500lb # [ 0.880690] thunder_bgx, ver 1.01501lb # [ 0.880712] nicpf, ver 1.01502client # [ 0.923687] Loading compiled-in X.509 certificates1503worker1 # [ 0.880813] e1000: Copyright (c) 1999-2006 Intel Corporation.1504lb # [ 0.881294] e1000: Intel(R) PRO/1000 Network Driver1505worker1 # [ 0.880836] e1000e: Intel(R) PRO/1000 Network Driver1506lb # [ 0.881301] e1000: Copyright (c) 1999-2006 Intel Corporation.1507worker1 # [ 0.880844] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1508lb # [ 0.881326] e1000e: Intel(R) PRO/1000 Network Driver1509worker1 # [ 0.880873] igb: Intel(R) Gigabit Ethernet Network Driver1510lb # [ 0.881333] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1511worker1 # [ 0.880878] igb: Copyright (c) 2007-2014 Intel Corporation.1512lb # [ 0.881365] igb: Intel(R) Gigabit Ethernet Network Driver1513worker1 # [ 0.880917] igbvf: Intel(R) Gigabit Virtual Function Network Driver1514lb # [ 0.881371] igb: Copyright (c) 2007-2014 Intel Corporation.1515worker1 # [ 0.880922] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1516lb # [ 0.881395] igbvf: Intel(R) Gigabit Virtual Function Network Driver1517worker1 # [ 0.881055] sky2: driver version 1.301518lb # [ 0.881401] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1519client # [ 0.932352] Demotion targets for Node 0: null1520lb # [ 0.881550] sky2: driver version 1.301521client # [ 0.932456] Key type .fscrypt registered1522worker1 # [ 0.890156] ehci-pci 0000:00:07.0: EHCI Host Controller1523client # [ 0.932462] Key type fscrypt-provisioning registered1524worker1 # [ 0.890192] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11525client # [ 0.932558] ima: No TPM chip found, activating TPM-bypass!1526client # [ 0.932577] ima: Allocated hash algorithm: sha11527worker1 # [ 0.890410] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001528client # [ 0.932597] ima: No architecture policies found1529worker2 # [ 0.867490] Freeing initrd memory: 26944K1530lb # [ 0.890296] usbcore: registered new interface driver usb-storage1531lb # [ 0.890362] usbcore: registered new interface driver usbserial_generic1532client # [ 0.936679] input: gpio-keys as /devices/platform/gpio-keys/input/input01533lb # [ 0.890384] usbserial: USB Serial support registered for generic1534worker1 # [ 0.893029] usbcore: registered new interface driver usb-storage1535lb # [ 0.891758] ehci-pci 0000:00:07.0: EHCI Host Controller1536worker1 # [ 0.893130] usbcore: registered new interface driver usbserial_generic1537lb # [ 0.891789] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11538worker1 # [ 0.893144] usbserial: USB Serial support registered for generic1539lb # [ 0.891992] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001540worker1 # [ 0.893737] hv_vmbus: registering driver hyperv_keyboard1541lb # [ 0.896690] hv_vmbus: registering driver hyperv_keyboard1542worker1 # [ 0.897547] rtc-pl031 9010000.pl031: registered as rtc01543worker2 # [ 0.873522] tun: Universal TUN/TAP device driver, 1.61544lb # [ 0.898220] rtc-pl031 9010000.pl031: registered as rtc01545worker1 # [ 0.897575] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:58:25 UTC (1790096305)1546worker1 # [ 0.897876] i2c_dev: i2c /dev entries driver1547lb # [ 0.898248] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:58:25 UTC (1790096305)1548lb # [ 0.898548] i2c_dev: i2c /dev entries driver1549worker2 # [ 0.877325] thunder_xcv, ver 1.01550worker2 # [ 0.877370] thunder_bgx, ver 1.01551worker2 # [ 0.877409] nicpf, ver 1.01552worker2 # [ 0.877987] e1000: Intel(R) PRO/1000 Network Driver1553worker2 # [ 0.877995] e1000: Copyright (c) 1999-2006 Intel Corporation.1554worker2 # [ 0.878019] e1000e: Intel(R) PRO/1000 Network Driver1555worker1 # [ 0.903179] sdhci: Secure Digital Host Controller Interface driver1556worker2 # [ 0.878028] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1557worker1 # [ 0.903190] sdhci: Copyright(c) Pierre Ossman1558lb # [ 0.903944] sdhci: Secure Digital Host Controller Interface driver1559worker2 # [ 0.878056] igb: Intel(R) Gigabit Ethernet Network Driver1560worker1 # [ 0.903457] Synopsys Designware Multimedia Card Interface Driver1561lb # [ 0.903956] sdhci: Copyright(c) Pierre Ossman1562worker1 # [ 0.903823] sdhci-pltfm: SDHCI platform and OF driver helper1563worker2 # [ 0.878062] igb: Copyright (c) 2007-2014 Intel Corporation.1564lb # [ 0.904231] Synopsys Designware Multimedia Card Interface Driver1565worker1 # [ 0.905318] hid: raw HID events driver (C) Jiri Kosina1566worker2 # [ 0.878084] igbvf: Intel(R) Gigabit Virtual Function Network Driver1567lb # [ 0.904603] sdhci-pltfm: SDHCI platform and OF driver helper1568worker1 # [ 0.905560] usbcore: registered new interface driver usbhid1569worker2 # [ 0.878090] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1570lb # [ 0.906150] hid: raw HID events driver (C) Jiri Kosina1571worker1 # [ 0.905566] usbhid: USB HID core driver1572worker2 # [ 0.878223] sky2: driver version 1.301573lb # [ 0.906394] usbcore: registered new interface driver usbhid1574lb # [ 0.906401] usbhid: USB HID core driver1575client # [ 0.954465] clk: Disabling unused clocks1576worker1 # [ 0.910839] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001577client # [ 0.954490] PM: genpd: Disabling unused power domains1578lb # [ 0.911331] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001579worker1 # [ 0.911179] hub 1-0:1.0: USB hub found1580worker2 # [ 0.886856] usbcore: registered new interface driver usb-storage1581worker1 # [ 0.911199] hub 1-0:1.0: 6 ports detected1582lb # [ 0.912351] hub 1-0:1.0: USB hub found1583lb # [ 0.912840] hub 1-0:1.0: 6 ports detected1584worker2 # [ 0.888233] ehci-pci 0000:00:07.0: EHCI Host Controller1585worker2 # [ 0.888264] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11586worker1 # [ 0.913325] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1587worker2 # [ 0.888520] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001588lb # [ 0.914310] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1589client # [ 0.958616] Freeing unused kernel memory: 4736K1590client # [ 0.958811] Run /init as init process1591worker1 # [ 0.915992] drop_monitor: Initializing network drop monitor service1592worker1 # [ 0.916225] NET: Registered PF_INET6 protocol family1593worker2 # [ 0.891006] usbcore: registered new interface driver usbserial_generic1594lb # [ 0.917000] drop_monitor: Initializing network drop monitor service1595worker1 # [ 0.918153] Segment Routing with IPv61596worker2 # [ 0.891023] usbserial: USB Serial support registered for generic1597lb # [ 0.917174] NET: Registered PF_INET6 protocol family1598worker1 # [ 0.918172] In-situ OAM (IOAM) with IPv61599worker2 # [ 0.891719] hv_vmbus: registering driver hyperv_keyboard1600worker1 # [ 0.918200] NET: Registered PF_PACKET protocol family1601lb # [ 0.919081] Segment Routing with IPv61602lb # [ 0.919099] In-situ OAM (IOAM) with IPv61603worker2 # [ 0.894743] rtc-pl031 9010000.pl031: registered as rtc01604lb # [ 0.919126] NET: Registered PF_PACKET protocol family1605worker2 # [ 0.894772] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:58:25 UTC (1790096305)1606worker1 # [ 0.919843] 9pnet: Installing 9P2000 support1607worker2 # [ 0.896875] i2c_dev: i2c /dev entries driver1608worker1 # [ 0.919893] Key type dns_resolver registered1609lb # [ 0.920762] 9pnet: Installing 9P2000 support1610lb # [ 0.920812] Key type dns_resolver registered1611worker2 # [ 0.900626] sdhci: Secure Digital Host Controller Interface driver1612worker2 # [ 0.900638] sdhci: Copyright(c) Pierre Ossman1613worker2 # [ 0.900904] Synopsys Designware Multimedia Card Interface Driver1614client # [ 0.972681] systemd[1]: Successfully made /usr/ read-only.1615worker2 # [ 0.901275] sdhci-pltfm: SDHCI platform and OF driver helper1616worker1 # [ 0.926800] registered taskstats version 11617worker2 # [ 0.902808] hid: raw HID events driver (C) Jiri Kosina1618worker1 # [ 0.926950] Loading compiled-in X.509 certificates1619lb # [ 0.927611] registered taskstats version 11620worker2 # [ 0.906212] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001621lb # [ 0.927758] Loading compiled-in X.509 certificates1622worker2 # [ 0.907288] hub 1-0:1.0: USB hub found1623worker2 # [ 0.907762] hub 1-0:1.0: 6 ports detected1624worker2 # [ 0.908854] usbcore: registered new interface driver usbhid1625worker2 # [ 0.908864] usbhid: USB HID core driver1626worker1 # [ 0.935736] Demotion targets for Node 0: null1627worker1 # [ 0.935847] Key type .fscrypt registered1628worker1 # [ 0.935854] Key type fscrypt-provisioning registered1629lb # [ 0.936605] Demotion targets for Node 0: null1630worker1 # [ 0.935947] ima: No TPM chip found, activating TPM-bypass!1631lb # [ 0.936715] Key type .fscrypt registered1632worker1 # [ 0.935967] ima: Allocated hash algorithm: sha11633lb # [ 0.936722] Key type fscrypt-provisioning registered1634worker2 # [ 0.911254] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1635worker1 # [ 0.935989] ima: No architecture policies found1636lb # [ 0.936813] ima: No TPM chip found, activating TPM-bypass!1637worker2 # [ 0.912831] drop_monitor: Initializing network drop monitor service1638lb # [ 0.936832] ima: Allocated hash algorithm: sha11639worker2 # [ 0.913026] NET: Registered PF_INET6 protocol family1640worker1 # [ 0.939929] input: gpio-keys as /devices/platform/gpio-keys/input/input01641lb # [ 0.936854] ima: No architecture policies found1642worker2 # [ 0.915983] Segment Routing with IPv61643worker2 # [ 0.916003] In-situ OAM (IOAM) with IPv61644lb # [ 0.940868] input: gpio-keys as /devices/platform/gpio-keys/input/input01645worker2 # [ 0.916032] NET: Registered PF_PACKET protocol family1646worker2 # [ 0.917670] 9pnet: Installing 9P2000 support1647worker2 # [ 0.917724] Key type dns_resolver registered1648worker2 # [ 0.924407] registered taskstats version 11649worker2 # [ 0.924548] Loading compiled-in X.509 certificates1650worker1 # [ 0.958406] clk: Disabling unused clocks1651worker1 # [ 0.958429] PM: genpd: Disabling unused power domains1652worker2 # [ 0.933455] Demotion targets for Node 0: null1653lb # [ 0.958794] clk: Disabling unused clocks1654worker2 # [ 0.933567] Key type .fscrypt registered1655lb # [ 0.958822] PM: genpd: Disabling unused power domains1656worker2 # [ 0.933574] Key type fscrypt-provisioning registered1657worker2 # [ 0.933670] ima: No TPM chip found, activating TPM-bypass!1658worker2 # [ 0.933688] ima: Allocated hash algorithm: sha11659worker2 # [ 0.933709] ima: No architecture policies found1660worker1 # [ 0.962642] Freeing unused kernel memory: 4736K1661worker2 # [ 0.937767] input: gpio-keys as /devices/platform/gpio-keys/input/input01662worker1 # [ 0.962846] Run /init as init process1663lb # [ 0.963013] Freeing unused kernel memory: 4736K1664lb # [ 0.963205] Run /init as init process1665lb # [ 0.977171] systemd[1]: Successfully made /usr/ read-only.1666worker1 # [ 0.980677] systemd[1]: Successfully made /usr/ read-only.1667worker2 # [ 0.955574] clk: Disabling unused clocks1668worker2 # [ 0.955606] PM: genpd: Disabling unused power domains1669worker2 # [ 0.959717] Freeing unused kernel memory: 4736K1670worker2 # [ 0.959921] Run /init as init process1671worker2 # [ 0.977034] systemd[1]: Successfully made /usr/ read-only.1672client # [ 1.150565] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker1 # [ 1.158528] usb 1-1: new high-speed USB device number 2 using ehci-pci1674lb # [ 1.159029] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker2 # [ 1.154951] usb 1-1: new high-speed USB device number 2 using ehci-pci1676client # [ 1.303031] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677client # [ 1.309053] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1678client # [ 1.309115] systemd[1]: Detected virtualization qemu.1679client # [ 1.309192] systemd[1]: Detected architecture arm64.1680client # [ 1.309215] systemd[1]: Running in initrd.1681client # [ 1.310183] systemd[1]: Initializing machine ID from random generator.1682client # [ 1.310456] systemd[1]: Hostname set to <client>.1683lb # [ 1.311470] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11684worker1 # [ 1.311060] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11685lb # [ 1.316531] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1686lb # [ 1.316584] systemd[1]: Detected virtualization qemu.1687worker1 # [ 1.317119] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1688lb # [ 1.316653] systemd[1]: Detected architecture arm64.1689lb # [ 1.316674] systemd[1]: Running in initrd.1690worker1 # [ 1.317176] systemd[1]: Detected virtualization qemu.1691lb # [ 1.317530] systemd[1]: Initializing machine ID from random generator.1692lb # [ 1.317780] systemd[1]: Hostname set to <lb>.1693worker1 # [ 1.317250] systemd[1]: Detected architecture arm64.1694worker1 # [ 1.317274] systemd[1]: Running in initrd.1695worker1 # [ 1.318204] systemd[1]: Initializing machine ID from random generator.1696worker2 # [ 1.307449] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11697worker1 # [ 1.336658] systemd[1]: Hostname set to <worker1>.1698client # [ 1.390746] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01699worker2 # [ 1.313481] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1700worker2 # [ 1.313538] systemd[1]: Detected virtualization qemu.1701worker2 # [ 1.313617] systemd[1]: Detected architecture arm64.1702worker2 # [ 1.313640] systemd[1]: Running in initrd.1703worker2 # [ 1.314547] systemd[1]: Initializing machine ID from random generator.1704worker2 # [ 1.314854] systemd[1]: Hostname set to <worker2>.1705lb # [ 1.391128] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker1 # [ 1.394616] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker2 # [ 1.391138] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708client # [ 1.510574] usb 1-2: new high-speed USB device number 3 using ehci-pci1709lb # [ 1.510980] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker1 # [ 1.514526] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker2 # [ 1.510923] usb 1-2: new high-speed USB device number 3 using ehci-pci1712lb # [ 1.567066] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713worker1 # [ 1.592099] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714client # [ 1.645949] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715lb # [ 1.622211] systemd[1]: Queued start job for default target Initrd Default Target.1716client # [ 1.668972] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21717lb # [ 1.632912] systemd[1]: Created slice Slice /system/modprobe.1718lb # [ 1.633868] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1719client # [ 1.674706] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720lb # [ 1.634924] systemd[1]: Expecting device /dev/disk/by-label/nixos...1721lb # [ 1.636233] systemd[1]: Reached target Path Units.1722lb # [ 1.636263] systemd[1]: Reached target Slice Units.1723lb # [ 1.636280] systemd[1]: Reached target Swaps.1724lb # [ 1.636296] systemd[1]: Reached target Timer Units.1725lb # [ 1.636443] systemd[1]: Listening on D-Bus System Message Bus Socket.1726lb # [ 1.636594] systemd[1]: Listening on Journal Socket (/dev/log).1727lb # [ 1.636716] systemd[1]: Listening on Journal Sockets.1728lb # [ 1.636829] systemd[1]: Listening on udev Control Socket.1729lb # [ 1.636929] systemd[1]: Listening on udev Kernel Socket.1730lb # [ 1.636948] systemd[1]: Reached target Socket Units.1731lb # [ 1.645026] systemd[1]: Starting Create List of Static Device Nodes...1732lb # [ 1.646004] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1733lb # [ 1.655113] systemd[1]: Mounting Kernel Configuration File System...1734lb # [ 1.662138] systemd[1]: Starting Journal Service...1735worker2 # [ 1.637353] systemd[1]: bpf-restrict-fs: LSM BPF program attached1736worker1 # [ 1.673444] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21737lb # [ 1.676105] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21738worker1 # [ 1.676203] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01739lb # [ 1.677868] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01740lb # [ 1.683239] systemd[1]: Starting Load Kernel Modules...1741lb # [ 1.683347] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1742worker1 # [ 1.687551] systemd[1]: Queued start job for default target Initrd Default Target.1743worker2 # [ 1.663414] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21744worker1 # [ 1.697119] systemd[1]: Created slice Slice /system/modprobe.1745lb # [ 1.696112] systemd[1]: Starting Coldplug All udev Devices...1746worker2 # [ 1.671122] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01747worker1 # [ 1.698574] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1748worker1 # [ 1.698633] systemd[1]: Expecting device /dev/disk/by-label/nixos...1749worker1 # [ 1.698671] systemd[1]: Reached target Path Units.1750worker1 # [ 1.698693] systemd[1]: Reached target Slice Units.1751worker1 # [ 1.698715] systemd[1]: Reached target Swaps.1752worker1 # [ 1.698738] systemd[1]: Reached target Timer Units.1753worker1 # [ 1.698948] systemd[1]: Listening on D-Bus System Message Bus Socket.1754worker1 # [ 1.699120] systemd[1]: Listening on Journal Socket (/dev/log).1755worker1 # [ 1.699286] systemd[1]: Listening on Journal Sockets.1756worker1 # [ 1.699438] systemd[1]: Listening on udev Control Socket.1757worker1 # [ 1.699560] systemd[1]: Listening on udev Kernel Socket.1758worker1 # [ 1.699585] systemd[1]: Reached target Socket Units.1759worker1 # [ 1.713587] systemd[1]: Starting Create List of Static Device Nodes...1760worker1 # [ 1.714918] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1761client # [ 1.759305] systemd[1]: Queued start job for default target Initrd Default Target.1762lb # [ 1.715062] systemd[1]: Finished Create List of Static Device Nodes.1763lb # [ 1.715909] systemd[1]: Mounted Kernel Configuration File System.1764worker1 # [ 1.726330] systemd[1]: Mounting Kernel Configuration File System...1765client # [ 1.770245] systemd[1]: Created slice Slice /system/modprobe.1766client # [ 1.771541] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1767client # [ 1.772843] systemd[1]: Expecting device /dev/disk/by-label/nixos...1768client # [ 1.773914] systemd[1]: Reached target Path Units.1769lb # [ 1.727338] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1770client # [ 1.774783] systemd[1]: Reached target Slice Units.1771client # [ 1.775611] systemd[1]: Reached target Swaps.1772client # [ 1.776398] systemd[1]: Reached target Timer Units.1773client # [ 1.777407] systemd[1]: Listening on D-Bus System Message Bus Socket.1774client # [ 1.778665] systemd[1]: Listening on Journal Socket (/dev/log).1775client # [ 1.779784] systemd[1]: Listening on Journal Sockets.1776client # [ 1.780775] systemd[1]: Listening on udev Control Socket.1777client # [ 1.781807] systemd[1]: Listening on udev Kernel Socket.1778client # [ 1.782748] systemd[1]: Reached target Socket Units.1779client # [ 1.785158] systemd[1]: Starting Create List of Static Device Nodes...1780client # [ 1.786319] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1781lb # [ 1.748745] systemd-journald[72]: Collecting audit messages is disabled.1782client # [ 1.794664] systemd[1]: Mounting Kernel Configuration File System...1783worker1 # [ 1.750642] systemd[1]: Starting Journal Service...1784client # [ 1.802821] systemd[1]: Starting Journal Service...1785lb # [ 1.758844] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1786worker1 # [ 1.761128] systemd[1]: Starting Load Kernel Modules...1787worker1 # [ 1.762073] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1788lb # [ 1.767583] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1789worker2 # [ 1.744582] systemd[1]: Queued start job for default target Initrd Default Target.1790lb # [ 1.770089] systemd[1]: Starting Create Static Device Nodes in /dev...1791worker1 # [ 1.774881] systemd[1]: Starting Coldplug All udev Devices...1792worker2 # [ 1.753170] systemd[1]: Created slice Slice /system/modprobe.1793worker2 # [ 1.754320] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1794lb # [ 1.779312] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1795worker2 # [ 1.755687] systemd[1]: Expecting device /dev/disk/by-label/nixos...1796worker2 # [ 1.756702] systemd[1]: Reached target Path Units.1797worker2 # [ 1.757489] systemd[1]: Reached target Slice Units.1798worker2 # [ 1.758274] systemd[1]: Reached target Swaps.1799client # [ 1.826762] systemd[1]: Starting Load Kernel Modules...1800worker2 # [ 1.759022] systemd[1]: Reached target Timer Units.1801client # [ 1.827730] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1802worker2 # [ 1.759993] systemd[1]: Listening on D-Bus System Message Bus Socket.1803worker2 # [ 1.761204] systemd[1]: Listening on Journal Socket (/dev/log).1804worker2 # [ 1.762301] systemd[1]: Listening on Journal Sockets.1805worker2 # [ 1.763325] systemd[1]: Listening on udev Control Socket.1806worker1 # [ 1.786619] systemd[1]: Finished Create List of Static Device Nodes.1807worker1 # [ 1.787405] systemd[1]: Mounted Kernel Configuration File System.1808worker2 # [ 1.764425] systemd[1]: Listening on udev Kernel Socket.1809worker2 # [ 1.765295] systemd[1]: Reached target Socket Units.1810lb # [ 1.788167] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01811lb # [ 1.788407] [drm] features: -virgl +edid -resource_blob -host_visible1812lb # [ 1.788416] [drm] features: -context_init1813worker2 # [ 1.767741] systemd[1]: Starting Create List of Static Device Nodes...1814lb # [ 1.789119] [drm] number of scanouts: 11815lb # [ 1.789137] [drm] number of cap sets: 01816worker2 # [ 1.768866] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1817worker2 # [ 1.776189] systemd[1]: Mounting Kernel Configuration File System...1818client # [ 1.849112] systemd-journald[72]: Collecting audit messages is disabled.1819worker1 # [ 1.810722] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1820worker2 # [ 1.787169] systemd[1]: Starting Journal Service...1821worker1 # [ 1.812713] systemd-journald[72]: Collecting audit messages is disabled.1822client # [ 1.859159] systemd[1]: Starting Coldplug All udev Devices...1823lb # [ 1.815533] systemd[1]: Finished Create Static Device Nodes in /dev.1824lb # [ 1.815896] systemd[1]: Reached target Preparation for Local File Systems.1825lb # [ 1.815924] systemd[1]: Reached target Local File Systems.1826client # [ 1.861373] systemd[1]: Finished Create List of Static Device Nodes.1827client # [ 1.862043] systemd[1]: Mounted Kernel Configuration File System.1828lb # [ 1.819808] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1829lb # [ 1.819822] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01830lb # [ 1.827949] systemd[1]: Starting Rule-based Manager for Device Events and Files...1831client # [ 1.871611] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1832worker1 # [ 1.834835] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1833worker2 # [ 1.811301] systemd[1]: Starting Load Kernel Modules...1834worker2 # [ 1.811417] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1835worker1 # [ 1.846522] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1836worker1 # [ 1.849291] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1837lb # [ 1.843318] Console: switching to colour frame buffer device 160x501838worker2 # [ 1.830629] systemd[1]: Starting Coldplug All udev Devices...1839worker1 # [ 1.856539] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01840worker1 # [ 1.856773] [drm] features: -virgl +edid -resource_blob -host_visible1841worker1 # [ 1.856783] [drm] features: -context_init1842worker1 # [ 1.857475] [drm] number of scanouts: 11843worker1 # [ 1.857493] [drm] number of cap sets: 01844worker1 # [ 1.861160] systemd[1]: Starting Create Static Device Nodes in /dev...1845worker2 # [ 1.840309] systemd-journald[72]: Collecting audit messages is disabled.1846client # [ 1.915185] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1847client # [ 1.918821] systemd[1]: Starting Create Static Device Nodes in /dev...1848lb # [ 1.875541] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1849lb # [ 1.884355] systemd[1]: Finished Load Kernel Modules.1850worker2 # [ 1.859001] systemd[1]: Finished Create List of Static Device Nodes.1851worker2 # [ 1.859736] systemd[1]: Mounted Kernel Configuration File System.1852worker1 # [ 1.878836] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1853worker1 # [ 1.878853] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01854worker2 # [ 1.862614] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1855client # [ 1.936995] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1856worker1 # [ 1.899030] systemd[1]: Finished Create Static Device Nodes in /dev.1857worker1 # [ 1.899356] systemd[1]: Reached target Preparation for Local File Systems.1858worker1 # [ 1.899384] systemd[1]: Reached target Local File Systems.1859lb # [ 1.899304] systemd[1]: Starting Apply Kernel Variables...1860worker1 # [ 1.903294] systemd[1]: Starting Rule-based Manager for Device Events and Files...1861worker1 # [ 1.913022] Console: switching to colour frame buffer device 160x501862worker2 # [ 1.892759] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1863client # [ 1.958907] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1864client # [ 1.965165] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01865client # [ 1.965423] [drm] features: -virgl +edid -resource_blob -host_visible1866client # [ 1.965432] [drm] features: -context_init1867client # [ 1.966172] [drm] number of scanouts: 11868client # [ 1.966190] [drm] number of cap sets: 01869lb # [ 1.931677] systemd[1]: Finished Apply Kernel Variables.1870worker2 # [ 1.907531] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1871worker2 # [ 1.909725] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1872lb # [ 1.936769] systemd[1]: Started Journal Service.1873worker1 # [ 1.939026] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1874worker2 # [ 1.912202] systemd[1]: Starting Create Static Device Nodes in /dev...1875client # [ 1.983202] systemd[1]: Finished Create Static Device Nodes in /dev.1876client # [ 1.983553] systemd[1]: Reached target Preparation for Local File Systems.1877client # [ 1.983583] systemd[1]: Reached target Local File Systems.1878worker2 # [ 1.920994] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01879lb # [ 1.932292] systemd-modules-load[73]: Inserted module 'dm_mod'1880worker2 # [ 1.921260] [drm] features: -virgl +edid -resource_blob -host_visible1881worker2 # [ 1.921270] [drm] features: -context_init1882worker2 # [ 1.921994] [drm] number of scanouts: 11883lb # [ 1.933987] systemd-modules-load[73]: Module 'virtio_balloon' is built in1884worker2 # [ 1.922012] [drm] number of cap sets: 01885lb # [ 1.935081] systemd-modules-load[73]: Module 'virtio_console' is built in1886client # [ 1.990852] systemd[1]: Starting Rule-based Manager for Device Events and Files...1887worker1 # [ 1.954462] systemd[1]: Finished Load Kernel Modules.1888lb # [ 1.941262] systemd-modules-load[73]: Inserted module 'virtio_gpu'1889lb # [ 1.942371] systemd-modules-load[73]: Module 'virtio_rng' is built in1890worker1 # [ 1.957603] systemd[1]: Starting Apply Kernel Variables...1891lb # [ 1.943647] systemd-udevd[79]: Using default interface naming scheme 'v261'.1892client # [ 1.998819] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1893client # [ 1.998836] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01894lb # [ 1.956889] systemd[1]: Starting Create System Files and Directories...1895worker2 # [ 1.943205] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1896worker2 # [ 1.943223] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01897client # [ 2.018735] Console: switching to colour frame buffer device 160x501898lb # [ 1.976172] systemd[1]: Finished Create System Files and Directories.1899client # [ 2.020569] systemd-modules-load[74]: Inserted module 'dm_mod'1900client # [ 2.021826] systemd-modules-load[74]: Module 'virtio_balloon' is built in1901client # [ 2.022914] systemd-modules-load[74]: Module 'virtio_console' is built in1902worker2 # [ 1.969391] systemd[1]: Finished Create Static Device Nodes in /dev.1903worker2 # [ 1.969761] systemd[1]: Reached target Preparation for Local File Systems.1904client # [ 2.039491] systemd[1]: Started Journal Service.1905worker2 # [ 1.969788] systemd[1]: Reached target Local File Systems.1906worker2 # [ 1.973692] systemd[1]: Starting Rule-based Manager for Device Events and Files...1907lb # [ 1.984498] systemd[1]: Started Rule-based Manager for Device Events and Files.1908worker1 # [ 1.999408] systemd[1]: Started Journal Service.1909worker1 # [ 1.988174] systemd-modules-load[73]: Inserted module 'dm_mod'1910worker1 # [ 1.989302] systemd-modules-load[73]: Module 'virtio_balloon' is built in1911worker1 # [ 1.990373] systemd-modules-load[73]: Module 'virtio_console' is built in1912worker1 # [ 1.991411] systemd-modules-load[73]: Inserted module 'virtio_gpu'1913worker2 # [ 1.975103] Console: switching to colour frame buffer device 160x501914client # [ 2.052373] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1915client # [ 2.040769] systemd[1]: Starting Create System Files and Directories...1916client # [ 2.048628] systemd-modules-load[74]: Inserted module 'virtio_gpu'1917client # [ 2.052163] systemd-modules-load[74]: Module 'virtio_rng' is built in1918worker1 # [ 2.010010] systemd-modules-load[73]: Module 'virtio_rng' is built in1919worker1 # [ 2.011207] systemd[1]: Starting Create System Files and Directories...1920client # [ 2.057599] systemd[1]: Finished Load Kernel Modules.1921worker2 # [ 2.003700] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1922worker1 # [ 2.016732] systemd-udevd[78]: Using default interface naming scheme 'v261'.1923worker1 # [ 2.027198] systemd[1]: Finished Apply Kernel Variables.1924worker2 # [ 2.019572] systemd[1]: Finished Load Kernel Modules.1925client # [ 2.080400] systemd-udevd[78]: Using default interface naming scheme 'v261'.1926client # [ 2.084193] systemd[1]: Starting Apply Kernel Variables...1927worker2 # [ 2.027251] systemd[1]: Starting Apply Kernel Variables...1928worker1 # [ 2.045072] systemd[1]: Finished Create System Files and Directories.1929worker2 # [ 2.015374] systemd-modules-load[74]: Inserted module 'dm_mod'1930worker2 # [ 2.016650] systemd-modules-load[74]: Module 'virtio_balloon' is built in1931lb # [ 2.047417] systemd[1]: Starting Virtual Console Setup...1932worker2 # [ 2.024463] systemd-modules-load[74]: Module 'virtio_console' is built in1933worker2 # [ 2.044572] systemd[1]: Started Journal Service.1934worker2 # [ 2.032314] systemd-modules-load[74]: Inserted module 'virtio_gpu'1935worker1 # [ 2.062702] systemd[1]: Started Rule-based Manager for Device Events and Files.1936worker2 # [ 2.035419] systemd-modules-load[74]: Module 'virtio_rng' is built in1937client # [ 2.116294] systemd[1]: Finished Create System Files and Directories.1938client # [ 2.120570] systemd[1]: Finished Apply Kernel Variables.1939worker2 # [ 2.053198] systemd[1]: Starting Create System Files and Directories...1940client # [ 2.130392] systemd[1]: Started Rule-based Manager for Device Events and Files.1941worker2 # [ 2.062227] systemd-udevd[78]: Using default interface naming scheme 'v261'.1942worker2 # [ 2.073069] systemd[1]: Finished Apply Kernel Variables.1943lb # [ 2.120257] systemd-vconsole-setup[102]: Configuration of first virtual console was skipped, ignoring remaining ones.1944worker1 # [ 2.122721] systemd[1]: Starting Virtual Console Setup...1945lb # [ 2.123850] systemd[1]: Finished Virtual Console Setup.1946worker2 # [ 2.095872] systemd[1]: Finished Create System Files and Directories.1947worker2 # [ 2.109273] systemd[1]: Started Rule-based Manager for Device Events and Files.1948client # [ 2.183925] systemd[1]: Starting Virtual Console Setup...1949worker1 # [ 2.172483] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1950worker1 # [ 2.175875] systemd[1]: Finished Virtual Console Setup.1951client # [ 2.232439] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1952client # [ 2.235650] systemd[1]: Finished Virtual Console Setup.1953worker2 # [ 2.172103] systemd[1]: Starting Virtual Console Setup...1954worker2 # [ 2.216474] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1955worker2 # [ 2.219879] systemd[1]: Finished Virtual Console Setup.1956lb # [ 2.746713] systemd[1]: Finished Coldplug All udev Devices.1957lb # [ 2.747720] systemd[1]: Reached target System Initialization.1958lb # [ 2.752152] systemd[1]: Reached target Basic System.1959worker1 # [ 2.780118] systemd[1]: Finished Coldplug All udev Devices.1960worker1 # [ 2.781073] systemd[1]: Reached target System Initialization.1961worker1 # [ 2.782090] systemd[1]: Reached target Basic System.1962client # [ 2.834160] systemd[1]: Finished Coldplug All udev Devices.1963client # [ 2.835121] systemd[1]: Reached target System Initialization.1964client # [ 2.835941] systemd[1]: Reached target Basic System.1965worker2 # [ 2.865449] systemd[1]: Finished Coldplug All udev Devices.1966worker2 # [ 2.866367] systemd[1]: Reached target System Initialization.1967worker2 # [ 2.867184] systemd[1]: Reached target Basic System.1968lb # [ 2.908431] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969lb # [ 2.922062] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1970client # [ 2.967036] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 2.932222] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972lb # [ 2.946471] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1973worker1 # [ 2.961840] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1974client # [ 3.008618] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975worker1 # [ 2.972151] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1976client # [ 3.015892] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1977lb # [ 3.000071] systemd[1]: Found device /dev/disk/by-label/nixos.1978lb # [ 3.005419] systemd[1]: Reached target Initrd Root Device.1979lb # [ 3.009796] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1980client # [ 3.065467] systemd[1]: Found device /dev/disk/by-label/nixos.1981client # [ 3.068153] systemd[1]: Reached target Initrd Root Device.1982client # [ 3.071010] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1983worker1 # [ 3.032886] systemd[1]: Found device /dev/disk/by-label/nixos.1984worker1 # [ 3.042527] systemd[1]: Reached target Initrd Root Device.1985worker1 # [ 3.046918] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986worker2 # [ 3.029903] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1987lb # [ 3.064290] systemd-fsck[111]: nixos: clean, 12/65536 files, 13019/262144 blocks1988worker2 # [ 3.039584] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1989lb # [ 3.069802] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1990worker2 # [ 3.044741] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1991lb # [ 3.074186] systemd[1]: Mounting /sysroot...1992client # [ 3.125627] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1993client # [ 3.131031] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1994worker1 # [ 3.092931] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1995worker1 # [ 3.099552] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1996client # [ 3.146250] systemd[1]: Mounting /sysroot...1997worker1 # [ 3.104791] systemd[1]: Mounting /sysroot...1998lb # [ 3.123279] EXT4-fs (vda): mounted filesystem ecc6f207-dc8e-431d-b120-fb330412fd42 r/w with ordered data mode. Quota mode: none.1999lb # [ 3.117556] systemd[1]: Mounted /sysroot.2000lb # [ 3.118619] systemd[1]: Reached target Initrd Root File System.
loading 2,082 hidden lines…
4083client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4084worker1 # [ 40.043847] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4085client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4086client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4087client # farm-b> LOG-farm-b-t14088worker1 # [ 41.511366] nix-daemon[831]: reaped child process 895, status = succeeded4089worker1 # [ 41.626768] nix-daemon[831]: accepted connection from pid 912, user nix-grpc-daemon (trusted)4090worker1 # [ 41.647546] nix-daemon[831]: reaped child process 915, status = succeeded4091lb # [ 41.645709] niks3-server[964]: 2026/09/22 16:59:06 INFO Received uploads request method=POST path=/api/pending_closures4092worker1 # [ 41.656830] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.273Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4093worker1 # [ 41.659900] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.273Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4094lb # [ 41.689594] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4095worker1 # [ 41.704104] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.320Z level=INFO msg="Uploading 1 narinfos"4096lb # [ 41.705773] niks3-server[964]: 2026/09/22 16:59:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4097lb # [ 41.715302] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4098lb # [ 41.723758] niks3-server[964]: 2026/09/22 16:59:06 INFO Signed narinfos id=4 count=14099lb # [ 41.731485] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4100worker1 # [ 41.732668] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.349Z level=INFO msg="Upload complete. (277ms)"4101lb # [ 41.735873] niks3-server[964]: 2026/09/22 16:59:06 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4102lb # [ 41.741076] niks3-server[964]: 2026/09/22 16:59:06 INFO Completed upload id=44103lb # [ 41.743709] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4104worker1 # [ 41.745198] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14105worker2 # [ 41.723763] nix-grpc-daemon[774]: ts=2026-09-22T16:59:06Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04106worker2 # [ 41.729807] nix-grpc-daemon[774]: ts=2026-09-22T16:59:06Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14107worker1 # [ 41.763728] nix-daemon[831]: reaped child process 893, status = succeeded4108worker1 # [ 41.797113] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184109worker1 # [ 41.814088] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184110worker1 # [ 41.830022] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.446Z level=INFO msg="All 2 paths already cached"4111client # copying 1 paths to 'lb:50051'...4112worker1 # [ 41.841408] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532184113client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4114worker1 # [ 41.859169] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4115worker1 # [ 42.022624] nix-daemon[831]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4116worker1 # [ 42.040907] nix-daemon[831]: reaped child process 922, status = succeeded4117lb # [ 42.046421] niks3-server[964]: 2026/09/22 16:59:06 INFO Received uploads request method=POST path=/api/pending_closures4118worker1 # [ 42.055412] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.671Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4119worker1 # [ 42.057222] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.671Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4120lb # [ 42.066148] niks3-server[964]: 2026/09/22 16:59:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4121worker1 # [ 42.072001] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.688Z level=INFO msg="Uploading 1 narinfos"4122lb # [ 42.073699] niks3-server[964]: 2026/09/22 16:59:06 INFO Signed narinfos id=5 count=14123lb # [ 42.077834] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4124lb # [ 42.085498] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4125worker1 # [ 42.090811] nix-grpc-daemon[844]: time=2026-09-22T16:59:06.707Z level=INFO msg="Upload complete. (214ms)"4126lb # [ 42.089910] niks3-server[964]: 2026/09/22 16:59:06 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4127lb # [ 42.093129] niks3-server[964]: 2026/09/22 16:59:06 INFO Completed upload id=54128worker1 # [ 42.096549] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=7524129lb # [ 42.096588] niks3-server[964]: 2026/09/22 16:59:06 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4130worker1 # [ 42.104807] nix-grpc-daemon[767]: ts=2026-09-22T16:59:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184131worker1 # [ 42.114724] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4132worker1 # [ 42.118825] nix-daemon[831]: reaped child process 917, status = succeeded4133client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4134worker1 # [ 42.130451] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4135client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4136client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4137client # farm-top> LOG-farm-top-t14138worker1 # [ 43.220386] nix-daemon[831]: reaped child process 926, status = succeeded4139worker1 # [ 43.292946] nix-daemon[831]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4140worker1 # [ 43.312761] nix-daemon[831]: reaped child process 945, status = succeeded4141lb # [ 43.314163] niks3-server[964]: 2026/09/22 16:59:07 INFO Received uploads request method=POST path=/api/pending_closures4142worker1 # [ 43.328417] nix-grpc-daemon[844]: time=2026-09-22T16:59:07.943Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4143worker1 # [ 43.330062] nix-grpc-daemon[844]: time=2026-09-22T16:59:07.943Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4144lb # [ 43.352743] niks3-server[964]: 2026/09/22 16:59:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4145worker1 # [ 43.362880] nix-grpc-daemon[844]: time=2026-09-22T16:59:07.979Z level=INFO msg="Uploading 1 narinfos"4146lb # [ 43.368342] niks3-server[964]: 2026/09/22 16:59:07 INFO Signed narinfos id=6 count=14147lb # [ 43.376712] niks3-server[964]: 2026/09/22 16:59:07 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4148lb # [ 43.384930] niks3-server[964]: 2026/09/22 16:59:07 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4149lb # [ 43.393051] niks3-server[964]: 2026/09/22 16:59:08 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4150lb # [ 43.403751] niks3-server[964]: 2026/09/22 16:59:08 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4151worker1 # [ 43.411964] nix-grpc-daemon[844]: time=2026-09-22T16:59:08.028Z level=INFO msg="Upload complete. (272ms)"4152lb # [ 43.410015] niks3-server[964]: 2026/09/22 16:59:08 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4153lb # [ 43.419021] niks3-server[964]: 2026/09/22 16:59:08 INFO Completed upload id=64154worker1 # [ 43.425000] nix-grpc-daemon[767]: ts=2026-09-22T16:59:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14155worker1 # [ 43.443272] nix-daemon[831]: reaped child process 924, status = succeeded4156client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 5.84 seconds)4157client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4158client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds)4159client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4160client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.02 seconds)4161client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4162client # copying 3 paths...4163client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4164client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4165client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4166client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds)4167(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.36 seconds)4168subtest: repeat is answered Cached by the scheduler without building4169worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4170worker1 # finding garbage collector roots...4171worker1 # deleting specified paths...4172worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4173worker1 # deleting unused links...4174worker1 # note: hard linking is currently saving 0.0 KiB4175worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4176worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4177worker2 # finding garbage collector roots...4178worker2 # deleting specified paths...4179worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4180worker2 # deleting unused links...4181worker2 # note: hard linking is currently saving 0.0 KiB4182worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)4183client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24184worker1 # [ 44.687921] nix-grpc-daemon[767]: ts=2026-09-22T16:59:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:532184185client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4186client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4187worker1 # [ 44.704684] nix-grpc-daemon[767]: ts=2026-09-22T16:59:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184188worker1 # [ 44.715714] nix-grpc-daemon[844]: time=2026-09-22T16:59:09.331Z level=INFO msg="All 2 paths already cached"4189worker2 # [ 44.705100] nix-grpc-daemon[774]: ts=2026-09-22T16:59:09Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14190client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.31 seconds)4191client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4192client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds)4193lb: must succeed: curl -sf localhost:9901/clusters4194lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4195lb: must succeed: curl -sf localhost:9901/clusters4196lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4197worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4198worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4199(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.20 seconds)4200subtest: inputs already in the cache are not sent to the scheduler4201lb: must succeed: curl -sf localhost:9901/clusters4202lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4203lb: must succeed: curl -sf localhost:9901/clusters4204lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4205worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4206worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4207client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24208worker2 # [ 45.586650] nix-grpc-daemon[774]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442404209client # don't know how to build these paths:4210client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4211worker2 # [ 45.596458] nix-grpc-daemon[774]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404212worker2 # [ 45.622825] nix-grpc-daemon[774]: ts=2026-09-22T16:59:10Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04213worker2 # [ 45.628358] nix-grpc-daemon[774]: ts=2026-09-22T16:59:10Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14214worker1 # [ 45.698577] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184215worker1 # [ 45.713921] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184216worker1 # [ 45.729538] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.345Z level=INFO msg="All 2 paths already cached"4217client # copying 1 paths to 'lb:50051'...4218client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4219worker1 # [ 45.740548] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532184220worker1 # [ 45.754843] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4221worker1 # [ 45.915880] nix-daemon[831]: accepted connection from pid 963, user nix-grpc-daemon (trusted)4222worker1 # [ 45.936542] nix-daemon[831]: reaped child process 966, status = succeeded4223lb # [ 45.942188] niks3-server[964]: 2026/09/22 16:59:10 INFO Received uploads request method=POST path=/api/pending_closures4224worker1 # [ 45.953353] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.569Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4225worker1 # [ 45.954854] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.569Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4226lb # [ 45.970088] niks3-server[964]: 2026/09/22 16:59:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4227worker1 # [ 45.981078] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.597Z level=INFO msg="Uploading 1 narinfos"4228lb # [ 45.982475] niks3-server[964]: 2026/09/22 16:59:10 INFO Signed narinfos id=7 count=14229lb # [ 45.992074] niks3-server[964]: 2026/09/22 16:59:10 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4230lb # [ 46.007278] niks3-server[964]: 2026/09/22 16:59:10 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4231lb # [ 46.020300] niks3-server[964]: 2026/09/22 16:59:10 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4232worker1 # [ 46.028786] nix-grpc-daemon[844]: time=2026-09-22T16:59:10.645Z level=INFO msg="Upload complete. (256ms)"4233lb # [ 46.030045] niks3-server[964]: 2026/09/22 16:59:10 INFO Completed upload id=74234lb # [ 46.033929] niks3-server[964]: 2026/09/22 16:59:10 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4235worker1 # [ 46.037332] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=7524236worker1 # [ 46.054952] nix-grpc-daemon[767]: ts=2026-09-22T16:59:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184237worker1 # [ 46.062068] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4238worker1 # [ 46.067300] nix-daemon[831]: reaped child process 961, status = succeeded4239client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4240worker1 # [ 46.080234] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4241client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4242client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4243client # farm-top> LOG-farm-top-t1b4244worker1 # [ 47.518366] nix-daemon[831]: reaped child process 970, status = succeeded4245worker1 # [ 47.628095] nix-daemon[831]: accepted connection from pid 986, user nix-grpc-daemon (trusted)4246worker1 # [ 47.645093] nix-daemon[831]: reaped child process 989, status = succeeded4247lb # [ 47.650575] niks3-server[964]: 2026/09/22 16:59:12 INFO Received uploads request method=POST path=/api/pending_closures4248worker1 # [ 47.664815] nix-grpc-daemon[844]: time=2026-09-22T16:59:12.281Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4249worker1 # [ 47.666336] nix-grpc-daemon[844]: time=2026-09-22T16:59:12.281Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4250lb # [ 47.705980] niks3-server[964]: 2026/09/22 16:59:12 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4251lb # [ 47.719261] niks3-server[964]: 2026/09/22 16:59:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4252worker1 # [ 47.726749] nix-grpc-daemon[844]: time=2026-09-22T16:59:12.343Z level=INFO msg="Uploading 1 narinfos"4253lb # [ 47.728567] niks3-server[964]: 2026/09/22 16:59:12 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4254lb # [ 47.736794] niks3-server[964]: 2026/09/22 16:59:12 INFO Signed narinfos id=8 count=14255lb # [ 47.738711] niks3-server[964]: 2026/09/22 16:59:12 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4256lb # [ 47.750093] niks3-server[964]: 2026/09/22 16:59:12 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4257lb # [ 47.755690] niks3-server[964]: 2026/09/22 16:59:12 INFO Completed upload id=84258worker1 # [ 47.757090] nix-grpc-daemon[844]: time=2026-09-22T16:59:12.373Z level=INFO msg="Upload complete. (266ms)"4259lb # [ 47.759242] niks3-server[964]: 2026/09/22 16:59:12 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4260worker1 # [ 47.771466] nix-grpc-daemon[767]: ts=2026-09-22T16:59:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14261worker1 # [ 47.793473] nix-daemon[831]: reaped child process 968, status = succeeded4262client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.42 seconds)4263client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4264client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.23 seconds)4265lb: must succeed: curl -sf localhost:9901/clusters4266lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4267lb: must succeed: curl -sf localhost:9901/clusters4268lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4269worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4270worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4271(finished: subtest: inputs already in the cache are not sent to the scheduler, in 3.05 seconds)4272subtest: outputs read back through any worker carry the cache signature4273client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24274worker1 # [ 48.511541] nix-grpc-daemon[767]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:532184275client # don't know how to build these paths:4276client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4277worker2 # [ 48.497497] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404278worker2 # [ 48.543906] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04279worker2 # [ 48.554841] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14280worker2 # [ 48.596366] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404281worker2 # [ 48.619474] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404282client # copying 1 paths to 'lb:50051'...4283client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4284worker2 # [ 48.642035] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442404285worker2 # [ 48.661698] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4286worker2 # [ 48.694408] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=info event=niks3_push_started pid=9124287worker2 # [ 48.898890] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.522Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4288worker2 # [ 49.024744] nix-daemon[859]: accepted connection from pid 924, user nix-grpc-daemon (trusted)4289worker2 # [ 49.049007] nix-daemon[859]: reaped child process 927, status = succeeded4290lb # [ 49.079361] niks3-server[964]: 2026/09/22 16:59:13 INFO Received uploads request method=POST path=/api/pending_closures4291worker2 # [ 49.067026] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.689Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4292worker2 # [ 49.071221] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.689Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4293lb # [ 49.141444] niks3-server[964]: 2026/09/22 16:59:13 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4294worker2 # [ 49.126985] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.749Z level=INFO msg="Uploading 1 narinfos"4295lb # [ 49.155221] niks3-server[964]: 2026/09/22 16:59:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4296lb # [ 49.167452] niks3-server[964]: 2026/09/22 16:59:13 INFO Signed narinfos id=9 count=14297lb # [ 49.183848] niks3-server[964]: 2026/09/22 16:59:13 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4298lb # [ 49.191751] niks3-server[964]: 2026/09/22 16:59:13 INFO Completed upload id=94299worker2 # [ 49.163241] nix-grpc-daemon[912]: time=2026-09-22T16:59:13.786Z level=INFO msg="Upload complete. (260ms)"4300lb # [ 49.196775] niks3-server[964]: 2026/09/22 16:59:13 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4301worker2 # [ 49.176409] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=5124302lb # [ 49.207406] niks3-server[964]: 2026/09/22 16:59:13 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4303worker2 # [ 49.187100] nix-grpc-daemon[774]: ts=2026-09-22T16:59:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404304worker2 # [ 49.203661] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4305worker2 # [ 49.209977] nix-daemon[859]: reaped child process 910, status = succeeded4306client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4307client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4308worker2 # [ 49.227103] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4309lb # [ 49.938400] envoy[919]: conn peer=[2001:db8:1::1]:52346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9493 tx=4368 ms=56614310lb # [ 49.943997] envoy[919]: conn peer=[2001:db8:1::1]:48152 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=684311lb # [ 49.951029] envoy[919]: conn peer=[2001:db8:1::1]:48154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1895 ms=21694312client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4313worker1 # [ 50.189463] nix-grpc-daemon[767]: E0000 00:00:1790096354.802841 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4314client # farm-a> LOG-farm-a-sig4315worker1 # [ 50.246286] nix-grpc-daemon[767]: E0000 00:00:1790096354.862633 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4316worker2 # [ 50.257441] nix-daemon[859]: reaped child process 931, status = succeeded4317worker1 # [ 50.295931] nix-grpc-daemon[767]: E0000 00:00:1790096354.868476 806 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4318worker1 # [ 50.300144] nix-grpc-daemon[767]: E0000 00:00:1790096354.908422 806 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4319worker2 # [ 50.363081] nix-daemon[859]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4320worker2 # [ 50.382442] nix-daemon[859]: reaped child process 950, status = succeeded4321lb # [ 50.422215] niks3-server[964]: 2026/09/22 16:59:15 INFO Received uploads request method=POST path=/api/pending_closures4322worker2 # [ 50.406498] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.028Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4323worker2 # [ 50.407973] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.028Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4324lb # [ 50.491360] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4325lb # [ 50.499971] niks3-server[964]: 2026/09/22 16:59:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4326lb # [ 50.506323] niks3-server[964]: 2026/09/22 16:59:15 INFO Signed narinfos id=10 count=14327worker2 # [ 50.476783] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.100Z level=INFO msg="Uploading 1 narinfos"4328lb # [ 50.513631] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4329lb # [ 50.519234] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4330lb # [ 50.528923] niks3-server[964]: 2026/09/22 16:59:15 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4331lb # [ 50.536518] niks3-server[964]: 2026/09/22 16:59:15 INFO Completed upload id=104332worker2 # [ 50.508444] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.132Z level=INFO msg="Upload complete. (269ms)"4333lb # [ 50.540273] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4334worker2 # [ 50.533620] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14335worker2 # [ 50.551227] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04336worker2 # [ 50.555918] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14337worker2 # [ 50.561316] nix-daemon[859]: reaped child process 929, status = succeeded4338worker2 # [ 50.574844] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404339worker2 # [ 50.595768] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404340worker2 # [ 50.615981] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.239Z level=INFO msg="All 1 paths already cached"4341client # copying 1 paths to 'lb:50051'...4342client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4343worker2 # [ 50.630289] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442404344worker2 # [ 50.646998] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4345worker2 # [ 50.806847] nix-daemon[859]: accepted connection from pid 956, user nix-grpc-daemon (trusted)4346worker2 # [ 50.828869] nix-daemon[859]: reaped child process 959, status = succeeded4347lb # [ 50.861425] niks3-server[964]: 2026/09/22 16:59:15 INFO Received uploads request method=POST path=/api/pending_closures4348worker2 # [ 50.843546] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.467Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4349worker2 # [ 50.845880] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.467Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4350lb # [ 50.895154] niks3-server[964]: 2026/09/22 16:59:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4351worker2 # [ 50.872495] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.496Z level=INFO msg="Uploading 1 narinfos"4352lb # [ 50.907042] niks3-server[964]: 2026/09/22 16:59:15 INFO Signed narinfos id=11 count=14353lb # [ 50.911211] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4354lb # [ 50.930146] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4355worker2 # [ 50.907596] nix-grpc-daemon[912]: time=2026-09-22T16:59:15.531Z level=INFO msg="Upload complete. (242ms)"4356lb # [ 50.937202] niks3-server[964]: 2026/09/22 16:59:15 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4357lb # [ 50.940792] niks3-server[964]: 2026/09/22 16:59:15 INFO Completed upload id=114358lb # [ 50.943836] niks3-server[964]: 2026/09/22 16:59:15 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4359worker2 # [ 50.916319] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=6244360worker2 # [ 50.928412] nix-grpc-daemon[774]: ts=2026-09-22T16:59:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404361worker2 # [ 50.941285] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4362worker2 # [ 50.946489] nix-daemon[859]: reaped child process 954, status = succeeded4363client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4364worker2 # [ 50.957459] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4365client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4366client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4367client # farm-b> LOG-farm-b-sig4368worker2 # [ 51.848832] nix-daemon[859]: reaped child process 963, status = succeeded4369worker2 # [ 51.941163] nix-daemon[859]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4370worker2 # [ 51.960498] nix-daemon[859]: reaped child process 982, status = succeeded4371lb # [ 51.995925] niks3-server[964]: 2026/09/22 16:59:16 INFO Received uploads request method=POST path=/api/pending_closures4372worker2 # [ 51.977481] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.600Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4373worker2 # [ 51.979143] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.600Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4374lb # [ 52.040860] niks3-server[964]: 2026/09/22 16:59:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4375worker2 # [ 52.022677] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.646Z level=INFO msg="Uploading 1 narinfos"4376lb # [ 52.057259] niks3-server[964]: 2026/09/22 16:59:16 INFO Signed narinfos id=12 count=14377lb # [ 52.065201] niks3-server[964]: 2026/09/22 16:59:16 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4378lb # [ 52.078308] niks3-server[964]: 2026/09/22 16:59:16 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4379worker2 # [ 52.053880] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.677Z level=INFO msg="Upload complete. (284ms)"4380lb # [ 52.086869] niks3-server[964]: 2026/09/22 16:59:16 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4381lb # [ 52.092377] niks3-server[964]: 2026/09/22 16:59:16 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4382lb # [ 52.096680] niks3-server[964]: 2026/09/22 16:59:16 INFO Completed upload id=124383lb # [ 52.099595] niks3-server[964]: 2026/09/22 16:59:16 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4384worker2 # [ 52.071563] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14385worker2 # [ 52.089948] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04386worker2 # [ 52.094089] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14387worker2 # [ 52.098531] nix-daemon[859]: reaped child process 961, status = succeeded4388worker2 # [ 52.128124] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404389worker2 # [ 52.143090] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404390worker2 # [ 52.157623] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.781Z level=INFO msg="All 2 paths already cached"4391client # copying 1 paths to 'lb:50051'...4392client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4393worker2 # [ 52.168711] nix-grpc-daemon[774]: ts=2026-09-22T16:59:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442404394worker2 # [ 52.182576] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4395worker2 # [ 52.338496] nix-daemon[859]: accepted connection from pid 988, user nix-grpc-daemon (trusted)4396worker2 # [ 52.358062] nix-daemon[859]: reaped child process 991, status = succeeded4397lb # [ 52.395027] niks3-server[964]: 2026/09/22 16:59:17 INFO Received uploads request method=POST path=/api/pending_closures4398worker2 # [ 52.373073] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.996Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4399worker2 # [ 52.374590] nix-grpc-daemon[912]: time=2026-09-22T16:59:16.996Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4400worker2 # [ 52.398688] nix-grpc-daemon[912]: time=2026-09-22T16:59:17.022Z level=INFO msg="Uploading 1 narinfos"4401lb # [ 52.427484] niks3-server[964]: 2026/09/22 16:59:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4402lb # [ 52.435596] niks3-server[964]: 2026/09/22 16:59:17 INFO Signed narinfos id=13 count=14403lb # [ 52.440890] niks3-server[964]: 2026/09/22 16:59:17 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4404lb # [ 52.449455] niks3-server[964]: 2026/09/22 16:59:17 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4405worker2 # [ 52.427281] nix-grpc-daemon[912]: time=2026-09-22T16:59:17.051Z level=INFO msg="Upload complete. (227ms)"4406lb # [ 52.458161] niks3-server[964]: 2026/09/22 16:59:17 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4407lb # [ 52.464353] niks3-server[964]: 2026/09/22 16:59:17 INFO Completed upload id=134408worker2 # [ 52.435703] nix-grpc-daemon[774]: ts=2026-09-22T16:59:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=7524409lb # [ 52.466356] niks3-server[964]: 2026/09/22 16:59:17 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4410worker2 # [ 52.447466] nix-grpc-daemon[774]: ts=2026-09-22T16:59:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404411worker2 # [ 52.456227] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4412worker2 # [ 52.458963] nix-daemon[859]: reaped child process 986, status = succeeded4413client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4414worker2 # [ 52.472375] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4415client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4416client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4417client # farm-top> LOG-farm-top-sig4418worker2 # [ 53.364833] nix-daemon[859]: reaped child process 995, status = succeeded4419worker2 # [ 53.445061] nix-daemon[859]: accepted connection from pid 1011, user nix-grpc-daemon (trusted)4420worker2 # [ 53.466296] nix-daemon[859]: reaped child process 1014, status = succeeded4421lb # [ 53.494705] niks3-server[964]: 2026/09/22 16:59:18 INFO Received uploads request method=POST path=/api/pending_closures4422worker2 # [ 53.476606] nix-grpc-daemon[912]: time=2026-09-22T16:59:18.099Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4423worker2 # [ 53.480393] nix-grpc-daemon[912]: time=2026-09-22T16:59:18.099Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4424lb # [ 53.536636] niks3-server[964]: 2026/09/22 16:59:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4425worker2 # [ 53.515890] nix-grpc-daemon[912]: time=2026-09-22T16:59:18.139Z level=INFO msg="Uploading 1 narinfos"4426lb # [ 53.552164] niks3-server[964]: 2026/09/22 16:59:18 INFO Signed narinfos id=14 count=14427lb # [ 53.554894] niks3-server[964]: 2026/09/22 16:59:18 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4428lb # [ 53.571874] niks3-server[964]: 2026/09/22 16:59:18 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4429worker2 # [ 53.548742] nix-grpc-daemon[912]: time=2026-09-22T16:59:18.172Z level=INFO msg="Upload complete. (266ms)"4430lb # [ 53.578585] niks3-server[964]: 2026/09/22 16:59:18 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4431lb # [ 53.581851] niks3-server[964]: 2026/09/22 16:59:18 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4432lb # [ 53.586047] niks3-server[964]: 2026/09/22 16:59:18 INFO Completed upload id=144433lb # [ 53.588456] niks3-server[964]: 2026/09/22 16:59:18 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4434worker2 # [ 53.561110] nix-grpc-daemon[774]: ts=2026-09-22T16:59:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14435worker2 # [ 53.579145] nix-daemon[859]: reaped child process 993, status = succeeded4436client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.45 seconds)4437client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4438client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.20 seconds)4439client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4440worker1 # [ 54.018995] nix-grpc-daemon[767]: ts=2026-09-22T16:59:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:392224441client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4442client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4443client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4444client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4445worker1 # [ 54.059169] nix-grpc-daemon[767]: ts=2026-09-22T16:59:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:392224446worker1 # [ 54.068666] nix-grpc-daemon[767]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4447worker1 # [ 54.086012] nix-grpc-daemon[767]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4448worker1 # [ 54.100538] nix-grpc-daemon[767]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4449client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.25 seconds)4450client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4451worker2 # [ 54.297036] nix-grpc-daemon[774]: ts=2026-09-22T16:59:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:410624452worker2 # [ 54.338827] nix-grpc-daemon[774]: ts=2026-09-22T16:59:18Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:410624453client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds)4454client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4455client # finding garbage collector roots...4456client # removing stale temporary roots file "/nix/var/nix/temproots/896"4457client # removing stale temporary roots file "/nix/var/nix/temproots/941"4458client # removing stale temporary roots file "/nix/var/nix/temproots/905"4459client # removing stale temporary roots file "/nix/var/nix/temproots/884"4460client # removing stale temporary roots file "/nix/var/nix/temproots/918"4461client # removing stale temporary roots file "/nix/var/nix/temproots/926"4462client # removing stale temporary roots file "/nix/var/nix/temproots/863"4463client # removing stale temporary roots file "/nix/var/nix/temproots/846"4464client # deleting specified paths...4465client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4466client # deleting unused links...4467client # note: hard linking is currently saving 0.0 KiB4468worker2 # [ 54.737595] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442404469worker2 # [ 54.746164] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404470worker2 # [ 54.757520] nix-grpc-daemon[912]: time=2026-09-22T16:59:19.380Z level=INFO msg="All 1 paths already cached"4471worker1 # [ 54.796101] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532184472worker2 # [ 54.776277] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442404473worker2 # [ 54.785041] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442404474client # copying 3 paths...4475worker1 # [ 54.830779] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532184476client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4477worker1 # [ 54.861083] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:532184478worker1 # [ 54.868686] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_out=2324479worker1 # [ 54.877261] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:532184480worker1 # [ 54.882597] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_out=1764481worker2 # [ 54.861251] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442404482worker2 # [ 54.868515] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=1284483client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4484client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4485client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.53 seconds)4486(finished: subtest: outputs read back through any worker carry the cache signature, in 6.72 seconds)4487subtest: two clients wanting the same drv build it once4488client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4489client # [ 55.052811] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4490client # Running as unit: dedup1.service; invocation ID: 0edca4d2b8df40c29eef9d643341df504491client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.06 seconds)4492client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4493worker1 # [ 55.193860] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:532184494client # [ 55.244063] nix[994]: don't know how to build these paths:4495client # [ 55.245977] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4496worker2 # [ 55.182439] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04497worker2 # [ 55.188589] nix-grpc-daemon[774]: ts=2026-09-22T16:59:19Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14498worker1 # [ 55.257278] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184499worker1 # [ 55.272989] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184500client # [ 55.329196] nix[994]: copying 1 paths to 'lb:50051'...4501worker1 # [ 55.287472] nix-grpc-daemon[767]: ts=2026-09-22T16:59:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532184502client # [ 55.333554] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4503worker1 # [ 55.305197] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4504worker1 # [ 55.483569] nix-daemon[831]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4505worker1 # [ 55.507870] nix-daemon[831]: reaped child process 999, status = succeeded4506lb # [ 55.506817] niks3-server[964]: 2026/09/22 16:59:20 INFO Received uploads request method=POST path=/api/pending_closures4507worker1 # [ 55.518021] nix-grpc-daemon[844]: time=2026-09-22T16:59:20.134Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4508worker1 # [ 55.519491] nix-grpc-daemon[844]: time=2026-09-22T16:59:20.134Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4509worker1 # [ 55.549885] nix-grpc-daemon[844]: time=2026-09-22T16:59:20.166Z level=INFO msg="Uploading 1 narinfos"4510lb # [ 55.540882] niks3-server[964]: 2026/09/22 16:59:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4511lb # [ 55.552349] niks3-server[964]: 2026/09/22 16:59:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4512lb # [ 55.562641] niks3-server[964]: 2026/09/22 16:59:20 INFO Signed narinfos id=15 count=14513lb # [ 55.572867] niks3-server[964]: 2026/09/22 16:59:20 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4514worker1 # [ 55.580771] nix-grpc-daemon[844]: time=2026-09-22T16:59:20.197Z level=INFO msg="Upload complete. (256ms)"4515lb # [ 55.581483] niks3-server[964]: 2026/09/22 16:59:20 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4516worker1 # [ 55.588548] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=4324517lb # [ 55.591484] niks3-server[964]: 2026/09/22 16:59:20 INFO Completed upload id=154518lb # [ 55.595596] niks3-server[964]: 2026/09/22 16:59:20 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4519worker1 # [ 55.601176] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184520worker1 # [ 55.613922] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4521worker1 # [ 55.619878] nix-daemon[831]: reaped child process 994, status = succeeded4522worker1 # [ 55.630092] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4523client # [ 55.672472] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4524client # [ 56.142733] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4525client # Running as unit: dedup2.service; invocation ID: d21e033d5bdb4b6c94f137f3f24732764526client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4527worker1 # [ 56.337841] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:532184528client # [ 56.392668] nix[1010]: this derivation will be built:4529worker1 # [ 56.350905] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532184530client # [ 56.400078] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4531worker2 # [ 56.336599] nix-grpc-daemon[774]: ts=2026-09-22T16:59:20Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04532worker1 # [ 56.369960] nix-grpc-daemon[767]: ts=2026-09-22T16:59:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184533client # [ 56.417273] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4534client: must succeed: sleep 24535client: (finished: must succeed: sleep 2, in 2.05 seconds)4536client: waiting for success: ! systemctl is-active dedup1 dedup24537worker1 # [ 59.719740] nix-daemon[831]: reaped child process 1003, status = succeeded4538worker1 # [ 59.785306] nix-daemon[831]: accepted connection from pid 1033, user nix-grpc-daemon (trusted)4539worker1 # [ 59.801713] nix-daemon[831]: reaped child process 1036, status = succeeded4540lb # [ 59.809067] niks3-server[964]: 2026/09/22 16:59:24 INFO Received uploads request method=POST path=/api/pending_closures4541worker1 # [ 59.820389] nix-grpc-daemon[844]: time=2026-09-22T16:59:24.435Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4542worker1 # [ 59.821965] nix-grpc-daemon[844]: time=2026-09-22T16:59:24.435Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4543lb # [ 59.858191] niks3-server[964]: 2026/09/22 16:59:24 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4544lb # [ 59.869132] niks3-server[964]: 2026/09/22 16:59:24 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4545worker1 # [ 59.879538] nix-grpc-daemon[844]: time=2026-09-22T16:59:24.495Z level=INFO msg="Uploading 1 narinfos"4546lb # [ 59.881744] niks3-server[964]: 2026/09/22 16:59:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4547lb # [ 59.890715] niks3-server[964]: 2026/09/22 16:59:24 INFO Signed narinfos id=16 count=14548lb # [ 59.896412] niks3-server[964]: 2026/09/22 16:59:24 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4549lb # [ 59.907008] niks3-server[964]: 2026/09/22 16:59:24 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4550lb # [ 59.913382] niks3-server[964]: 2026/09/22 16:59:24 INFO Completed upload id=164551worker1 # [ 59.913058] nix-grpc-daemon[844]: time=2026-09-22T16:59:24.529Z level=INFO msg="Upload complete. (269ms)"4552lb # [ 59.917583] niks3-server[964]: 2026/09/22 16:59:24 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4553worker1 # [ 59.925121] nix-grpc-daemon[767]: ts=2026-09-22T16:59:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14554lb # [ 59.940756] envoy[919]: conn peer=[2001:db8:1::1]:48160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9472 tx=4355 ms=51434555lb # [ 59.944640] envoy[919]: conn peer=[2001:db8:1::1]:56766 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2928 ms=1794556lb # [ 59.948446] envoy[919]: conn peer=[2001:db8:1::1]:56770 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=409 ms=864557worker1 # [ 59.944422] nix-grpc-daemon[767]: ts=2026-09-22T16:59:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14558worker1 # [ 59.953469] nix-daemon[831]: reaped child process 1001, status = succeeded4559lb # [ 59.952589] envoy[919]: conn peer=[2001:db8:1::1]:56792 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=544560lb # [ 59.955787] envoy[919]: conn peer=[2001:db8:1::1]:56784 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=704561lb # [ 59.958792] envoy[919]: conn peer=[2001:db8:1::1]:56796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1563 ms=47504562client # [ 60.005836] systemd[1]: dedup1.service: Deactivated successfully.4563client # [ 60.011226] systemd[1]: dedup1.service: Consumed 171ms CPU time over 4.957s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.4564client # [ 60.046379] systemd[1]: dedup2.service: Deactivated successfully.4565client # [ 60.054907] systemd[1]: dedup2.service: Consumed 213ms CPU time over 3.910s wall clock time, 35.3M memory peak, 6.5K incoming IP traffic, 5.9K outgoing IP traffic.4566client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4567client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4568client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4569worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4570worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)4571worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4572worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4573worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4574worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4575worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4576worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4577(finished: subtest: two clients wanting the same drv build it once, in 6.09 seconds)4578subtest: --timeout and --max-silent-time reach the builder4579worker1 # [ 61.273633] nix-grpc-daemon[767]: ts=2026-09-22T16:59:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:532184580worker2 # [ 61.260974] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04581worker2 # [ 61.261383] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14582worker2 # [ 61.303971] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404583worker2 # [ 61.321076] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404584worker2 # [ 61.336746] nix-grpc-daemon[774]: ts=2026-09-22T16:59:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442404585worker2 # [ 61.353599] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4586worker2 # [ 61.509854] nix-daemon[859]: accepted connection from pid 1045, user nix-grpc-daemon (trusted)4587worker2 # [ 61.527678] nix-daemon[859]: reaped child process 1048, status = succeeded4588lb # [ 61.566159] niks3-server[964]: 2026/09/22 16:59:26 INFO Received uploads request method=POST path=/api/pending_closures4589worker2 # [ 61.545286] nix-grpc-daemon[912]: time=2026-09-22T16:59:26.168Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4590worker2 # [ 61.546833] nix-grpc-daemon[912]: time=2026-09-22T16:59:26.168Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4591lb # [ 61.595420] niks3-server[964]: 2026/09/22 16:59:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4592worker2 # [ 61.575812] nix-grpc-daemon[912]: time=2026-09-22T16:59:26.199Z level=INFO msg="Uploading 1 narinfos"4593lb # [ 61.610499] niks3-server[964]: 2026/09/22 16:59:26 INFO Signed narinfos id=17 count=14594lb # [ 61.618894] niks3-server[964]: 2026/09/22 16:59:26 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4595worker2 # [ 61.604742] nix-grpc-daemon[912]: time=2026-09-22T16:59:26.228Z level=INFO msg="Upload complete. (234ms)"4596lb # [ 61.633368] niks3-server[964]: 2026/09/22 16:59:26 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4597lb # [ 61.643973] niks3-server[964]: 2026/09/22 16:59:26 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4598worker2 # [ 61.614095] nix-grpc-daemon[774]: ts=2026-09-22T16:59:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=4244599lb # [ 61.647550] niks3-server[964]: 2026/09/22 16:59:26 INFO Completed upload id=174600lb # [ 61.650595] niks3-server[964]: 2026/09/22 16:59:26 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4601worker2 # [ 61.624796] nix-grpc-daemon[774]: ts=2026-09-22T16:59:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404602worker2 # [ 61.635471] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4603worker2 # [ 61.641283] nix-daemon[859]: reaped child process 1043, status = succeeded4604worker2 # [ 61.652148] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4605worker2 # [ 67.468943] nix-grpc-daemon[774]: ts=2026-09-22T16:59:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04606worker2 # [ 67.495170] nix-daemon[859]: reaped child process 1050, status = succeeded4607worker2 # [ 67.497061] nix-daemon[859]: reaped child process 1052, status = succeeded4608worker2 # [ 67.779377] nix-grpc-daemon[774]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442404609worker2 # [ 67.794680] nix-grpc-daemon[774]: ts=2026-09-22T16:59:32Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04610worker2 # [ 67.801109] nix-grpc-daemon[774]: ts=2026-09-22T16:59:32Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14611worker1 # [ 67.835664] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184612worker1 # [ 67.851548] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184613worker1 # [ 67.865938] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532184614worker1 # [ 67.881113] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4615worker1 # [ 68.045522] nix-daemon[831]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)4616worker1 # [ 68.066360] nix-daemon[831]: reaped child process 1052, status = succeeded4617lb # [ 68.067430] niks3-server[964]: 2026/09/22 16:59:32 INFO Received uploads request method=POST path=/api/pending_closures4618worker1 # [ 68.078437] nix-grpc-daemon[844]: time=2026-09-22T16:59:32.694Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4619worker1 # [ 68.080098] nix-grpc-daemon[844]: time=2026-09-22T16:59:32.694Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4620lb # [ 68.104523] niks3-server[964]: 2026/09/22 16:59:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4621worker1 # [ 68.114287] nix-grpc-daemon[844]: time=2026-09-22T16:59:32.730Z level=INFO msg="Uploading 1 narinfos"4622lb # [ 68.117397] niks3-server[964]: 2026/09/22 16:59:32 INFO Signed narinfos id=18 count=14623lb # [ 68.124452] niks3-server[964]: 2026/09/22 16:59:32 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4624lb # [ 68.134542] niks3-server[964]: 2026/09/22 16:59:32 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4625worker1 # [ 68.144118] nix-grpc-daemon[844]: time=2026-09-22T16:59:32.760Z level=INFO msg="Upload complete. (246ms)"4626lb # [ 68.143374] niks3-server[964]: 2026/09/22 16:59:32 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4627lb # [ 68.146995] niks3-server[964]: 2026/09/22 16:59:32 INFO Completed upload id=184628lb # [ 68.151616] niks3-server[964]: 2026/09/22 16:59:32 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4629worker1 # [ 68.156138] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=4244630worker1 # [ 68.172677] nix-grpc-daemon[767]: ts=2026-09-22T16:59:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184631worker1 # [ 68.180397] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4632worker1 # [ 68.185922] nix-daemon[831]: reaped child process 1047, status = succeeded4633worker1 # [ 68.197274] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4634lb # [ 69.939747] envoy[919]: conn peer=[2001:db8:1::1]:56802 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1579 tx=1598 ms=36524635lb # [ 69.945181] envoy[919]: conn peer=[2001:db8:1::1]:42492 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3027 tx=978 ms=63234636worker1 # [ 74.413866] nix-grpc-daemon[767]: ts=2026-09-22T16:59:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04637worker1 # [ 74.431256] nix-daemon[831]: reaped child process 1054, status = succeeded4638worker1 # [ 74.432563] nix-daemon[831]: reaped child process 1056, status = succeeded4639(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.57 seconds)4640subtest: the first client leaving does not fail the second4641client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4642client # [ 74.733483] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4643client # Running as unit: share1.service; invocation ID: d954108ada2d4cbf9a1690ddcdb4ee124644client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4645client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4646worker1 # [ 74.877073] nix-grpc-daemon[767]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:532184647client # [ 74.927411] nix[1083]: don't know how to build these paths:4648client # [ 74.928501] nix[1083]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4649worker2 # [ 74.870178] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04650worker2 # [ 74.879140] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14651worker2 # [ 74.887651] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404652worker2 # [ 74.895379] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404653client # [ 74.976251] nix[1083]: copying 1 paths to 'lb:50051'...4654client # [ 74.979185] nix[1083]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4655worker2 # [ 74.907682] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442404656worker2 # [ 74.922959] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4657worker2 # [ 75.074929] nix-daemon[859]: accepted connection from pid 1079, user nix-grpc-daemon (trusted)4658worker2 # [ 75.096344] nix-daemon[859]: reaped child process 1082, status = succeeded4659lb # [ 75.128692] niks3-server[964]: 2026/09/22 16:59:39 INFO Received uploads request method=POST path=/api/pending_closures4660worker2 # [ 75.110350] nix-grpc-daemon[912]: time=2026-09-22T16:59:39.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4661worker2 # [ 75.111879] nix-grpc-daemon[912]: time=2026-09-22T16:59:39.733Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4662lb # [ 75.162326] niks3-server[964]: 2026/09/22 16:59:39 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4663worker2 # [ 75.146203] nix-grpc-daemon[912]: time=2026-09-22T16:59:39.769Z level=INFO msg="Uploading 1 narinfos"4664lb # [ 75.176398] niks3-server[964]: 2026/09/22 16:59:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4665lb # [ 75.186731] niks3-server[964]: 2026/09/22 16:59:39 INFO Signed narinfos id=19 count=14666lb # [ 75.192891] niks3-server[964]: 2026/09/22 16:59:39 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4667lb # [ 75.203827] niks3-server[964]: 2026/09/22 16:59:39 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4668worker2 # [ 75.181804] nix-grpc-daemon[912]: time=2026-09-22T16:59:39.805Z level=INFO msg="Upload complete. (242ms)"4669lb # [ 75.210976] niks3-server[964]: 2026/09/22 16:59:39 INFO Completed upload id=194670lb # [ 75.215803] niks3-server[964]: 2026/09/22 16:59:39 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4671worker2 # [ 75.192378] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=4324672worker2 # [ 75.211942] nix-grpc-daemon[774]: ts=2026-09-22T16:59:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404673worker2 # [ 75.219983] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4674worker2 # [ 75.226742] nix-daemon[859]: reaped child process 1077, status = succeeded4675client # [ 75.312555] nix[1083]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4676worker2 # [ 75.241365] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4677client # [ 75.829260] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4678client # Running as unit: share2.service; invocation ID: 9c8bd7543c074f289c01a62c4f9cfe054679client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4680client: must succeed: sleep 2; systemctl kill -s INT share14681worker2 # [ 75.968284] nix-grpc-daemon[774]: ts=2026-09-22T16:59:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:442404682client # [ 76.050615] nix[1099]: this derivation will be built:4683worker1 # [ 76.013836] nix-grpc-daemon[767]: ts=2026-09-22T16:59:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532184684worker1 # [ 76.025769] nix-grpc-daemon[767]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' from 'http://lb:5751'...4685client # [ 76.086933] nix[1099]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4686worker2 # [ 76.022444] nix-grpc-daemon[774]: ts=2026-09-22T16:59:40Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04687worker2 # [ 76.025344] nix-grpc-daemon[774]: ts=2026-09-22T16:59:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404688client # [ 76.102101] nix[1099]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4689client # [ 78.052674] systemd[1]: share1.service: Sent signal SIGINT to main process 1083 (nix) on client request.4690client # [ 78.062793] nix[1083]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4691client # [ 78.076428] nix[1083]: error: interrupted by the user4692client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4693client: waiting for success: ! systemctl is-active share14694client # [ 78.132827] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4695client # [ 78.136591] systemd[1]: share1.service: Failed with result 'exit-code'.4696client # [ 78.139171] systemd[1]: share1.service: Consumed 162ms CPU time over 3.404s wall clock time, 41.4M memory peak, 6.1K incoming IP traffic, 8K outgoing IP traffic.4697client: (finished: waiting for success: ! systemctl is-active share1, in 1.11 seconds)4698client: waiting for success: ! systemctl is-active share24699worker2 # [ 79.492577] nix-daemon[859]: reaped child process 1086, status = succeeded4700worker2 # [ 79.608814] nix-daemon[859]: accepted connection from pid 1118, user nix-grpc-daemon (trusted)4701worker2 # [ 79.626459] nix-daemon[859]: reaped child process 1121, status = succeeded4702lb # [ 79.661622] niks3-server[964]: 2026/09/22 16:59:44 INFO Received uploads request method=POST path=/api/pending_closures4703worker2 # [ 79.644179] nix-grpc-daemon[912]: time=2026-09-22T16:59:44.265Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4704worker2 # [ 79.645640] nix-grpc-daemon[912]: time=2026-09-22T16:59:44.265Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4705lb # [ 79.696515] niks3-server[964]: 2026/09/22 16:59:44 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4706lb # [ 79.712438] niks3-server[964]: 2026/09/22 16:59:44 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4707lb # [ 79.721453] niks3-server[964]: 2026/09/22 16:59:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4708worker2 # [ 79.699245] nix-grpc-daemon[912]: time=2026-09-22T16:59:44.322Z level=INFO msg="Uploading 1 narinfos"4709lb # [ 79.732337] niks3-server[964]: 2026/09/22 16:59:44 INFO Signed narinfos id=20 count=14710lb # [ 79.740121] niks3-server[964]: 2026/09/22 16:59:44 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4711worker2 # [ 79.724225] nix-grpc-daemon[912]: time=2026-09-22T16:59:44.348Z level=INFO msg="Upload complete. (261ms)"4712lb # [ 79.752548] niks3-server[964]: 2026/09/22 16:59:44 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4713lb # [ 79.758837] niks3-server[964]: 2026/09/22 16:59:44 INFO Completed upload id=204714lb # [ 79.764378] niks3-server[964]: 2026/09/22 16:59:44 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4715worker2 # [ 79.736957] nix-grpc-daemon[774]: ts=2026-09-22T16:59:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14716worker2 # [ 79.754951] nix-grpc-daemon[774]: ts=2026-09-22T16:59:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=3 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14717worker2 # [ 79.761245] nix-daemon[859]: reaped child process 1084, status = succeeded4718client # [ 79.883807] systemd[1]: share2.service: Deactivated successfully.4719client # [ 79.888553] systemd[1]: share2.service: Consumed 184ms CPU time over 4.060s wall clock time, 33.3M memory peak, 6.6K incoming IP traffic, 5.9K outgoing IP traffic.4720lb # [ 79.941182] envoy[919]: conn peer=[2001:db8:1::1]:42496 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3002 tx=952 ms=66664721lb # [ 79.946119] envoy[919]: conn peer=[2001:db8:1::1]:57688 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2947 tx=772 ms=31644722lb # [ 79.950927] envoy[919]: conn peer=[2001:db8:1::1]:57694 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1605 tx=1598 ms=38204723client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4724client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4725client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds)4726(finished: subtest: the first client leaving does not fail the second, in 5.96 seconds)4727subtest: build hook: nix-daemon with builders = grpc://lb4728client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4729client # [ 80.764092] systemd[1]: Started Nix Daemon.4730client # [ 80.877715] nix-daemon[1151]: accepted connection from pid 1148, user root (trusted)4731worker2 # [ 80.953284] nix-grpc-daemon[774]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:442404732worker2 # [ 80.959611] nix-grpc-daemon[774]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404733worker2 # [ 80.978718] nix-grpc-daemon[774]: ts=2026-09-22T16:59:45Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04734worker2 # [ 80.984829] nix-grpc-daemon[774]: ts=2026-09-22T16:59:45Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14735worker1 # [ 81.014895] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184736worker1 # [ 81.030799] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184737worker1 # [ 81.045112] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532184738worker1 # [ 81.060160] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4739worker1 # [ 81.224347] nix-daemon[831]: accepted connection from pid 1092, user nix-grpc-daemon (trusted)4740worker1 # [ 81.246252] nix-daemon[831]: reaped child process 1095, status = succeeded4741lb # [ 81.244739] niks3-server[964]: 2026/09/22 16:59:45 INFO Received uploads request method=POST path=/api/pending_closures4742worker1 # [ 81.258896] nix-grpc-daemon[844]: time=2026-09-22T16:59:45.874Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4743worker1 # [ 81.260572] nix-grpc-daemon[844]: time=2026-09-22T16:59:45.875Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4744worker1 # [ 81.285987] nix-grpc-daemon[844]: time=2026-09-22T16:59:45.902Z level=INFO msg="Uploading 1 narinfos"4745lb # [ 81.279435] niks3-server[964]: 2026/09/22 16:59:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4746lb # [ 81.288989] niks3-server[964]: 2026/09/22 16:59:45 INFO Signed narinfos id=21 count=14747lb # [ 81.290969] niks3-server[964]: 2026/09/22 16:59:45 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4748lb # [ 81.299792] niks3-server[964]: 2026/09/22 16:59:45 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4749lb # [ 81.309929] niks3-server[964]: 2026/09/22 16:59:45 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4750lb # [ 81.316205] niks3-server[964]: 2026/09/22 16:59:45 INFO Completed upload id=214751worker1 # [ 81.315727] nix-grpc-daemon[844]: time=2026-09-22T16:59:45.932Z level=INFO msg="Upload complete. (238ms)"4752lb # [ 81.319477] niks3-server[964]: 2026/09/22 16:59:45 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4753worker1 # [ 81.324317] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=5124754worker1 # [ 81.342142] nix-grpc-daemon[767]: ts=2026-09-22T16:59:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184755worker1 # [ 81.349088] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4756worker1 # [ 81.354355] nix-daemon[831]: reaped child process 1090, status = succeeded4757worker1 # [ 81.365299] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4758worker1 # [ 82.225937] nix-daemon[831]: reaped child process 1099, status = succeeded4759worker1 # [ 82.356491] nix-daemon[831]: accepted connection from pid 1115, user nix-grpc-daemon (trusted)4760worker1 # [ 82.375173] nix-daemon[831]: reaped child process 1118, status = succeeded4761lb # [ 82.381879] niks3-server[964]: 2026/09/22 16:59:46 INFO Received uploads request method=POST path=/api/pending_closures4762worker1 # [ 82.392448] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.007Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4763worker1 # [ 82.395439] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.007Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4764lb # [ 82.425879] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4765worker1 # [ 82.438086] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.054Z level=INFO msg="Uploading 1 narinfos"4766lb # [ 82.441561] niks3-server[964]: 2026/09/22 16:59:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4767lb # [ 82.450126] niks3-server[964]: 2026/09/22 16:59:47 INFO Signed narinfos id=22 count=14768lb # [ 82.458108] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4769worker1 # [ 82.465006] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.081Z level=INFO msg="Upload complete. (286ms)"4770lb # [ 82.464376] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4771lb # [ 82.468567] niks3-server[964]: 2026/09/22 16:59:47 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4772lb # [ 82.474907] niks3-server[964]: 2026/09/22 16:59:47 INFO Completed upload id=224773worker1 # [ 82.475715] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14774lb # [ 82.481322] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4775worker2 # [ 82.453150] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:442404776worker1 # [ 82.494787] nix-daemon[831]: reaped child process 1097, status = succeeded4777worker2 # [ 82.466075] nix-grpc-daemon[774]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4778worker2 # [ 82.502836] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442404779worker2 # [ 82.509443] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=1284780worker2 # [ 82.787323] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:442404781worker1 # [ 82.821161] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184782worker1 # [ 82.831461] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.448Z level=INFO msg="All 1 paths already cached"4783worker2 # [ 82.820131] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04784worker2 # [ 82.826439] nix-grpc-daemon[774]: ts=2026-09-22T16:59:47Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14785worker1 # [ 82.859507] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184786worker1 # [ 82.874932] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184787worker1 # [ 82.889235] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.505Z level=INFO msg="All 1 paths already cached"4788worker1 # [ 82.898958] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532184789worker1 # [ 82.913036] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4790worker1 # [ 83.071098] nix-daemon[831]: accepted connection from pid 1122, user nix-grpc-daemon (trusted)4791worker1 # [ 83.092648] nix-daemon[831]: reaped child process 1125, status = succeeded4792lb # [ 83.089348] niks3-server[964]: 2026/09/22 16:59:47 INFO Received uploads request method=POST path=/api/pending_closures4793worker1 # [ 83.099435] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.715Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4794worker1 # [ 83.101117] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.715Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4795lb # [ 83.119845] niks3-server[964]: 2026/09/22 16:59:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4796worker1 # [ 83.127361] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.743Z level=INFO msg="Uploading 1 narinfos"4797lb # [ 83.134108] niks3-server[964]: 2026/09/22 16:59:47 INFO Signed narinfos id=23 count=14798lb # [ 83.143113] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4799lb # [ 83.153381] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4800lb # [ 83.165775] niks3-server[964]: 2026/09/22 16:59:47 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4801worker1 # [ 83.172405] nix-grpc-daemon[844]: time=2026-09-22T16:59:47.789Z level=INFO msg="Upload complete. (242ms)"4802lb # [ 83.171035] niks3-server[964]: 2026/09/22 16:59:47 INFO Completed upload id=234803lb # [ 83.175622] niks3-server[964]: 2026/09/22 16:59:47 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4804worker1 # [ 83.184349] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=6244805worker1 # [ 83.200377] nix-grpc-daemon[767]: ts=2026-09-22T16:59:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184806worker1 # [ 83.210756] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4807worker1 # [ 83.218164] nix-daemon[831]: reaped child process 1120, status = succeeded4808worker1 # [ 83.231319] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4809worker1 # [ 84.274329] nix-daemon[831]: reaped child process 1129, status = succeeded4810worker1 # [ 84.395863] nix-daemon[831]: accepted connection from pid 1145, user nix-grpc-daemon (trusted)4811worker1 # [ 84.411236] nix-daemon[831]: reaped child process 1148, status = succeeded4812lb # [ 84.418795] niks3-server[964]: 2026/09/22 16:59:49 INFO Received uploads request method=POST path=/api/pending_closures4813worker1 # [ 84.432456] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.045Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4814worker1 # [ 84.434087] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.045Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4815lb # [ 84.467134] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4816worker1 # [ 84.479266] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.095Z level=INFO msg="Uploading 1 narinfos"4817lb # [ 84.480932] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4818lb # [ 84.490800] niks3-server[964]: 2026/09/22 16:59:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4819lb # [ 84.500529] niks3-server[964]: 2026/09/22 16:59:49 INFO Signed narinfos id=24 count=14820lb # [ 84.503121] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4821worker1 # [ 84.512845] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.129Z level=INFO msg="Upload complete. (267ms)"4822lb # [ 84.514112] niks3-server[964]: 2026/09/22 16:59:49 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4823lb # [ 84.517869] niks3-server[964]: 2026/09/22 16:59:49 INFO Completed upload id=244824lb # [ 84.521424] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4825worker1 # [ 84.525278] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14826worker1 # [ 84.544380] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532184827worker1 # [ 84.551909] nix-daemon[831]: reaped child process 1127, status = succeeded4828worker1 # [ 84.573321] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:532184829worker1 # [ 84.580137] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_out=1764830worker2 # [ 84.839830] nix-grpc-daemon[774]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:442404831worker2 # [ 84.850384] nix-grpc-daemon[774]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404832worker2 # [ 84.871032] nix-grpc-daemon[912]: time=2026-09-22T16:59:49.493Z level=INFO msg="All 1 paths already cached"4833worker2 # [ 84.891417] nix-grpc-daemon[774]: ts=2026-09-22T16:59:49Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04834worker1 # [ 84.924738] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184835worker2 # [ 84.898108] nix-grpc-daemon[774]: ts=2026-09-22T16:59:49Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14836worker1 # [ 84.940834] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532184837worker1 # [ 84.956166] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.572Z level=INFO msg="All 2 paths already cached"4838worker1 # [ 84.966940] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532184839worker1 # [ 84.982562] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4840worker1 # [ 85.149333] nix-daemon[831]: accepted connection from pid 1152, user nix-grpc-daemon (trusted)4841worker1 # [ 85.173891] nix-daemon[831]: reaped child process 1155, status = succeeded4842lb # [ 85.171012] niks3-server[964]: 2026/09/22 16:59:49 INFO Received uploads request method=POST path=/api/pending_closures4843worker1 # [ 85.181912] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.798Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4844worker1 # [ 85.183795] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.798Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4845lb # [ 85.206103] niks3-server[964]: 2026/09/22 16:59:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4846worker1 # [ 85.214334] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.830Z level=INFO msg="Uploading 1 narinfos"4847lb # [ 85.221477] niks3-server[964]: 2026/09/22 16:59:49 INFO Signed narinfos id=25 count=14848lb # [ 85.228896] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4849lb # [ 85.239786] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4850worker1 # [ 85.246374] nix-grpc-daemon[844]: time=2026-09-22T16:59:49.863Z level=INFO msg="Upload complete. (247ms)"4851lb # [ 85.246908] niks3-server[964]: 2026/09/22 16:59:49 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4852lb # [ 85.252047] niks3-server[964]: 2026/09/22 16:59:49 INFO Completed upload id=254853lb # [ 85.255526] niks3-server[964]: 2026/09/22 16:59:49 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4854worker1 # [ 85.257310] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=0 paths=1 nar_bytes_in=7524855worker1 # [ 85.272229] nix-grpc-daemon[767]: ts=2026-09-22T16:59:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532184856worker1 # [ 85.280720] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4857worker1 # [ 85.283166] nix-daemon[831]: reaped child process 1150, status = succeeded4858worker1 # [ 85.296462] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4859worker1 # [ 86.568991] nix-daemon[831]: reaped child process 1159, status = succeeded4860worker1 # [ 86.695770] nix-daemon[831]: accepted connection from pid 1175, user nix-grpc-daemon (trusted)4861worker1 # [ 86.720266] nix-daemon[831]: reaped child process 1178, status = succeeded4862lb # [ 86.740525] niks3-server[964]: 2026/09/22 16:59:51 INFO Received uploads request method=POST path=/api/pending_closures4863worker1 # [ 86.753015] nix-grpc-daemon[844]: time=2026-09-22T16:59:51.369Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4864worker1 # [ 86.760992] nix-grpc-daemon[844]: time=2026-09-22T16:59:51.372Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4865lb # [ 86.810628] niks3-server[964]: 2026/09/22 16:59:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4866lb # [ 86.818042] niks3-server[964]: 2026/09/22 16:59:51 INFO Signed narinfos id=26 count=14867lb # [ 86.821144] niks3-server[964]: 2026/09/22 16:59:51 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4868worker1 # [ 86.820697] nix-grpc-daemon[844]: time=2026-09-22T16:59:51.436Z level=INFO msg="Uploading 1 narinfos"4869lb # [ 86.828851] niks3-server[964]: 2026/09/22 16:59:51 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4870lb # [ 86.836797] niks3-server[964]: 2026/09/22 16:59:51 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4871lb # [ 86.851416] niks3-server[964]: 2026/09/22 16:59:51 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4872lb # [ 86.857722] niks3-server[964]: 2026/09/22 16:59:51 INFO Completed upload id=264873worker1 # [ 86.857756] nix-grpc-daemon[844]: time=2026-09-22T16:59:51.474Z level=INFO msg="Upload complete. (350ms)"4874lb # [ 86.860158] niks3-server[964]: 2026/09/22 16:59:51 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4875worker1 # [ 86.877990] nix-grpc-daemon[767]: ts=2026-09-22T16:59:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14876worker1 # [ 86.910748] nix-daemon[831]: reaped child process 1157, status = succeeded4877worker1 # [ 86.929767] nix-grpc-daemon[767]: ts=2026-09-22T16:59:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532184878worker2 # [ 86.954619] nix-grpc-daemon[774]: ts=2026-09-22T16:59:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442404879worker2 # [ 86.980994] nix-grpc-daemon[774]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4880worker2 # [ 87.008865] nix-grpc-daemon[774]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4881worker2 # [ 87.033806] nix-grpc-daemon[774]: ts=2026-09-22T16:59:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=2324882client # [ 87.282617] nix-daemon[1151]: reaped child process 1153, status = succeeded4883client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.68 seconds)4884client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4885client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.05 seconds)4886client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4887client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4888client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4889client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4890(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.81 seconds)4891subtest: an input only one worker has still reaches the builder4892client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4893client # this derivation will be built:4894client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4895client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4896client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)4897client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4898client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4899worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4900worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.18 seconds)4901worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4902worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4903client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24904client # [ 88.264280] nix-daemon[1151]: accepted connection from pid 1234, user root (trusted)4905client # this derivation will be built:4906client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4907worker2 # [ 88.339746] nix-grpc-daemon[774]: ts=2026-09-22T16:59:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:442404908client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4909worker2 # [ 88.348574] nix-grpc-daemon[774]: ts=2026-09-22T16:59:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404910client # copying 1 paths...4911client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4912worker2 # [ 88.359477] nix-grpc-daemon[774]: ts=2026-09-22T16:59:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442404913worker2 # [ 88.376933] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4914worker2 # [ 88.552167] nix-daemon[859]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)4915worker2 # [ 88.574985] nix-daemon[859]: reaped child process 1140, status = succeeded4916lb # [ 88.603715] niks3-server[964]: 2026/09/22 16:59:53 INFO Received uploads request method=POST path=/api/pending_closures4917worker2 # [ 88.585803] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.208Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4918worker2 # [ 88.587399] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.208Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4919lb # [ 88.634829] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4920worker2 # [ 88.618036] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.241Z level=INFO msg="Uploading 1 narinfos"4921lb # [ 88.651624] niks3-server[964]: 2026/09/22 16:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4922lb # [ 88.660783] niks3-server[964]: 2026/09/22 16:59:53 INFO Signed narinfos id=27 count=14923lb # [ 88.669080] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4924worker2 # [ 88.645679] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.269Z level=INFO msg="Upload complete. (248ms)"4925lb # [ 88.678573] niks3-server[964]: 2026/09/22 16:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4926lb # [ 88.684265] niks3-server[964]: 2026/09/22 16:59:53 INFO Completed upload id=274927worker2 # [ 88.656313] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=1204928lb # [ 88.688067] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4929worker2 # [ 88.671932] nix-daemon[859]: reaped child process 1135, status = succeeded4930worker2 # [ 88.676457] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04931worker2 # [ 88.682115] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14932worker2 # [ 88.723711] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404933worker2 # [ 88.739034] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442404934client # copying 2 paths to 'lb:50051'...4935worker2 # [ 88.756738] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442404936client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4937client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4938worker2 # [ 88.774025] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4939worker2 # [ 88.932114] nix-daemon[859]: accepted connection from pid 1146, user nix-grpc-daemon (trusted)4940worker2 # [ 88.953654] nix-daemon[859]: reaped child process 1149, status = succeeded4941lb # [ 88.984871] niks3-server[964]: 2026/09/22 16:59:53 INFO Received uploads request method=POST path=/api/pending_closures4942lb # [ 88.994359] niks3-server[964]: 2026/09/22 16:59:53 INFO Received uploads request method=POST path=/api/pending_closures4943worker2 # [ 88.978381] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.600Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4944worker2 # [ 88.981477] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.600Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4945worker2 # [ 88.984162] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.600Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4946lb # [ 89.033669] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4947lb # [ 89.051516] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4948lb # [ 89.065726] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4949lb # [ 89.075598] niks3-server[964]: 2026/09/22 16:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4950worker2 # [ 89.048115] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.671Z level=INFO msg="Uploading 3 narinfos"4951lb # [ 89.085727] niks3-server[964]: 2026/09/22 16:59:53 INFO Signed narinfos id=28 count=14952lb # [ 89.091120] niks3-server[964]: 2026/09/22 16:59:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4953lb # [ 89.098873] niks3-server[964]: 2026/09/22 16:59:53 INFO Signed narinfos id=29 count=24954lb # [ 89.103655] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4955lb # [ 89.115442] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4956lb # [ 89.124617] niks3-server[964]: 2026/09/22 16:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4957lb # [ 89.133411] niks3-server[964]: 2026/09/22 16:59:53 INFO Completed upload id=284958lb # [ 89.136671] niks3-server[964]: 2026/09/22 16:59:53 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4959worker2 # [ 89.107317] nix-grpc-daemon[912]: time=2026-09-22T16:59:53.731Z level=INFO msg="Upload complete. (314ms)"4960lb # [ 89.139193] niks3-server[964]: 2026/09/22 16:59:53 INFO Completed upload id=294961lb # [ 89.141719] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4962lb # [ 89.146013] niks3-server[964]: 2026/09/22 16:59:53 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4963worker2 # [ 89.120335] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=2 nar_bytes_in=9844964worker2 # [ 89.132379] nix-grpc-daemon[774]: ts=2026-09-22T16:59:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442404965worker2 # [ 89.147751] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4966worker2 # [ 89.155168] nix-daemon[859]: reaped child process 1144, status = succeeded4967client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4968worker2 # [ 89.170665] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4969client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4970client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4971worker2 # [ 89.807304] nix-daemon[859]: reaped child process 1153, status = succeeded4972worker2 # [ 89.905714] nix-daemon[859]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)4973lb # [ 89.942957] envoy[919]: conn peer=[2001:db8:1::1]:57806 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3397 tx=1926 ms=16174974lb # [ 89.948091] envoy[919]: conn peer=[2001:db8:1::1]:57822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=684975worker2 # [ 89.923655] nix-daemon[859]: reaped child process 1172, status = succeeded4976lb # [ 89.952716] envoy[919]: conn peer=[2001:db8:1::1]:57832 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3900 tx=2155 ms=18214977lb # [ 89.961624] envoy[919]: conn peer=[2001:db8:1::1]:57838 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=694978lb # [ 89.966968] envoy[919]: conn peer=[2001:db8:1::1]:57840 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4369 tx=2407 ms=22624979worker2 # [ 89.946447] nix-grpc-daemon[912]: time=2026-09-22T16:59:54.568Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4980lb # [ 89.973301] envoy[919]: conn peer=[2001:db8:1::1]:57844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=407 ms=1684981lb # [ 89.984987] niks3-server[964]: 2026/09/22 16:59:54 INFO Received uploads request method=POST path=/api/pending_closures4982lb # [ 90.006650] niks3-server[964]: 2026/09/22 16:59:54 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4983lb # [ 90.014982] niks3-server[964]: 2026/09/22 16:59:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign4984worker2 # [ 89.990904] nix-grpc-daemon[912]: time=2026-09-22T16:59:54.614Z level=INFO msg="Uploading 1 narinfos"4985lb # [ 90.022121] niks3-server[964]: 2026/09/22 16:59:54 INFO Signed narinfos id=30 count=14986lb # [ 90.027631] niks3-server[964]: 2026/09/22 16:59:54 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4987lb # [ 90.040048] niks3-server[964]: 2026/09/22 16:59:54 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete4988lb # [ 90.045793] niks3-server[964]: 2026/09/22 16:59:54 INFO Completed upload id=304989worker2 # [ 90.015809] nix-grpc-daemon[912]: time=2026-09-22T16:59:54.639Z level=INFO msg="Upload complete. (230ms)"4990lb # [ 90.048890] niks3-server[964]: 2026/09/22 16:59:54 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4991worker2 # [ 90.029143] nix-grpc-daemon[774]: ts=2026-09-22T16:59:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=14992worker2 # [ 90.043694] nix-daemon[859]: reaped child process 1151, status = succeeded4993client # copying 1 paths...4994worker1 # [ 90.110051] nix-grpc-daemon[767]: ts=2026-09-22T16:59:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532184995worker1 # [ 90.118230] nix-grpc-daemon[767]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4996client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4997worker2 # [ 90.133323] nix-grpc-daemon[774]: ts=2026-09-22T16:59:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442404998worker2 # [ 90.140572] nix-grpc-daemon[774]: ts=2026-09-22T16:59:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=1204999client # [ 90.300915] nix-daemon[1151]: reaped child process 1237, status = succeeded5000client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.23 seconds)5001client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25002client # [ 90.472930] nix-daemon[1151]: accepted connection from pid 1256, user root (trusted)5003client # this derivation will be built:5004client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5005worker1 # [ 90.577174] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:532185006client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5007worker1 # [ 90.585446] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532185008worker1 # [ 90.593021] nix-grpc-daemon[844]: time=2026-09-22T16:59:55.207Z level=INFO msg="All 1 paths already cached"5009worker2 # [ 90.574916] nix-grpc-daemon[774]: ts=2026-09-22T16:59:55Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05010worker1 # [ 90.608155] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532185011worker2 # [ 90.581819] nix-grpc-daemon[774]: ts=2026-09-22T16:59:55Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15012worker1 # [ 90.623069] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:532185013client # copying 1 paths to 'lb:50051'...5014client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5015worker1 # [ 90.641867] nix-grpc-daemon[767]: ts=2026-09-22T16:59:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:532185016worker1 # [ 90.657381] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5017worker1 # [ 90.679080] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)5018worker1 # [ 90.720793] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 146 ms (attempt 2/5)5019worker1 # [ 90.868891] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 292 ms (attempt 3/5)5020worker1 # [ 91.164632] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 523 ms (attempt 4/5)5021worker1 # [ 91.688856] nix-grpc-daemon[767]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5022worker1 # [ 91.698518] nix-grpc-daemon[767]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5023worker1 # [ 91.705679] nix-grpc-daemon[767]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5024worker1 # [ 91.878050] nix-daemon[831]: accepted connection from pid 1193, user nix-grpc-daemon (trusted)5025worker1 # [ 91.899004] nix-daemon[831]: reaped child process 1196, status = succeeded5026lb # [ 91.903166] niks3-server[964]: 2026/09/22 16:59:56 INFO Received uploads request method=POST path=/api/pending_closures5027worker1 # [ 91.914075] nix-grpc-daemon[844]: time=2026-09-22T16:59:56.529Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5028worker1 # [ 91.915836] nix-grpc-daemon[844]: time=2026-09-22T16:59:56.530Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5029lb # [ 91.931057] niks3-server[964]: 2026/09/22 16:59:56 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5030lb # [ 91.944081] niks3-server[964]: 2026/09/22 16:59:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5031worker1 # [ 91.952647] nix-grpc-daemon[844]: time=2026-09-22T16:59:56.569Z level=INFO msg="Uploading 1 narinfos"5032lb # [ 91.957505] niks3-server[964]: 2026/09/22 16:59:56 INFO Signed narinfos id=31 count=15033lb # [ 91.965167] niks3-server[964]: 2026/09/22 16:59:56 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5034lb # [ 91.979801] niks3-server[964]: 2026/09/22 16:59:56 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5035worker1 # [ 91.986591] nix-grpc-daemon[844]: time=2026-09-22T16:59:56.603Z level=INFO msg="Upload complete. (267ms)"5036lb # [ 91.988241] niks3-server[964]: 2026/09/22 16:59:56 INFO Completed upload id=315037lb # [ 91.993041] niks3-server[964]: 2026/09/22 16:59:56 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5038worker1 # [ 91.996369] nix-grpc-daemon[767]: ts=2026-09-22T16:59:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 paths=1 nar_bytes_in=5685039worker1 # [ 92.011163] nix-grpc-daemon[767]: ts=2026-09-22T16:59:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:532185040worker1 # [ 92.025872] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5041worker1 # [ 92.031519] nix-daemon[831]: reaped child process 1186, status = succeeded5042client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5043worker1 # [ 92.048192] nix-daemon[831]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5044client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5045client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5046worker1 # [ 92.914770] nix-daemon[831]: reaped child process 1200, status = succeeded5047worker1 # [ 93.051863] nix-daemon[831]: accepted connection from pid 1216, user nix-grpc-daemon (trusted)5048worker1 # [ 93.069845] nix-daemon[831]: reaped child process 1219, status = succeeded5049lb # [ 93.073768] niks3-server[964]: 2026/09/22 16:59:57 INFO Received uploads request method=POST path=/api/pending_closures5050worker1 # [ 93.084443] nix-grpc-daemon[844]: time=2026-09-22T16:59:57.700Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5051lb # [ 93.099558] niks3-server[964]: 2026/09/22 16:59:57 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5052lb # [ 93.115529] niks3-server[964]: 2026/09/22 16:59:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5053worker1 # [ 93.125172] nix-grpc-daemon[844]: time=2026-09-22T16:59:57.741Z level=INFO msg="Uploading 1 narinfos"5054lb # [ 93.127649] niks3-server[964]: 2026/09/22 16:59:57 INFO Signed narinfos id=32 count=15055lb # [ 93.133514] niks3-server[964]: 2026/09/22 16:59:57 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5056worker1 # [ 93.144911] nix-grpc-daemon[844]: time=2026-09-22T16:59:57.761Z level=INFO msg="Upload complete. (284ms)"5057lb # [ 93.144644] niks3-server[964]: 2026/09/22 16:59:57 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5058lb # [ 93.147800] niks3-server[964]: 2026/09/22 16:59:57 INFO Completed upload id=325059lb # [ 93.153702] niks3-server[964]: 2026/09/22 16:59:57 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5060worker1 # [ 93.157052] nix-grpc-daemon[767]: ts=2026-09-22T16:59:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53218 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15061worker1 # [ 93.173871] nix-daemon[831]: reaped child process 1198, status = succeeded5062client # copying 1 paths...5063worker1 # [ 93.202419] nix-grpc-daemon[767]: ts=2026-09-22T16:59:57Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532185064client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5065worker2 # [ 93.206751] nix-grpc-daemon[774]: ts=2026-09-22T16:59:57Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442405066worker2 # [ 93.220085] nix-grpc-daemon[774]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5067worker2 # [ 93.237821] nix-grpc-daemon[774]: ts=2026-09-22T16:59:57Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=1205068client # [ 93.439455] nix-daemon[1151]: reaped child process 1259, status = succeeded5069client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.14 seconds)5070client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25071client # [ 93.609907] nix-daemon[1151]: accepted connection from pid 1277, user root (trusted)5072client # this derivation will be built:5073client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5074worker1 # [ 93.715602] nix-grpc-daemon[767]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:532185075client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5076worker2 # [ 93.690169] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442405077worker2 # [ 93.701017] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.324Z level=INFO msg="All 1 paths already cached"5078worker2 # [ 93.718045] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05079worker2 # [ 93.727138] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15080worker2 # [ 93.735776] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442405081worker2 # [ 93.743283] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:442405082worker2 # [ 93.753490] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.377Z level=INFO msg="All 1 paths already cached"5083client # copying 1 paths to 'lb:50051'...5084client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5085worker2 # [ 93.763750] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:442405086worker2 # [ 93.779265] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5087worker2 # [ 93.926459] nix-daemon[859]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)5088worker2 # [ 93.946721] nix-daemon[859]: reaped child process 1184, status = succeeded5089lb # [ 93.978916] niks3-server[964]: 2026/09/22 16:59:58 INFO Received uploads request method=POST path=/api/pending_closures5090worker2 # [ 93.959671] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.583Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5091worker2 # [ 93.961846] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.583Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5092lb # [ 94.011291] niks3-server[964]: 2026/09/22 16:59:58 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5093worker2 # [ 93.993977] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.617Z level=INFO msg="Uploading 1 narinfos"5094lb # [ 94.024439] niks3-server[964]: 2026/09/22 16:59:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5095lb # [ 94.035047] niks3-server[964]: 2026/09/22 16:59:58 INFO Signed narinfos id=33 count=15096lb # [ 94.042778] niks3-server[964]: 2026/09/22 16:59:58 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5097worker2 # [ 94.021239] nix-grpc-daemon[912]: time=2026-09-22T16:59:58.645Z level=INFO msg="Upload complete. (226ms)"5098lb # [ 94.052957] niks3-server[964]: 2026/09/22 16:59:58 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5099lb # [ 94.058029] niks3-server[964]: 2026/09/22 16:59:58 INFO Completed upload id=335100lb # [ 94.061760] niks3-server[964]: 2026/09/22 16:59:58 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5101worker2 # [ 94.032267] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_in=5685102worker2 # [ 94.044543] nix-grpc-daemon[774]: ts=2026-09-22T16:59:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:442405103worker2 # [ 94.052894] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5104worker2 # [ 94.059145] nix-daemon[859]: reaped child process 1179, status = succeeded5105worker2 # [ 94.071921] nix-daemon[859]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5106client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5107client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5108client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5109worker2 # [ 95.183906] nix-daemon[859]: reaped child process 1188, status = succeeded5110worker2 # [ 95.256125] nix-daemon[859]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5111worker2 # [ 95.279327] nix-daemon[859]: reaped child process 1207, status = succeeded5112lb # [ 95.309993] niks3-server[964]: 2026/09/22 16:59:59 INFO Received uploads request method=POST path=/api/pending_closures5113worker2 # [ 95.292510] nix-grpc-daemon[912]: time=2026-09-22T16:59:59.914Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5114lb # [ 95.342375] niks3-server[964]: 2026/09/22 16:59:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5115worker2 # [ 95.322327] nix-grpc-daemon[912]: time=2026-09-22T16:59:59.946Z level=INFO msg="Uploading 1 narinfos"5116lb # [ 95.356659] niks3-server[964]: 2026/09/22 16:59:59 INFO Signed narinfos id=34 count=15117lb # [ 95.364729] niks3-server[964]: 2026/09/22 16:59:59 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5118lb # [ 95.373312] niks3-server[964]: 2026/09/22 16:59:59 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5119worker2 # [ 95.354682] nix-grpc-daemon[912]: time=2026-09-22T16:59:59.978Z level=INFO msg="Upload complete. (269ms)"5120lb # [ 95.383751] niks3-server[964]: 2026/09/22 16:59:59 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5121lb # [ 95.390682] niks3-server[964]: 2026/09/22 16:59:59 INFO Completed upload id=345122lb # [ 95.394432] niks3-server[964]: 2026/09/22 17:00:00 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5123worker2 # [ 95.369054] nix-grpc-daemon[774]: ts=2026-09-22T16:59:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15124client # copying 1 paths...5125worker2 # [ 95.386400] nix-daemon[859]: reaped child process 1186, status = succeeded5126worker1 # [ 95.415483] nix-grpc-daemon[767]: ts=2026-09-22T17:00:00Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532185127worker1 # [ 95.428756] nix-grpc-daemon[767]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5128client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5129worker2 # [ 95.442492] nix-grpc-daemon[774]: ts=2026-09-22T17:00:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:442405130worker2 # [ 95.449735] nix-grpc-daemon[774]: ts=2026-09-22T17:00:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:44240 duration_s=0 paths=1 nar_bytes_out=1205131client # [ 95.605432] nix-daemon[1151]: reaped child process 1280, status = succeeded5132client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.16 seconds)5133(finished: subtest: an input only one worker has still reaches the builder, in 8.18 seconds)5134subtest: upload whose reference the worker lacks is completed from the cache5135client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5136client # this derivation will be built:5137client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5138client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5139client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5140client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5141client # this derivation will be built:5142client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5143client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5144client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.29 seconds)5145client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5146client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5147worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5148worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5149worker2 # [ 96.550764] systemd[1]: Stopping Nix worker-protocol over gRPC...5150worker2 # [ 97.248651] nix-grpc-daemon[774]: ts=2026-09-22T17:00:01Z level=info event=signal_exit5151lb # [ 97.366518] niks3-server[964]: 2026/09/22 17:00:01 INFO lead: released remote=192.168.1.4:376285152worker1 # [ 98.370414] nix-grpc-daemon[767]: ts=2026-09-22T17:00:02Z level=info event=scheduler_restarting addr=lb:500515153worker2 # [ 98.347389] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=scheduler_restarting5154worker2 # [ 98.351638] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=scheduler_restarting addr=lb:500515155worker2 # [ 98.355767] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442345156worker2 # [ 98.361093] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=worker_gone addr=192.168.1.4:500515157worker2 # [ 98.365617] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:442345158worker2 # [ 98.372491] nix-grpc-daemon[774]: ts=2026-09-22T17:00:02Z level=info event=worker_gone addr=192.168.1.3:500515159worker2 # [ 98.376823] nix-grpc-daemon[774]: E0000 00:00:1790096402.995925 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5160worker2 # [ 98.403030] nix-daemon[859]: reaped child process 861, status = succeeded5161worker2 # [ 98.406338] nix-grpc-daemon[774]: E0000 00:00:1790096403.030092 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5162worker1 # [ 98.570601] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_connected addr=lb:500515163worker1 # [ 98.576158] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5164worker2 # [ 98.565887] nix-grpc-daemon[774]: ts=2026-09-22T17:00:03Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5165worker1 # [ 98.776820] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_connected addr=lb:500515166worker1 # [ 98.781943] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5167worker1 # [ 98.982556] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_connected addr=lb:500515168worker1 # [ 98.987634] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5169worker2 # [ 99.090550] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5170worker2 # [ 99.097103] systemd[1]: Stopped Nix worker-protocol over gRPC.5171worker2 # [ 99.099626] systemd[1]: nix-grpc-daemon.service: Consumed 2.943s CPU time over 1min 19.062s wall clock time, 65.7M memory peak, 93.2K incoming IP traffic, 224.6K outgoing IP traffic.5172worker2 # [ 99.118816] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5173lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5174lb: must succeed: curl -sf localhost:9901/clusters5175worker2 # [ 99.131024] systemd[1]: Closed Nix worker-protocol over gRPC.5176worker2 # [ 99.133011] systemd[1]: nix-grpc-daemon.socket: Consumed 786us CPU time over 1min 29.380s wall clock time, 256K memory peak, 114.2K incoming IP traffic, 95.6K outgoing IP traffic.5177worker1 # [ 99.188525] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_connected addr=lb:500515178worker1 # [ 99.193529] nix-grpc-daemon[767]: ts=2026-09-22T17:00:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5179lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5180lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5181client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25182worker1 # [ 99.395963] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:500515183worker1 # [ 99.399943] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5184worker1 # [ 99.453757] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:532185185worker1 # [ 99.463048] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:532185186client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5187client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5188worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5189worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5190worker2 # [ 99.557995] systemd[1]: Listening on Nix worker-protocol over gRPC.5191worker1 # [ 99.596482] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:500515192worker1 # [ 99.602835] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5193worker1 # [ 99.668692] systemd[1]: Stopping Nix worker-protocol over gRPC...5194worker1 # [ 99.802509] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:500515195worker1 # [ 99.807728] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5196lb # [ 99.945492] envoy[919]: conn peer=[2001:db8:1::1]:57876 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=325197lb # [ 99.950512] envoy[919]: conn peer=[2001:db8:1::1]:57860 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4238 tx=2056 ms=18245198lb # [ 99.955431] envoy[919]: conn peer=[2001:db8:1::1]:57888 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3753 tx=2124 ms=27475199lb # [ 99.961090] envoy[919]: conn peer=[2001:db8:1::1]:42726 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=945200lb # [ 99.965751] envoy[919]: conn peer=[2001:db8:1::1]:42740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2101 ms=17805201lb # [ 99.970516] envoy[919]: conn peer=[2001:db8:1::1]:42748 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=285202lb # [ 99.976065] envoy[919]: conn peer=[2001:db8:1::4]:37094 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=51196 tx=2969 ms=782655203lb # [ 99.980978] envoy[919]: conn peer=[2001:db8:1::1]:42754 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=285204worker1 # [ 100.008467] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:500515205worker1 # [ 100.013687] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5206worker2 # [ 100.087987] systemd[1]: Starting scheduler-token.service...5207worker1 # [ 100.203009] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=signal_exit5208worker2 # [ 100.175547] systemd[1]: scheduler-token.service: Deactivated successfully.5209worker2 # [ 100.184800] systemd[1]: Finished scheduler-token.service.5210worker1 # [ 100.214074] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_connected addr=lb:500515211worker2 # [ 100.187032] systemd[1]: scheduler-token.service: Consumed 58ms CPU time over 91ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5212worker1 # [ 100.219170] nix-grpc-daemon[767]: ts=2026-09-22T17:00:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5213worker2 # [ 100.211939] systemd[1]: Starting Nix worker-protocol over gRPC...5214worker2 # [ 100.335191] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5215worker2 # [ 100.337589] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5216worker2 # [ 100.341514] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15217worker2 # [ 100.347146] nix-grpc-daemon[1231]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5218worker2 # [ 100.367445] nix-grpc-daemon[1231]: E0000 00:00:1790096404.970522 1231 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5219lb # [ 100.396447] niks3-server[964]: 2026/09/22 17:00:05 INFO lead: acquired remote=192.168.1.4:543665220worker1 # [ 100.421074] nix-grpc-daemon[767]: ts=2026-09-22T17:00:05Z level=info event=scheduler_connected addr=lb:500515221worker2 # [ 100.394184] nix-grpc-daemon[1231]: E0000 00:00:1790096404.974351 1231 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5222worker2 # [ 100.402430] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5223worker2 # [ 100.404537] nix-grpc-daemon[1231]: E0000 00:00:1790096404.986307 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5224worker2 # [ 100.407052] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:04Z level=info event=scheduler_take_over5225worker2 # [ 100.408205] nix-grpc-daemon[1231]: E0000 00:00:1790096405.006854 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5226worker2 # [ 100.413193] nix-grpc-daemon[1231]: E0000 00:00:1790096405.007000 1272 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5227worker2 # [ 100.418002] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:600805228worker2 # [ 100.420351] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05229worker2 # [ 100.422739] systemd[1]: Started Nix worker-protocol over gRPC.5230worker2 # [ 100.423919] nix-grpc-daemon[1231]: E0000 00:00:1790096405.025599 1272 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5231worker2 # [ 100.428270] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=scheduler_connected addr=lb:500515232worker2 # [ 100.429598] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=oidc_jwks provider=mock keys=15233worker2 # [ 100.430839] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:600805234worker2 # [ 100.432578] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05235worker1 # [ 101.379386] nix-grpc-daemon[767]: ts=2026-09-22T17:00:05Z level=info event=scheduler_restarting5236worker2 # [ 101.362658] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600805237worker2 # [ 101.362934] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:05Z level=info event=worker_gone addr=192.168.1.3:500515238worker1 # [ 101.424666] nix-grpc-daemon[767]: ts=2026-09-22T17:00:06Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5239worker1 # [ 101.442841] nix-daemon[831]: reaped child process 834, status = succeeded5240worker1 # [ 102.375779] nix-grpc-daemon[767]: E0000 00:00:1790096406.992214 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5241worker1 # [ 102.382322] nix-grpc-daemon[767]: E0000 00:00:1790096406.992445 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5242worker1 # [ 103.060981] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5243worker1 # [ 103.068534] systemd[1]: Stopped Nix worker-protocol over gRPC.5244worker1 # [ 103.071117] systemd[1]: nix-grpc-daemon.service: Consumed 3.710s CPU time over 1min 23.117s wall clock time, 69M memory peak, 97.4K incoming IP traffic, 177.6K outgoing IP traffic.5245worker1 # [ 103.088513] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5246worker1 # [ 103.090822] systemd[1]: Closed Nix worker-protocol over gRPC.5247lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5248lb: must succeed: curl -sf localhost:9901/clusters5249worker1 # [ 103.102236] systemd[1]: nix-grpc-daemon.socket: Consumed 675us CPU time over 1min 32.814s wall clock time, 256K memory peak, 102.4K incoming IP traffic, 91.2K outgoing IP traffic.5250lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5251lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5252client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25253worker2 # [ 103.366619] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344305254worker2 # [ 103.382069] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5255worker2 # [ 103.411461] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5)5256worker2 # [ 103.444604] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5)5257worker2 # [ 103.467398] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 359 ms (attempt 3/5)5258worker2 # [ 103.835864] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 619 ms (attempt 4/5)5259worker2 # [ 104.458401] nix-grpc-daemon[1231]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5260worker2 # [ 104.464098] nix-grpc-daemon[1231]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5261client # copying 2 paths...5262client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5263worker2 # [ 104.488832] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344305264client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5265worker2 # [ 104.507610] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5266worker2 # [ 104.538638] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:09Z level=info event=niks3_push_started pid=12915267worker2 # [ 104.696510] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.319Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5268worker2 # [ 104.804622] nix-daemon[859]: accepted connection from pid 1304, user nix-grpc-daemon (trusted)5269worker2 # [ 104.822954] nix-daemon[859]: reaped child process 1307, status = succeeded5270lb # [ 104.857213] niks3-server[964]: 2026/09/22 17:00:09 INFO Received uploads request method=POST path=/api/pending_closures5271lb # [ 104.869170] niks3-server[964]: 2026/09/22 17:00:09 INFO Received uploads request method=POST path=/api/pending_closures5272worker2 # [ 104.852094] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.474Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5273worker2 # [ 104.853983] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.474Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5274worker2 # [ 104.855857] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.474Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5275lb # [ 104.944280] niks3-server[964]: 2026/09/22 17:00:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5276worker2 # [ 104.931972] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.553Z level=INFO msg="Uploading 3 narinfos"5277lb # [ 104.958026] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5278lb # [ 104.968129] niks3-server[964]: 2026/09/22 17:00:09 INFO Signed narinfos id=35 count=15279lb # [ 104.974540] niks3-server[964]: 2026/09/22 17:00:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5280lb # [ 104.981110] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5281lb # [ 104.983520] niks3-server[964]: 2026/09/22 17:00:09 INFO Signed narinfos id=36 count=25282lb # [ 104.989819] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5283lb # [ 104.999151] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5284lb # [ 105.006348] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5285lb # [ 105.014697] niks3-server[964]: 2026/09/22 17:00:09 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5286lb # [ 105.024073] niks3-server[964]: 2026/09/22 17:00:09 INFO Completed upload id=355287lb # [ 105.026223] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5288worker2 # [ 105.000908] nix-grpc-daemon[1291]: time=2026-09-22T17:00:09.624Z level=INFO msg="Upload complete. (301ms)"5289lb # [ 105.032284] niks3-server[964]: 2026/09/22 17:00:09 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5290lb # [ 105.037529] niks3-server[964]: 2026/09/22 17:00:09 INFO Completed upload id=365291lb # [ 105.040591] niks3-server[964]: 2026/09/22 17:00:09 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5292worker2 # [ 105.011930] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=0 paths=2 nar_bytes_in=3125293worker2 # [ 105.029322] nix-daemon[859]: reaped child process 1289, status = succeeded5294client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.90 seconds)5295worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5296worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5297worker1 # [ 105.181094] systemd[1]: Listening on Nix worker-protocol over gRPC.5298worker1 # [ 105.188602] systemd[1]: Starting Nix worker-protocol over gRPC...5299worker1 # [ 105.341869] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5300worker1 # [ 105.344772] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5301worker1 # [ 105.348869] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15302worker1 # [ 105.354506] nix-grpc-daemon[1243]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5303worker1 # [ 105.355944] nix-grpc-daemon[1243]: E0000 00:00:1790096409.970441 1243 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5304lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5305lb: must succeed: curl -sf localhost:9901/clusters5306worker1 # [ 105.401589] nix-grpc-daemon[1243]: E0000 00:00:1790096409.976671 1243 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5307worker1 # [ 105.404502] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5308worker1 # [ 105.406276] nix-grpc-daemon[1243]: E0000 00:00:1790096409.983292 1285 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5309worker1 # [ 105.408686] nix-grpc-daemon[1243]: E0000 00:00:1790096410.000823 1285 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5310worker1 # [ 105.410885] nix-grpc-daemon[1243]: E0000 00:00:1790096410.001905 1285 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5311worker1 # [ 105.413317] nix-grpc-daemon[1243]: E0000 00:00:1790096410.017378 1285 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5312worker1 # [ 105.418627] systemd[1]: Started Nix worker-protocol over gRPC.5313worker1 # [ 105.421861] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:10Z level=info event=scheduler_connected addr=lb:500515314worker2 # [ 105.392949] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:600805315worker2 # [ 105.399333] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05316lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5317lb: must succeed: curl -sf localhost:9901/clusters5318lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5319lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5320(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 10.97 seconds)5321subtest: a niks3 restart does not cost the leader its role5322lb: must succeed: curl -sf localhost:9901/clusters5323lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5324lb: must succeed: curl -sf localhost:9901/clusters5325lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5326worker1 # [ 106.664795] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:11Z level=info event=scheduler_lock_lost status=2005327lb # [ 106.663616] niks3-server[964]: 2026/09/22 17:00:11 INFO Shutdown signal received, draining in-flight requests timeout=10s5328lb # [ 106.671223] systemd[1]: Stopping niks3 server...5329lb # [ 106.672824] niks3-server[964]: 2026/09/22 17:00:11 INFO lead: released remote=192.168.1.4:543665330worker2 # [ 106.644569] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:11Z level=info event=scheduler_lock_lost status=2005331lb # [ 106.694130] systemd[1]: niks3.service: Deactivated successfully.5332lb # [ 106.695894] systemd[1]: Stopped niks3 server.5333lb # [ 106.697619] systemd[1]: niks3.service: Consumed 670ms CPU time over 1min 24.573s wall clock time, 27.8M memory peak, 100.3K incoming IP traffic, 143.7K outgoing IP traffic.5334lb # [ 106.708427] systemd[1]: Starting niks3 server...5335lb # [ 106.767416] niks3-server[1105]: 2026/09/22 17:00:11 goose: no migrations to run. current version: 202609200000005336lb # [ 106.772538] niks3-server[1105]: 2026/09/22 17:00:11 OK 1_commit_pending_closure.sql (6.63ms)5337lb # [ 106.776260] niks3-server[1105]: 2026/09/22 17:00:11 OK 2_object_stats_trigger.sql (3.37ms)5338lb # [ 106.777438] niks3-server[1105]: 2026/09/22 17:00:11 goose: up to current file version: 25339lb # [ 106.779092] niks3-server[1105]: 2026/09/22 17:00:11 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5340lb # [ 106.790449] niks3-server[1105]: 2026/09/22 17:00:11 INFO Read proxy enabled — serving cache objects from S35341lb # [ 106.794883] niks3-server[1105]: 2026/09/22 17:00:11 INFO Using socket-activated listener address=0.0.0.0:57515342lb # [ 106.797410] niks3-server[1105]: 2026/09/22 17:00:11 INFO systemd watchdog enabled interval=15s5343lb # [ 106.799679] niks3-server[1105]: 2026/09/22 17:00:11 INFO Starting HTTP server address=0.0.0.0:57515344lb # [ 106.802264] systemd[1]: Started niks3 server.5345lb: waiting for unit niks3.service5346lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5347lb: must succeed: curl -sf localhost:9901/clusters5348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5349worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5350worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5351lb: must succeed: curl -sf localhost:9901/clusters5352lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5353lb: must succeed: curl -sf localhost:9901/clusters5354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5355worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5356worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5357client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25358worker1 # [ 107.294233] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401505359worker1 # [ 107.309320] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5360client # don't know how to build these paths:5361client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5362worker1 # [ 107.332673] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401505363worker1 # [ 107.349009] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 58 ms (attempt 1/5)5364worker1 # [ 107.408854] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 2/5)5365worker1 # [ 107.483685] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 3/5)5366lb # [ 107.677123] niks3-server[1105]: 2026/09/22 17:00:12 INFO lead: acquired remote=192.168.1.4:488065367worker1 # [ 107.681551] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 206 ms (attempt 4/5)5368worker1 # [ 107.890963] nix-grpc-daemon[1243]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5369worker1 # [ 107.896668] nix-grpc-daemon[1243]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5370worker2 # [ 107.916265] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05371worker2 # [ 107.916580] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15372worker2 # [ 107.960474] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344305373worker2 # [ 107.978114] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344305374client # copying 1 paths to 'lb:50051'...5375client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5376worker2 # [ 107.992675] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344305377worker2 # [ 108.008098] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5378worker2 # [ 108.174276] nix-daemon[859]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5379worker2 # [ 108.191519] nix-daemon[859]: reaped child process 1329, status = succeeded5380lb # [ 108.227362] niks3-server[1105]: 2026/09/22 17:00:12 INFO Received uploads request method=POST path=/api/pending_closures5381worker2 # [ 108.213364] nix-grpc-daemon[1291]: time=2026-09-22T17:00:12.836Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5382worker2 # [ 108.215095] nix-grpc-daemon[1291]: time=2026-09-22T17:00:12.836Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5383lb # [ 108.266981] niks3-server[1105]: 2026/09/22 17:00:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5384lb # [ 108.300759] niks3-server[1105]: 2026/09/22 17:00:12 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5385lb # [ 108.313127] niks3-server[1105]: 2026/09/22 17:00:12 INFO Signed narinfos id=37 count=15386worker2 # [ 108.288111] nix-grpc-daemon[1291]: time=2026-09-22T17:00:12.911Z level=INFO msg="Uploading 1 narinfos"5387lb # [ 108.325890] niks3-server[1105]: 2026/09/22 17:00:12 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5388lb # [ 108.334911] niks3-server[1105]: 2026/09/22 17:00:12 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5389lb # [ 108.340083] niks3-server[1105]: 2026/09/22 17:00:12 INFO Completed upload id=375390worker2 # [ 108.311781] nix-grpc-daemon[1291]: time=2026-09-22T17:00:12.935Z level=INFO msg="Upload complete. (285ms)"5391lb # [ 108.344534] niks3-server[1105]: 2026/09/22 17:00:12 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5392worker2 # [ 108.322708] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=0 paths=1 nar_bytes_in=5445393worker2 # [ 108.336341] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344305394worker2 # [ 108.350605] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5395worker2 # [ 108.356700] nix-daemon[859]: reaped child process 1324, status = succeeded5396client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5397worker2 # [ 108.369172] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5398client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5399client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5400client # farm-a> LOG-farm-a-after-niks3-restart5401worker2 # [ 109.879817] nix-daemon[859]: reaped child process 1333, status = succeeded5402lb # [ 109.945509] envoy[919]: conn peer=192.168.1.3:39116 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=11455 tx=3263 ms=802535403lb # [ 109.950565] envoy[919]: conn peer=[2001:db8:1::1]:47384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=1006 tx=304 ms=16635404worker2 # [ 109.951369] nix-daemon[859]: accepted connection from pid 1349, user nix-grpc-daemon (trusted)5405worker2 # [ 109.968852] nix-daemon[859]: reaped child process 1352, status = succeeded5406lb # [ 110.003983] niks3-server[1105]: 2026/09/22 17:00:14 INFO Received uploads request method=POST path=/api/pending_closures5407worker2 # [ 109.988482] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.610Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5408worker2 # [ 109.990392] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.610Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5409lb # [ 110.051885] niks3-server[1105]: 2026/09/22 17:00:14 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5410worker2 # [ 110.039404] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.662Z level=INFO msg="Uploading 1 narinfos"5411lb # [ 110.070076] niks3-server[1105]: 2026/09/22 17:00:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5412lb # [ 110.081144] niks3-server[1105]: 2026/09/22 17:00:14 INFO Signed narinfos id=38 count=15413lb # [ 110.086667] niks3-server[1105]: 2026/09/22 17:00:14 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5414lb # [ 110.100991] niks3-server[1105]: 2026/09/22 17:00:14 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5415lb # [ 110.108206] niks3-server[1105]: 2026/09/22 17:00:14 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5416lb # [ 110.112438] niks3-server[1105]: 2026/09/22 17:00:14 INFO Completed upload id=385417worker2 # [ 110.071045] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.694Z level=INFO msg="Upload complete. (288ms)"5418lb # [ 110.114860] niks3-server[1105]: 2026/09/22 17:00:14 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5419worker2 # [ 110.089448] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15420worker2 # [ 110.112419] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05421worker2 # [ 110.118371] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15422worker2 # [ 110.123161] nix-daemon[859]: reaped child process 1331, status = succeeded5423worker2 # [ 110.156219] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344305424worker2 # [ 110.172271] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344305425worker2 # [ 110.187265] nix-grpc-daemon[1291]: time=2026-09-22T17:00:14.811Z level=INFO msg="All 1 paths already cached"5426client # copying 1 paths to 'lb:50051'...5427client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5428worker2 # [ 110.198094] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344305429worker2 # [ 110.211910] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5430worker2 # [ 110.371241] nix-daemon[859]: accepted connection from pid 1358, user nix-grpc-daemon (trusted)5431worker2 # [ 110.390200] nix-daemon[859]: reaped child process 1361, status = succeeded5432lb # [ 110.424178] niks3-server[1105]: 2026/09/22 17:00:15 INFO Received uploads request method=POST path=/api/pending_closures5433worker2 # [ 110.406791] nix-grpc-daemon[1291]: time=2026-09-22T17:00:15.030Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5434worker2 # [ 110.408554] nix-grpc-daemon[1291]: time=2026-09-22T17:00:15.030Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5435lb # [ 110.459610] niks3-server[1105]: 2026/09/22 17:00:15 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5436worker2 # [ 110.441698] nix-grpc-daemon[1291]: time=2026-09-22T17:00:15.065Z level=INFO msg="Uploading 1 narinfos"5437lb # [ 110.473429] niks3-server[1105]: 2026/09/22 17:00:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5438lb # [ 110.483344] niks3-server[1105]: 2026/09/22 17:00:15 INFO Signed narinfos id=39 count=15439lb # [ 110.489517] niks3-server[1105]: 2026/09/22 17:00:15 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5440worker2 # [ 110.473659] nix-grpc-daemon[1291]: time=2026-09-22T17:00:15.097Z level=INFO msg="Upload complete. (245ms)"5441lb # [ 110.502591] niks3-server[1105]: 2026/09/22 17:00:15 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5442lb # [ 110.509962] niks3-server[1105]: 2026/09/22 17:00:15 INFO Completed upload id=395443worker2 # [ 110.484327] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=0 paths=1 nar_bytes_in=6565444lb # [ 110.515029] niks3-server[1105]: 2026/09/22 17:00:15 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5445worker2 # [ 110.499635] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344305446worker2 # [ 110.507047] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5447worker2 # [ 110.512646] nix-daemon[859]: reaped child process 1356, status = succeeded5448client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5449client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5450worker2 # [ 110.527253] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5451client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5452client # farm-b> LOG-farm-b-after-niks3-restart5453worker2 # [ 111.390644] nix-daemon[859]: reaped child process 1365, status = succeeded5454worker2 # [ 111.531292] nix-daemon[859]: accepted connection from pid 1381, user nix-grpc-daemon (trusted)5455worker2 # [ 111.553244] nix-daemon[859]: reaped child process 1384, status = succeeded5456lb # [ 111.585643] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received uploads request method=POST path=/api/pending_closures5457worker2 # [ 111.567989] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.190Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5458worker2 # [ 111.570629] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.190Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5459lb # [ 111.620151] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5460worker2 # [ 111.597255] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.220Z level=INFO msg="Uploading 1 narinfos"5461lb # [ 111.628058] niks3-server[1105]: 2026/09/22 17:00:16 INFO Signed narinfos id=40 count=15462lb # [ 111.634811] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5463lb # [ 111.642626] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5464lb # [ 111.653609] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5465lb # [ 111.660307] niks3-server[1105]: 2026/09/22 17:00:16 INFO Completed upload id=405466worker2 # [ 111.630218] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.254Z level=INFO msg="Upload complete. (291ms)"5467lb # [ 111.662847] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5468lb # [ 111.669354] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5469worker2 # [ 111.648623] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15470worker2 # [ 111.664403] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05471worker2 # [ 111.669121] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15472worker2 # [ 111.673112] nix-daemon[859]: reaped child process 1363, status = succeeded5473worker2 # [ 111.691780] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344305474worker2 # [ 111.706966] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:344305475worker2 # [ 111.723045] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.346Z level=INFO msg="All 2 paths already cached"5476client # copying 1 paths to 'lb:50051'...5477client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5478worker2 # [ 111.735323] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:344305479worker2 # [ 111.749684] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5480worker2 # [ 111.901808] nix-daemon[859]: accepted connection from pid 1390, user nix-grpc-daemon (trusted)5481worker2 # [ 111.919147] nix-daemon[859]: reaped child process 1393, status = succeeded5482lb # [ 111.953786] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received uploads request method=POST path=/api/pending_closures5483worker2 # [ 111.935866] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.559Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5484worker2 # [ 111.937792] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.559Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5485worker2 # [ 111.969654] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.593Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 111.988204] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5487lb # [ 112.003076] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5488lb # [ 112.015684] niks3-server[1105]: 2026/09/22 17:00:16 INFO Signed narinfos id=41 count=15489lb # [ 112.024926] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5490lb # [ 112.031864] niks3-server[1105]: 2026/09/22 17:00:16 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5491lb # [ 112.039211] niks3-server[1105]: 2026/09/22 17:00:16 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5492lb # [ 112.043811] niks3-server[1105]: 2026/09/22 17:00:16 INFO Completed upload id=415493worker2 # [ 112.015609] nix-grpc-daemon[1291]: time=2026-09-22T17:00:16.639Z level=INFO msg="Upload complete. (250ms)"5494worker2 # [ 112.028378] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=0 paths=1 nar_bytes_in=7845495worker2 # [ 112.042949] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:344305496worker2 # [ 112.053850] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5497worker2 # [ 112.062149] nix-daemon[859]: reaped child process 1388, status = succeeded5498client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5499worker2 # [ 112.071207] nix-daemon[859]: accepted connection from pid 1231, user nix-grpc-daemon (trusted)5500client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5501client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5502client # farm-top> LOG-farm-top-after-niks3-restart5503worker2 # [ 113.302291] nix-daemon[859]: reaped child process 1397, status = succeeded5504worker2 # [ 113.416585] nix-daemon[859]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)5505worker2 # [ 113.435180] nix-daemon[859]: reaped child process 1416, status = succeeded5506lb # [ 113.470708] niks3-server[1105]: 2026/09/22 17:00:18 INFO Received uploads request method=POST path=/api/pending_closures5507worker2 # [ 113.455370] nix-grpc-daemon[1291]: time=2026-09-22T17:00:18.078Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5508worker2 # [ 113.457110] nix-grpc-daemon[1291]: time=2026-09-22T17:00:18.078Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5509lb # [ 113.509988] niks3-server[1105]: 2026/09/22 17:00:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5510worker2 # [ 113.490816] nix-grpc-daemon[1291]: time=2026-09-22T17:00:18.114Z level=INFO msg="Uploading 1 narinfos"5511lb # [ 113.526667] niks3-server[1105]: 2026/09/22 17:00:18 INFO Signed narinfos id=42 count=15512lb # [ 113.534625] niks3-server[1105]: 2026/09/22 17:00:18 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5513lb # [ 113.543081] niks3-server[1105]: 2026/09/22 17:00:18 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5514lb # [ 113.548234] niks3-server[1105]: 2026/09/22 17:00:18 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5515lb # [ 113.551049] niks3-server[1105]: 2026/09/22 17:00:18 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5516lb # [ 113.560113] niks3-server[1105]: 2026/09/22 17:00:18 INFO Completed upload id=425517worker2 # [ 113.531881] nix-grpc-daemon[1291]: time=2026-09-22T17:00:18.155Z level=INFO msg="Upload complete. (246ms)"5518lb # [ 113.563591] niks3-server[1105]: 2026/09/22 17:00:18 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5519worker2 # [ 113.545109] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:34430 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15520worker2 # [ 113.564640] nix-daemon[859]: reaped child process 1395, status = succeeded5521client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.54 seconds)5522client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5523client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds)5524(finished: subtest: a niks3 restart does not cost the leader its role, in 7.26 seconds)5525subtest: scheduler down: the other node takes the lock and keeps it5526lb: must succeed: curl -sf localhost:9901/clusters5527lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5528lb: must succeed: curl -sf localhost:9901/clusters5529lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5530lb: must succeed: curl -sf localhost:9901/clusters5531lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5532lb: must succeed: curl -sf localhost:9901/clusters5533lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5534worker2 # [ 114.019188] systemd[1]: Stopping Nix worker-protocol over gRPC...5535worker2 # [ 114.363317] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:18Z level=info event=signal_exit5536lb # [ 114.685337] niks3-server[1105]: 2026/09/22 17:00:19 INFO lead: released remote=192.168.1.4:488065537worker1 # [ 115.686823] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_restarting addr=lb:500515538worker2 # [ 115.662681] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=scheduler_restarting5539worker2 # [ 115.666805] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=scheduler_restarting addr=lb:500515540worker2 # [ 115.671581] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600805541worker2 # [ 115.676712] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=worker_gone addr=192.168.1.4:500515542worker2 # [ 115.681859] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:600805543worker2 # [ 115.688342] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=worker_gone addr=192.168.1.3:500515544worker2 # [ 115.713167] nix-daemon[859]: reaped child process 1283, status = succeeded5545worker1 # [ 115.886652] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_connected addr=lb:500515546worker1 # [ 115.892325] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5547worker2 # [ 115.877621] nix-grpc-daemon[1231]: ts=2026-09-22T17:00:20Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5548worker1 # [ 116.092829] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_connected addr=lb:500515549worker1 # [ 116.099300] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5550worker1 # [ 116.299180] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_connected addr=lb:500515551worker1 # [ 116.304462] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5552worker2 # [ 116.412434] nix-grpc-daemon[1231]: E0000 00:00:1790096421.036236 1276 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5553worker2 # [ 116.420121] nix-grpc-daemon[1231]: E0000 00:00:1790096421.043574 1276 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5554worker1 # [ 116.505272] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:500515555worker1 # [ 116.510479] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5556worker1 # [ 116.713068] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:500515557worker1 # [ 116.717192] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5558worker1 # [ 116.913614] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:500515559worker1 # [ 116.918386] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5560worker2 # [ 116.936009] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5561worker2 # [ 116.942974] systemd[1]: Stopped Nix worker-protocol over gRPC.5562worker2 # [ 116.949712] systemd[1]: nix-grpc-daemon.service: Consumed 1.410s CPU time over 16.741s wall clock time, 39.1M memory peak, 46.1K incoming IP traffic, 78.2K outgoing IP traffic.5563worker2 # [ 116.967524] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5564worker2 # [ 116.973187] systemd[1]: Closed Nix worker-protocol over gRPC.5565worker2 # [ 116.974951] systemd[1]: nix-grpc-daemon.socket: Consumed 833us CPU time over 17.416s wall clock time, 256K memory peak, 38.2K incoming IP traffic, 28.4K outgoing IP traffic.5566worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5567worker1 # [ 117.119127] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:500515568worker1 # [ 117.124535] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5569worker1 # [ 117.325315] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_connected addr=lb:500515570worker1 # [ 117.330488] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5571worker1 # [ 117.531040] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_connected addr=lb:500515572worker1 # [ 117.536402] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5573worker1 # [ 117.671259] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_take_over5574lb # [ 117.670870] niks3-server[1105]: 2026/09/22 17:00:22 INFO lead: acquired remote=192.168.1.3:590165575worker1 # [ 117.738865] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_connected addr=lb:500515576worker1 # [ 117.743028] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5577worker1 # [ 117.939615] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_connected addr=lb:500515578worker1 # [ 117.944789] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5579worker1 # [ 118.147649] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=scheduler_connected addr=lb:500515580worker1 # [ 118.159212] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:388765581worker1 # [ 118.168605] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:22Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05582worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.18 seconds)5583worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5584worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5585client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25586client # don't know how to build these paths:5587client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5588worker1 # [ 118.456384] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:401505589worker1 # [ 118.466162] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401505590worker1 # [ 118.484999] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05591worker1 # [ 118.491174] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15592worker1 # [ 118.529521] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401505593worker1 # [ 118.544792] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401505594client # copying 1 paths to 'lb:50051'...5595client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5596worker1 # [ 118.561105] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401505597worker1 # [ 118.577146] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5598worker1 # [ 118.602941] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=info event=niks3_push_started pid=13225599worker1 # [ 118.751565] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.367Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5600worker1 # [ 118.858817] nix-daemon[831]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)5601worker1 # [ 118.875215] nix-daemon[831]: reaped child process 1337, status = succeeded5602lb # [ 118.882240] niks3-server[1105]: 2026/09/22 17:00:23 INFO Received uploads request method=POST path=/api/pending_closures5603worker1 # [ 118.893725] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.509Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5604worker1 # [ 118.895198] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.509Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5605lb # [ 118.920426] niks3-server[1105]: 2026/09/22 17:00:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5606worker1 # [ 118.930504] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.546Z level=INFO msg="Uploading 1 narinfos"5607lb # [ 118.932153] niks3-server[1105]: 2026/09/22 17:00:23 INFO Signed narinfos id=43 count=15608lb # [ 118.939780] niks3-server[1105]: 2026/09/22 17:00:23 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5609lb # [ 118.954696] niks3-server[1105]: 2026/09/22 17:00:23 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5610lb # [ 118.964401] niks3-server[1105]: 2026/09/22 17:00:23 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5611lb # [ 118.971802] niks3-server[1105]: 2026/09/22 17:00:23 INFO Completed upload id=435612worker1 # [ 118.971736] nix-grpc-daemon[1322]: time=2026-09-22T17:00:23.588Z level=INFO msg="Upload complete. (216ms)"5613lb # [ 118.977264] niks3-server[1105]: 2026/09/22 17:00:23 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5614worker1 # [ 118.985052] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 paths=1 nar_bytes_in=5205615worker1 # [ 118.995413] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401505616worker1 # [ 119.004681] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5617worker1 # [ 119.007470] nix-daemon[831]: reaped child process 1320, status = succeeded5618client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5619client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5620worker1 # [ 119.020886] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5621client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5622client # farm-a> LOG-farm-a-on-standby5623worker1 # [ 119.654853] nix-daemon[831]: reaped child process 1341, status = succeeded5624worker1 # [ 119.747077] nix-daemon[831]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5625worker1 # [ 119.765094] nix-daemon[831]: reaped child process 1360, status = succeeded5626lb # [ 119.770029] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received uploads request method=POST path=/api/pending_closures5627worker1 # [ 119.784432] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5628worker1 # [ 119.785884] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.399Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5629lb # [ 119.805703] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5630lb # [ 119.826779] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5631worker1 # [ 119.835928] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.452Z level=INFO msg="Uploading 1 narinfos"5632lb # [ 119.836616] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5633lb # [ 119.839539] niks3-server[1105]: 2026/09/22 17:00:24 INFO Signed narinfos id=44 count=15634lb # [ 119.847654] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5635lb # [ 119.855463] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5636worker1 # [ 119.860200] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.476Z level=INFO msg="Upload complete. (229ms)"5637lb # [ 119.860800] niks3-server[1105]: 2026/09/22 17:00:24 INFO Completed upload id=445638lb # [ 119.863118] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5639worker1 # [ 119.878082] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15640worker1 # [ 119.894288] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05641worker1 # [ 119.898445] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15642worker1 # [ 119.902695] nix-daemon[831]: reaped child process 1339, status = succeeded5643worker1 # [ 119.925765] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401505644worker1 # [ 119.956965] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401505645lb # [ 119.951156] envoy[919]: conn peer=[2001:db8:1::1]:47392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9736 tx=4453 ms=62955646lb # [ 119.962974] envoy[919]: conn peer=[2001:db8:1::4]:35734 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1691 tx=425 ms=154535647worker1 # [ 119.978246] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.594Z level=INFO msg="All 1 paths already cached"5648client # copying 1 paths to 'lb:50051'...5649client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5650worker1 # [ 119.990523] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401505651worker1 # [ 120.005686] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5652worker1 # [ 120.169419] nix-daemon[831]: accepted connection from pid 1366, user nix-grpc-daemon (trusted)5653worker1 # [ 120.190579] nix-daemon[831]: reaped child process 1369, status = succeeded5654lb # [ 120.187667] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received uploads request method=POST path=/api/pending_closures5655worker1 # [ 120.198486] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.814Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5656worker1 # [ 120.199963] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.814Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5657lb # [ 120.217140] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5658worker1 # [ 120.226359] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.842Z level=INFO msg="Uploading 1 narinfos"5659lb # [ 120.232522] niks3-server[1105]: 2026/09/22 17:00:24 INFO Signed narinfos id=45 count=15660lb # [ 120.235800] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5661lb # [ 120.258281] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5662worker1 # [ 120.263332] nix-grpc-daemon[1322]: time=2026-09-22T17:00:24.880Z level=INFO msg="Upload complete. (242ms)"5663lb # [ 120.265118] niks3-server[1105]: 2026/09/22 17:00:24 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5664lb # [ 120.269378] niks3-server[1105]: 2026/09/22 17:00:24 INFO Completed upload id=455665lb # [ 120.272385] niks3-server[1105]: 2026/09/22 17:00:24 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5666worker1 # [ 120.272336] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 paths=1 nar_bytes_in=6405667worker1 # [ 120.292166] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401505668worker1 # [ 120.301969] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5669worker1 # [ 120.305019] nix-daemon[831]: reaped child process 1364, status = succeeded5670client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5671client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5672worker1 # [ 120.318505] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5673client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5674client # farm-b> LOG-farm-b-on-standby5675worker1 # [ 120.977841] nix-daemon[831]: reaped child process 1373, status = succeeded5676worker1 # [ 121.088335] nix-daemon[831]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)5677worker1 # [ 121.109639] nix-daemon[831]: reaped child process 1392, status = succeeded5678lb # [ 121.112663] niks3-server[1105]: 2026/09/22 17:00:25 INFO Received uploads request method=POST path=/api/pending_closures5679worker1 # [ 121.124543] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.739Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5680worker1 # [ 121.126142] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.740Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5681lb # [ 121.154505] niks3-server[1105]: 2026/09/22 17:00:25 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5682worker1 # [ 121.168568] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.784Z level=INFO msg="Uploading 1 narinfos"5683lb # [ 121.168979] niks3-server[1105]: 2026/09/22 17:00:25 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5684lb # [ 121.178682] niks3-server[1105]: 2026/09/22 17:00:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5685lb # [ 121.187883] niks3-server[1105]: 2026/09/22 17:00:25 INFO Signed narinfos id=46 count=15686lb # [ 121.196749] niks3-server[1105]: 2026/09/22 17:00:25 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5687worker1 # [ 121.198080] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.814Z level=INFO msg="Upload complete. (269ms)"5688lb # [ 121.201218] niks3-server[1105]: 2026/09/22 17:00:25 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5689lb # [ 121.208645] niks3-server[1105]: 2026/09/22 17:00:25 INFO Completed upload id=465690lb # [ 121.211823] niks3-server[1105]: 2026/09/22 17:00:25 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5691worker1 # [ 121.216546] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15692worker1 # [ 121.234773] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05693worker1 # [ 121.241569] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15694worker1 # [ 121.246690] nix-daemon[831]: reaped child process 1371, status = succeeded5695worker1 # [ 121.281730] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401505696worker1 # [ 121.296948] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401505697worker1 # [ 121.311004] nix-grpc-daemon[1322]: time=2026-09-22T17:00:25.927Z level=INFO msg="All 2 paths already cached"5698client # copying 1 paths to 'lb:50051'...5699client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5700worker1 # [ 121.322052] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401505701worker1 # [ 121.336123] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5702worker1 # [ 121.491144] nix-daemon[831]: accepted connection from pid 1398, user nix-grpc-daemon (trusted)5703worker1 # [ 121.510163] nix-daemon[831]: reaped child process 1401, status = succeeded5704lb # [ 121.511825] niks3-server[1105]: 2026/09/22 17:00:26 INFO Received uploads request method=POST path=/api/pending_closures5705worker1 # [ 121.522103] nix-grpc-daemon[1322]: time=2026-09-22T17:00:26.138Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5706worker1 # [ 121.523635] nix-grpc-daemon[1322]: time=2026-09-22T17:00:26.138Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5707lb # [ 121.542843] niks3-server[1105]: 2026/09/22 17:00:26 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5708worker1 # [ 121.554217] nix-grpc-daemon[1322]: time=2026-09-22T17:00:26.170Z level=INFO msg="Uploading 1 narinfos"5709lb # [ 121.553234] niks3-server[1105]: 2026/09/22 17:00:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5710lb # [ 121.565212] niks3-server[1105]: 2026/09/22 17:00:26 INFO Signed narinfos id=47 count=15711lb # [ 121.571985] niks3-server[1105]: 2026/09/22 17:00:26 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5712lb # [ 121.587764] niks3-server[1105]: 2026/09/22 17:00:26 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5713worker1 # [ 121.594474] nix-grpc-daemon[1322]: time=2026-09-22T17:00:26.211Z level=INFO msg="Upload complete. (242ms)"5714lb # [ 121.594574] niks3-server[1105]: 2026/09/22 17:00:26 INFO Completed upload id=475715lb # [ 121.598607] niks3-server[1105]: 2026/09/22 17:00:26 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5716worker1 # [ 121.604397] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 paths=1 nar_bytes_in=7685717worker1 # [ 121.623185] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401505718worker1 # [ 121.630581] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5719worker1 # [ 121.636903] nix-daemon[831]: reaped child process 1396, status = succeeded5720client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5721client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5722worker1 # [ 121.653659] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5723client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5724client # farm-top> LOG-farm-top-on-standby5725worker1 # [ 122.922634] nix-daemon[831]: reaped child process 1405, status = succeeded5726worker1 # [ 123.052232] nix-daemon[831]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5727worker1 # [ 123.070449] nix-daemon[831]: reaped child process 1424, status = succeeded5728lb # [ 123.078961] niks3-server[1105]: 2026/09/22 17:00:27 INFO Received uploads request method=POST path=/api/pending_closures5729worker1 # [ 123.091788] nix-grpc-daemon[1322]: time=2026-09-22T17:00:27.706Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5730worker1 # [ 123.093805] nix-grpc-daemon[1322]: time=2026-09-22T17:00:27.706Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5731lb # [ 123.118045] niks3-server[1105]: 2026/09/22 17:00:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5732worker1 # [ 123.130886] nix-grpc-daemon[1322]: time=2026-09-22T17:00:27.747Z level=INFO msg="Uploading 1 narinfos"5733lb # [ 123.132813] niks3-server[1105]: 2026/09/22 17:00:27 INFO Signed narinfos id=48 count=15734lb # [ 123.143658] niks3-server[1105]: 2026/09/22 17:00:27 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5735lb # [ 123.154492] niks3-server[1105]: 2026/09/22 17:00:27 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5736lb # [ 123.163947] niks3-server[1105]: 2026/09/22 17:00:27 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5737worker1 # [ 123.169763] nix-grpc-daemon[1322]: time=2026-09-22T17:00:27.786Z level=INFO msg="Upload complete. (302ms)"5738lb # [ 123.172115] niks3-server[1105]: 2026/09/22 17:00:27 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5739lb # [ 123.178704] niks3-server[1105]: 2026/09/22 17:00:27 INFO Completed upload id=485740lb # [ 123.182424] niks3-server[1105]: 2026/09/22 17:00:27 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5741worker1 # [ 123.181265] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15742worker1 # [ 123.202751] nix-daemon[831]: reaped child process 1403, status = succeeded5743client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.00 seconds)5744client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5745client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.19 seconds)5746worker2 # [ 123.479269] systemd[1]: Listening on Nix worker-protocol over gRPC.5747worker2 # [ 123.489485] systemd[1]: Starting scheduler-token.service...5748worker2 # [ 123.572383] systemd[1]: scheduler-token.service: Deactivated successfully.5749worker2 # [ 123.576895] systemd[1]: Finished scheduler-token.service.5750worker2 # [ 123.578593] systemd[1]: scheduler-token.service: Consumed 48ms CPU time over 87ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5751worker2 # [ 123.595862] systemd[1]: Starting Nix worker-protocol over gRPC...5752worker2 # [ 123.715234] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5753worker2 # [ 123.717568] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5754worker2 # [ 123.721291] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15755worker2 # [ 123.727813] nix-grpc-daemon[1437]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5756lb: must succeed: curl -sf localhost:9901/clusters5757worker2 # [ 123.737798] nix-grpc-daemon[1437]: E0000 00:00:1790096428.349587 1437 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5758worker2 # [ 123.756121] nix-grpc-daemon[1437]: E0000 00:00:1790096428.351690 1437 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5759worker2 # [ 123.758449] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5760worker2 # [ 123.777231] nix-grpc-daemon[1437]: E0000 00:00:1790096428.366144 1476 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5761worker2 # [ 123.780208] systemd[1]: Started Nix worker-protocol over gRPC.5762worker2 # [ 123.781192] nix-grpc-daemon[1437]: E0000 00:00:1790096428.385439 1476 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5763worker2 # [ 123.783435] nix-grpc-daemon[1437]: E0000 00:00:1790096428.385648 1476 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5764worker2 # [ 123.789082] nix-grpc-daemon[1437]: E0000 00:00:1790096428.400404 1476 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5765worker2 # [ 123.793303] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=info event=scheduler_connected addr=lb:500515766worker1 # [ 123.837881] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=info event=oidc_jwks provider=mock keys=15767worker1 # [ 123.838253] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:388765768lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5769worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5770worker1 # [ 123.847610] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05771worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5772lb: must succeed: curl -sf localhost:9901/clusters5773lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5774lb: must succeed: curl -sf localhost:9901/clusters5775lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5776worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5777worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5778(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.28 seconds)5779subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5780client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5781client # [ 124.194636] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5782client # Running as unit: rs.service; invocation ID: 29dde1ba2f3f43909d28834975bbf36a5783client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5784worker2 # [ 124.309334] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380885785worker2 # [ 124.322188] nix-daemon[859]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)5786client # [ 124.411474] nix[1404]: don't know how to build these paths:5787client # [ 124.413033] nix[1404]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5788worker1 # [ 124.379727] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05789worker1 # [ 124.385414] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:28Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15790worker1 # [ 124.425356] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401505791worker1 # [ 124.441624] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:401505792client # [ 124.499405] nix[1404]: copying 1 paths to 'lb:50051'...5793worker1 # [ 124.456113] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:401505794client # [ 124.501922] nix[1404]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5795worker1 # [ 124.470993] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5796worker1 # [ 124.632267] nix-daemon[831]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)5797worker1 # [ 124.652508] nix-daemon[831]: reaped child process 1440, status = succeeded5798lb # [ 124.655154] niks3-server[1105]: 2026/09/22 17:00:29 INFO Received uploads request method=POST path=/api/pending_closures5799worker1 # [ 124.665172] nix-grpc-daemon[1322]: time=2026-09-22T17:00:29.281Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5800worker1 # [ 124.666718] nix-grpc-daemon[1322]: time=2026-09-22T17:00:29.281Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5801lb # [ 124.683967] niks3-server[1105]: 2026/09/22 17:00:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5802worker1 # [ 124.692997] nix-grpc-daemon[1322]: time=2026-09-22T17:00:29.309Z level=INFO msg="Uploading 1 narinfos"5803lb # [ 124.696071] niks3-server[1105]: 2026/09/22 17:00:29 INFO Signed narinfos id=49 count=15804lb # [ 124.704912] niks3-server[1105]: 2026/09/22 17:00:29 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5805lb # [ 124.714241] niks3-server[1105]: 2026/09/22 17:00:29 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5806worker1 # [ 124.729016] nix-grpc-daemon[1322]: time=2026-09-22T17:00:29.345Z level=INFO msg="Upload complete. (240ms)"5807lb # [ 124.728488] niks3-server[1105]: 2026/09/22 17:00:29 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5808lb # [ 124.736616] niks3-server[1105]: 2026/09/22 17:00:29 INFO Completed upload id=495809worker1 # [ 124.737216] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=0 paths=1 nar_bytes_in=4165810lb # [ 124.738700] niks3-server[1105]: 2026/09/22 17:00:29 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5811worker1 # [ 124.748328] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:401505812worker1 # [ 124.760412] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5813worker1 # [ 124.765840] nix-daemon[831]: reaped child process 1435, status = succeeded5814client # [ 124.818693] nix[1404]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5815worker1 # [ 124.778922] nix-daemon[831]: accepted connection from pid 1243, user nix-grpc-daemon (trusted)5816lb: must succeed: curl -sf localhost:9901/clusters5817lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5818lb: must succeed: curl -sf localhost:9901/clusters5819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5820lb: must succeed: curl -sf localhost:9901/clusters5821lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5822lb: must succeed: curl -sf localhost:9901/clusters5823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5824worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5825worker2 # [ 126.942097] systemd-journald[262]: Received client request to rotate journal, rotating.5826worker2 # [ 126.961564] systemd-journald[262]: Vacuuming done, freed 0B of archived journals from /var/log/journal/695a74b3b1d743e7954e01a7d8cef4b5.5827worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5828worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5829worker1 # [ 127.039741] systemd[1]: Stopping Nix worker-protocol over gRPC...5830worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5831client: must succeed: sleep 5.05832worker1 # [ 127.373320] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:31Z level=info event=draining5833lb # [ 129.946561] envoy[919]: conn peer=[2001:db8:1::1]:36190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9563 tx=4394 ms=47525834client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5835worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5836worker1 # [ 132.488095] nix-daemon[831]: reaped child process 1444, status = succeeded5837worker1 # [ 132.558619] nix-daemon[831]: accepted connection from pid 1485, user nix-grpc-daemon (trusted)5838worker1 # [ 132.576461] nix-daemon[831]: reaped child process 1488, status = succeeded5839lb # [ 132.581646] niks3-server[1105]: 2026/09/22 17:00:37 INFO Received uploads request method=POST path=/api/pending_closures5840worker1 # [ 132.595007] nix-grpc-daemon[1322]: time=2026-09-22T17:00:37.209Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5841worker1 # [ 132.596732] nix-grpc-daemon[1322]: time=2026-09-22T17:00:37.209Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5842lb # [ 132.639704] niks3-server[1105]: 2026/09/22 17:00:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5843worker1 # [ 132.649965] nix-grpc-daemon[1322]: time=2026-09-22T17:00:37.266Z level=INFO msg="Uploading 1 narinfos"5844lb # [ 132.654408] niks3-server[1105]: 2026/09/22 17:00:37 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5845lb # [ 132.667779] niks3-server[1105]: 2026/09/22 17:00:37 INFO Signed narinfos id=50 count=15846lb # [ 132.671974] niks3-server[1105]: 2026/09/22 17:00:37 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5847lb # [ 132.683895] niks3-server[1105]: 2026/09/22 17:00:37 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5848lb # [ 132.693711] niks3-server[1105]: 2026/09/22 17:00:37 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5849lb # [ 132.699484] niks3-server[1105]: 2026/09/22 17:00:37 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5850lb # [ 132.704769] niks3-server[1105]: 2026/09/22 17:00:37 INFO Completed upload id=505851worker1 # [ 132.706128] nix-grpc-daemon[1322]: time=2026-09-22T17:00:37.322Z level=INFO msg="Upload complete. (309ms)"5852worker1 # [ 132.721187] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40150 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15853worker1 # [ 132.739970] nix-daemon[831]: reaped child process 1442, status = succeeded5854client # [ 132.793927] systemd[1]: rs.service: Deactivated successfully.5855client # [ 132.799226] systemd[1]: rs.service: Consumed 178ms CPU time over 8.600s wall clock time, 39.3M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.5856worker1 # [ 133.375643] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:37Z level=info event=signal_exit5857lb # [ 133.678413] niks3-server[1105]: 2026/09/22 17:00:38 INFO lead: released remote=192.168.1.3:590165858lb # [ 133.769934] niks3-server[1105]: 2026/09/22 17:00:38 INFO lead: acquired remote=192.168.1.4:416005859worker2 # [ 133.746873] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:38Z level=info event=scheduler_take_over5860worker2 # [ 134.646819] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:39Z level=info event=scheduler_restarting addr=lb:500515861worker1 # [ 134.681634] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=scheduler_restarting5862worker1 # [ 134.682070] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=scheduler_restarting addr=lb:500515863worker1 # [ 134.682198] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:388765864worker1 # [ 134.682311] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=worker_gone addr=192.168.1.3:500515865worker1 # [ 134.682423] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:388765866worker1 # [ 134.682591] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=worker_gone addr=192.168.1.4:500515867worker1 # [ 134.730287] nix-daemon[831]: reaped child process 1295, status = succeeded5868worker2 # [ 134.849591] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:39Z level=info event=scheduler_connected addr=lb:500515869worker1 # [ 134.880601] nix-grpc-daemon[1243]: E0000 00:00:1790096439.497209 1288 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5870worker1 # [ 134.906693] nix-grpc-daemon[1243]: ts=2026-09-22T17:00:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5871worker1 # [ 135.115442] nix-grpc-daemon[1243]: E0000 00:00:1790096439.732008 1288 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5872worker1 # [ 135.132502] nix-grpc-daemon[1243]: E0000 00:00:1790096439.749189 1288 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5873worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.24 seconds)5874client: waiting for success: ! systemctl is-active rs5875client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5876client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5877client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5878client: must fail: journalctl -u rs | grep -q 'reconnecting'5879client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5880lb: must succeed: curl -sf localhost:9901/clusters5881lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5882worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5883worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5884worker1 # [ 135.951900] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5885worker1 # [ 135.958390] systemd[1]: Stopped Nix worker-protocol over gRPC.5886worker2 # [ 135.933191] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=oidc_jwks provider=mock keys=15887worker1 # [ 135.964360] systemd[1]: nix-grpc-daemon.service: Consumed 1.602s CPU time over 30.766s wall clock time, 38.9M memory peak, 47.6K incoming IP traffic, 91.2K outgoing IP traffic.5888worker1 # [ 135.983535] systemd[1]: Starting Nix worker-protocol over gRPC...5889worker1 # [ 136.124424] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5890worker1 # [ 136.126770] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5891worker1 # [ 136.130551] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15892worker1 # [ 136.135868] nix-grpc-daemon[1495]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5893worker1 # [ 136.156132] nix-grpc-daemon[1495]: E0000 00:00:1790096440.751861 1495 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5894worker1 # [ 136.158497] nix-grpc-daemon[1495]: E0000 00:00:1790096440.755466 1495 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5895worker1 # [ 136.180665] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5896worker1 # [ 136.182847] nix-grpc-daemon[1495]: E0000 00:00:1790096440.761463 1537 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5897worker1 # [ 136.185536] systemd[1]: Started Nix worker-protocol over gRPC.5898worker1 # [ 136.186363] nix-grpc-daemon[1495]: E0000 00:00:1790096440.781701 1537 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5899worker1 # [ 136.191543] nix-grpc-daemon[1495]: E0000 00:00:1790096440.781861 1537 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5900worker1 # [ 136.195992] nix-grpc-daemon[1495]: E0000 00:00:1790096440.796485 1537 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5901worker1 # [ 136.198430] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:40Z level=info event=scheduler_connected addr=lb:500515902worker2 # [ 135.938646] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:489165903worker2 # [ 135.938808] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05904worker2 # [ 136.167200] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:489165905worker2 # [ 136.183303] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05906lb: must succeed: curl -sf localhost:9901/clusters5907lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5908worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5909worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5910(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.91 seconds)5911subtest: scheduler restart mid-build: build finishes, no double build5912client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5913client # [ 137.099796] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5914client # Running as unit: mid.service; invocation ID: c5b69b5c34544c7c819645c08dcc7d3f5915client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.08 seconds)5916worker2 # [ 137.234745] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:380885917client # [ 137.315162] nix[1440]: don't know how to build these paths:5918client # [ 137.316447] nix[1440]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5919worker2 # [ 137.254715] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:41Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05920worker2 # [ 137.261910] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:41Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15921worker2 # [ 137.299727] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380885922worker2 # [ 137.314717] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 40 ms (attempt 1/5)5923worker2 # [ 137.363982] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 195 ms (attempt 2/5)5924worker2 # [ 137.561006] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 211 ms (attempt 3/5)5925worker2 # [ 137.773858] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 489 ms (attempt 4/5)5926worker2 # [ 138.268675] nix-grpc-daemon[1437]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5927worker2 # [ 138.280720] nix-grpc-daemon[1437]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5928worker2 # [ 138.300474] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:380885929client # [ 138.385403] nix[1440]: copying 1 paths to 'lb:50051'...5930worker2 # [ 138.316091] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:380885931client # [ 138.390417] nix[1440]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5932worker2 # [ 138.331776] nix-daemon[859]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)5933worker2 # [ 138.366656] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:42Z level=info event=niks3_push_started pid=15695934worker2 # [ 138.514844] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.138Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5935worker2 # [ 138.622639] nix-daemon[859]: accepted connection from pid 1582, user nix-grpc-daemon (trusted)5936worker2 # [ 138.642608] nix-daemon[859]: reaped child process 1585, status = succeeded5937lb # [ 138.678073] niks3-server[1105]: 2026/09/22 17:00:43 INFO Received uploads request method=POST path=/api/pending_closures5938worker2 # [ 138.659335] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.282Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5939worker2 # [ 138.662711] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.282Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5940lb # [ 138.716450] niks3-server[1105]: 2026/09/22 17:00:43 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5941worker2 # [ 138.696477] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.320Z level=INFO msg="Uploading 1 narinfos"5942lb # [ 138.725409] niks3-server[1105]: 2026/09/22 17:00:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5943lb # [ 138.737733] niks3-server[1105]: 2026/09/22 17:00:43 INFO Signed narinfos id=51 count=15944lb # [ 138.747387] niks3-server[1105]: 2026/09/22 17:00:43 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5945lb # [ 138.760000] niks3-server[1105]: 2026/09/22 17:00:43 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5946lb # [ 138.767671] niks3-server[1105]: 2026/09/22 17:00:43 INFO Completed upload id=515947worker2 # [ 138.737452] nix-grpc-daemon[1569]: time=2026-09-22T17:00:43.361Z level=INFO msg="Upload complete. (218ms)"5948lb # [ 138.772584] niks3-server[1105]: 2026/09/22 17:00:43 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5949worker2 # [ 138.748008] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:38088 duration_s=0 paths=1 nar_bytes_in=4245950worker2 # [ 138.756302] nix-grpc-daemon[1437]: ts=2026-09-22T17:00:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:380885951worker2 # [ 138.767687] nix-daemon[859]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)5952worker2 # [ 138.772879] nix-daemon[859]: reaped child process 1566, status = succeeded5953worker2 # [ 138.783111] nix-daemon[859]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)5954client # [ 138.854802] nix[1440]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5955lb # [ 139.947371] envoy[919]: conn peer=[2001:db8:1::1]:47346 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1541 ms=83935956lb # [ 139.951947] envoy[919]: conn peer=[2001:db8:1::3]:33604 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2953 tx=948 ms=294745957lb: must succeed: curl -sf localhost:9901/clusters5958lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5959lb: must succeed: curl -sf localhost:9901/clusters5960lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5961worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5962worker2 # [ 141.014399] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1437 (nix-grpc-daemon) on client request.5963worker2 # [ 141.015962] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5964worker2 # [ 141.038291] systemd[1]: Starting scheduler-token.service...5965lb # [ 141.065184] niks3-server[1105]: 2026/09/22 17:00:45 INFO lead: released remote=192.168.1.4:416005966client # [ 141.117191] nix[1440]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5967worker1 # [ 141.073902] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5968client # [ 141.123435] nix[1440]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5969worker2 # [ 141.054633] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5970worker2 # [ 141.061358] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5971worker2 # [ 141.062643] systemd[1]: nix-grpc-daemon.service: Consumed 488ms CPU time over 17.463s wall clock time, 34.7M memory peak, 14.4K incoming IP traffic, 23.3K outgoing IP traffic.5972worker2 # [ 141.095435] nix-daemon[859]: reaped child process 1496, status = succeeded5973worker2 # [ 141.100418] nix-daemon[859]: reaped child process 1587, status = succeeded5974worker2 # [ 141.113601] nix-daemon[1589]: unexpected Nix daemon error: error: interrupted by the user5975lb # [ 141.144758] niks3-server[1105]: 2026/09/22 17:00:45 INFO lead: acquired remote=192.168.1.3:564505976worker2 # [ 141.119090] nix-daemon[859]: reaped child process 1589, status = failed with exit code 15977worker1 # [ 141.150279] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:45Z level=info event=scheduler_take_over5978worker2 # [ 141.133063] systemd[1]: scheduler-token.service: Deactivated successfully.5979worker2 # [ 141.135924] systemd[1]: Finished scheduler-token.service.5980worker2 # [ 141.138698] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 96ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5981worker2 # [ 141.142527] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5982worker2 # [ 141.147607] systemd[1]: Starting scheduler-token.service...5983worker2 # [ 141.188625] systemd[1]: scheduler-token.service: Deactivated successfully.5984worker2 # [ 141.192158] systemd[1]: Finished scheduler-token.service.5985worker2 # [ 141.192941] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 48ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5986worker2 # [ 141.196495] systemd[1]: Starting Nix worker-protocol over gRPC...5987worker1 # [ 141.274058] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:45Z level=info event=scheduler_connected addr=lb:500515988worker2 # [ 141.293611] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5989worker2 # [ 141.295794] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5990worker2 # [ 141.299545] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15991worker2 # [ 141.304069] nix-grpc-daemon[1632]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5992worker2 # [ 141.305539] nix-grpc-daemon[1632]: E0000 00:00:1790096445.928043 1632 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5993worker1 # [ 141.355513] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5994worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.36 seconds)5995client: must succeed: sleep 5.05996worker2 # [ 141.352065] nix-grpc-daemon[1632]: E0000 00:00:1790096445.934271 1632 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5997worker2 # [ 141.354386] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5998worker2 # [ 141.358429] nix-grpc-daemon[1632]: E0000 00:00:1790096445.944954 1671 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5999worker2 # [ 141.361401] nix-grpc-daemon[1632]: E0000 00:00:1790096445.962923 1671 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6000worker2 # [ 141.363840] nix-grpc-daemon[1632]: E0000 00:00:1790096445.963068 1671 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6001worker2 # [ 141.369235] systemd[1]: Started Nix worker-protocol over gRPC.6002worker2 # [ 141.372160] nix-grpc-daemon[1632]: E0000 00:00:1790096445.981713 1671 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6003worker2 # [ 141.374986] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=scheduler_connected addr=lb:500516004worker2 # [ 141.376360] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:45Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6005worker1 # [ 141.556422] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:500516006worker1 # [ 141.561787] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6007client # [ 141.621635] nix[1440]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting6008worker2 # [ 141.576232] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:500516009worker2 # [ 141.582857] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6010worker1 # [ 141.762405] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:500516011worker1 # [ 141.767965] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6012worker2 # [ 141.783979] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:500516013worker2 # [ 141.790623] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6014worker1 # [ 141.968752] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:500516015worker1 # [ 141.975880] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6016worker2 # [ 141.991821] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:500516017worker1 # [ 142.044235] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=oidc_jwks provider=mock keys=16018worker1 # [ 142.048703] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:429926019worker1 # [ 142.054042] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06020worker1 # [ 142.177490] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_connected addr=lb:500516021worker1 # [ 142.193645] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:46Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6022worker1 # [ 142.394710] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:47Z level=info event=scheduler_connected addr=lb:500516023worker1 # [ 142.400927] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:429926024worker1 # [ 142.405712] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06025client # [ 142.627985] nix[1440]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting6026worker1 # [ 144.596434] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:49Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06027worker1 # [ 144.596820] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:49Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16028worker2 # [ 144.617887] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353786029worker2 # [ 144.632959] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6030worker2 # [ 144.651586] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6031worker2 # [ 144.672201] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6032client # [ 144.743845] nix[1440]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6033client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6034client: waiting for success: ! systemctl is-active mid6035worker2 # [ 146.692536] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:51Z level=info event=niks3_push_started pid=17136036worker2 # [ 146.729118] nix-daemon[859]: reaped child process 1688, status = succeeded6037worker2 # [ 146.824522] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.448Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6038worker2 # [ 146.930465] nix-daemon[859]: accepted connection from pid 1725, user nix-grpc-daemon (trusted)6039worker2 # [ 146.949737] nix-daemon[859]: reaped child process 1728, status = succeeded6040lb # [ 146.979516] niks3-server[1105]: 2026/09/22 17:00:51 INFO Received uploads request method=POST path=/api/pending_closures6041worker2 # [ 146.963402] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.585Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6042worker2 # [ 146.967129] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.585Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6043lb # [ 147.034328] niks3-server[1105]: 2026/09/22 17:00:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6044worker2 # [ 147.015342] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.638Z level=INFO msg="Uploading 1 narinfos"6045lb # [ 147.051986] niks3-server[1105]: 2026/09/22 17:00:51 INFO Signed narinfos id=52 count=16046lb # [ 147.063076] niks3-server[1105]: 2026/09/22 17:00:51 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6047lb # [ 147.079991] niks3-server[1105]: 2026/09/22 17:00:51 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6048worker2 # [ 147.060309] nix-grpc-daemon[1713]: time=2026-09-22T17:00:51.684Z level=INFO msg="Upload complete. (232ms)"6049lb # [ 147.094630] niks3-server[1105]: 2026/09/22 17:00:51 INFO Completed upload id=526050lb # [ 147.104621] niks3-server[1105]: 2026/09/22 17:00:51 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6051lb # [ 147.111361] niks3-server[1105]: 2026/09/22 17:00:51 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6052worker2 # [ 147.072283] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35378 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16053lb # [ 147.116374] niks3-server[1105]: 2026/09/22 17:00:51 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6054worker2 # [ 147.091256] nix-daemon[859]: reaped child process 1686, status = succeeded6055client # [ 147.180796] systemd[1]: mid.service: Deactivated successfully.6056client # [ 147.185870] systemd[1]: mid.service: Consumed 213ms CPU time over 10.082s wall clock time, 41.4M memory peak, 9.3K incoming IP traffic, 11.3K outgoing IP traffic.6057client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6058client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6059client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)6060(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.85 seconds)6061subtest: interrupting the client stops the build on the worker6062client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6063client # [ 147.923106] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6064client # Running as unit: intr.service; invocation ID: 944ac6f3a5d64fc883c401a251bfcf6c6065client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6066worker1 # [ 148.067326] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429946067worker1 # [ 148.081552] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6068client # [ 148.142929] nix[1476]: don't know how to build these paths:6069client # [ 148.144463] nix[1476]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6070worker1 # [ 148.108868] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:52Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06071worker1 # [ 148.115727] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:52Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=16072worker1 # [ 148.161912] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946073worker1 # [ 148.176857] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 54 ms (attempt 1/5)6074worker1 # [ 148.239673] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 153 ms (attempt 2/5)6075worker1 # [ 148.394652] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 39 ms (attempt 3/5)6076worker1 # [ 148.435615] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 549 ms (attempt 4/5)6077worker1 # [ 148.986635] nix-grpc-daemon[1495]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6078worker1 # [ 148.991702] nix-grpc-daemon[1495]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6079worker1 # [ 149.008767] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946080client # [ 149.067621] nix[1476]: copying 1 paths to 'lb:50051'...6081client # [ 149.068125] nix[1476]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6082worker1 # [ 149.024346] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946083worker1 # [ 149.044353] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6084worker1 # [ 149.085426] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:53Z level=info event=niks3_push_started pid=15946085worker1 # [ 149.247045] nix-grpc-daemon[1594]: time=2026-09-22T17:00:53.863Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6086worker1 # [ 149.353852] nix-daemon[831]: accepted connection from pid 1607, user nix-grpc-daemon (trusted)6087worker1 # [ 149.369928] nix-daemon[831]: reaped child process 1610, status = succeeded6088lb # [ 149.377898] niks3-server[1105]: 2026/09/22 17:00:53 INFO Received uploads request method=POST path=/api/pending_closures6089worker1 # [ 149.392068] nix-grpc-daemon[1594]: time=2026-09-22T17:00:54.008Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6090worker1 # [ 149.393572] nix-grpc-daemon[1594]: time=2026-09-22T17:00:54.008Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6091lb # [ 149.424532] niks3-server[1105]: 2026/09/22 17:00:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6092worker1 # [ 149.434688] nix-grpc-daemon[1594]: time=2026-09-22T17:00:54.050Z level=INFO msg="Uploading 1 narinfos"6093lb # [ 149.438548] niks3-server[1105]: 2026/09/22 17:00:54 INFO Signed narinfos id=53 count=16094lb # [ 149.447204] niks3-server[1105]: 2026/09/22 17:00:54 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6095lb # [ 149.461151] niks3-server[1105]: 2026/09/22 17:00:54 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6096lb # [ 149.470002] niks3-server[1105]: 2026/09/22 17:00:54 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6097lb # [ 149.477747] niks3-server[1105]: 2026/09/22 17:00:54 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6098lb # [ 149.482533] niks3-server[1105]: 2026/09/22 17:00:54 INFO Completed upload id=536099worker1 # [ 149.483155] nix-grpc-daemon[1594]: time=2026-09-22T17:00:54.099Z level=INFO msg="Upload complete. (232ms)"6100worker1 # [ 149.493357] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=4246101worker1 # [ 149.508290] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946102worker1 # [ 149.518725] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6103worker1 # [ 149.525475] nix-daemon[831]: reaped child process 1591, status = succeeded6104worker1 # [ 149.536251] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6105client # [ 149.579118] nix[1476]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6106lb # [ 149.945496] envoy[919]: conn peer=[2001:db8:1::4]:37976 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1918 tx=311 ms=172616107lb # [ 149.950520] envoy[919]: conn peer=[2001:db8:1::1]:58048 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4314 tx=2099 ms=98526108client: must succeed: systemctl kill -s INT intr6109client # [ 151.675126] systemd[1]: intr.service: Sent signal SIGINT to main process 1476 (nix) on client request.6110worker1 # [ 151.636405] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:56Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6111client # [ 151.684740] nix[1476]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6112client # [ 151.696962] nix[1476]: error: interrupted by the user6113client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6114client # [ 151.721863] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6115client # [ 151.726550] systemd[1]: intr.service: Failed with result 'exit-code'.6116client # [ 151.728789] systemd[1]: intr.service: Consumed 167ms CPU time over 3.801s wall clock time, 43.5M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.6117worker1 # [ 151.764328] nix-daemon[831]: reaped child process 1612, status = succeeded6118worker1 # [ 151.767981] nix-daemon[1614]: unexpected Nix daemon error: error: interrupted by the user6119worker1 # [ 151.774411] nix-daemon[831]: reaped child process 1614, status = failed with exit code 16120(finished: subtest: interrupting the client stops the build on the worker, in 5.09 seconds)6121subtest: a deploy mid-build drains: the build finishes, the new generation takes over6122client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6123client # [ 153.032790] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6124client # Running as unit: sw.service; invocation ID: 07a7f0811b714c8bbe091c72bad7efe26125client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)6126client # [ 153.222720] nix[1497]: don't know how to build these paths:6127client # [ 153.224391] nix[1497]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6128worker1 # [ 153.179076] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429946129worker1 # [ 153.194387] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:57Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06130worker1 # [ 153.200312] nix-grpc-daemon[1495]: ts=2026-09-22T17:00:57Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=16131worker2 # [ 153.213476] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353786132worker2 # [ 153.229246] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 88 ms (attempt 1/5)6133worker2 # [ 153.326635] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 9 ms (attempt 2/5)6134worker2 # [ 153.345549] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 3/5)6135worker2 # [ 153.496998] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 225 ms (attempt 4/5)6136worker2 # [ 153.724270] nix-grpc-daemon[1632]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6137worker2 # [ 153.730733] nix-grpc-daemon[1632]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6138worker2 # [ 153.750010] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:353786139client # [ 153.839944] nix[1497]: copying 1 paths to 'lb:50051'...6140worker2 # [ 153.766989] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:353786141client # [ 153.840385] nix[1497]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6142worker2 # [ 153.782099] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6143worker2 # [ 153.951500] nix-daemon[859]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6144worker2 # [ 153.971747] nix-daemon[859]: reaped child process 1767, status = succeeded6145lb # [ 153.999070] niks3-server[1105]: 2026/09/22 17:00:58 INFO Received uploads request method=POST path=/api/pending_closures6146worker2 # [ 153.986613] nix-grpc-daemon[1713]: time=2026-09-22T17:00:58.609Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6147worker2 # [ 153.988221] nix-grpc-daemon[1713]: time=2026-09-22T17:00:58.609Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6148lb # [ 154.039748] niks3-server[1105]: 2026/09/22 17:00:58 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6149worker2 # [ 154.024034] nix-grpc-daemon[1713]: time=2026-09-22T17:00:58.646Z level=INFO msg="Uploading 1 narinfos"6150lb # [ 154.055073] niks3-server[1105]: 2026/09/22 17:00:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6151lb # [ 154.064902] niks3-server[1105]: 2026/09/22 17:00:58 INFO Signed narinfos id=54 count=16152lb # [ 154.067625] niks3-server[1105]: 2026/09/22 17:00:58 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6153lb # [ 154.082088] niks3-server[1105]: 2026/09/22 17:00:58 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6154lb # [ 154.090467] niks3-server[1105]: 2026/09/22 17:00:58 INFO Completed upload id=546155worker2 # [ 154.061048] nix-grpc-daemon[1713]: time=2026-09-22T17:00:58.684Z level=INFO msg="Upload complete. (261ms)"6156lb # [ 154.093045] niks3-server[1105]: 2026/09/22 17:00:58 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6157worker2 # [ 154.076526] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35378 duration_s=0 paths=1 nar_bytes_in=4166158worker2 # [ 154.088551] nix-grpc-daemon[1632]: ts=2026-09-22T17:00:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:353786159worker2 # [ 154.098214] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6160worker2 # [ 154.104498] nix-daemon[859]: reaped child process 1762, status = succeeded6161client # [ 154.186649] nix[1497]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6162worker2 # [ 154.117369] nix-daemon[859]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6163worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26164worker2 # Checking switch inhibitors... done6165worker2 # [ 155.971162] nixos[1803]: switching to system configuration /nix/store/x9vw27xc29pg9q8wzyqzk0qs9m96ksmm-nixos-system-worker2-test6166worker2 # activating the configuration...6167worker2 # [ 155.978890] systemd[1]: Stopped target Remote File Systems.6168worker2 # [ 155.979684] systemd[1]: Stopped target Local File Systems.6169worker2 # setting up /etc...6170worker2 # [ 156.537478] systemd[1]: Reload requested from client PID 1803 ('.switch-to-conf') (unit backdoor.service)...6171worker2 # [ 156.538991] systemd[1]: Reloading...6172lb # [ 159.948989] envoy[919]: conn peer=[2001:db8:1::1]:38880 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=827 ms=36126173worker2 # [ 161.367385] systemd[1]: Reloading finished in 4817 ms.6174worker2 # restarting sysinit-reactivation.target6175worker2 # [ 161.493074] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6176worker2 # [ 161.619301] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6177worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6178worker2 # [ 161.635040] systemd[1]: Reached target Reactivate sysinit units.6179worker2 # [ 161.635934] systemd[1]: Reloading D-Bus System Message Bus...6180worker2 # [ 161.641466] dbus-broker-launch[487]: Caught SIGHUP, trigger reload.6181worker2 # [ 161.642457] systemd[1]: Reloading Nix worker-protocol over gRPC...6182worker2 # [ 161.649521] dbus-broker-launch[487]: Looking up NSS user entry for 'systemd-timesync'...6183worker2 # [ 161.656593] dbus-broker-launch[487]: NSS returned no entry for 'systemd-timesync'6184worker2 # [ 161.666513] dbus-broker-launch[487]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6185worker2 # [ 161.677598] systemd[1]: Reloaded Nix worker-protocol over gRPC.6186worker2 # [ 161.694431] systemd[1]: Reloaded D-Bus System Message Bus.6187worker2 # [ 161.700838] systemd[1]: Reached target Remote File Systems.6188worker2 # [ 161.705354] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6189worker2 # [ 161.706637] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6190worker2 # [ 161.716240] systemd[1]: Reached target Local File Systems.6191worker2 # [ 161.721364] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6192worker2 # [ 161.722790] systemd[1]: Starting Create SUID/SGID Wrappers...6193worker2 # [ 161.723569] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6194worker2 # [ 161.736122] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676195worker2 # [ 161.738236] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6196worker2 # [ 161.739734] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6197worker2 # [ 161.744559] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6198worker2 # [ 162.065970] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6199worker2 # [ 162.067047] systemd[1]: Finished Create SUID/SGID Wrappers.6200worker2 # [ 162.067789] systemd[1]: Starting linger-users.service...6201worker2 # [ 162.114407] systemd[1]: linger-users.service: Deactivated successfully.6202worker2 # [ 162.115373] systemd[1]: Finished linger-users.service.6203worker2 # [ 162.319162] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:06Z level=info event=draining6204worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6205worker2 # [ 162.368289] nixos[1803]: finished switching to system configuration /nix/store/x9vw27xc29pg9q8wzyqzk0qs9m96ksmm-nixos-system-worker2-test6206worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.84 seconds)6207worker2: must succeed: pgrep -f 'read -t [9]0 x'6208worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6209client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26210client # don't know how to build these paths:6211client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6212worker2 # [ 162.655367] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:353786213worker1 # [ 162.683107] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429946214worker1 # [ 162.708234] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06215worker1 # [ 162.714976] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16216worker1 # [ 162.767041] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946217worker1 # [ 162.782060] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946218client # copying 1 paths to 'lb:50051'...6219worker1 # [ 162.798384] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946220client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6221worker1 # [ 162.822009] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6222worker1 # [ 162.982789] nix-daemon[831]: accepted connection from pid 1669, user nix-grpc-daemon (trusted)6223worker1 # [ 163.003937] nix-daemon[831]: reaped child process 1672, status = succeeded6224lb # [ 163.007383] niks3-server[1105]: 2026/09/22 17:01:07 INFO Received uploads request method=POST path=/api/pending_closures6225worker1 # [ 163.021189] nix-grpc-daemon[1594]: time=2026-09-22T17:01:07.636Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6226worker1 # [ 163.022956] nix-grpc-daemon[1594]: time=2026-09-22T17:01:07.636Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6227lb # [ 163.046339] niks3-server[1105]: 2026/09/22 17:01:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6228worker1 # [ 163.055549] nix-grpc-daemon[1594]: time=2026-09-22T17:01:07.671Z level=INFO msg="Uploading 1 narinfos"6229lb # [ 163.059126] niks3-server[1105]: 2026/09/22 17:01:07 INFO Signed narinfos id=55 count=16230lb # [ 163.067657] niks3-server[1105]: 2026/09/22 17:01:07 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6231lb # [ 163.082001] niks3-server[1105]: 2026/09/22 17:01:07 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6232worker1 # [ 163.087129] nix-grpc-daemon[1594]: time=2026-09-22T17:01:07.703Z level=INFO msg="Upload complete. (247ms)"6233lb # [ 163.089368] niks3-server[1105]: 2026/09/22 17:01:07 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6234lb # [ 163.094630] niks3-server[1105]: 2026/09/22 17:01:07 INFO Completed upload id=556235lb # [ 163.098669] niks3-server[1105]: 2026/09/22 17:01:07 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6236worker1 # [ 163.099717] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=5286237worker1 # [ 163.116333] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946238worker1 # [ 163.126527] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6239worker1 # [ 163.130568] nix-daemon[831]: reaped child process 1667, status = succeeded6240worker1 # [ 163.144846] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6241client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6242client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6243client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6244client # farm-a> LOG-farm-a-during-drain6245worker1 # [ 164.190229] nix-daemon[831]: reaped child process 1676, status = succeeded6246worker1 # [ 164.291135] nix-daemon[831]: accepted connection from pid 1692, user nix-grpc-daemon (trusted)6247worker1 # [ 164.307202] nix-daemon[831]: reaped child process 1695, status = succeeded6248lb # [ 164.313355] niks3-server[1105]: 2026/09/22 17:01:08 INFO Received uploads request method=POST path=/api/pending_closures6249worker1 # [ 164.328507] nix-grpc-daemon[1594]: time=2026-09-22T17:01:08.944Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6250worker1 # [ 164.333679] nix-grpc-daemon[1594]: time=2026-09-22T17:01:08.944Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6251lb # [ 164.357581] niks3-server[1105]: 2026/09/22 17:01:08 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6252lb # [ 164.383288] niks3-server[1105]: 2026/09/22 17:01:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6253worker1 # [ 164.392538] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.008Z level=INFO msg="Uploading 1 narinfos"6254lb # [ 164.393631] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6255lb # [ 164.403000] niks3-server[1105]: 2026/09/22 17:01:09 INFO Signed narinfos id=56 count=16256lb # [ 164.411191] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6257worker1 # [ 164.416697] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.033Z level=INFO msg="Upload complete. (256ms)"6258lb # [ 164.417237] niks3-server[1105]: 2026/09/22 17:01:09 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6259lb # [ 164.420630] niks3-server[1105]: 2026/09/22 17:01:09 INFO Completed upload id=566260lb # [ 164.425626] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6261worker1 # [ 164.429197] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16262worker1 # [ 164.452851] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06263worker1 # [ 164.458727] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16264worker1 # [ 164.463276] nix-daemon[831]: reaped child process 1674, status = succeeded6265worker1 # [ 164.493395] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946266worker1 # [ 164.505767] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946267worker1 # [ 164.520220] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.136Z level=INFO msg="All 1 paths already cached"6268client # copying 1 paths to 'lb:50051'...6269worker1 # [ 164.536341] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946270client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6271worker1 # [ 164.558816] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6272worker1 # [ 164.724481] nix-daemon[831]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6273worker1 # [ 164.743398] nix-daemon[831]: reaped child process 1704, status = succeeded6274lb # [ 164.746260] niks3-server[1105]: 2026/09/22 17:01:09 INFO Received uploads request method=POST path=/api/pending_closures6275worker1 # [ 164.757429] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.373Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6276worker1 # [ 164.759001] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.373Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6277lb # [ 164.778361] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6278worker1 # [ 164.790388] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.406Z level=INFO msg="Uploading 1 narinfos"6279lb # [ 164.792410] niks3-server[1105]: 2026/09/22 17:01:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6280lb # [ 164.801383] niks3-server[1105]: 2026/09/22 17:01:09 INFO Signed narinfos id=57 count=16281lb # [ 164.803800] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6282worker1 # [ 164.816169] nix-grpc-daemon[1594]: time=2026-09-22T17:01:09.432Z level=INFO msg="Upload complete. (240ms)"6283lb # [ 164.815935] niks3-server[1105]: 2026/09/22 17:01:09 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6284lb # [ 164.823510] niks3-server[1105]: 2026/09/22 17:01:09 INFO Completed upload id=576285lb # [ 164.825329] niks3-server[1105]: 2026/09/22 17:01:09 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6286worker1 # [ 164.828123] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=6406287worker1 # [ 164.840198] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946288worker1 # [ 164.848229] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6289worker1 # [ 164.850694] nix-daemon[831]: reaped child process 1699, status = succeeded6290client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6291worker1 # [ 164.861125] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6292client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6293client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6294client # farm-b> LOG-farm-b-during-drain6295worker1 # [ 165.752633] nix-daemon[831]: reaped child process 1708, status = succeeded6296worker1 # [ 165.839649] nix-daemon[831]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6297worker1 # [ 165.858477] nix-daemon[831]: reaped child process 1727, status = succeeded6298lb # [ 165.864978] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received uploads request method=POST path=/api/pending_closures6299worker1 # [ 165.876436] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.492Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6300worker1 # [ 165.878477] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.492Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6301lb # [ 165.900379] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6302lb # [ 165.917450] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6303worker1 # [ 165.931216] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.547Z level=INFO msg="Uploading 1 narinfos"6304lb # [ 165.930171] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6305lb # [ 165.938636] niks3-server[1105]: 2026/09/22 17:01:10 INFO Signed narinfos id=58 count=16306lb # [ 165.944979] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6307lb # [ 165.954156] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6308worker1 # [ 165.962698] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.579Z level=INFO msg="Upload complete. (289ms)"6309lb # [ 165.964461] niks3-server[1105]: 2026/09/22 17:01:10 INFO Completed upload id=586310lb # [ 165.966753] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6311worker1 # [ 165.978831] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16312worker1 # [ 166.000413] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06313worker1 # [ 166.006168] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16314worker1 # [ 166.011865] nix-daemon[831]: reaped child process 1706, status = succeeded6315worker1 # [ 166.041392] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946316worker1 # [ 166.056264] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946317worker1 # [ 166.069829] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.686Z level=INFO msg="All 2 paths already cached"6318client # copying 1 paths to 'lb:50051'...6319worker1 # [ 166.080981] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946320client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6321worker1 # [ 166.103014] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6322worker1 # [ 166.257232] nix-daemon[831]: accepted connection from pid 1733, user nix-grpc-daemon (trusted)6323worker1 # [ 166.276558] nix-daemon[831]: reaped child process 1736, status = succeeded6324lb # [ 166.280417] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received uploads request method=POST path=/api/pending_closures6325worker1 # [ 166.289816] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.906Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6326worker1 # [ 166.291379] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.906Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6327worker1 # [ 166.322076] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.938Z level=INFO msg="Uploading 1 narinfos"6328lb # [ 166.312343] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6329lb # [ 166.326456] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6330lb # [ 166.338125] niks3-server[1105]: 2026/09/22 17:01:10 INFO Signed narinfos id=59 count=16331lb # [ 166.347871] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6332worker1 # [ 166.357506] nix-grpc-daemon[1594]: time=2026-09-22T17:01:10.974Z level=INFO msg="Upload complete. (237ms)"6333lb # [ 166.358427] niks3-server[1105]: 2026/09/22 17:01:10 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6334lb # [ 166.363604] niks3-server[1105]: 2026/09/22 17:01:10 INFO Completed upload id=596335lb # [ 166.367394] niks3-server[1105]: 2026/09/22 17:01:10 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6336worker1 # [ 166.368908] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=7686337worker1 # [ 166.383518] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946338worker1 # [ 166.393137] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6339worker1 # [ 166.396315] nix-daemon[831]: reaped child process 1731, status = succeeded6340client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6341worker1 # [ 166.408260] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6342client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6343client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6344client # farm-top> LOG-farm-top-during-drain6345worker1 # [ 167.911803] nix-daemon[831]: reaped child process 1740, status = succeeded6346worker1 # [ 167.978153] nix-daemon[831]: accepted connection from pid 1756, user nix-grpc-daemon (trusted)6347worker1 # [ 167.998158] nix-daemon[831]: reaped child process 1759, status = succeeded6348lb # [ 168.003105] niks3-server[1105]: 2026/09/22 17:01:12 INFO Received uploads request method=POST path=/api/pending_closures6349worker1 # [ 168.015669] nix-grpc-daemon[1594]: time=2026-09-22T17:01:12.630Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6350worker1 # [ 168.017503] nix-grpc-daemon[1594]: time=2026-09-22T17:01:12.630Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6351lb # [ 168.047690] niks3-server[1105]: 2026/09/22 17:01:12 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6352worker1 # [ 168.061257] nix-grpc-daemon[1594]: time=2026-09-22T17:01:12.677Z level=INFO msg="Uploading 1 narinfos"6353lb # [ 168.061644] niks3-server[1105]: 2026/09/22 17:01:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6354lb # [ 168.070251] niks3-server[1105]: 2026/09/22 17:01:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6355lb # [ 168.082868] niks3-server[1105]: 2026/09/22 17:01:12 INFO Signed narinfos id=60 count=16356worker1 # [ 168.090465] nix-grpc-daemon[1594]: time=2026-09-22T17:01:12.707Z level=INFO msg="Upload complete. (268ms)"6357lb # [ 168.090578] niks3-server[1105]: 2026/09/22 17:01:12 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6358lb # [ 168.095129] niks3-server[1105]: 2026/09/22 17:01:12 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6359lb # [ 168.101585] niks3-server[1105]: 2026/09/22 17:01:12 INFO Completed upload id=606360lb # [ 168.104742] niks3-server[1105]: 2026/09/22 17:01:12 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6361worker1 # [ 168.104287] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16362worker1 # [ 168.123851] nix-daemon[831]: reaped child process 1738, status = succeeded6363client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.69 seconds)6364client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6365client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6366worker2: must succeed: pkill -f 'read -t [9]0 x'6367worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6368client: waiting for success: ! systemctl is-active sw6369worker2 # [ 168.628449] nix-daemon[859]: reaped child process 1771, status = succeeded6370worker2 # [ 168.730388] nix-daemon[859]: accepted connection from pid 1944, user nix-grpc-daemon (trusted)6371worker2 # [ 168.748224] nix-daemon[859]: reaped child process 1947, status = succeeded6372lb # [ 168.781794] niks3-server[1105]: 2026/09/22 17:01:13 INFO Received uploads request method=POST path=/api/pending_closures6373worker2 # [ 168.763807] nix-grpc-daemon[1713]: time=2026-09-22T17:01:13.386Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6374worker2 # [ 168.766396] nix-grpc-daemon[1713]: time=2026-09-22T17:01:13.386Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6375lb # [ 168.826810] niks3-server[1105]: 2026/09/22 17:01:13 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6376lb # [ 168.838584] niks3-server[1105]: 2026/09/22 17:01:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6377worker2 # [ 168.816406] nix-grpc-daemon[1713]: time=2026-09-22T17:01:13.439Z level=INFO msg="Uploading 1 narinfos"6378lb # [ 168.848713] niks3-server[1105]: 2026/09/22 17:01:13 INFO Signed narinfos id=61 count=16379lb # [ 168.851608] niks3-server[1105]: 2026/09/22 17:01:13 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6380lb # [ 168.865074] niks3-server[1105]: 2026/09/22 17:01:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6381worker2 # [ 168.846635] nix-grpc-daemon[1713]: time=2026-09-22T17:01:13.470Z level=INFO msg="Upload complete. (297ms)"6382lb # [ 168.876505] niks3-server[1105]: 2026/09/22 17:01:13 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6383lb # [ 168.880794] niks3-server[1105]: 2026/09/22 17:01:13 INFO Completed upload id=616384lb # [ 168.886857] niks3-server[1105]: 2026/09/22 17:01:13 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6385worker2 # [ 168.861274] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35378 duration_s=14 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16386worker2 # [ 168.880462] nix-daemon[859]: reaped child process 1769, status = succeeded6387client # [ 168.971779] systemd[1]: sw.service: Deactivated successfully.6388client # [ 168.976281] systemd[1]: sw.service: Consumed 190ms CPU time over 15.943s wall clock time, 41.2M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.6389worker2 # [ 169.324532] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:13Z level=info event=signal_exit6390client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6391client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6392client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6393worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6394worker2 # [ 169.580702] systemd[1]: Starting scheduler-token.service...6395worker2 # [ 169.644356] systemd[1]: scheduler-token.service: Deactivated successfully.6396worker2 # [ 169.649444] systemd[1]: Finished scheduler-token.service.6397worker2 # [ 169.650931] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 67ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6398lb # [ 169.950907] envoy[919]: conn peer=[2001:db8:1::1]:44990 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9651 tx=4412 ms=54436399lb # [ 169.955815] envoy[919]: conn peer=[2001:db8:1::1]:32774 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3012 tx=1577 ms=157326400worker1 # [ 170.368324] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:14Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:429926401worker2 # [ 170.341045] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:14Z level=info event=scheduler_restarting6402worker1 # [ 170.368673] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:14Z level=info event=worker_gone addr=192.168.1.4:500516403worker2 # [ 170.363595] nix-grpc-daemon[1632]: ts=2026-09-22T17:01:14Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6404worker2 # [ 170.391305] nix-daemon[859]: reaped child process 1684, status = succeeded6405worker2 # [ 171.067732] nix-grpc-daemon[1632]: E0000 00:00:1790096475.691518 1677 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6406worker2 # [ 171.080723] nix-grpc-daemon[1632]: E0000 00:00:1790096475.704584 1677 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6407worker2 # [ 171.360819] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6408worker2 # [ 171.367003] systemd[1]: nix-grpc-daemon.service: Consumed 829ms CPU time over 30.313s wall clock time, 36.2M memory peak, 21.6K incoming IP traffic, 31.4K outgoing IP traffic.6409worker2 # [ 171.387750] systemd[1]: Starting Nix worker-protocol over gRPC...6410worker2 # [ 171.538771] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6411worker2 # [ 171.541205] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6412worker2 # [ 171.545062] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16413worker2 # [ 171.550299] nix-grpc-daemon[1964]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6414worker2 # [ 171.551715] nix-grpc-daemon[1964]: E0000 00:00:1790096476.173640 1964 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6415worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.10 seconds)6416lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6417lb: must succeed: curl -sf localhost:9901/clusters6418worker2 # [ 171.597402] nix-grpc-daemon[1964]: E0000 00:00:1790096476.179705 1964 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6419worker2 # [ 171.599604] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6420worker2 # [ 171.602164] nix-grpc-daemon[1964]: E0000 00:00:1790096476.187489 2006 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6421worker2 # [ 171.605211] nix-grpc-daemon[1964]: E0000 00:00:1790096476.203621 2006 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6422worker2 # [ 171.607623] nix-grpc-daemon[1964]: E0000 00:00:1790096476.204309 2006 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6423worker2 # [ 171.610071] nix-grpc-daemon[1964]: E0000 00:00:1790096476.221252 2006 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6424worker2 # [ 171.615332] systemd[1]: Started Nix worker-protocol over gRPC.6425worker2 # [ 171.618545] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:16Z level=info event=scheduler_connected addr=lb:500516426worker1 # [ 171.651503] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:429926427worker1 # [ 171.657460] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06428lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6429lb: must succeed: curl -sf localhost:9901/clusters6430lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6431lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)6432lb: must succeed: curl -sf localhost:9901/clusters6433lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6434worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6435worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6436(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 19.96 seconds)6437subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6438client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16439worker2 # [ 173.085744] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:17Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472586440worker2 # [ 173.098963] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6441worker2 # [ 173.119848] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472586442worker2 # [ 173.134560] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)6443worker2 # [ 173.178243] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 33 ms (attempt 2/5)6444worker2 # [ 173.212921] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 3/5)6445worker2 # [ 173.314363] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 493 ms (attempt 4/5)6446worker2 # [ 173.809732] nix-grpc-daemon[1964]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6447worker2 # [ 173.817132] nix-grpc-daemon[1964]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6448worker1 # [ 173.893369] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06449worker1 # [ 173.900636] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16450worker1 # [ 173.906975] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06451worker1 # [ 173.912317] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06452worker1 # [ 173.946078] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946453worker1 # [ 173.961505] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946454worker1 # [ 173.976113] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946455worker1 # [ 173.990541] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6456worker1 # [ 174.147439] nix-daemon[831]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)6457worker1 # [ 174.169859] nix-daemon[831]: reaped child process 1771, status = succeeded6458lb # [ 174.167882] niks3-server[1105]: 2026/09/22 17:01:18 INFO Received uploads request method=POST path=/api/pending_closures6459worker1 # [ 174.180507] nix-grpc-daemon[1594]: time=2026-09-22T17:01:18.796Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6460worker1 # [ 174.182811] nix-grpc-daemon[1594]: time=2026-09-22T17:01:18.796Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6461lb # [ 174.196359] niks3-server[1105]: 2026/09/22 17:01:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6462worker1 # [ 174.206961] nix-grpc-daemon[1594]: time=2026-09-22T17:01:18.823Z level=INFO msg="Uploading 1 narinfos"6463lb # [ 174.210407] niks3-server[1105]: 2026/09/22 17:01:18 INFO Signed narinfos id=62 count=16464lb # [ 174.219303] niks3-server[1105]: 2026/09/22 17:01:18 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6465lb # [ 174.229732] niks3-server[1105]: 2026/09/22 17:01:18 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6466worker1 # [ 174.243961] nix-grpc-daemon[1594]: time=2026-09-22T17:01:18.860Z level=INFO msg="Upload complete. (237ms)"6467lb # [ 174.244362] niks3-server[1105]: 2026/09/22 17:01:18 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6468worker1 # [ 174.252271] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=5126469lb # [ 174.254881] niks3-server[1105]: 2026/09/22 17:01:18 INFO Completed upload id=626470lb # [ 174.259808] niks3-server[1105]: 2026/09/22 17:01:18 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6471worker1 # [ 174.268412] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946472worker1 # [ 174.276584] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6473worker1 # [ 174.281879] nix-daemon[831]: reaped child process 1766, status = succeeded6474worker1 # [ 174.295397] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6475worker1 # [ 175.727960] nix-daemon[831]: reaped child process 1775, status = succeeded6476worker1 # [ 175.823256] nix-daemon[831]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)6477worker1 # [ 175.843610] nix-daemon[831]: reaped child process 1794, status = succeeded6478lb # [ 175.846784] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received uploads request method=POST path=/api/pending_closures6479worker1 # [ 175.859558] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.473Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6480worker1 # [ 175.861194] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.473Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6481lb # [ 175.881182] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6482lb # [ 175.896722] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6483lb # [ 175.908291] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6484worker1 # [ 175.916493] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.533Z level=INFO msg="Uploading 1 narinfos"6485lb # [ 175.918632] niks3-server[1105]: 2026/09/22 17:01:20 INFO Signed narinfos id=63 count=16486lb # [ 175.924310] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6487worker1 # [ 175.937648] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.554Z level=INFO msg="Upload complete. (231ms)"6488lb # [ 175.936698] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6489lb # [ 175.943296] niks3-server[1105]: 2026/09/22 17:01:20 INFO Completed upload id=636490lb # [ 175.947160] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6491worker1 # [ 175.950920] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16492worker1 # [ 175.973449] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16493worker1 # [ 175.978714] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946494worker1 # [ 175.984354] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06495worker1 # [ 175.989554] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946496worker1 # [ 175.994178] nix-daemon[831]: reaped child process 1773, status = succeeded6497worker1 # [ 175.997659] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946498worker1 # [ 176.007471] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6499worker1 # [ 176.136173] nix-daemon[831]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6500worker1 # [ 176.155973] nix-daemon[831]: reaped child process 1803, status = succeeded6501lb # [ 176.157293] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received uploads request method=POST path=/api/pending_closures6502worker1 # [ 176.167090] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.783Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6503worker1 # [ 176.168675] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.783Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6504lb # [ 176.186953] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6505worker1 # [ 176.196850] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.813Z level=INFO msg="Uploading 1 narinfos"6506lb # [ 176.201582] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6507lb # [ 176.212004] niks3-server[1105]: 2026/09/22 17:01:20 INFO Signed narinfos id=64 count=16508worker1 # [ 176.220068] nix-grpc-daemon[1594]: time=2026-09-22T17:01:20.836Z level=INFO msg="Upload complete. (202ms)"6509lb # [ 176.221100] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6510worker1 # [ 176.226871] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=5126511lb # [ 176.227944] niks3-server[1105]: 2026/09/22 17:01:20 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6512lb # [ 176.233463] niks3-server[1105]: 2026/09/22 17:01:20 INFO Completed upload id=646513lb # [ 176.235982] niks3-server[1105]: 2026/09/22 17:01:20 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6514worker1 # [ 176.239959] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946515worker1 # [ 176.243481] nix-daemon[831]: reaped child process 1798, status = succeeded6516worker1 # [ 176.250644] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6517worker1 # [ 176.264138] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6518worker1 # [ 176.933042] nix-daemon[831]: reaped child process 1807, status = succeeded6519worker1 # [ 177.051065] nix-daemon[831]: accepted connection from pid 1823, user nix-grpc-daemon (trusted)6520worker1 # [ 177.069941] nix-daemon[831]: reaped child process 1826, status = succeeded6521lb # [ 177.075028] niks3-server[1105]: 2026/09/22 17:01:21 INFO Received uploads request method=POST path=/api/pending_closures6522worker1 # [ 177.088428] nix-grpc-daemon[1594]: time=2026-09-22T17:01:21.702Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6523worker1 # [ 177.090024] nix-grpc-daemon[1594]: time=2026-09-22T17:01:21.702Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6524lb # [ 177.117602] niks3-server[1105]: 2026/09/22 17:01:21 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6525worker1 # [ 177.134511] nix-grpc-daemon[1594]: time=2026-09-22T17:01:21.750Z level=INFO msg="Uploading 1 narinfos"6526lb # [ 177.132948] niks3-server[1105]: 2026/09/22 17:01:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6527lb # [ 177.142520] niks3-server[1105]: 2026/09/22 17:01:21 INFO Signed narinfos id=65 count=16528lb # [ 177.151421] niks3-server[1105]: 2026/09/22 17:01:21 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6529lb # [ 177.159861] niks3-server[1105]: 2026/09/22 17:01:21 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6530lb # [ 177.169386] niks3-server[1105]: 2026/09/22 17:01:21 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6531lb # [ 177.174314] niks3-server[1105]: 2026/09/22 17:01:21 INFO Completed upload id=656532worker1 # [ 177.175636] nix-grpc-daemon[1594]: time=2026-09-22T17:01:21.792Z level=INFO msg="Upload complete. (300ms)"6533lb # [ 177.177743] niks3-server[1105]: 2026/09/22 17:01:21 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6534worker1 # [ 177.196153] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16535worker1 # [ 177.216361] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16536worker1 # [ 177.220754] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946537worker1 # [ 177.224888] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06538worker1 # [ 177.228592] nix-daemon[831]: reaped child process 1805, status = succeeded6539worker1 # [ 177.234780] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946540worker1 # [ 177.248407] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946541worker1 # [ 177.260441] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6542worker1 # [ 177.407754] nix-daemon[831]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)6543worker1 # [ 177.425116] nix-daemon[831]: reaped child process 1835, status = succeeded6544lb # [ 177.429165] niks3-server[1105]: 2026/09/22 17:01:22 INFO Received uploads request method=POST path=/api/pending_closures6545worker1 # [ 177.438776] nix-grpc-daemon[1594]: time=2026-09-22T17:01:22.055Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6546worker1 # [ 177.440436] nix-grpc-daemon[1594]: time=2026-09-22T17:01:22.055Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6547lb # [ 177.457543] niks3-server[1105]: 2026/09/22 17:01:22 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6548worker1 # [ 177.468140] nix-grpc-daemon[1594]: time=2026-09-22T17:01:22.084Z level=INFO msg="Uploading 1 narinfos"6549lb # [ 177.470266] niks3-server[1105]: 2026/09/22 17:01:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6550lb # [ 177.482048] niks3-server[1105]: 2026/09/22 17:01:22 INFO Signed narinfos id=66 count=16551lb # [ 177.490171] niks3-server[1105]: 2026/09/22 17:01:22 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6552worker1 # [ 177.498628] nix-grpc-daemon[1594]: time=2026-09-22T17:01:22.115Z level=INFO msg="Upload complete. (223ms)"6553lb # [ 177.500966] niks3-server[1105]: 2026/09/22 17:01:22 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6554lb # [ 177.506666] niks3-server[1105]: 2026/09/22 17:01:22 INFO Completed upload id=666555worker1 # [ 177.508238] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=5126556lb # [ 177.510054] niks3-server[1105]: 2026/09/22 17:01:22 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6557worker1 # [ 177.519202] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946558worker1 # [ 177.526266] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6559worker1 # [ 177.531232] nix-daemon[831]: reaped child process 1830, status = succeeded6560worker1 # [ 177.541844] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6561worker1 # [ 179.210746] nix-daemon[831]: reaped child process 1839, status = succeeded6562worker1 # [ 179.325784] nix-daemon[831]: accepted connection from pid 1855, user nix-grpc-daemon (trusted)6563worker1 # [ 179.343649] nix-daemon[831]: reaped child process 1858, status = succeeded6564lb # [ 179.352451] niks3-server[1105]: 2026/09/22 17:01:23 INFO Received uploads request method=POST path=/api/pending_closures6565worker1 # [ 179.363048] nix-grpc-daemon[1594]: time=2026-09-22T17:01:23.978Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6566worker1 # [ 179.364807] nix-grpc-daemon[1594]: time=2026-09-22T17:01:23.978Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6567lb # [ 179.392322] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6568worker1 # [ 179.403403] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.019Z level=INFO msg="Uploading 1 narinfos"6569lb # [ 179.407797] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6570lb # [ 179.418045] niks3-server[1105]: 2026/09/22 17:01:24 INFO Signed narinfos id=67 count=16571lb # [ 179.423095] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6572worker1 # [ 179.433243] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.049Z level=INFO msg="Upload complete. (279ms)"6573lb # [ 179.432423] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6574lb # [ 179.444445] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6575lb # [ 179.450788] niks3-server[1105]: 2026/09/22 17:01:24 INFO Completed upload id=676576worker1 # [ 179.450321] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16577lb # [ 179.453311] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6578worker1 # [ 179.469618] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16579worker1 # [ 179.481370] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946580worker1 # [ 179.494142] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06581worker1 # [ 179.504212] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946582worker1 # [ 179.507626] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6583worker1 # [ 179.513881] nix-daemon[831]: reaped child process 1837, status = succeeded6584worker1 # [ 179.518303] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.097Z level=INFO msg="All 1 paths already cached"6585worker1 # [ 179.522209] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946586worker1 # [ 179.643809] nix-daemon[831]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6587worker1 # [ 179.661594] nix-daemon[831]: reaped child process 1867, status = succeeded6588lb # [ 179.665448] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received uploads request method=POST path=/api/pending_closures6589worker1 # [ 179.675229] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.291Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6590worker1 # [ 179.677161] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.291Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6591lb # [ 179.694411] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6592worker1 # [ 179.701969] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.318Z level=INFO msg="Uploading 1 narinfos"6593lb # [ 179.707650] niks3-server[1105]: 2026/09/22 17:01:24 INFO Signed narinfos id=68 count=16594lb # [ 179.716170] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6595lb # [ 179.730951] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6596worker1 # [ 179.737431] nix-grpc-daemon[1594]: time=2026-09-22T17:01:24.354Z level=INFO msg="Upload complete. (224ms)"6597lb # [ 179.740378] niks3-server[1105]: 2026/09/22 17:01:24 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6598lb # [ 179.745699] niks3-server[1105]: 2026/09/22 17:01:24 INFO Completed upload id=686599worker1 # [ 179.748287] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=6246600lb # [ 179.750246] niks3-server[1105]: 2026/09/22 17:01:24 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6601worker1 # [ 179.760336] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946602worker1 # [ 179.770402] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6603worker1 # [ 179.773645] nix-daemon[831]: reaped child process 1862, status = succeeded6604worker1 # [ 179.788426] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6605lb # [ 179.951592] envoy[919]: conn peer=[2001:db8:1::4]:51512 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3760 tx=489 ms=289786606worker1 # [ 181.267994] nix-daemon[831]: reaped child process 1871, status = succeeded6607worker1 # [ 181.363679] nix-daemon[831]: accepted connection from pid 1887, user nix-grpc-daemon (trusted)6608worker1 # [ 181.382080] nix-daemon[831]: reaped child process 1890, status = succeeded6609lb # [ 181.388874] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received uploads request method=POST path=/api/pending_closures6610worker1 # [ 181.404086] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.017Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6611worker1 # [ 181.406039] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.017Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6612worker1 # [ 181.447611] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.063Z level=INFO msg="Uploading 1 narinfos"6613lb # [ 181.436533] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6614lb # [ 181.453919] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6615lb # [ 181.464694] niks3-server[1105]: 2026/09/22 17:01:26 INFO Signed narinfos id=69 count=16616lb # [ 181.467249] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6617lb # [ 181.479232] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6618lb # [ 181.484370] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6619lb # [ 181.487473] niks3-server[1105]: 2026/09/22 17:01:26 INFO Completed upload id=696620lb # [ 181.489689] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6621worker1 # [ 181.485023] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.101Z level=INFO msg="Upload complete. (285ms)"6622worker1 # [ 181.504214] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16623worker1 # [ 181.519880] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16624worker1 # [ 181.528905] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946625worker1 # [ 181.532856] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06626worker1 # [ 181.537447] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946627worker1 # [ 181.541743] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.157Z level=INFO msg="All 1 paths already cached"6628worker1 # [ 181.544292] nix-daemon[831]: reaped child process 1869, status = succeeded6629worker1 # [ 181.547718] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946630worker1 # [ 181.556568] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6631worker1 # [ 181.685893] nix-daemon[831]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)6632worker1 # [ 181.702027] nix-daemon[831]: reaped child process 1899, status = succeeded6633lb # [ 181.707646] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received uploads request method=POST path=/api/pending_closures6634worker1 # [ 181.717145] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.333Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6635worker1 # [ 181.718655] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.333Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6636lb # [ 181.738051] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6637worker1 # [ 181.744983] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.361Z level=INFO msg="Uploading 1 narinfos"6638lb # [ 181.753559] niks3-server[1105]: 2026/09/22 17:01:26 INFO Signed narinfos id=70 count=16639lb # [ 181.759808] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6640lb # [ 181.770671] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6641worker1 # [ 181.779807] nix-grpc-daemon[1594]: time=2026-09-22T17:01:26.396Z level=INFO msg="Upload complete. (214ms)"6642lb # [ 181.779762] niks3-server[1105]: 2026/09/22 17:01:26 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6643lb # [ 181.787282] niks3-server[1105]: 2026/09/22 17:01:26 INFO Completed upload id=706644worker1 # [ 181.787005] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=6246645lb # [ 181.790344] niks3-server[1105]: 2026/09/22 17:01:26 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6646worker1 # [ 181.802530] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946647worker1 # [ 181.808664] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6648worker1 # [ 181.813781] nix-daemon[831]: reaped child process 1894, status = succeeded6649worker1 # [ 181.826582] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6650worker1 # [ 182.922085] nix-daemon[831]: reaped child process 1903, status = succeeded6651worker1 # [ 183.001850] nix-daemon[831]: accepted connection from pid 1919, user nix-grpc-daemon (trusted)6652worker1 # [ 183.021433] nix-daemon[831]: reaped child process 1922, status = succeeded6653lb # [ 183.026888] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received uploads request method=POST path=/api/pending_closures6654worker1 # [ 183.039523] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.654Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6655worker1 # [ 183.041308] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.655Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6656lb # [ 183.068146] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6657worker1 # [ 183.078090] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.694Z level=INFO msg="Uploading 1 narinfos"6658lb # [ 183.081230] niks3-server[1105]: 2026/09/22 17:01:27 INFO Signed narinfos id=71 count=16659lb # [ 183.091308] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6660lb # [ 183.099427] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6661lb # [ 183.110287] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6662worker1 # [ 183.111209] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.727Z level=INFO msg="Upload complete. (274ms)"6663lb # [ 183.117977] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6664lb # [ 183.122493] niks3-server[1105]: 2026/09/22 17:01:27 INFO Completed upload id=716665lb # [ 183.126369] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6666worker1 # [ 183.128426] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16667worker1 # [ 183.148367] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16668worker1 # [ 183.156501] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946669worker1 # [ 183.162235] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06670worker1 # [ 183.168208] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946671worker1 # [ 183.173659] nix-daemon[831]: reaped child process 1901, status = succeeded6672worker1 # [ 183.175741] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.789Z level=INFO msg="All 1 paths already cached"6673worker1 # [ 183.188326] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946674worker1 # [ 183.197442] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6675worker1 # [ 183.318740] nix-daemon[831]: accepted connection from pid 1928, user nix-grpc-daemon (trusted)6676worker1 # [ 183.337260] nix-daemon[831]: reaped child process 1931, status = succeeded6677lb # [ 183.341398] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received uploads request method=POST path=/api/pending_closures6678worker1 # [ 183.350875] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.967Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6679worker1 # [ 183.352519] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.967Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6680lb # [ 183.372211] niks3-server[1105]: 2026/09/22 17:01:27 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6681worker1 # [ 183.381843] nix-grpc-daemon[1594]: time=2026-09-22T17:01:27.998Z level=INFO msg="Uploading 1 narinfos"6682lb # [ 183.383793] niks3-server[1105]: 2026/09/22 17:01:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6683lb # [ 183.396212] niks3-server[1105]: 2026/09/22 17:01:27 INFO Signed narinfos id=72 count=16684lb # [ 183.398893] niks3-server[1105]: 2026/09/22 17:01:28 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6685lb # [ 183.411981] niks3-server[1105]: 2026/09/22 17:01:28 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6686lb # [ 183.416759] niks3-server[1105]: 2026/09/22 17:01:28 INFO Completed upload id=726687worker1 # [ 183.416950] nix-grpc-daemon[1594]: time=2026-09-22T17:01:28.033Z level=INFO msg="Upload complete. (214ms)"6688lb # [ 183.419361] niks3-server[1105]: 2026/09/22 17:01:28 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6689worker1 # [ 183.424224] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=6246690worker1 # [ 183.436896] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946691worker1 # [ 183.448798] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6692worker1 # [ 183.454069] nix-daemon[831]: reaped child process 1926, status = succeeded6693worker1 # [ 183.467055] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6694worker1 # [ 184.977273] nix-daemon[831]: reaped child process 1935, status = succeeded6695worker1 # [ 185.048855] nix-daemon[831]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6696worker1 # [ 185.069766] nix-daemon[831]: reaped child process 1954, status = succeeded6697lb # [ 185.074505] niks3-server[1105]: 2026/09/22 17:01:29 INFO Received uploads request method=POST path=/api/pending_closures6698worker1 # [ 185.084468] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.699Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6699worker1 # [ 185.086256] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.700Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6700lb # [ 185.116555] niks3-server[1105]: 2026/09/22 17:01:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6701worker1 # [ 185.128297] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.744Z level=INFO msg="Uploading 1 narinfos"6702lb # [ 185.132329] niks3-server[1105]: 2026/09/22 17:01:29 INFO Signed narinfos id=73 count=16703lb # [ 185.134806] niks3-server[1105]: 2026/09/22 17:01:29 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6704lb # [ 185.148146] niks3-server[1105]: 2026/09/22 17:01:29 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6705lb # [ 185.150916] niks3-server[1105]: 2026/09/22 17:01:29 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6706lb # [ 185.156747] niks3-server[1105]: 2026/09/22 17:01:29 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6707lb # [ 185.163120] niks3-server[1105]: 2026/09/22 17:01:29 INFO Completed upload id=736708worker1 # [ 185.161999] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.778Z level=INFO msg="Upload complete. (284ms)"6709lb # [ 185.165102] niks3-server[1105]: 2026/09/22 17:01:29 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6710worker1 # [ 185.181161] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16711worker1 # [ 185.197213] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16712worker1 # [ 185.208586] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946713worker1 # [ 185.214068] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06714worker1 # [ 185.228237] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946715worker1 # [ 185.231346] nix-daemon[831]: reaped child process 1933, status = succeeded6716worker1 # [ 185.237664] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6717worker1 # [ 185.243182] nix-grpc-daemon[1594]: time=2026-09-22T17:01:29.829Z level=INFO msg="All 2 paths already cached"6718worker1 # [ 185.247582] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946719worker1 # [ 185.365627] nix-daemon[831]: accepted connection from pid 1960, user nix-grpc-daemon (trusted)6720worker1 # [ 185.385232] nix-daemon[831]: reaped child process 1963, status = succeeded6721lb # [ 185.382174] niks3-server[1105]: 2026/09/22 17:01:29 INFO Received uploads request method=POST path=/api/pending_closures6722worker1 # [ 185.391836] nix-grpc-daemon[1594]: time=2026-09-22T17:01:30.008Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6723worker1 # [ 185.393817] nix-grpc-daemon[1594]: time=2026-09-22T17:01:30.008Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6724lb # [ 185.409016] niks3-server[1105]: 2026/09/22 17:01:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6725worker1 # [ 185.415418] nix-grpc-daemon[1594]: time=2026-09-22T17:01:30.032Z level=INFO msg="Uploading 1 narinfos"6726lb # [ 185.421329] niks3-server[1105]: 2026/09/22 17:01:30 INFO Signed narinfos id=74 count=16727lb # [ 185.429961] niks3-server[1105]: 2026/09/22 17:01:30 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6728lb # [ 185.442324] niks3-server[1105]: 2026/09/22 17:01:30 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6729worker1 # [ 185.451080] nix-grpc-daemon[1594]: time=2026-09-22T17:01:30.067Z level=INFO msg="Upload complete. (214ms)"6730lb # [ 185.450742] niks3-server[1105]: 2026/09/22 17:01:30 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6731lb # [ 185.454563] niks3-server[1105]: 2026/09/22 17:01:30 INFO Completed upload id=746732lb # [ 185.459646] niks3-server[1105]: 2026/09/22 17:01:30 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6733worker1 # [ 185.461027] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=7526734worker1 # [ 185.472845] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946735worker1 # [ 185.481622] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6736worker1 # [ 185.484764] nix-daemon[831]: reaped child process 1958, status = succeeded6737worker1 # [ 185.496201] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6738worker1 # [ 186.601404] nix-daemon[831]: reaped child process 1967, status = succeeded6739worker1 # [ 186.669665] nix-daemon[831]: accepted connection from pid 1983, user nix-grpc-daemon (trusted)6740worker1 # [ 186.690948] nix-daemon[831]: reaped child process 1986, status = succeeded6741lb # [ 186.695973] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received uploads request method=POST path=/api/pending_closures6742worker1 # [ 186.710162] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.325Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6743worker1 # [ 186.711887] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.325Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6744lb # [ 186.745052] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6745worker1 # [ 186.752278] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.368Z level=INFO msg="Uploading 1 narinfos"6746lb # [ 186.755529] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6747lb # [ 186.765274] niks3-server[1105]: 2026/09/22 17:01:31 INFO Signed narinfos id=75 count=16748worker1 # [ 186.766319] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.383Z level=INFO msg="Upload complete. (260ms)"6749lb # [ 186.767265] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6750lb # [ 186.777434] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6751worker1 # [ 186.776332] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16752lb # [ 186.781160] niks3-server[1105]: 2026/09/22 17:01:31 INFO Completed upload id=756753lb # [ 186.783953] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6754lb # [ 186.786706] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6755worker1 # [ 186.794087] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16756worker1 # [ 186.798762] nix-daemon[831]: reaped child process 1965, status = succeeded6757worker1 # [ 186.821443] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946758worker1 # [ 186.836115] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946759worker1 # [ 186.850691] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.467Z level=INFO msg="All 2 paths already cached"6760worker1 # [ 186.860367] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946761worker1 # [ 186.873857] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6762worker1 # [ 187.017867] nix-daemon[831]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)6763worker1 # [ 187.036453] nix-daemon[831]: reaped child process 1993, status = succeeded6764lb # [ 187.043524] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received uploads request method=POST path=/api/pending_closures6765worker1 # [ 187.054269] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.670Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6766worker1 # [ 187.055777] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.670Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6767lb # [ 187.077335] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6768worker1 # [ 187.084094] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.700Z level=INFO msg="Uploading 1 narinfos"6769lb # [ 187.091393] niks3-server[1105]: 2026/09/22 17:01:31 INFO Signed narinfos id=76 count=16770lb # [ 187.099196] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6771worker1 # [ 187.113110] nix-grpc-daemon[1594]: time=2026-09-22T17:01:31.729Z level=INFO msg="Upload complete. (225ms)"6772lb # [ 187.112437] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6773worker1 # [ 187.120364] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=7526774lb # [ 187.122211] niks3-server[1105]: 2026/09/22 17:01:31 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6775lb # [ 187.126991] niks3-server[1105]: 2026/09/22 17:01:31 INFO Completed upload id=766776lb # [ 187.129672] niks3-server[1105]: 2026/09/22 17:01:31 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6777worker1 # [ 187.132503] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946778worker1 # [ 187.143985] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6779worker1 # [ 187.149229] nix-daemon[831]: reaped child process 1988, status = succeeded6780worker1 # [ 187.160139] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6781worker1 # [ 188.637003] nix-daemon[831]: reaped child process 1997, status = succeeded6782worker1 # [ 188.748138] nix-daemon[831]: accepted connection from pid 2013, user nix-grpc-daemon (trusted)6783worker1 # [ 188.765788] nix-daemon[831]: reaped child process 2016, status = succeeded6784lb # [ 188.773130] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received uploads request method=POST path=/api/pending_closures6785worker1 # [ 188.784543] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.399Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6786worker1 # [ 188.786881] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.399Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6787lb # [ 188.808794] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6788lb # [ 188.830243] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6789worker1 # [ 188.837021] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.453Z level=INFO msg="Uploading 1 narinfos"6790lb # [ 188.843000] niks3-server[1105]: 2026/09/22 17:01:33 INFO Signed narinfos id=77 count=16791lb # [ 188.847637] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6792lb # [ 188.855461] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6793lb # [ 188.861629] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6794lb # [ 188.866998] niks3-server[1105]: 2026/09/22 17:01:33 INFO Completed upload id=776795worker1 # [ 188.866137] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.482Z level=INFO msg="Upload complete. (292ms)"6796worker1 # [ 188.881222] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16797worker1 # [ 188.896293] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16798worker1 # [ 188.905524] nix-daemon[831]: reaped child process 1995, status = succeeded6799worker1 # [ 188.921291] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946800worker1 # [ 188.936153] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946801worker1 # [ 188.949882] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.566Z level=INFO msg="All 2 paths already cached"6802worker1 # [ 188.961076] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:429946803worker1 # [ 188.972959] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6804worker1 # [ 189.124869] nix-daemon[831]: accepted connection from pid 2020, user nix-grpc-daemon (trusted)6805worker1 # [ 189.141895] nix-daemon[831]: reaped child process 2023, status = succeeded6806lb # [ 189.148828] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received uploads request method=POST path=/api/pending_closures6807worker1 # [ 189.159779] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.776Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6808worker1 # [ 189.161503] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.776Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6809lb # [ 189.179435] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6810worker1 # [ 189.187995] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.804Z level=INFO msg="Uploading 1 narinfos"6811lb # [ 189.193895] niks3-server[1105]: 2026/09/22 17:01:33 INFO Signed narinfos id=78 count=16812lb # [ 189.201140] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6813lb # [ 189.218036] niks3-server[1105]: 2026/09/22 17:01:33 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6814worker1 # [ 189.224596] nix-grpc-daemon[1594]: time=2026-09-22T17:01:33.841Z level=INFO msg="Upload complete. (237ms)"6815lb # [ 189.226607] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6816lb # [ 189.233816] niks3-server[1105]: 2026/09/22 17:01:33 INFO Completed upload id=786817worker1 # [ 189.234155] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=0 paths=1 nar_bytes_in=7526818lb # [ 189.236459] niks3-server[1105]: 2026/09/22 17:01:33 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6819worker1 # [ 189.250510] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:429946820worker1 # [ 189.258141] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6821worker1 # [ 189.263883] nix-daemon[831]: reaped child process 2018, status = succeeded6822worker1 # [ 189.276299] nix-daemon[831]: accepted connection from pid 1495, user nix-grpc-daemon (trusted)6823worker1 # [ 190.787487] nix-daemon[831]: reaped child process 2027, status = succeeded6824worker1 # [ 190.862741] nix-daemon[831]: accepted connection from pid 2043, user nix-grpc-daemon (trusted)6825worker1 # [ 190.880302] nix-daemon[831]: reaped child process 2046, status = succeeded6826lb # [ 190.885306] niks3-server[1105]: 2026/09/22 17:01:35 INFO Received uploads request method=POST path=/api/pending_closures6827worker1 # [ 190.898506] nix-grpc-daemon[1594]: time=2026-09-22T17:01:35.514Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6828worker1 # [ 190.901036] nix-grpc-daemon[1594]: time=2026-09-22T17:01:35.514Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6829lb # [ 190.927227] niks3-server[1105]: 2026/09/22 17:01:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6830worker1 # [ 190.935647] nix-grpc-daemon[1594]: time=2026-09-22T17:01:35.552Z level=INFO msg="Uploading 1 narinfos"6831lb # [ 190.942877] niks3-server[1105]: 2026/09/22 17:01:35 INFO Signed narinfos id=79 count=16832lb # [ 190.951435] niks3-server[1105]: 2026/09/22 17:01:35 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6833lb # [ 190.961814] niks3-server[1105]: 2026/09/22 17:01:35 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6834lb # [ 190.972160] niks3-server[1105]: 2026/09/22 17:01:35 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6835lb # [ 190.976523] niks3-server[1105]: 2026/09/22 17:01:35 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6836lb # [ 190.981796] niks3-server[1105]: 2026/09/22 17:01:35 INFO Completed upload id=796837worker1 # [ 190.981805] nix-grpc-daemon[1594]: time=2026-09-22T17:01:35.598Z level=INFO msg="Upload complete. (291ms)"6838lb # [ 190.984842] niks3-server[1105]: 2026/09/22 17:01:35 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6839worker1 # [ 190.997252] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42994 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16840worker1 # [ 191.015569] nix-daemon[831]: reaped child process 2025, status = succeeded6841client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 18.18 seconds)6842worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6843worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6844client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6845client # [ 191.230616] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6846client # Running as unit: gpu.service; invocation ID: 05864bdb2e3a4213838aa445ceeb09326847client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6848lb: must succeed: curl -sf localhost:9901/clusters6849lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6850lb: must succeed: curl -sf localhost:9901/clusters6851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6852worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6853client # [ 191.429507] nix[1576]: don't know how to build these paths:6854client # [ 191.430481] nix[1576]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6855worker2 # [ 191.357139] nix-grpc-daemon[1964]: ts=2026-09-22T17:01:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:472586856worker1 # [ 191.393233] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946857worker1 # [ 191.414674] nix-grpc-daemon[1495]: ts=2026-09-22T17:01:36Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06858client # [ 191.458453] nix[1576]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6859worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6860lb: must succeed: curl -sf localhost:9901/clusters6861lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6862lb: must succeed: curl -sf localhost:9901/clusters6863lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6864worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6865worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6866client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6867lb # [ 199.955997] envoy[919]: conn peer=[2001:db8:1::1]:53712 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27153 tx=12353 ms=179006868client # [ 221.463673] nix[1576]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6869client # [ 221.476151] nix[1576]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6870client # [ 221.514726] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6871client # [ 221.522988] systemd[1]: gpu.service: Failed with result 'exit-code'.6872client # [ 221.526822] systemd[1]: gpu.service: Consumed 189ms CPU time over 30.289s wall clock time, 35.4M memory peak, 5.2K incoming IP traffic, 5K outgoing IP traffic.6873client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.08 seconds)6874client: must succeed: journalctl -u gpu -o cat6875client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6876lb: must succeed: curl -sf localhost:9901/clusters6877lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6878lb: must succeed: curl -sf localhost:9901/clusters6879lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6880worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6881worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6882(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 49.01 seconds)6883subtest: low disk drains a worker and builds go to the other6884worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6885worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.16 seconds)6886worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6887worker1 # [ 223.187100] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:07Z level=info event=unhealthy reason=min_free6888worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.15 seconds)6889client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26890worker1 # [ 224.439345] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:429946891client # don't know how to build these paths:6892client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6893worker1 # [ 224.454587] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:429946894worker1 # [ 224.492267] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:09Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06895worker1 # [ 224.499539] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:09Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16896worker2 # [ 224.518349] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472586897worker2 # [ 224.536341] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472586898client # copying 1 paths to 'lb:50051'...6899client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6900worker2 # [ 224.553557] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472586901worker2 # [ 224.569809] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6902worker2 # [ 224.600627] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=info event=niks3_push_started pid=20246903worker2 # [ 224.753714] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.377Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6904worker2 # [ 224.865215] nix-daemon[859]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)6905worker2 # [ 224.883983] nix-daemon[859]: reaped child process 2040, status = succeeded6906lb # [ 224.922697] niks3-server[1105]: 2026/09/22 17:02:09 INFO Received uploads request method=POST path=/api/pending_closures6907worker2 # [ 224.905092] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.527Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6908worker2 # [ 224.906603] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.527Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6909lb # [ 224.970235] niks3-server[1105]: 2026/09/22 17:02:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6910worker2 # [ 224.952078] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.575Z level=INFO msg="Uploading 1 narinfos"6911lb # [ 224.985815] niks3-server[1105]: 2026/09/22 17:02:09 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6912lb # [ 224.996219] niks3-server[1105]: 2026/09/22 17:02:09 INFO Signed narinfos id=80 count=16913lb # [ 224.998461] niks3-server[1105]: 2026/09/22 17:02:09 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6914lb # [ 225.010277] niks3-server[1105]: 2026/09/22 17:02:09 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6915lb # [ 225.015079] niks3-server[1105]: 2026/09/22 17:02:09 INFO Completed upload id=806916worker2 # [ 224.986375] nix-grpc-daemon[2024]: time=2026-09-22T17:02:09.610Z level=INFO msg="Upload complete. (228ms)"6917lb # [ 225.017292] niks3-server[1105]: 2026/09/22 17:02:09 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6918worker2 # [ 225.000199] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=0 paths=1 nar_bytes_in=5126919worker2 # [ 225.012079] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472586920worker2 # [ 225.017799] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6921worker2 # [ 225.022778] nix-daemon[859]: reaped child process 2022, status = succeeded6922worker2 # [ 225.032725] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6923client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6924client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6925client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6926client # farm-a> LOG-farm-a-drain6927worker2 # [ 226.112826] nix-daemon[859]: reaped child process 2044, status = succeeded6928worker2 # [ 226.215564] nix-daemon[859]: accepted connection from pid 2060, user nix-grpc-daemon (trusted)6929worker2 # [ 226.236984] nix-daemon[859]: reaped child process 2063, status = succeeded6930lb # [ 226.265034] niks3-server[1105]: 2026/09/22 17:02:10 INFO Received uploads request method=POST path=/api/pending_closures6931worker2 # [ 226.248554] nix-grpc-daemon[2024]: time=2026-09-22T17:02:10.871Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6932worker2 # [ 226.250257] nix-grpc-daemon[2024]: time=2026-09-22T17:02:10.871Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6933lb # [ 226.317668] niks3-server[1105]: 2026/09/22 17:02:10 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6934worker2 # [ 226.299662] nix-grpc-daemon[2024]: time=2026-09-22T17:02:10.923Z level=INFO msg="Uploading 1 narinfos"6935lb # [ 226.329971] niks3-server[1105]: 2026/09/22 17:02:10 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6936lb # [ 226.340427] niks3-server[1105]: 2026/09/22 17:02:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6937lb # [ 226.343460] niks3-server[1105]: 2026/09/22 17:02:10 INFO Signed narinfos id=81 count=16938lb # [ 226.349443] niks3-server[1105]: 2026/09/22 17:02:10 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6939lb # [ 226.357579] niks3-server[1105]: 2026/09/22 17:02:10 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete6940worker2 # [ 226.327874] nix-grpc-daemon[2024]: time=2026-09-22T17:02:10.951Z level=INFO msg="Upload complete. (283ms)"6941lb # [ 226.360629] niks3-server[1105]: 2026/09/22 17:02:10 INFO Completed upload id=816942lb # [ 226.362951] niks3-server[1105]: 2026/09/22 17:02:10 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6943worker2 # [ 226.341187] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=16944worker1 # [ 226.377136] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:10Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06945worker1 # [ 226.382588] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:10Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16946worker2 # [ 226.361230] nix-daemon[859]: reaped child process 2042, status = succeeded6947worker2 # [ 226.392137] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472586948worker2 # [ 226.407569] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472586949client # copying 1 paths to 'lb:50051'...6950worker2 # [ 226.422520] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.046Z level=INFO msg="All 1 paths already cached"6951client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6952worker2 # [ 226.428925] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472586953worker2 # [ 226.442979] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6954worker2 # [ 226.605478] nix-daemon[859]: accepted connection from pid 2067, user nix-grpc-daemon (trusted)6955worker2 # [ 226.625797] nix-daemon[859]: reaped child process 2070, status = succeeded6956lb # [ 226.652568] niks3-server[1105]: 2026/09/22 17:02:11 INFO Received uploads request method=POST path=/api/pending_closures6957worker2 # [ 226.632940] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.256Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6958worker2 # [ 226.634389] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.256Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6959lb # [ 226.679474] niks3-server[1105]: 2026/09/22 17:02:11 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6960worker2 # [ 226.665870] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.289Z level=INFO msg="Uploading 1 narinfos"6961lb # [ 226.693926] niks3-server[1105]: 2026/09/22 17:02:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign6962lb # [ 226.706273] niks3-server[1105]: 2026/09/22 17:02:11 INFO Signed narinfos id=82 count=16963lb # [ 226.713393] niks3-server[1105]: 2026/09/22 17:02:11 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6964lb # [ 226.727300] niks3-server[1105]: 2026/09/22 17:02:11 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete6965worker2 # [ 226.706504] nix-grpc-daemon[2024]: time=2026-09-22T17:02:11.330Z level=INFO msg="Upload complete. (246ms)"6966lb # [ 226.735825] niks3-server[1105]: 2026/09/22 17:02:11 INFO Completed upload id=826967lb # [ 226.739638] niks3-server[1105]: 2026/09/22 17:02:11 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6968worker2 # [ 226.716268] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=0 paths=1 nar_bytes_in=6326969worker2 # [ 226.731571] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472586970worker2 # [ 226.740331] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6971worker2 # [ 226.745210] nix-daemon[859]: reaped child process 2065, status = succeeded6972client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6973worker2 # [ 226.758282] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)6974client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6975client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6976client # farm-b> LOG-farm-b-drain6977worker2 # [ 228.185538] nix-daemon[859]: reaped child process 2074, status = succeeded6978worker2 # [ 228.282697] nix-daemon[859]: accepted connection from pid 2090, user nix-grpc-daemon (trusted)6979worker2 # [ 228.301110] nix-daemon[859]: reaped child process 2093, status = succeeded6980lb # [ 228.338354] niks3-server[1105]: 2026/09/22 17:02:12 INFO Received uploads request method=POST path=/api/pending_closures6981worker2 # [ 228.321148] nix-grpc-daemon[2024]: time=2026-09-22T17:02:12.944Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6982worker2 # [ 228.322628] nix-grpc-daemon[2024]: time=2026-09-22T17:02:12.944Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6983lb # [ 228.382371] niks3-server[1105]: 2026/09/22 17:02:12 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6984worker2 # [ 228.365223] nix-grpc-daemon[2024]: time=2026-09-22T17:02:12.988Z level=INFO msg="Uploading 1 narinfos"6985lb # [ 228.396480] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6986lb # [ 228.404766] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign6987lb # [ 228.414350] niks3-server[1105]: 2026/09/22 17:02:13 INFO Signed narinfos id=83 count=16988worker2 # [ 228.388097] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.012Z level=INFO msg="Upload complete. (216ms)"6989lb # [ 228.418816] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6990lb # [ 228.422959] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete6991lb # [ 228.426195] niks3-server[1105]: 2026/09/22 17:02:13 INFO Completed upload id=836992lb # [ 228.427608] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6993worker2 # [ 228.396931] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=16994worker1 # [ 228.436192] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:13Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06995worker2 # [ 228.415330] nix-daemon[859]: reaped child process 2072, status = succeeded6996worker1 # [ 228.441741] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:13Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16997worker2 # [ 228.452190] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472586998worker2 # [ 228.468128] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:472586999worker2 # [ 228.484087] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.107Z level=INFO msg="All 2 paths already cached"7000client # copying 1 paths to 'lb:50051'...7001client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7002worker2 # [ 228.495568] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:472587003worker2 # [ 228.509329] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)7004worker2 # [ 228.670239] nix-daemon[859]: accepted connection from pid 2097, user nix-grpc-daemon (trusted)7005worker2 # [ 228.692274] nix-daemon[859]: reaped child process 2100, status = succeeded7006lb # [ 228.719724] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received uploads request method=POST path=/api/pending_closures7007worker2 # [ 228.701295] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.324Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7008worker2 # [ 228.702753] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.325Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7009worker2 # [ 228.723865] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.347Z level=INFO msg="Uploading 1 narinfos"7010lb # [ 228.752084] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7011lb # [ 228.765803] niks3-server[1105]: 2026/09/22 17:02:13 INFO Signed narinfos id=84 count=17012lb # [ 228.773573] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7013lb # [ 228.786778] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7014worker2 # [ 228.763345] nix-grpc-daemon[2024]: time=2026-09-22T17:02:13.387Z level=INFO msg="Upload complete. (237ms)"7015lb # [ 228.795745] niks3-server[1105]: 2026/09/22 17:02:13 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7016lb # [ 228.800917] niks3-server[1105]: 2026/09/22 17:02:13 INFO Completed upload id=847017worker2 # [ 228.772345] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=0 paths=1 nar_bytes_in=7607018lb # [ 228.804565] niks3-server[1105]: 2026/09/22 17:02:13 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7019worker2 # [ 228.788756] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:472587020worker2 # [ 228.795714] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)7021worker2 # [ 228.801505] nix-daemon[859]: reaped child process 2095, status = succeeded7022client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7023worker2 # [ 228.814888] nix-daemon[859]: accepted connection from pid 1964, user nix-grpc-daemon (trusted)7024client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7025client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7026client # farm-top> LOG-farm-top-drain7027lb # [ 229.956659] envoy[919]: conn peer=[2001:db8:1::1]:51160 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=579 ms=300657028worker2 # [ 230.100278] nix-daemon[859]: reaped child process 2104, status = succeeded7029worker2 # [ 230.191121] nix-daemon[859]: accepted connection from pid 2120, user nix-grpc-daemon (trusted)7030worker2 # [ 230.210648] nix-daemon[859]: reaped child process 2123, status = succeeded7031lb # [ 230.239345] niks3-server[1105]: 2026/09/22 17:02:14 INFO Received uploads request method=POST path=/api/pending_closures7032worker2 # [ 230.220535] nix-grpc-daemon[2024]: time=2026-09-22T17:02:14.843Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7033worker2 # [ 230.222059] nix-grpc-daemon[2024]: time=2026-09-22T17:02:14.843Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7034lb # [ 230.281127] niks3-server[1105]: 2026/09/22 17:02:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7035worker2 # [ 230.259852] nix-grpc-daemon[2024]: time=2026-09-22T17:02:14.883Z level=INFO msg="Uploading 1 narinfos"7036lb # [ 230.297538] niks3-server[1105]: 2026/09/22 17:02:14 INFO Signed narinfos id=85 count=17037lb # [ 230.308470] niks3-server[1105]: 2026/09/22 17:02:14 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7038lb # [ 230.317591] niks3-server[1105]: 2026/09/22 17:02:14 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7039worker2 # [ 230.300624] nix-grpc-daemon[2024]: time=2026-09-22T17:02:14.924Z level=INFO msg="Upload complete. (275ms)"7040lb # [ 230.330189] niks3-server[1105]: 2026/09/22 17:02:14 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7041lb # [ 230.335788] niks3-server[1105]: 2026/09/22 17:02:14 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7042lb # [ 230.340896] niks3-server[1105]: 2026/09/22 17:02:14 INFO Completed upload id=857043worker2 # [ 230.313144] nix-grpc-daemon[1964]: ts=2026-09-22T17:02:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:47258 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=17044lb # [ 230.343431] niks3-server[1105]: 2026/09/22 17:02:14 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7045worker2 # [ 230.330850] nix-daemon[859]: reaped child process 2102, status = succeeded7046client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 6.19 seconds)7047client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7048client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7049worker1: must succeed: rm /nix/.rw-store/fill7050worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.15 seconds)7051worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7052worker1 # [ 231.190516] nix-grpc-daemon[1495]: ts=2026-09-22T17:02:15Z level=info event=healthy reason=min_free7053worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7054(finished: subtest: low disk drains a worker and builds go to the other, in 10.03 seconds)7055(finished: run the VM test script, in 232.81 seconds)7056test script finished in 233.06s7057cleanup7058kill QemuMachine (pid 49)7059client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7060client # [2026-09-22T17:02:16Z INFO virtiofsd] Client disconnected, shutting down7061client # [2026-09-22T17:02:16Z INFO virtiofsd] Client disconnected, shutting down7062client # [2026-09-22T17:02:16Z INFO virtiofsd] Client disconnected, shutting down7063kill QemuMachine (pid 50)7064lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7065lb # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7066lb # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7067lb # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7068kill QemuMachine (pid 51)7069worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7070worker1 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7071worker1 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7072worker1 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7073kill QemuMachine (pid 52)7074worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7075worker2 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7076worker2 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7077worker2 # [2026-09-22T17:02:17Z INFO virtiofsd] Client disconnected, shutting down7078(finished: cleanup, in 1.11 seconds)7079additionally exposed symbols:7080 client, lb, worker1, worker2,7081 vlan1,7082 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh