vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #507
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Formatting '/build/vm-state-worker1/tmp.xFQxbUuM1N', fmt=raw size=107374182414worker1 # mke2fs 1.47.4 (6-Mar-2025)15lb # Disk image does not exist, creating the virtualisation disk image...16worker1 # Discarding device blocks: 0/262144 done17worker2 # Disk image does not exist, creating the virtualisation disk image...18worker2: QEMU running (pid 52)19client # Disk image does not exist, creating the virtualisation disk image...20client: QEMU running (pid 49)21lb: QEMU running (pid 50)22lb # Formatting '/build/vm-state-lb/tmp.HSDNWdOjbH', fmt=raw size=107374182423client # Formatting '/build/vm-state-client/tmp.19klYC5uS8', fmt=raw size=107374182424worker2 # Formatting '/build/vm-state-worker2/tmp.de7i2iGXpJ', fmt=raw size=107374182425lb # mke2fs 1.47.4 (6-Mar-2025)26client # mke2fs 1.47.4 (6-Mar-2025)27worker2 # mke2fs 1.47.4 (6-Mar-2025)28lb # Discarding device blocks: 0/262144 done29client # Discarding device blocks: 0/262144 done30worker2 # Discarding device blocks: 0/262144 done31lb # Creating filesystem with 262144 4k blocks and 65536 inodes32client # Creating filesystem with 262144 4k blocks and 65536 inodes33worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes34lb # Filesystem UUID: 43eeae1b-fdb2-40d2-be15-b4a34cfa4c0d35client # Filesystem UUID: b8f4d9fb-9828-4d03-a7aa-d8b67cf041ea36worker2 # Filesystem UUID: fbdf700a-8a52-42f9-bc71-8eb921ea047937lb # Superblock backups stored on blocks:38client # Superblock backups stored on blocks:39worker2 # Superblock backups stored on blocks:40lb # 32768, 98304, 163840, 22937641client # 32768, 98304, 163840, 22937642worker2 # 32768, 98304, 163840, 22937643lb # 44client # 45worker2 # 46lb # Allocating group tables: 0/8 done47client # Allocating group tables: 0/8 done48worker2 # Allocating group tables: 0/8 done49lb # Writing inode tables: 0/8 done50client # Writing inode tables: 0/8 done51worker2 # Writing inode tables: 0/8 done52lb # Creating journal (8192 blocks): done53client # Creating journal (8192 blocks): done54worker2 # Creating journal (8192 blocks): done55lb # Writing superblocks and filesystem accounting information: 0/8 done56client # Writing superblocks and filesystem accounting information: 0/8 done57worker2 # Writing superblocks and filesystem accounting information: 0/8 done58lb # 59client # 60worker2 # 61lb # Virtualisation disk image created.62client # Virtualisation disk image created.63worker2 # Virtualisation disk image created.64lb # Starting virtiofs daemons...65client # Starting virtiofs daemons...66worker2 # Starting virtiofs daemons...67lb # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)68client # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69worker2 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)70lb # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71client # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72worker2 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether73lb # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74client # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...75worker2 # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...76lb # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77client # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78worker2 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79lb # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...80client # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether81worker2 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82lb # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...83client # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...84worker2 # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...85lb # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86client # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)87worker2 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)88lb # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether89client # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether90worker2 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether91lb # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...92client # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker2 # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...94lb # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests95client # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests96worker2 # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests97lb # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests98client # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests99worker2 # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests100lb # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests101client # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests102worker2 # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests103worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes104worker1: QEMU running (pid 51)105worker1 # Filesystem UUID: 4e060669-9984-4396-8f73-f143f0318f33106worker1 # Superblock backups stored on blocks:107worker1 # 32768, 98304, 163840, 229376108worker1 # 109worker1 # Allocating group tables: 0/8 done110(finished: start all VMs, in 0.51 seconds)111worker1 # Writing inode tables: 0/8 done112client: waiting for unit network-addresses-eth1.service113worker1 # Creating journal (8192 blocks): done114client: waiting for the VM to finish booting115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker1 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker1 # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...125worker1 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126worker1 # [2026-09-22T16:53:39Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127worker1 # [2026-09-22T16:53:39Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-22T16:53:39Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133worker1 # [ 0.000000] KASLR enabled134worker1 # [ 0.000000] random: crng init done135worker1 # [ 0.000000] Machine model: linux,dummy-virt136worker1 # [ 0.000000] efi: UEFI not found.137worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT138worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]139worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]140worker1 # [ 0.000000] Zone ranges:141worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]142worker1 # [ 0.000000] DMA32 empty143worker1 # [ 0.000000] Normal empty144worker1 # [ 0.000000] Device empty145worker1 # [ 0.000000] Movable zone start for each node146worker1 # [ 0.000000] Early memory node ranges147worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]148worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]149worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000150worker1 # [ 0.000000] psci: probing for conduit method from DT.151worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.152worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs153worker1 # [ 0.000000] psci: Trusted OS migration not required154worker1 # [ 0.000000] psci: SMC Calling Convention v1.1155worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)156worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296157worker1 # [ 0.000000] Detected PIPT I-cache on CPU0158worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)159worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface160worker1 # [ 0.000000] CPU features: detected: Spectre-v4161worker1 # [ 0.000000] CPU features: detected: Spectre-BHB162worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38163worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23164worker1 # [ 0.000000] alternatives: applying boot alternatives165lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]166worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yms5bvxhm2a2zidkzddvi0xxaqsl6a35-nixos-system-worker1-test/init regInfo=/nix/store/vnq6mm2mwl8jxprixhfy00r13gsa7l45-closure-info/registration console=ttyAMA0,115200n8 console=tty0167lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026168lb # [ 0.000000] KASLR enabled169worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/vnq6mm2mwl8jxprixhfy00r13gsa7l45-closure-info/registration", will be passed to user space.170lb # [ 0.000000] random: crng init done171lb # [ 0.000000] Machine model: linux,dummy-virt172worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes173lb # [ 0.000000] efi: UEFI not found.174lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT175worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)176lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]177worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)178lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]179worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB180lb # [ 0.000000] Zone ranges:181worker1 # [ 0.000000] software IO TLB: area num 1.182lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]183worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB184lb # [ 0.000000] DMA32 empty185lb # [ 0.000000] Normal empty186lb # [ 0.000000] Device empty187worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)188lb # [ 0.000000] Movable zone start for each node189worker1 # [ 0.000000] Fallback order for Node 0: 0190lb # [ 0.000000] Early memory node ranges191worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216192lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]193worker1 # [ 0.000000] Policy zone: DMA194lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]195worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off196lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000197worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1198lb # [ 0.000000] psci: probing for conduit method from DT.199worker1 # [ 0.000000] allocated 3145728 bytes of page_ext200lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.201worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages202lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs203worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups204lb # [ 0.000000] psci: Trusted OS migration not required205worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.206lb # [ 0.000000] psci: SMC Calling Convention v1.1207worker1 # [ 0.000000] rcu: RCU event tracing is enabled.208lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)209worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.210worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.211lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296212worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.213lb # [ 0.000000] Detected PIPT I-cache on CPU0214worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.215lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)216worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.217lb # [ 0.000000] CPU features: detected: GICv3 CPU interface218worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1219lb # [ 0.000000] CPU features: detected: Spectre-v4220lb # [ 0.000000] CPU features: detected: Spectre-BHB221worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.222lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38223lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23224worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.225lb # [ 0.000000] alternatives: applying boot alternatives226worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.227worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0228worker1 # [ 0.000000] GICv3: 256 SPIs implemented229worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented230worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq231worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI232lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qvl0xxkr124pdkzdvanphvcpy7197xra-nixos-system-lb-test/init regInfo=/nix/store/4y9byqv3ifqd7wzrdp7iwk6bnwmvvvvc-closure-info/registration console=ttyAMA0,115200n8 console=tty0233worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0234worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000235worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]236worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]237lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4y9byqv3ifqd7wzrdp7iwk6bnwmvvvvc-closure-info/registration", will be passed to user space.238worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)239worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026240worker2 # [ 0.000000] KASLR enabled241lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes242worker2 # [ 0.000000] random: crng init done243worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)244worker2 # [ 0.000000] Machine model: linux,dummy-virt245lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)246worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000247worker2 # [ 0.000000] efi: UEFI not found.248lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)249worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000250worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT251lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB252worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.253worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]254lb # [ 0.000000] software IO TLB: area num 1.255worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]256lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB257worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns258worker2 # [ 0.000000] Zone ranges:259worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).260worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]261lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)262worker2 # [ 0.000000] DMA32 empty263lb # [ 0.000000] Fallback order for Node 0: 0264worker2 # [ 0.000000] Normal empty265worker2 # [ 0.000000] Device empty266worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns267lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216268worker2 # [ 0.000000] Movable zone start for each node269lb # [ 0.000000] Policy zone: DMA270worker2 # [ 0.000000] Early memory node ranges271worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns272lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off273client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]274worker1 # [ 0.000032] arm-pv: using stolen time PV275worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]276lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1277worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]278lb # [ 0.000000] allocated 3145728 bytes of page_ext279worker1 # [ 0.000650] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)280client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026281worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000282client # [ 0.000000] KASLR enabled283lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages284worker1 # [ 0.000814] Console: colour dummy device 80x25285worker2 # [ 0.000000] psci: probing for conduit method from DT.286client # [ 0.000000] random: crng init done287lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups288worker1 # [ 0.000822] printk: legacy console [tty0] enabled289worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.290client # [ 0.000000] Machine model: linux,dummy-virt291lb # [ 0.000000] rcu: Hierarchical RCU implementation.292client # [ 0.000000] efi: UEFI not found.293worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs294lb # [ 0.000000] rcu: RCU event tracing is enabled.295worker1 # [ 0.001015] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)296worker2 # [ 0.000000] psci: Trusted OS migration not required297client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT298worker1 # [ 0.001021] pid_max: default: 32768 minimum: 301299lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.300worker2 # [ 0.000000] psci: SMC Calling Convention v1.1301lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.302client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]303worker1 # [ 0.001101] LSM: initializing lsm=capability,landlock,yama,bpf,ima304worker1 # [ 0.001258] landlock: Up and running.305lb # [ 0.000000] Rude variant of Tasks RCU enabled.306worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)307client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]308worker1 # [ 0.001261] Yama: becoming mindful.309client # [ 0.000000] Zone ranges:310lb # [ 0.000000] Tracing variant of Tasks RCU enabled.311worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296312worker1 # [ 0.001907] LSM support for eBPF active313worker2 # [ 0.000000] Detected PIPT I-cache on CPU0314client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]315lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.316client # [ 0.000000] DMA32 empty317worker1 # [ 0.002077] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)318client # [ 0.000000] Normal empty319lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1320worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)321client # [ 0.000000] Device empty322worker1 # [ 0.002109] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)323client # [ 0.000000] Movable zone start for each node324worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface325worker1 # [ 0.003998] rcu: Hierarchical SRCU implementation.326client # [ 0.000000] Early memory node ranges327lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.328worker2 # [ 0.000000] CPU features: detected: Spectre-v4329worker1 # [ 0.004027] rcu: Max phase no-delay instances is 1000.330worker2 # [ 0.000000] CPU features: detected: Spectre-BHB331client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]332lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.333worker1 # [ 0.005276] fsl-mc MSI: its@8080000 domain created334worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38335worker1 # [ 0.005367] EFI services will not be available.336client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]337worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23338lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.339worker1 # [ 0.005431] smp: Bringing up secondary CPUs ...340client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000341worker2 # [ 0.000000] alternatives: applying boot alternatives342worker1 # [ 0.005440] smp: Brought up 1 node, 1 CPU343client # [ 0.000000] psci: probing for conduit method from DT.344lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0345worker1 # [ 0.005443] SMP: Total of 1 processors activated.346lb # [ 0.000000] GICv3: 256 SPIs implemented347client # [ 0.000000] psci: PSCIv1.3 detected in firmware.348worker1 # [ 0.005448] CPU: All CPU(s) started at EL1349lb # [ 0.000000] GICv3: 0 Extended SPIs implemented350client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs351lb # [ 0.000000] Root IRQ handler: gic_handle_irq352worker1 # [ 0.005459] CPU features: detected: Branch Target Identification353client # [ 0.000000] psci: Trusted OS migration not required354lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI355client # [ 0.000000] psci: SMC Calling Convention v1.1356worker1 # [ 0.005464] CPU features: detected: ARMv8.4 Translation Table Level357lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0358lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000359client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)360worker1 # [ 0.005467] CPU features: detected: Instruction cache invalidation not required for I/D coherence361worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/j24yd38mc0ckic4mmr56r3ka0bx8c52x-nixos-system-worker2-test/init regInfo=/nix/store/jw51lbh9icv07w9h8il9pf744i6nicyg-closure-info/registration console=ttyAMA0,115200n8 console=tty0362lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]363client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296364worker1 # [ 0.005471] CPU features: detected: Data cache clean to the PoU not required for I/D coherence365client # [ 0.000000] Detected PIPT I-cache on CPU0366lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)367worker1 # [ 0.005474] CPU features: detected: Common not Private translations368worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/jw51lbh9icv07w9h8il9pf744i6nicyg-closure-info/registration", will be passed to user space.369client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)370worker1 # [ 0.005477] CPU features: detected: CRC32 instructions371worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes372lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)373client # [ 0.000000] CPU features: detected: GICv3 CPU interface374worker1 # [ 0.005480] CPU features: detected: Data cache clean to Point of Deep Persistence375client # [ 0.000000] CPU features: detected: Spectre-v4376lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000377worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)378worker1 # [ 0.005484] CPU features: detected: Data cache clean to Point of Persistence379client # [ 0.000000] CPU features: detected: Spectre-BHB380lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000381worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)382client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38383worker1 # [ 0.005487] CPU features: detected: Data independent timing control (DIT)384lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.385worker1 # [ 0.005490] CPU features: detected: E0PD386worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB387client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23388worker2 # [ 0.000000] software IO TLB: area num 1.389worker1 # [ 0.005492] CPU features: detected: Enhanced Counter Virtualization390client # [ 0.000000] alternatives: applying boot alternatives391lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns392worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB393worker1 # [ 0.005495] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)394lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).395worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)396worker1 # [ 0.005499] CPU features: detected: Enhanced Virtualization Traps397worker2 # [ 0.000000] Fallback order for Node 0: 0398worker1 # [ 0.005502] CPU features: detected: Fine Grained Traps399lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns400worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216401worker2 # [ 0.000000] Policy zone: DMA402worker1 # [ 0.005505] CPU features: detected: Generic authentication (architected QARMA5 algorithm)403lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns404lb # [ 0.000030] arm-pv: using stolen time PV405worker1 # [ 0.005509] CPU features: detected: RCpc load-acquire (LDAPR)406worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off407client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/c7dwh1z1cl7zl3zmm2gm23k4b20rxnfn-nixos-system-client-test/init regInfo=/nix/store/bvnhgdmv5qvfh2jgky9d95hczimkx039-closure-info/registration console=ttyAMA0,115200n8 console=tty0408worker1 # [ 0.005512] CPU features: detected: LSE atomic instructions409worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1410lb # [ 0.000511] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)411worker1 # [ 0.005515] CPU features: detected: Privileged Access Never412worker2 # [ 0.000000] allocated 3145728 bytes of page_ext413lb # [ 0.000669] Console: colour dummy device 80x25414worker1 # [ 0.005518] CPU features: detected: PMUv3415client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/bvnhgdmv5qvfh2jgky9d95hczimkx039-closure-info/registration", will be passed to user space.416worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages417lb # [ 0.000676] printk: legacy console [tty0] enabled418worker1 # [ 0.005521] CPU features: detected: RAS Extension Support419worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups420client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes421worker1 # [ 0.005524] CPU features: detected: RASv1p1 Extension Support422worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.423lb # [ 0.000866] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)424worker2 # [ 0.000000] rcu: RCU event tracing is enabled.425client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)426worker1 # [ 0.005526] CPU features: detected: Random Number Generator427lb # [ 0.000873] pid_max: default: 32768 minimum: 301428worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.429worker1 # [ 0.005529] CPU features: detected: Speculation barrier (SB)430client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)431lb # [ 0.000953] LSM: initializing lsm=capability,landlock,yama,bpf,ima432worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.433worker1 # [ 0.005531] CPU features: detected: Stage-2 Force Write-Back434lb # [ 0.001148] landlock: Up and running.435client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB436worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.437lb # [ 0.001151] Yama: becoming mindful.438client # [ 0.000000] software IO TLB: area num 1.439worker1 # [ 0.005534] CPU features: detected: TLB range maintenance instructions440lb # [ 0.001688] LSM support for eBPF active441worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.442client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB443worker1 # [ 0.005538] CPU features: detected: Speculative Store Bypassing Safe (SSBS)444lb # [ 0.001829] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)445worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.446worker1 # [ 0.005578] alternatives: applying system-wide alternatives447client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)448lb # [ 0.001862] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)449worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1450client # [ 0.000000] Fallback order for Node 0: 0451worker1 # [ 0.008543] CPU features: detected: BBM Level 2 without TLB conflict abort452lb # [ 0.003796] rcu: Hierarchical SRCU implementation.453worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.454client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216455lb # [ 0.003802] rcu: Max phase no-delay instances is 1000.456client # [ 0.000000] Policy zone: DMA457lb # [ 0.005052] fsl-mc MSI: its@8080000 domain created458worker1 # [ 0.008751] Memory: 1405672K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)459worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.460client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off461lb # [ 0.005149] EFI services will not be available.462worker1 # [ 0.009123] devtmpfs: initialized463lb # [ 0.005240] smp: Bringing up secondary CPUs ...464client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1465worker1 # [ 0.010949] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)466worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.467lb # [ 0.005247] smp: Brought up 1 node, 1 CPU468client # [ 0.000000] allocated 3145728 bytes of page_ext469worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0470lb # [ 0.005250] SMP: Total of 1 processors activated.471client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages472worker1 # [ 0.010972] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).473worker2 # [ 0.000000] GICv3: 256 SPIs implemented474lb # [ 0.005256] CPU: All CPU(s) started at EL1475client # [ 0.000000] ftrace: allocated 294 pages with 4 groups476worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented477worker1 # [ 0.011142] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL478lb # [ 0.005266] CPU features: detected: Branch Target Identification479worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq480worker1 # [ 0.011146] 0 pages in range for non-PLT usage481client # [ 0.000000] rcu: Hierarchical RCU implementation.482lb # [ 0.005270] CPU features: detected: ARMv8.4 Translation Table Level483worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI484client # [ 0.000000] rcu: RCU event tracing is enabled.485worker1 # [ 0.011148] 508272 pages in range for PLT usage486worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0487worker1 # [ 0.011262] pinctrl core: initialized pinctrl subsystem488client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.489lb # [ 0.005273] CPU features: detected: Instruction cache invalidation not required for I/D coherence490worker1 # [ 0.012014] DMI not present or invalid.491worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000492client # [ 0.000000] Trampoline variant of Tasks RCU enabled.493worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]494worker1 # [ 0.015787] NET: Registered PF_NETLINK/PF_ROUTE protocol family495client # [ 0.000000] Rude variant of Tasks RCU enabled.496lb # [ 0.005277] CPU features: detected: Data cache clean to the PoU not required for I/D coherence497client # [ 0.000000] Tracing variant of Tasks RCU enabled.498worker1 # [ 0.017988] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations499lb # [ 0.005281] CPU features: detected: Common not Private translations500worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)501lb # [ 0.005284] CPU features: detected: CRC32 instructions502client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.503worker1 # [ 0.018174] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations504worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)505lb # [ 0.005287] CPU features: detected: Data cache clean to Point of Deep Persistence506client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1507worker1 # [ 0.018376] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations508worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000509lb # [ 0.005290] CPU features: detected: Data cache clean to Point of Persistence510worker1 # [ 0.018397] audit: initializing netlink subsys (disabled)511client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.512worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000513lb # [ 0.005293] CPU features: detected: Data independent timing control (DIT)514worker1 # [ 0.018943] thermal_sys: Registered thermal governor 'fair_share'515client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.516worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.517worker1 # [ 0.018945] thermal_sys: Registered thermal governor 'bang_bang'518worker1 # [ 0.018949] thermal_sys: Registered thermal governor 'step_wise'519client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.520worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns521worker1 # [ 0.018951] thermal_sys: Registered thermal governor 'user_space'522client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0523worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).524client # [ 0.000000] GICv3: 256 SPIs implemented525worker1 # [ 0.018957] thermal_sys: Registered thermal governor 'power_allocator'526client # [ 0.000000] GICv3: 0 Extended SPIs implemented527worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns528client # [ 0.000000] Root IRQ handler: gic_handle_irq529worker1 # [ 0.018989] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1530worker1 # [ 0.018998] cpuidle: using governor ladder531client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI532worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns533worker1 # [ 0.019004] cpuidle: using governor menu534client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0535worker2 # [ 0.000029] arm-pv: using stolen time PV536worker1 # [ 0.019205] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.537client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000538worker1 # [ 0.019221] ASID allocator initialised with 65536 entries539client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]540worker2 # [ 0.000383] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)541worker1 # [ 0.020453] Serial: AMBA PL011 UART driver542worker2 # [ 0.000538] Console: colour dummy device 80x25543client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)544worker2 # [ 0.000546] printk: legacy console [tty0] enabled545worker1 # [ 0.025839] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1546worker1 # [ 0.025998] printk: console [ttyAMA0] enabled547worker2 # [ 0.000745] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)548client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)549lb # [ 0.005296] CPU features: detected: E0PD550worker2 # [ 0.000751] pid_max: default: 32768 minimum: 301551client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000552lb # [ 0.005299] CPU features: detected: Enhanced Counter Virtualization553worker2 # [ 0.000841] LSM: initializing lsm=capability,landlock,yama,bpf,ima554client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000555worker2 # [ 0.000974] landlock: Up and running.556lb # [ 0.005302] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)557worker2 # [ 0.000977] Yama: becoming mindful.558client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.559lb # [ 0.005306] CPU features: detected: Enhanced Virtualization Traps560worker2 # [ 0.001404] LSM support for eBPF active561lb # [ 0.005309] CPU features: detected: Fine Grained Traps562worker2 # [ 0.001527] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)563client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns564lb # [ 0.005312] CPU features: detected: Generic authentication (architected QARMA5 algorithm)565client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).566worker2 # [ 0.001559] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)567lb # [ 0.005317] CPU features: detected: RCpc load-acquire (LDAPR)568worker2 # [ 0.003348] rcu: Hierarchical SRCU implementation.569lb # [ 0.005320] CPU features: detected: LSE atomic instructions570worker2 # [ 0.003352] rcu: Max phase no-delay instances is 1000.571client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns572lb # [ 0.005323] CPU features: detected: Privileged Access Never573worker2 # [ 0.004589] fsl-mc MSI: its@8080000 domain created574lb # [ 0.005326] CPU features: detected: PMUv3575client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns576worker2 # [ 0.004675] EFI services will not be available.577client # [ 0.000026] arm-pv: using stolen time PV578lb # [ 0.005328] CPU features: detected: RAS Extension Support579worker2 # [ 0.004745] smp: Bringing up secondary CPUs ...580worker2 # [ 0.004753] smp: Brought up 1 node, 1 CPU581worker1 # [ 0.144401] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages582lb # [ 0.005331] CPU features: detected: RASv1p1 Extension Support583worker2 # [ 0.004756] SMP: Total of 1 processors activated.584client # [ 0.000376] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)585lb # [ 0.005334] CPU features: detected: Random Number Generator586worker1 # [ 0.144417] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page587worker2 # [ 0.004761] CPU: All CPU(s) started at EL1588client # [ 0.000524] Console: colour dummy device 80x25589lb # [ 0.005336] CPU features: detected: Speculation barrier (SB)590worker1 # [ 0.144422] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages591worker2 # [ 0.004771] CPU features: detected: Branch Target Identification592client # [ 0.000531] printk: legacy console [tty0] enabled593lb # [ 0.005339] CPU features: detected: Stage-2 Force Write-Back594worker1 # [ 0.144426] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page595worker2 # [ 0.004775] CPU features: detected: ARMv8.4 Translation Table Level596lb # [ 0.005342] CPU features: detected: TLB range maintenance instructions597client # [ 0.000715] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)598worker1 # [ 0.144431] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages599worker2 # [ 0.004779] CPU features: detected: Instruction cache invalidation not required for I/D coherence600client # [ 0.000721] pid_max: default: 32768 minimum: 301601lb # [ 0.005347] CPU features: detected: Speculative Store Bypassing Safe (SSBS)602worker1 # [ 0.144435] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page603client # [ 0.000814] LSM: initializing lsm=capability,landlock,yama,bpf,ima604lb # [ 0.005384] alternatives: applying system-wide alternatives605worker2 # [ 0.004782] CPU features: detected: Data cache clean to the PoU not required for I/D coherence606worker1 # [ 0.144439] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages607client # [ 0.000930] landlock: Up and running.608client # [ 0.000933] Yama: becoming mindful.609lb # [ 0.008273] CPU features: detected: BBM Level 2 without TLB conflict abort610worker2 # [ 0.004786] CPU features: detected: Common not Private translations611worker1 # [ 0.144443] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page612client # [ 0.001358] LSM support for eBPF active613worker2 # [ 0.004789] CPU features: detected: CRC32 instructions614client # [ 0.001504] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)615worker2 # [ 0.004792] CPU features: detected: Data cache clean to Point of Deep Persistence616lb # [ 0.008465] Memory: 1405636K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)617lb # [ 0.008812] devtmpfs: initialized618client # [ 0.001536] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)619worker2 # [ 0.004796] CPU features: detected: Data cache clean to Point of Persistence620worker1 # [ 0.151607] fbcon: Taking over console621client # [ 0.003374] rcu: Hierarchical SRCU implementation.622lb # [ 0.010626] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)623worker2 # [ 0.004798] CPU features: detected: Data independent timing control (DIT)624worker1 # [ 0.151623] ACPI: Interpreter disabled.625client # [ 0.003378] rcu: Max phase no-delay instances is 1000.626worker2 # [ 0.004802] CPU features: detected: E0PD627lb # [ 0.010647] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).628client # [ 0.004590] fsl-mc MSI: its@8080000 domain created629worker2 # [ 0.004804] CPU features: detected: Enhanced Counter Virtualization630client # [ 0.004680] EFI services will not be available.631lb # [ 0.010834] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL632lb # [ 0.010838] 0 pages in range for non-PLT usage633worker2 # [ 0.004808] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)634client # [ 0.004743] smp: Bringing up secondary CPUs ...635client # [ 0.004751] smp: Brought up 1 node, 1 CPU636lb # [ 0.010839] 508272 pages in range for PLT usage637worker2 # [ 0.004811] CPU features: detected: Enhanced Virtualization Traps638client # [ 0.004754] SMP: Total of 1 processors activated.639lb # [ 0.010948] pinctrl core: initialized pinctrl subsystem640worker2 # [ 0.004814] CPU features: detected: Fine Grained Traps641client # [ 0.004759] CPU: All CPU(s) started at EL1642lb # [ 0.011703] DMI not present or invalid.643client # [ 0.004769] CPU features: detected: Branch Target Identification644worker2 # [ 0.004818] CPU features: detected: Generic authentication (architected QARMA5 algorithm)645lb # [ 0.015187] NET: Registered PF_NETLINK/PF_ROUTE protocol family646client # [ 0.004772] CPU features: detected: ARMv8.4 Translation Table Level647worker2 # [ 0.004823] CPU features: detected: RCpc load-acquire (LDAPR)648lb # [ 0.017349] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations649worker2 # [ 0.004826] CPU features: detected: LSE atomic instructions650client # [ 0.004776] CPU features: detected: Instruction cache invalidation not required for I/D coherence651lb # [ 0.017537] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations652worker2 # [ 0.004828] CPU features: detected: Privileged Access Never653worker2 # [ 0.004831] CPU features: detected: PMUv3654lb # [ 0.017733] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations655client # [ 0.004779] CPU features: detected: Data cache clean to the PoU not required for I/D coherence656worker2 # [ 0.004834] CPU features: detected: RAS Extension Support657lb # [ 0.017754] audit: initializing netlink subsys (disabled)658client # [ 0.004783] CPU features: detected: Common not Private translations659worker2 # [ 0.004837] CPU features: detected: RASv1p1 Extension Support660worker1 # [ 0.160390] iommu: Default domain type: Translated661lb # [ 0.018333] thermal_sys: Registered thermal governor 'fair_share'662client # [ 0.004786] CPU features: detected: CRC32 instructions663worker2 # [ 0.004839] CPU features: detected: Random Number Generator664worker1 # [ 0.160399] iommu: DMA domain TLB invalidation policy: strict mode665lb # [ 0.018335] thermal_sys: Registered thermal governor 'bang_bang'666client # [ 0.004789] CPU features: detected: Data cache clean to Point of Deep Persistence667worker2 # [ 0.004842] CPU features: detected: Speculation barrier (SB)668worker1 # [ 0.160772] SCSI subsystem initialized669lb # [ 0.018339] thermal_sys: Registered thermal governor 'step_wise'670worker2 # [ 0.004845] CPU features: detected: Stage-2 Force Write-Back671client # [ 0.004793] CPU features: detected: Data cache clean to Point of Persistence672lb # [ 0.018341] thermal_sys: Registered thermal governor 'user_space'673worker1 # [ 0.162684] usbcore: registered new interface driver usbfs674worker2 # [ 0.004847] CPU features: detected: TLB range maintenance instructions675client # [ 0.004796] CPU features: detected: Data independent timing control (DIT)676lb # [ 0.018346] thermal_sys: Registered thermal governor 'power_allocator'677worker1 # [ 0.162717] usbcore: registered new interface driver hub678client # [ 0.004799] CPU features: detected: E0PD679worker2 # [ 0.004852] CPU features: detected: Speculative Store Bypassing Safe (SSBS)680worker1 # [ 0.162733] usbcore: registered new device driver usb681lb # [ 0.018384] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1682client # [ 0.004802] CPU features: detected: Enhanced Counter Virtualization683worker2 # [ 0.004887] alternatives: applying system-wide alternatives684worker1 # [ 0.163019] pps_core: LinuxPPS API ver. 1 registered685lb # [ 0.018393] cpuidle: using governor ladder686worker2 # [ 0.007751] CPU features: detected: BBM Level 2 without TLB conflict abort687lb # [ 0.018399] cpuidle: using governor menu688client # [ 0.004805] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)689worker1 # [ 0.163025] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>690client # [ 0.004808] CPU features: detected: Enhanced Virtualization Traps691lb # [ 0.018602] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.692worker1 # [ 0.163035] PTP clock support registered693worker1 # [ 0.163084] EDAC MC: Ver: 3.0.0694client # [ 0.004812] CPU features: detected: Fine Grained Traps695lb # [ 0.018617] ASID allocator initialised with 65536 entries696worker2 # [ 0.007910] Memory: 1405652K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)697lb # [ 0.019852] Serial: AMBA PL011 UART driver698worker2 # [ 0.008274] devtmpfs: initialized699worker1 # [ 0.167484] scmi_core: SCMI protocol bus registered700client # [ 0.004816] CPU features: detected: Generic authentication (architected QARMA5 algorithm)701worker1 # [ 0.168412] FPGA manager framework702worker2 # [ 0.010047] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)703lb # [ 0.025223] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1704client # [ 0.004820] CPU features: detected: RCpc load-acquire (LDAPR)705lb # [ 0.025338] printk: console [ttyAMA0] enabled706worker1 # [ 0.169323] vgaarb: loaded707client # [ 0.004823] CPU features: detected: LSE atomic instructions708worker2 # [ 0.010069] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).709client # [ 0.004826] CPU features: detected: Privileged Access Never710worker2 # [ 0.010278] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL711worker1 # [ 0.169915] clocksource: Switched to clocksource arch_sys_counter712client # [ 0.004829] CPU features: detected: PMUv3713lb # [ 0.145164] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages714client # [ 0.004832] CPU features: detected: RAS Extension Support715lb # [ 0.145180] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page716worker2 # [ 0.010283] 0 pages in range for non-PLT usage717client # [ 0.004835] CPU features: detected: RASv1p1 Extension Support718worker2 # [ 0.010284] 508272 pages in range for PLT usage719lb # [ 0.145186] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages720client # [ 0.004838] CPU features: detected: Random Number Generator721worker2 # [ 0.010396] pinctrl core: initialized pinctrl subsystem722lb # [ 0.145190] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page723worker2 # [ 0.011135] DMI not present or invalid.724lb # [ 0.145194] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages725client # [ 0.004840] CPU features: detected: Speculation barrier (SB)726worker2 # [ 0.014240] NET: Registered PF_NETLINK/PF_ROUTE protocol family727worker1 # [ 0.173648] VFS: Disk quotas dquot_6.6.0728lb # [ 0.145199] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page729client # [ 0.004843] CPU features: detected: Stage-2 Force Write-Back730worker2 # [ 0.016359] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations731worker1 # [ 0.173685] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)732lb # [ 0.145203] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages733client # [ 0.004846] CPU features: detected: TLB range maintenance instructions734worker2 # [ 0.016528] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations735lb # [ 0.145207] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page736client # [ 0.004851] CPU features: detected: Speculative Store Bypassing Safe (SSBS)737worker2 # [ 0.016719] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations738client # [ 0.004887] alternatives: applying system-wide alternatives739worker2 # [ 0.016739] audit: initializing netlink subsys (disabled)740worker1 # [ 0.177265] netfs: FS-Cache loaded741client # [ 0.007711] CPU features: detected: BBM Level 2 without TLB conflict abort742worker2 # [ 0.017279] thermal_sys: Registered thermal governor 'fair_share'743worker1 # [ 0.177394] pnp: PnP ACPI: disabled744lb # [ 0.152433] fbcon: Taking over console745worker2 # [ 0.017280] thermal_sys: Registered thermal governor 'bang_bang'746lb # [ 0.152445] ACPI: Interpreter disabled.747worker2 # [ 0.017284] thermal_sys: Registered thermal governor 'step_wise'748client # [ 0.007901] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)749client # [ 0.008251] devtmpfs: initialized750worker2 # [ 0.017287] thermal_sys: Registered thermal governor 'user_space'751client # [ 0.009986] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)752worker2 # [ 0.017292] thermal_sys: Registered thermal governor 'power_allocator'753worker2 # [ 0.017315] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1754client # [ 0.010006] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).755worker2 # [ 0.017325] cpuidle: using governor ladder756worker1 # [ 0.181312] NET: Registered PF_INET protocol family757client # [ 0.010187] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL758worker2 # [ 0.017330] cpuidle: using governor menu759client # [ 0.010191] 0 pages in range for non-PLT usage760worker1 # [ 0.181586] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)761worker2 # [ 0.017514] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.762client # [ 0.010192] 508272 pages in range for PLT usage763worker2 # [ 0.017528] ASID allocator initialised with 65536 entries764client # [ 0.010278] pinctrl core: initialized pinctrl subsystem765worker2 # [ 0.018699] Serial: AMBA PL011 UART driver766client # [ 0.011013] DMI not present or invalid.767lb # [ 0.159739] iommu: Default domain type: Translated768client # [ 0.014005] NET: Registered PF_NETLINK/PF_ROUTE protocol family769worker2 # [ 0.023808] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1770lb # [ 0.159748] iommu: DMA domain TLB invalidation policy: strict mode771worker2 # [ 0.023926] printk: console [ttyAMA0] enabled772client # [ 0.016147] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations773lb # [ 0.161376] SCSI subsystem initialized774client # [ 0.016322] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations775lb # [ 0.162090] usbcore: registered new interface driver usbfs776client # [ 0.016517] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations777lb # [ 0.162122] usbcore: registered new interface driver hub778client # [ 0.016537] audit: initializing netlink subsys (disabled)779lb # [ 0.162138] usbcore: registered new device driver usb780client # [ 0.017078] thermal_sys: Registered thermal governor 'fair_share'781lb # [ 0.162417] pps_core: LinuxPPS API ver. 1 registered782client # [ 0.017080] thermal_sys: Registered thermal governor 'bang_bang'783lb # [ 0.162423] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>784client # [ 0.017084] thermal_sys: Registered thermal governor 'step_wise'785lb # [ 0.162433] PTP clock support registered786client # [ 0.017086] thermal_sys: Registered thermal governor 'user_space'787lb # [ 0.162480] EDAC MC: Ver: 3.0.0788client # [ 0.017092] thermal_sys: Registered thermal governor 'power_allocator'789lb # [ 0.167043] scmi_core: SCMI protocol bus registered790worker2 # [ 0.139306] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages791client # [ 0.017132] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1792worker2 # [ 0.139321] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page793lb # [ 0.167990] FPGA manager framework794client # [ 0.017143] cpuidle: using governor ladder795lb # [ 0.168939] vgaarb: loaded796worker2 # [ 0.139326] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages797client # [ 0.017148] cpuidle: using governor menu798lb # [ 0.169535] clocksource: Switched to clocksource arch_sys_counter799worker2 # [ 0.139330] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page800client # [ 0.017333] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.801client # [ 0.017347] ASID allocator initialised with 65536 entries802worker2 # [ 0.139334] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages803client # [ 0.018540] Serial: AMBA PL011 UART driver804worker2 # [ 0.139338] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page805client # [ 0.023782] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1806worker2 # [ 0.139343] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages807client # [ 0.023933] printk: console [ttyAMA0] enabled808worker2 # [ 0.139347] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page809lb # [ 0.174419] VFS: Disk quotas dquot_6.6.0810worker2 # [ 0.146611] fbcon: Taking over console811lb # [ 0.174457] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)812worker2 # [ 0.146625] ACPI: Interpreter disabled.813lb # [ 0.176087] netfs: FS-Cache loaded814lb # [ 0.176211] pnp: PnP ACPI: disabled815client # [ 0.141929] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages816client # [ 0.141945] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page817client # [ 0.141951] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages818client # [ 0.141955] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page819lb # [ 0.179991] NET: Registered PF_INET protocol family820client # [ 0.141960] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages821lb # [ 0.180245] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)822client # [ 0.141964] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page823worker2 # [ 0.155594] iommu: Default domain type: Translated824client # [ 0.141969] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages825worker2 # [ 0.155603] iommu: DMA domain TLB invalidation policy: strict mode826client # [ 0.141973] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page827worker2 # [ 0.155919] SCSI subsystem initialized828worker2 # [ 0.157906] usbcore: registered new interface driver usbfs829worker2 # [ 0.157938] usbcore: registered new interface driver hub830client # [ 0.149498] fbcon: Taking over console831client # [ 0.149513] ACPI: Interpreter disabled.832worker2 # [ 0.157953] usbcore: registered new device driver usb833worker2 # [ 0.158234] pps_core: LinuxPPS API ver. 1 registered834worker2 # [ 0.158239] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>835worker2 # [ 0.158250] PTP clock support registered836worker2 # [ 0.158305] EDAC MC: Ver: 3.0.0837worker2 # [ 0.162861] scmi_core: SCMI protocol bus registered838worker2 # [ 0.163812] FPGA manager framework839worker2 # [ 0.164729] vgaarb: loaded840worker2 # [ 0.165366] clocksource: Switched to clocksource arch_sys_counter841worker1 # [ 0.210010] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)842worker1 # [ 0.210046] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)843worker1 # [ 0.210072] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)844client # [ 0.158507] iommu: Default domain type: Translated845worker1 # [ 0.210143] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)846client # [ 0.158517] iommu: DMA domain TLB invalidation policy: strict mode847client # [ 0.158873] SCSI subsystem initialized848worker2 # [ 0.169351] VFS: Disk quotas dquot_6.6.0849worker1 # [ 0.210287] TCP: Hash tables configured (established 16384 bind 16384)850worker1 # [ 0.210384] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)851client # [ 0.160870] usbcore: registered new interface driver usbfs852worker1 # [ 0.210415] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)853client # [ 0.160900] usbcore: registered new interface driver hub854client # [ 0.160916] usbcore: registered new device driver usb855worker1 # [ 0.210450] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)856worker2 # [ 0.171864] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)857client # [ 0.161191] pps_core: LinuxPPS API ver. 1 registered858worker1 # [ 0.210566] NET: Registered PF_UNIX/PF_LOCAL protocol family859worker2 # [ 0.172964] netfs: FS-Cache loaded860worker1 # [ 0.210586] NET: Registered PF_XDP protocol family861worker2 # [ 0.173074] pnp: PnP ACPI: disabled862client # [ 0.161198] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>863worker1 # [ 0.210601] PCI: CLS 0 bytes, default 64864client # [ 0.161208] PTP clock support registered865worker1 # [ 0.210803] Trying to unpack rootfs image as initramfs...866client # [ 0.161259] EDAC MC: Ver: 3.0.0867client # [ 0.165864] scmi_core: SCMI protocol bus registered868client # [ 0.166851] FPGA manager framework869client # [ 0.167792] vgaarb: loaded870worker2 # [ 0.176821] NET: Registered PF_INET protocol family871client # [ 0.168436] clocksource: Switched to clocksource arch_sys_counter872worker2 # [ 0.177080] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)873worker1 # [ 0.227460] kvm [1]: HYP mode not available874client # [ 0.174630] VFS: Disk quotas dquot_6.6.0875client # [ 0.174674] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)876client # [ 0.176296] netfs: FS-Cache loaded877client # [ 0.176416] pnp: PnP ACPI: disabled878lb # [ 0.208074] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)879lb # [ 0.208113] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)880lb # [ 0.208137] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)881lb # [ 0.208206] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)882client # [ 0.180235] NET: Registered PF_INET protocol family883lb # [ 0.208348] TCP: Hash tables configured (established 16384 bind 16384)884lb # [ 0.208424] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)885lb # [ 0.208457] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)886lb # [ 0.208492] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)887lb # [ 0.208601] NET: Registered PF_UNIX/PF_LOCAL protocol family888lb # [ 0.208621] NET: Registered PF_XDP protocol family889client # [ 0.181132] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)890lb # [ 0.208636] PCI: CLS 0 bytes, default 64891lb # [ 0.208855] Trying to unpack rootfs image as initramfs...892lb # [ 0.223171] kvm [1]: HYP mode not available893worker2 # [ 0.205137] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)894worker2 # [ 0.205176] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)895worker2 # [ 0.205202] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)896worker2 # [ 0.205275] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)897worker2 # [ 0.209239] TCP: Hash tables configured (established 16384 bind 16384)898worker2 # [ 0.209316] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)899worker2 # [ 0.209348] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker2 # [ 0.211815] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)901worker2 # [ 0.211916] NET: Registered PF_UNIX/PF_LOCAL protocol family902worker2 # [ 0.211935] NET: Registered PF_XDP protocol family903worker2 # [ 0.211951] PCI: CLS 0 bytes, default 64904worker2 # [ 0.212163] Trying to unpack rootfs image as initramfs...905worker2 # [ 0.220535] kvm [1]: HYP mode not available906client # [ 0.209573] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)907client # [ 0.209616] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)908client # [ 0.209644] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)909client # [ 0.209711] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)910client # [ 0.209854] TCP: Hash tables configured (established 16384 bind 16384)911client # [ 0.209933] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)912client # [ 0.209965] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)913client # [ 0.210025] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)914client # [ 0.210122] NET: Registered PF_UNIX/PF_LOCAL protocol family915client # [ 0.210147] NET: Registered PF_XDP protocol family916client # [ 0.210163] PCI: CLS 0 bytes, default 64917client # [ 0.210397] Trying to unpack rootfs image as initramfs...918client # [ 0.226315] kvm [1]: HYP mode not available919worker1 # [ 0.318801] Initialise system trusted keyrings920worker1 # [ 0.319549] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker1 # [ 0.320822] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker1 # [ 0.321584] 9p: Installing v9fs 9p2000 file system support923lb # [ 0.314922] Initialise system trusted keyrings924lb # [ 0.315650] workingset: timestamp_bits=42 max_order=19 bucket_order=0925lb # [ 0.316901] squashfs: version 4.0 (2009/01/31) Phillip Lougher926lb # [ 0.317688] 9p: Installing v9fs 9p2000 file system support927worker1 # [ 0.350262] Key type asymmetric registered928worker1 # [ 0.350289] Asymmetric key parser 'x509' registered929worker1 # [ 0.350354] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)930worker1 # [ 0.352588] io scheduler mq-deadline registered931worker1 # [ 0.352600] io scheduler kyber registered932worker1 # [ 0.362045] pl061_gpio 9030000.pl061: PL061 GPIO chip registered933worker2 # [ 0.312012] Initialise system trusted keyrings934worker2 # [ 0.312737] workingset: timestamp_bits=42 max_order=19 bucket_order=0935worker1 # [ 0.363423] ledtrig-cpu: registered to indicate activity on CPUs936worker2 # [ 0.314009] squashfs: version 4.0 (2009/01/31) Phillip Lougher937worker1 # [ 0.363842] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:938worker1 # [ 0.363860] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000939worker1 # [ 0.363885] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000940worker2 # [ 0.314788] 9p: Installing v9fs 9p2000 file system support941worker1 # [ 0.363894] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000942lb # [ 0.345678] Key type asymmetric registered943lb # [ 0.345700] Asymmetric key parser 'x509' registered944worker1 # [ 0.363914] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits945client # [ 0.309707] Initialise system trusted keyrings946lb # [ 0.345764] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)947worker1 # [ 0.363937] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]948lb # [ 0.347853] io scheduler mq-deadline registered949worker1 # [ 0.364011] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00950lb # [ 0.347865] io scheduler kyber registered951worker1 # [ 0.364021] pci_bus 0000:00: root bus resource [bus 00-ff]952worker1 # [ 0.364027] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]953worker1 # [ 0.364032] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]954worker1 # [ 0.364037] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]955worker1 # [ 0.364092] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint956worker1 # [ 0.364521] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint957client # [ 0.316481] workingset: timestamp_bits=42 max_order=19 bucket_order=0958worker1 # [ 0.364702] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]959worker1 # [ 0.364719] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]960client # [ 0.317827] squashfs: version 4.0 (2009/01/31) Phillip Lougher961worker1 # [ 0.364748] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]962worker1 # [ 0.364763] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]963worker1 # [ 0.365220] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint964client # [ 0.318612] 9p: Installing v9fs 9p2000 file system support965worker1 # [ 0.365399] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]966lb # [ 0.357690] pl061_gpio 9030000.pl061: PL061 GPIO chip registered967worker1 # [ 0.365415] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]968worker1 # [ 0.365443] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]969worker1 # [ 0.365891] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint970lb # [ 0.359152] ledtrig-cpu: registered to indicate activity on CPUs971lb # [ 0.359533] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:972lb # [ 0.359550] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000973lb # [ 0.359562] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000974worker1 # [ 0.386030] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]975lb # [ 0.359571] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000976worker1 # [ 0.386050] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]977worker1 # [ 0.386080] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]978worker2 # [ 0.338881] Key type asymmetric registered979lb # [ 0.359591] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits980worker2 # [ 0.338900] Asymmetric key parser 'x509' registered981worker1 # [ 0.386549] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint982lb # [ 0.359613] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]983worker2 # [ 0.338958] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)984worker1 # [ 0.386730] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]985lb # [ 0.359703] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00986worker2 # [ 0.340947] io scheduler mq-deadline registered987worker1 # [ 0.386746] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]988worker2 # [ 0.340958] io scheduler kyber registered989lb # [ 0.359712] pci_bus 0000:00: root bus resource [bus 00-ff]990worker1 # [ 0.386775] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]991lb # [ 0.359718] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]992worker1 # [ 0.386794] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]993lb # [ 0.359723] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]994worker1 # [ 0.387287] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint995lb # [ 0.359728] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]996worker1 # [ 0.387473] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]997lb # [ 0.359783] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint998worker1 # [ 0.387503] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]999lb # [ 0.360221] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1000worker1 # [ 0.387950] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1001lb # [ 0.360405] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1002worker1 # [ 0.388136] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1003lb # [ 0.360422] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1004worker1 # [ 0.388165] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1005lb # [ 0.360452] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1006client # [ 0.339236] Key type asymmetric registered1007worker1 # [ 0.388562] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1008lb # [ 0.360468] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1009client # [ 0.339253] Asymmetric key parser 'x509' registered1010worker1 # [ 0.388743] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1011client # [ 0.339307] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)1012lb # [ 0.360923] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1013worker1 # [ 0.388992] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1014lb # [ 0.361105] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1015worker1 # [ 0.389177] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1016lb # [ 0.361120] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1017worker1 # [ 0.389206] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1018lb # [ 0.361149] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1019worker1 # [ 0.389658] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1020client # [ 0.344484] io scheduler mq-deadline registered1021worker1 # [ 0.389843] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1022client # [ 0.344500] io scheduler kyber registered1023worker1 # [ 0.389872] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024worker2 # [ 0.354245] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1025worker2 # [ 0.355510] ledtrig-cpu: registered to indicate activity on CPUs1026worker2 # [ 0.355863] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1027worker2 # [ 0.355880] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001028worker2 # [ 0.355892] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001029worker2 # [ 0.355900] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001030lb # [ 0.385650] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1031worker2 # [ 0.355920] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1032lb # [ 0.385844] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1033lb # [ 0.385861] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1034worker2 # [ 0.355941] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1035worker1 # [ 0.414488] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1036lb # [ 0.385890] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037worker2 # [ 0.356017] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001038worker1 # [ 0.414692] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1039lb # [ 0.386377] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1040worker2 # [ 0.356026] pci_bus 0000:00: root bus resource [bus 00-ff]1041worker1 # [ 0.414723] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042lb # [ 0.386559] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1043worker2 # [ 0.356032] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1044lb # [ 0.386575] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1045worker1 # [ 0.415218] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1046worker2 # [ 0.356037] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1047worker1 # [ 0.415510] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1048lb # [ 0.386604] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049worker2 # [ 0.356041] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1050worker1 # [ 0.415527] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1051lb # [ 0.386622] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1052worker2 # [ 0.356140] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1053worker1 # [ 0.415557] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054lb # [ 0.387071] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1055worker2 # [ 0.356563] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1056client # [ 0.356582] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1057lb # [ 0.387257] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1058worker1 # [ 0.416013] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1059worker2 # [ 0.356742] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1060client # [ 0.357224] ledtrig-cpu: registered to indicate activity on CPUs1061worker1 # [ 0.416195] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1062lb # [ 0.387286] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1063worker2 # [ 0.356758] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1064worker1 # [ 0.416210] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1065client # [ 0.357604] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1066lb # [ 0.387732] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1067worker2 # [ 0.356787] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1068worker1 # [ 0.416240] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1069lb # [ 0.387916] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1070client # [ 0.357621] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001071worker2 # [ 0.356803] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1072worker1 # [ 0.416804] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1073lb # [ 0.387946] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074client # [ 0.357633] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001075worker2 # [ 0.357236] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1076worker1 # [ 0.416815] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1077lb # [ 0.388339] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1078worker2 # [ 0.357431] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1079client # [ 0.357641] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001080worker1 # [ 0.416821] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1081lb # [ 0.388519] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1082worker2 # [ 0.357446] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1083client # [ 0.357662] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1084worker1 # [ 0.416866] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1085worker2 # [ 0.357475] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086lb # [ 0.388769] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1087worker1 # [ 0.416913] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1088worker2 # [ 0.357920] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1089client # [ 0.357684] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1090lb # [ 0.388955] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1091worker2 # [ 0.358099] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1092lb # [ 0.388985] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093client # [ 0.357754] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001094worker1 # [ 0.416959] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1095worker2 # [ 0.358115] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1096client # [ 0.357764] pci_bus 0000:00: root bus resource [bus 00-ff]1097lb # [ 0.389434] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1098worker1 # [ 0.417006] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1099worker2 # [ 0.358144] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100client # [ 0.357770] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1101lb # [ 0.389633] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1102worker1 # [ 0.417053] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1103client # [ 0.357775] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1104worker2 # [ 0.358601] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1105lb # [ 0.389662] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1106worker2 # [ 0.358783] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1107worker1 # [ 0.417099] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1108client # [ 0.357780] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1109lb # [ 0.390118] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1110worker2 # [ 0.358798] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1111worker1 # [ 0.417146] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1112lb # [ 0.390306] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1113client # [ 0.357848] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1114worker2 # [ 0.358827] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115worker1 # [ 0.417191] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1116lb # [ 0.390335] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1117client # [ 0.358285] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1118worker2 # [ 0.358845] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1119client # [ 0.358470] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1120worker1 # [ 0.417237] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1121lb # [ 0.390797] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1122worker2 # [ 0.359289] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1123client # [ 0.358486] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1124lb # [ 0.391046] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1125worker1 # [ 0.417304] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1126worker2 # [ 0.359473] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1127lb # [ 0.391063] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1128client # [ 0.358515] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129worker1 # [ 0.417350] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1130worker2 # [ 0.359503] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1131client # [ 0.358531] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1132lb # [ 0.391092] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1133worker1 # [ 0.417371] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1134worker2 # [ 0.359950] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1135client # [ 0.358980] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1136lb # [ 0.391549] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1137worker1 # [ 0.417392] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1138worker2 # [ 0.360134] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1139client # [ 0.359162] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1140lb # [ 0.391731] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1141worker1 # [ 0.417413] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1142worker2 # [ 0.360164] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143client # [ 0.359177] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1144lb # [ 0.391747] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1145worker1 # [ 0.417435] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1146worker2 # [ 0.360543] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1147client # [ 0.359206] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148lb # [ 0.391776] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker1 # [ 0.417456] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1150worker2 # [ 0.360722] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1151lb # [ 0.392338] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1152client # [ 0.359650] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1153worker1 # [ 0.417477] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1154client # [ 0.359831] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1155worker2 # [ 0.360992] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1156lb # [ 0.392349] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1157worker1 # [ 0.417503] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1158worker2 # [ 0.361177] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1159client # [ 0.359846] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1160lb # [ 0.392354] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1161worker1 # [ 0.417524] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1162worker2 # [ 0.361206] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1163client # [ 0.359875] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164worker1 # [ 0.417546] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1165lb # [ 0.392397] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1166client # [ 0.360318] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1167worker1 # [ 0.417568] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1168lb # [ 0.392443] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1169client # [ 0.360522] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1170worker1 # [ 0.417590] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1171client # [ 0.360538] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1172lb # [ 0.392488] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1173worker1 # [ 0.417610] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1174client # [ 0.360568] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1175lb # [ 0.392533] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1176worker1 # [ 0.417631] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1177client # [ 0.360586] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1178worker1 # [ 0.417652] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1179lb # [ 0.392579] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1180worker1 # [ 0.417673] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1181client # [ 0.361041] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1182lb # [ 0.392624] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1183worker1 # [ 0.417694] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1184client # [ 0.361244] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1185lb # [ 0.392670] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1186worker1 # [ 0.417715] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1187client # [ 0.361273] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1188worker1 # [ 0.417741] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1189lb # [ 0.392715] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1190client # [ 0.361730] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1191worker1 # [ 0.417750] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1192client # [ 0.361916] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1193lb # [ 0.392761] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1194worker1 # [ 0.417755] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1195client # [ 0.361945] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1196lb # [ 0.392829] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1197worker1 # [ 0.418575] pci 0000:00:07.0: enabling device (0000 -> 0002)1198lb # [ 0.392874] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1199client # [ 0.362359] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1200client # [ 0.362538] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1201lb # [ 0.392896] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1202worker2 # [ 0.401748] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1203lb # [ 0.392917] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1204client # [ 0.362781] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1205worker2 # [ 0.401947] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1206lb # [ 0.392938] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1207client # [ 0.362965] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1208worker2 # [ 0.401978] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1209lb # [ 0.392959] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1210client # [ 0.362994] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1211worker2 # [ 0.402445] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1212lb # [ 0.392981] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1213worker2 # [ 0.402631] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1214client # [ 0.363436] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1215lb # [ 0.393002] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1216client # [ 0.363620] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1217worker2 # [ 0.402661] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1218lb # [ 0.393026] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1219client # [ 0.363649] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1220worker2 # [ 0.403121] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1221lb # [ 0.393047] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1222worker2 # [ 0.403367] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1223client # [ 0.364101] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1224lb # [ 0.393069] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1225worker2 # [ 0.403384] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1226client # [ 0.364285] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1227lb # [ 0.393091] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1228worker2 # [ 0.403413] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1229client # [ 0.364314] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1230lb # [ 0.393113] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1231worker2 # [ 0.403872] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1232lb # [ 0.393135] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1233worker2 # [ 0.404051] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1234lb # [ 0.393156] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1235worker2 # [ 0.404067] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1236lb # [ 0.393177] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1237worker2 # [ 0.404097] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1238lb # [ 0.393198] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1239worker2 # [ 0.404646] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1240lb # [ 0.393219] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1241worker2 # [ 0.404657] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1242lb # [ 0.393240] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1243lb # [ 0.393265] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1244worker2 # [ 0.404663] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1245lb # [ 0.393275] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1246worker2 # [ 0.404705] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1247lb # [ 0.393279] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1248worker2 # [ 0.404751] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1249worker2 # [ 0.404797] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1250worker2 # [ 0.404844] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1251worker2 # [ 0.404891] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1252worker2 # [ 0.404937] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1253worker2 # [ 0.404984] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1254worker2 # [ 0.405029] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1255worker2 # [ 0.405074] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1256client # [ 0.412870] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1257worker2 # [ 0.405141] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1258client # [ 0.413146] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1259lb # [ 0.454157] pci 0000:00:07.0: enabling device (0000 -> 0002)1260client # [ 0.413164] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1261worker2 # [ 0.405185] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1262worker1 # [ 0.480193] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1263worker2 # [ 0.405207] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1264client # [ 0.413194] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1265worker2 # [ 0.405228] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1266client # [ 0.413648] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1267worker2 # [ 0.405248] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1268client # [ 0.413828] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1269client # [ 0.413843] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1270worker2 # [ 0.405269] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1271worker2 # [ 0.405290] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1272client # [ 0.413873] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1273worker2 # [ 0.405312] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1274client # [ 0.414433] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1275worker2 # [ 0.405336] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1276client # [ 0.414445] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1277worker2 # [ 0.405358] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1278client # [ 0.414450] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1279client # [ 0.414494] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1280client # [ 0.414539] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1281client # [ 0.414584] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1282client # [ 0.414630] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1283client # [ 0.414675] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1284client # [ 0.414721] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1285worker1 # [ 0.490123] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1286worker1 # [ 0.493491] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1287client # [ 0.414767] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1288client # [ 0.414812] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1289worker2 # [ 0.441427] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1290client # [ 0.414857] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1291worker2 # [ 0.441461] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1292client # [ 0.414924] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1293worker2 # [ 0.441484] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1294client # [ 0.414970] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1295worker2 # [ 0.441506] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1296client # [ 0.414991] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1297worker2 # [ 0.441527] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1298client # [ 0.415012] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1299worker2 # [ 0.441548] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1300client # [ 0.415032] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1301worker2 # [ 0.441569] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1302client # [ 0.415053] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1303worker2 # [ 0.441591] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1304client # [ 0.415074] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1305worker2 # [ 0.441613] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1306worker2 # [ 0.441641] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1307client # [ 0.415094] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1308worker2 # [ 0.441650] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1309client # [ 0.415119] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1310worker2 # [ 0.441655] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1311client # [ 0.415140] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1312worker2 # [ 0.442474] pci 0000:00:07.0: enabling device (0000 -> 0002)1313client # [ 0.415161] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1314lb # [ 0.480306] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1315worker1 # [ 0.503242] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1316client # [ 0.415183] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1317client # [ 0.415205] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1318client # [ 0.415225] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1319client # [ 0.415245] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1320client # [ 0.415266] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1321worker1 # [ 0.507754] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1322client # [ 0.415286] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1323lb # [ 0.483375] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1324worker1 # [ 0.509814] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1325client # [ 0.415307] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1326lb # [ 0.485295] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1327client # [ 0.415327] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1328client # [ 0.415352] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1329client # [ 0.415361] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1330client # [ 0.415365] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1331client # [ 0.416155] pci 0000:00:07.0: enabling device (0000 -> 0002)1332lb # [ 0.494921] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1333lb # [ 0.496882] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1334worker1 # [ 0.523302] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1335worker2 # [ 0.472230] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1336worker1 # [ 0.525153] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1337worker1 # [ 0.528477] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1338lb # [ 0.508249] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1339worker1 # [ 0.531148] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1340worker1 # [ 0.533685] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1341worker2 # [ 0.482588] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1342client # [ 0.472464] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1343client # [ 0.474499] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1344worker2 # [ 0.486780] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1345lb # [ 0.510765] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1346worker2 # [ 0.488645] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1347lb # [ 0.512476] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1348lb # [ 0.514270] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1349lb # [ 0.516096] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1350client # [ 0.484533] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1351client # [ 0.486892] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1352worker1 # [ 0.551166] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1353worker1 # [ 0.553654] msm_serial: driver initialized1354worker2 # [ 0.500268] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1355worker1 # [ 0.553809] SuperH (H)SCI(F) driver initialized1356worker1 # [ 0.553863] STM32 USART driver initialized1357worker2 # [ 0.503006] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1358lb # [ 0.530634] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1359worker2 # [ 0.504652] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1360client # [ 0.496989] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1361client # [ 0.499377] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1362client # [ 0.501112] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1363client # [ 0.502899] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1364worker2 # [ 0.514647] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1365worker2 # [ 0.516537] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1366lb # [ 0.546742] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1367lb # [ 0.549257] msm_serial: driver initialized1368lb # [ 0.549397] SuperH (H)SCI(F) driver initialized1369lb # [ 0.549452] STM32 USART driver initialized1370worker2 # [ 0.527329] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1371worker2 # [ 0.530454] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1372worker1 # [ 0.587156] loop: module loaded1373worker1 # [ 0.587329] virtio_blk virtio2: 1/0/0 default/read/poll queues1374client # [ 0.520646] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1375worker1 # [ 0.588111] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1376client # [ 0.522521] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1377client # [ 0.524695] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1378worker2 # [ 0.543231] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1379worker1 # [ 0.594482] megasas: 07.734.00.00-rc11380worker1 # [ 0.595269] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1381client # [ 0.537653] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1382worker1 # [ 0.597421] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001383worker1 # [ 0.597457] Intel/Sharp Extended Query Table at 0x00311384worker2 # [ 0.545669] msm_serial: driver initialized1385client # [ 0.540052] msm_serial: driver initialized1386worker2 # [ 0.545813] SuperH (H)SCI(F) driver initialized1387client # [ 0.540202] SuperH (H)SCI(F) driver initialized1388worker2 # [ 0.545866] STM32 USART driver initialized1389client # [ 0.540255] STM32 USART driver initialized1390lb # [ 0.579093] loop: module loaded1391lb # [ 0.579254] virtio_blk virtio2: 1/0/0 default/read/poll queues1392lb # [ 0.579999] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1393worker1 # [ 0.606854] Using buffer write method1394worker1 # [ 0.606918] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1395worker1 # [ 0.609766] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001396worker1 # [ 0.609787] Intel/Sharp Extended Query Table at 0x00311397lb # [ 0.586127] megasas: 07.734.00.00-rc11398lb # [ 0.586827] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399lb # [ 0.588919] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001400lb # [ 0.588942] Intel/Sharp Extended Query Table at 0x00311401worker1 # [ 0.623051] Using buffer write method1402worker1 # [ 0.623105] Concatenating MTD devices:1403worker1 # [ 0.623109] (0): "0.flash"1404worker1 # [ 0.623113] (1): "0.flash"1405worker1 # [ 0.623119] into device "0.flash"1406lb # [ 0.598489] Using buffer write method1407lb # [ 0.598561] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1408lb # [ 0.601368] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001409lb # [ 0.601391] Intel/Sharp Extended Query Table at 0x00311410worker2 # [ 0.577592] loop: module loaded1411worker2 # [ 0.577755] virtio_blk virtio2: 1/0/0 default/read/poll queues1412worker2 # [ 0.578531] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1413client # [ 0.571352] loop: module loaded1414client # [ 0.571520] virtio_blk virtio2: 1/0/0 default/read/poll queues1415client # [ 0.572337] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1416lb # [ 0.614584] Using buffer write method1417lb # [ 0.614617] Concatenating MTD devices:1418lb # [ 0.614622] (0): "0.flash"1419lb # [ 0.614625] (1): "0.flash"1420lb # [ 0.614635] into device "0.flash"1421worker2 # [ 0.585926] megasas: 07.734.00.00-rc11422client # [ 0.576974] megasas: 07.734.00.00-rc11423worker2 # [ 0.586664] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1424client # [ 0.577735] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1425worker2 # [ 0.588816] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001426client # [ 0.579840] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001427worker2 # [ 0.588861] Intel/Sharp Extended Query Table at 0x00311428client # [ 0.579864] Intel/Sharp Extended Query Table at 0x00311429worker2 # [ 0.593878] Using buffer write method1430worker2 # [ 0.593945] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1431worker2 # [ 0.595715] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001432worker2 # [ 0.595737] Intel/Sharp Extended Query Table at 0x00311433client # [ 0.589403] Using buffer write method1434client # [ 0.589477] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1435client # [ 0.592240] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001436client # [ 0.592264] Intel/Sharp Extended Query Table at 0x00311437worker2 # [ 0.609427] Using buffer write method1438client # [ 0.601360] Using buffer write method1439worker2 # [ 0.609453] Concatenating MTD devices:1440worker2 # [ 0.609457] (0): "0.flash"1441client # [ 0.601386] Concatenating MTD devices:1442worker2 # [ 0.609461] (1): "0.flash"1443client # [ 0.601390] (0): "0.flash"1444client # [ 0.601394] (1): "0.flash"1445worker2 # [ 0.609469] into device "0.flash"1446client # [ 0.601404] into device "0.flash"1447worker1 # [ 0.848932] Freeing initrd memory: 26976K1448worker1 # [ 0.854984] tun: Universal TUN/TAP device driver, 1.61449lb # [ 0.833966] Freeing initrd memory: 26948K1450worker1 # [ 0.858666] thunder_xcv, ver 1.01451worker1 # [ 0.858711] thunder_bgx, ver 1.01452worker1 # [ 0.858734] nicpf, ver 1.01453worker1 # [ 0.859342] e1000: Intel(R) PRO/1000 Network Driver1454worker1 # [ 0.859350] e1000: Copyright (c) 1999-2006 Intel Corporation.1455worker1 # [ 0.859375] e1000e: Intel(R) PRO/1000 Network Driver1456worker1 # [ 0.859383] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1457lb # [ 0.839983] tun: Universal TUN/TAP device driver, 1.61458worker1 # [ 0.859412] igb: Intel(R) Gigabit Ethernet Network Driver1459worker1 # [ 0.859418] igb: Copyright (c) 2007-2014 Intel Corporation.1460worker1 # [ 0.859441] igbvf: Intel(R) Gigabit Virtual Function Network Driver1461worker1 # [ 0.859446] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1462worker1 # [ 0.859584] sky2: driver version 1.301463worker1 # [ 0.861151] usbcore: registered new interface driver usb-storage1464lb # [ 0.843611] thunder_xcv, ver 1.01465lb # [ 0.843654] thunder_bgx, ver 1.01466worker1 # [ 0.861232] usbcore: registered new interface driver usbserial_generic1467lb # [ 0.843678] nicpf, ver 1.01468worker1 # [ 0.861246] usbserial: USB Serial support registered for generic1469lb # [ 0.844259] e1000: Intel(R) PRO/1000 Network Driver1470lb # [ 0.844266] e1000: Copyright (c) 1999-2006 Intel Corporation.1471worker1 # [ 0.861842] hv_vmbus: registering driver hyperv_keyboard1472lb # [ 0.844295] e1000e: Intel(R) PRO/1000 Network Driver1473worker1 # [ 0.862684] ehci-pci 0000:00:07.0: EHCI Host Controller1474lb # [ 0.844302] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1475worker1 # [ 0.862710] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11476lb # [ 0.844333] igb: Intel(R) Gigabit Ethernet Network Driver1477lb # [ 0.844339] igb: Copyright (c) 2007-2014 Intel Corporation.1478worker1 # [ 0.862913] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001479lb # [ 0.844362] igbvf: Intel(R) Gigabit Virtual Function Network Driver1480worker1 # [ 0.874546] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001481lb # [ 0.844368] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1482worker1 # [ 0.874850] hub 1-0:1.0: USB hub found1483lb # [ 0.844512] sky2: driver version 1.301484worker1 # [ 0.874866] hub 1-0:1.0: 6 ports detected1485lb # [ 0.853289] usbcore: registered new interface driver usb-storage1486worker1 # [ 0.878416] rtc-pl031 9010000.pl031: registered as rtc01487lb # [ 0.853447] usbcore: registered new interface driver usbserial_generic1488worker1 # [ 0.878444] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:53:41 UTC (1790096021)1489lb # [ 0.853461] usbserial: USB Serial support registered for generic1490worker1 # [ 0.878748] i2c_dev: i2c /dev entries driver1491lb # [ 0.854208] ehci-pci 0000:00:07.0: EHCI Host Controller1492lb # [ 0.854236] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11493lb # [ 0.854491] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001494lb # [ 0.859670] hv_vmbus: registering driver hyperv_keyboard1495worker1 # [ 0.884131] sdhci: Secure Digital Host Controller Interface driver1496lb # [ 0.861188] rtc-pl031 9010000.pl031: registered as rtc01497worker1 # [ 0.884142] sdhci: Copyright(c) Pierre Ossman1498worker1 # [ 0.884407] Synopsys Designware Multimedia Card Interface Driver1499lb # [ 0.861218] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:53:41 UTC (1790096021)1500worker1 # [ 0.884789] sdhci-pltfm: SDHCI platform and OF driver helper1501lb # [ 0.861520] i2c_dev: i2c /dev entries driver1502worker2 # [ 0.837086] Freeing initrd memory: 26944K1503worker1 # [ 0.888975] hid: raw HID events driver (C) Jiri Kosina1504worker1 # [ 0.889240] usbcore: registered new interface driver usbhid1505worker1 # [ 0.889247] usbhid: USB HID core driver1506worker1 # [ 0.892113] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1507lb # [ 0.866919] sdhci: Secure Digital Host Controller Interface driver1508lb # [ 0.866929] sdhci: Copyright(c) Pierre Ossman1509worker1 # [ 0.893644] drop_monitor: Initializing network drop monitor service1510client # [ 0.832323] Freeing initrd memory: 26972K1511worker2 # [ 0.842975] tun: Universal TUN/TAP device driver, 1.61512lb # [ 0.867205] Synopsys Designware Multimedia Card Interface Driver1513worker1 # [ 0.893792] NET: Registered PF_INET6 protocol family1514lb # [ 0.867591] sdhci-pltfm: SDHCI platform and OF driver helper1515worker1 # [ 0.896781] Segment Routing with IPv61516lb # [ 0.869147] hid: raw HID events driver (C) Jiri Kosina1517worker1 # [ 0.896798] In-situ OAM (IOAM) with IPv61518lb # [ 0.869394] usbcore: registered new interface driver usbhid1519worker1 # [ 0.896826] NET: Registered PF_PACKET protocol family1520lb # [ 0.869400] usbhid: USB HID core driver1521lb # [ 0.874026] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001522lb # [ 0.875015] hub 1-0:1.0: USB hub found1523client # [ 0.838231] tun: Universal TUN/TAP device driver, 1.61524lb # [ 0.875536] hub 1-0:1.0: 6 ports detected1525worker1 # [ 0.898663] 9pnet: Installing 9P2000 support1526worker1 # [ 0.898709] Key type dns_resolver registered1527worker2 # [ 0.846576] thunder_xcv, ver 1.01528lb # [ 0.877403] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1529worker2 # [ 0.846619] thunder_bgx, ver 1.01530worker2 # [ 0.846642] nicpf, ver 1.01531worker2 # [ 0.847182] e1000: Intel(R) PRO/1000 Network Driver1532worker2 # [ 0.847189] e1000: Copyright (c) 1999-2006 Intel Corporation.1533lb # [ 0.880102] drop_monitor: Initializing network drop monitor service1534worker2 # [ 0.847217] e1000e: Intel(R) PRO/1000 Network Driver1535worker1 # [ 0.905314] registered taskstats version 11536lb # [ 0.880274] NET: Registered PF_INET6 protocol family1537worker2 # [ 0.847225] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1538worker1 # [ 0.905465] Loading compiled-in X.509 certificates1539worker2 # [ 0.847255] igb: Intel(R) Gigabit Ethernet Network Driver1540client # [ 0.841855] thunder_xcv, ver 1.01541lb # [ 0.882177] Segment Routing with IPv61542client # [ 0.841897] thunder_bgx, ver 1.01543lb # [ 0.882194] In-situ OAM (IOAM) with IPv61544worker2 # [ 0.847260] igb: Copyright (c) 2007-2014 Intel Corporation.1545client # [ 0.841918] nicpf, ver 1.01546lb # [ 0.882221] NET: Registered PF_PACKET protocol family1547worker2 # [ 0.847284] igbvf: Intel(R) Gigabit Virtual Function Network Driver1548client # [ 0.842479] e1000: Intel(R) PRO/1000 Network Driver1549worker2 # [ 0.847290] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1550client # [ 0.842486] e1000: Copyright (c) 1999-2006 Intel Corporation.1551worker2 # [ 0.847427] sky2: driver version 1.301552client # [ 0.842509] e1000e: Intel(R) PRO/1000 Network Driver1553worker2 # [ 0.848988] usbcore: registered new interface driver usb-storage1554client # [ 0.842519] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1555lb # [ 0.883773] 9pnet: Installing 9P2000 support1556lb # [ 0.883822] Key type dns_resolver registered1557worker2 # [ 0.849068] usbcore: registered new interface driver usbserial_generic1558client # [ 0.842547] igb: Intel(R) Gigabit Ethernet Network Driver1559client # [ 0.842552] igb: Copyright (c) 2007-2014 Intel Corporation.1560worker2 # [ 0.849082] usbserial: USB Serial support registered for generic1561worker2 # [ 0.850066] ehci-pci 0000:00:07.0: EHCI Host Controller1562client # [ 0.842575] igbvf: Intel(R) Gigabit Virtual Function Network Driver1563client # [ 0.842580] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1564worker2 # [ 0.850089] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11565client # [ 0.842713] sky2: driver version 1.301566worker2 # [ 0.850355] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001567client # [ 0.844271] usbcore: registered new interface driver usb-storage1568worker1 # [ 0.913984] Demotion targets for Node 0: null1569worker1 # [ 0.914081] Key type .fscrypt registered1570client # [ 0.844355] usbcore: registered new interface driver usbserial_generic1571worker2 # [ 0.861788] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001572worker1 # [ 0.914087] Key type fscrypt-provisioning registered1573worker2 # [ 0.862084] hub 1-0:1.0: USB hub found1574client # [ 0.844370] usbserial: USB Serial support registered for generic1575worker1 # [ 0.914174] ima: No TPM chip found, activating TPM-bypass!1576worker2 # [ 0.862103] hub 1-0:1.0: 6 ports detected1577client # [ 0.845160] ehci-pci 0000:00:07.0: EHCI Host Controller1578worker1 # [ 0.914193] ima: Allocated hash algorithm: sha11579worker1 # [ 0.914214] ima: No architecture policies found1580client # [ 0.845191] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11581worker2 # [ 0.864387] hv_vmbus: registering driver hyperv_keyboard1582client # [ 0.845380] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001583lb # [ 0.890382] registered taskstats version 11584worker1 # [ 0.918156] input: gpio-keys as /devices/platform/gpio-keys/input/input01585lb # [ 0.890525] Loading compiled-in X.509 certificates1586client # [ 0.857055] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001587worker2 # [ 0.865946] rtc-pl031 9010000.pl031: registered as rtc01588client # [ 0.857342] hub 1-0:1.0: USB hub found1589client # [ 0.857356] hub 1-0:1.0: 6 ports detected1590worker2 # [ 0.865969] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:53:41 UTC (1790096021)1591worker2 # [ 0.866276] i2c_dev: i2c /dev entries driver1592client # [ 0.859921] hv_vmbus: registering driver hyperv_keyboard1593client # [ 0.861405] rtc-pl031 9010000.pl031: registered as rtc01594lb # [ 0.899322] Demotion targets for Node 0: null1595lb # [ 0.899421] Key type .fscrypt registered1596client # [ 0.861430] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T16:53:41 UTC (1790096021)1597lb # [ 0.899427] Key type fscrypt-provisioning registered1598client # [ 0.861727] i2c_dev: i2c /dev entries driver1599worker2 # [ 0.871522] sdhci: Secure Digital Host Controller Interface driver1600lb # [ 0.899521] ima: No TPM chip found, activating TPM-bypass!1601worker2 # [ 0.871532] sdhci: Copyright(c) Pierre Ossman1602lb # [ 0.899540] ima: Allocated hash algorithm: sha11603worker2 # [ 0.871790] Synopsys Designware Multimedia Card Interface Driver1604lb # [ 0.899559] ima: No architecture policies found1605worker2 # [ 0.872152] sdhci-pltfm: SDHCI platform and OF driver helper1606lb # [ 0.903509] input: gpio-keys as /devices/platform/gpio-keys/input/input01607worker2 # [ 0.876275] hid: raw HID events driver (C) Jiri Kosina1608client # [ 0.866994] sdhci: Secure Digital Host Controller Interface driver1609worker2 # [ 0.876535] usbcore: registered new interface driver usbhid1610client # [ 0.867003] sdhci: Copyright(c) Pierre Ossman1611worker2 # [ 0.876541] usbhid: USB HID core driver1612client # [ 0.867271] Synopsys Designware Multimedia Card Interface Driver1613client # [ 0.867634] sdhci-pltfm: SDHCI platform and OF driver helper1614client # [ 0.871861] hid: raw HID events driver (C) Jiri Kosina1615worker2 # [ 0.879384] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1616client # [ 0.872116] usbcore: registered new interface driver usbhid1617worker2 # [ 0.880883] drop_monitor: Initializing network drop monitor service1618client # [ 0.872123] usbhid: USB HID core driver1619worker2 # [ 0.881018] NET: Registered PF_INET6 protocol family1620worker2 # [ 0.884019] Segment Routing with IPv61621worker2 # [ 0.884040] In-situ OAM (IOAM) with IPv61622worker2 # [ 0.884067] NET: Registered PF_PACKET protocol family1623worker1 # [ 0.935904] clk: Disabling unused clocks1624worker1 # [ 0.935929] PM: genpd: Disabling unused power domains1625client # [ 0.875033] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1626client # [ 0.877643] drop_monitor: Initializing network drop monitor service1627worker1 # [ 0.939998] Freeing unused kernel memory: 4736K1628client # [ 0.877814] NET: Registered PF_INET6 protocol family1629worker1 # [ 0.940188] Run /init as init process1630client # [ 0.879710] Segment Routing with IPv61631client # [ 0.879728] In-situ OAM (IOAM) with IPv61632worker2 # [ 0.885901] 9pnet: Installing 9P2000 support1633worker2 # [ 0.885984] Key type dns_resolver registered1634client # [ 0.879755] NET: Registered PF_PACKET protocol family1635worker2 # [ 0.892326] registered taskstats version 11636client # [ 0.881594] 9pnet: Installing 9P2000 support1637lb # [ 0.920507] clk: Disabling unused clocks1638worker2 # [ 0.892482] Loading compiled-in X.509 certificates1639client # [ 0.881637] Key type dns_resolver registered1640lb # [ 0.920530] PM: genpd: Disabling unused power domains1641lb # [ 0.924584] Freeing unused kernel memory: 4736K1642client # [ 0.888251] registered taskstats version 11643lb # [ 0.924783] Run /init as init process1644client # [ 0.888424] Loading compiled-in X.509 certificates1645worker2 # [ 0.900865] Demotion targets for Node 0: null1646worker2 # [ 0.900962] Key type .fscrypt registered1647worker2 # [ 0.900968] Key type fscrypt-provisioning registered1648worker2 # [ 0.901058] ima: No TPM chip found, activating TPM-bypass!1649worker2 # [ 0.901078] ima: Allocated hash algorithm: sha11650worker1 # [ 0.956076] systemd[1]: Successfully made /usr/ read-only.1651worker2 # [ 0.901097] ima: No architecture policies found1652worker2 # [ 0.905060] input: gpio-keys as /devices/platform/gpio-keys/input/input01653client # [ 0.896849] Demotion targets for Node 0: null1654client # [ 0.896944] Key type .fscrypt registered1655client # [ 0.896950] Key type fscrypt-provisioning registered1656client # [ 0.897037] ima: No TPM chip found, activating TPM-bypass!1657client # [ 0.897056] ima: Allocated hash algorithm: sha11658client # [ 0.897076] ima: No architecture policies found1659client # [ 0.901159] input: gpio-keys as /devices/platform/gpio-keys/input/input01660lb # [ 0.939645] systemd[1]: Successfully made /usr/ read-only.1661worker2 # [ 0.922014] clk: Disabling unused clocks1662worker2 # [ 0.922037] PM: genpd: Disabling unused power domains1663worker2 # [ 0.926099] Freeing unused kernel memory: 4736K1664worker2 # [ 0.926298] Run /init as init process1665client # [ 0.918054] clk: Disabling unused clocks1666client # [ 0.918073] PM: genpd: Disabling unused power domains1667client # [ 0.922146] Freeing unused kernel memory: 4736K1668client # [ 0.922352] Run /init as init process1669worker2 # [ 0.943455] systemd[1]: Successfully made /usr/ read-only.1670client # [ 0.938503] systemd[1]: Successfully made /usr/ read-only.1671worker1 # [ 1.121991] usb 1-1: new high-speed USB device number 2 using ehci-pci1672lb # [ 1.121613] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker2 # [ 1.109441] usb 1-1: new high-speed USB device number 2 using ehci-pci1674client # [ 1.104509] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker1 # [ 1.274536] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker1 # [ 1.290854] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1677lb # [ 1.274067] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11678worker1 # [ 1.302731] systemd[1]: Detected virtualization qemu.1679worker1 # [ 1.304722] systemd[1]: Detected architecture arm64.1680worker1 # [ 1.306677] systemd[1]: Running in initrd.1681worker1 # [ 1.309217] systemd[1]: Initializing machine ID from random generator.1682worker1 # [ 1.312146] systemd[1]: Hostname set to <worker1>.1683lb # [ 1.280194] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1684lb # [ 1.292648] systemd[1]: Detected virtualization qemu.1685lb # [ 1.294858] systemd[1]: Detected architecture arm64.1686worker2 # [ 1.263931] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11687lb # [ 1.296760] systemd[1]: Running in initrd.1688client # [ 1.257011] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11689lb # [ 1.299344] systemd[1]: Initializing machine ID from random generator.1690lb # [ 1.302150] systemd[1]: Hostname set to <lb>.1691worker2 # [ 1.278191] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1692worker2 # [ 1.290089] systemd[1]: Detected virtualization qemu.1693worker2 # [ 1.292126] systemd[1]: Detected architecture arm64.1694client # [ 1.273239] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1695worker2 # [ 1.294082] systemd[1]: Running in initrd.1696client # [ 1.285187] systemd[1]: Detected virtualization qemu.1697client # [ 1.287131] systemd[1]: Detected architecture arm64.1698worker2 # [ 1.296491] systemd[1]: Initializing machine ID from random generator.1699client # [ 1.289026] systemd[1]: Running in initrd.1700worker2 # [ 1.299287] systemd[1]: Hostname set to <worker2>.1701client # [ 1.291526] systemd[1]: Initializing machine ID from random generator.1702client # [ 1.294362] systemd[1]: Hostname set to <client>.1703worker1 # [ 1.358262] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704lb # [ 1.357787] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705worker2 # [ 1.345652] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706client # [ 1.340712] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707worker1 # [ 1.477971] usb 1-2: new high-speed USB device number 3 using ehci-pci1708lb # [ 1.477589] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.469414] usb 1-2: new high-speed USB device number 3 using ehci-pci1710client # [ 1.464486] usb 1-2: new high-speed USB device number 3 using ehci-pci1711worker1 # [ 1.612029] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712lb # [ 1.605500] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713worker1 # [ 1.638757] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21714worker1 # [ 1.646063] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01715client # [ 1.592210] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716worker2 # [ 1.606067] systemd[1]: bpf-restrict-fs: LSM BPF program attached1717lb # [ 1.634358] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718lb # [ 1.639734] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01719client # [ 1.621337] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21720worker2 # [ 1.633990] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721client # [ 1.626949] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722worker2 # [ 1.639334] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723worker1 # [ 1.722029] systemd[1]: Queued start job for default target Initrd Default Target.1724worker1 # [ 1.732277] systemd[1]: Created slice Slice /system/modprobe.1725worker1 # [ 1.733462] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1726worker1 # [ 1.734891] systemd[1]: Expecting device /dev/disk/by-label/nixos...1727worker1 # [ 1.734928] systemd[1]: Reached target Path Units.1728worker1 # [ 1.734948] systemd[1]: Reached target Slice Units.1729worker1 # [ 1.734969] systemd[1]: Reached target Swaps.1730worker1 # [ 1.734987] systemd[1]: Reached target Timer Units.1731worker1 # [ 1.735178] systemd[1]: Listening on D-Bus System Message Bus Socket.1732worker1 # [ 1.735327] systemd[1]: Listening on Journal Socket (/dev/log).1733worker1 # [ 1.735467] systemd[1]: Listening on Journal Sockets.1734worker1 # [ 1.735595] systemd[1]: Listening on udev Control Socket.1735worker1 # [ 1.735706] systemd[1]: Listening on udev Kernel Socket.1736worker1 # [ 1.735728] systemd[1]: Reached target Socket Units.1737worker1 # [ 1.746685] systemd[1]: Starting Create List of Static Device Nodes...1738lb # [ 1.721870] systemd[1]: Queued start job for default target Initrd Default Target.1739worker1 # [ 1.747868] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1740worker1 # [ 1.755260] systemd[1]: Mounting Kernel Configuration File System...1741lb # [ 1.732001] systemd[1]: Created slice Slice /system/modprobe.1742lb # [ 1.733226] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1743lb # [ 1.734723] systemd[1]: Expecting device /dev/disk/by-label/nixos...1744lb # [ 1.734764] systemd[1]: Reached target Path Units.1745lb # [ 1.734785] systemd[1]: Reached target Slice Units.1746lb # [ 1.734804] systemd[1]: Reached target Swaps.1747lb # [ 1.734824] systemd[1]: Reached target Timer Units.1748lb # [ 1.735021] systemd[1]: Listening on D-Bus System Message Bus Socket.1749worker1 # [ 1.766220] systemd[1]: Starting Journal Service...1750lb # [ 1.735176] systemd[1]: Listening on Journal Socket (/dev/log).1751lb # [ 1.735321] systemd[1]: Listening on Journal Sockets.1752lb # [ 1.735462] systemd[1]: Listening on udev Control Socket.1753lb # [ 1.735574] systemd[1]: Listening on udev Kernel Socket.1754lb # [ 1.735596] systemd[1]: Reached target Socket Units.1755lb # [ 1.746924] systemd[1]: Starting Create List of Static Device Nodes...1756client # [ 1.710045] systemd[1]: Queued start job for default target Initrd Default Target.1757worker2 # [ 1.720313] systemd[1]: Queued start job for default target Initrd Default Target.1758lb # [ 1.748131] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1759worker2 # [ 1.727598] systemd[1]: Created slice Slice /system/modprobe.1760client # [ 1.718829] systemd[1]: Created slice Slice /system/modprobe.1761worker2 # [ 1.728727] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1762client # [ 1.720058] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1763lb # [ 1.757736] systemd[1]: Mounting Kernel Configuration File System...1764worker2 # [ 1.730016] systemd[1]: Expecting device /dev/disk/by-label/nixos...1765worker2 # [ 1.730046] systemd[1]: Reached target Path Units.1766client # [ 1.721609] systemd[1]: Expecting device /dev/disk/by-label/nixos...1767worker2 # [ 1.730064] systemd[1]: Reached target Slice Units.1768client # [ 1.721649] systemd[1]: Reached target Path Units.1769worker2 # [ 1.730082] systemd[1]: Reached target Swaps.1770client # [ 1.721671] systemd[1]: Reached target Slice Units.1771worker2 # [ 1.730100] systemd[1]: Reached target Timer Units.1772client # [ 1.721693] systemd[1]: Reached target Swaps.1773worker2 # [ 1.730298] systemd[1]: Listening on D-Bus System Message Bus Socket.1774client # [ 1.721711] systemd[1]: Reached target Timer Units.1775worker2 # [ 1.730439] systemd[1]: Listening on Journal Socket (/dev/log).1776client # [ 1.721899] systemd[1]: Listening on D-Bus System Message Bus Socket.1777worker2 # [ 1.730569] systemd[1]: Listening on Journal Sockets.1778worker1 # [ 1.790356] systemd[1]: Starting Load Kernel Modules...1779lb # [ 1.765851] systemd[1]: Starting Journal Service...1780worker2 # [ 1.730692] systemd[1]: Listening on udev Control Socket.1781client # [ 1.722052] systemd[1]: Listening on Journal Socket (/dev/log).1782worker2 # [ 1.730795] systemd[1]: Listening on udev Kernel Socket.1783client # [ 1.722194] systemd[1]: Listening on Journal Sockets.1784worker1 # [ 1.791418] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1785worker2 # [ 1.730814] systemd[1]: Reached target Socket Units.1786client # [ 1.722332] systemd[1]: Listening on udev Control Socket.1787client # [ 1.722446] systemd[1]: Listening on udev Kernel Socket.1788worker2 # [ 1.741077] systemd[1]: Starting Create List of Static Device Nodes...1789client # [ 1.722469] systemd[1]: Reached target Socket Units.1790worker2 # [ 1.741157] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1791client # [ 1.733878] systemd[1]: Starting Create List of Static Device Nodes...1792client # [ 1.735067] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1793worker2 # [ 1.746744] systemd[1]: Mounting Kernel Configuration File System...1794client # [ 1.744632] systemd[1]: Mounting Kernel Configuration File System...1795worker1 # [ 1.810121] systemd[1]: Starting Coldplug All udev Devices...1796worker2 # [ 1.761799] systemd[1]: Starting Journal Service...1797client # [ 1.752733] systemd[1]: Starting Journal Service...1798lb # [ 1.789743] systemd[1]: Starting Load Kernel Modules...1799lb # [ 1.789854] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1800worker1 # [ 1.819994] systemd-journald[72]: Collecting audit messages is disabled.1801worker1 # [ 1.826387] systemd[1]: Finished Create List of Static Device Nodes.1802worker1 # [ 1.827037] systemd[1]: Mounted Kernel Configuration File System.1803client # [ 1.776239] systemd[1]: Starting Load Kernel Modules...1804worker2 # [ 1.785542] systemd[1]: Starting Load Kernel Modules...1805worker2 # [ 1.785641] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1806client # [ 1.777366] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1807lb # [ 1.817818] systemd[1]: Starting Coldplug All udev Devices...1808worker1 # [ 1.846114] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1809lb # [ 1.826102] systemd-journald[72]: Collecting audit messages is disabled.1810worker2 # [ 1.797573] systemd[1]: Starting Coldplug All udev Devices...1811worker2 # [ 1.806942] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1812worker2 # [ 1.808656] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1813client # [ 1.792647] systemd[1]: Starting Coldplug All udev Devices...1814worker1 # [ 1.855109] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1815lb # [ 1.837651] systemd[1]: Finished Create List of Static Device Nodes.1816lb # [ 1.841916] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1817lb # [ 1.842209] systemd[1]: Mounted Kernel Configuration File System.1818worker2 # [ 1.815942] systemd[1]: Finished Create List of Static Device Nodes.1819worker2 # [ 1.816604] systemd[1]: Mounted Kernel Configuration File System.1820client # [ 1.810271] systemd-journald[72]: Collecting audit messages is disabled.1821worker1 # [ 1.878538] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1822worker2 # [ 1.826832] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1823worker1 # [ 1.880885] systemd[1]: Starting Create Static Device Nodes in /dev...1824worker1 # [ 1.882840] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1825client # [ 1.824577] systemd[1]: Finished Create List of Static Device Nodes.1826worker2 # [ 1.834604] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01827worker2 # [ 1.834831] [drm] features: -virgl +edid -resource_blob -host_visible1828worker2 # [ 1.834840] [drm] features: -context_init1829worker2 # [ 1.835525] [drm] number of scanouts: 11830worker2 # [ 1.835542] [drm] number of cap sets: 01831lb # [ 1.867224] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1832worker1 # [ 1.895149] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01833worker1 # [ 1.895385] [drm] features: -virgl +edid -resource_blob -host_visible1834worker1 # [ 1.895395] [drm] features: -context_init1835client # [ 1.832741] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1836worker1 # [ 1.896076] [drm] number of scanouts: 11837worker1 # [ 1.896093] [drm] number of cap sets: 01838client # [ 1.832986] systemd[1]: Mounted Kernel Configuration File System.1839lb # [ 1.885638] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1840client # [ 1.845759] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1841lb # [ 1.888330] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1842worker2 # [ 1.853665] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1843worker2 # [ 1.853682] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01844lb # [ 1.890717] systemd[1]: Starting Create Static Device Nodes in /dev...1845worker1 # [ 1.910336] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1846worker1 # [ 1.910352] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01847worker2 # [ 1.870981] systemd-journald[72]: Collecting audit messages is disabled.1848client # [ 1.863980] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1849client # [ 1.866367] systemd[1]: Starting Create Static Device Nodes in /dev...1850lb # [ 1.902784] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01851lb # [ 1.903009] [drm] features: -virgl +edid -resource_blob -host_visible1852worker2 # [ 1.872553] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1853lb # [ 1.903018] [drm] features: -context_init1854lb # [ 1.903688] [drm] number of scanouts: 11855worker2 # [ 1.873980] systemd[1]: Starting Create Static Device Nodes in /dev...1856lb # [ 1.903705] [drm] number of cap sets: 01857worker1 # [ 1.934571] systemd[1]: Finished Create Static Device Nodes in /dev.1858worker1 # [ 1.934897] systemd[1]: Reached target Preparation for Local File Systems.1859worker1 # [ 1.934922] systemd[1]: Reached target Local File Systems.1860worker1 # [ 1.940894] systemd[1]: Starting Rule-based Manager for Device Events and Files...1861client # [ 1.876512] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1862client # [ 1.882459] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01863client # [ 1.882688] [drm] features: -virgl +edid -resource_blob -host_visible1864client # [ 1.882699] [drm] features: -context_init1865client # [ 1.883378] [drm] number of scanouts: 11866client # [ 1.883396] [drm] number of cap sets: 01867worker1 # [ 1.946377] Console: switching to colour frame buffer device 160x501868worker2 # [ 1.893956] Console: switching to colour frame buffer device 160x501869worker2 # [ 1.900520] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1870lb # [ 1.925910] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1871lb # [ 1.925927] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01872client # [ 1.900759] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1873worker1 # [ 1.970617] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1874client # [ 1.900775] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01875lb # [ 1.946192] systemd[1]: Finished Create Static Device Nodes in /dev.1876worker2 # [ 1.917990] systemd[1]: Finished Create Static Device Nodes in /dev.1877lb # [ 1.946378] systemd[1]: Reached target Preparation for Local File Systems.1878worker2 # [ 1.918200] systemd[1]: Reached target Preparation for Local File Systems.1879lb # [ 1.946404] systemd[1]: Reached target Local File Systems.1880worker2 # [ 1.918232] systemd[1]: Reached target Local File Systems.1881worker2 # [ 1.925728] systemd[1]: Starting Rule-based Manager for Device Events and Files...1882lb # [ 1.953865] systemd[1]: Starting Rule-based Manager for Device Events and Files...1883worker1 # [ 1.985629] systemd[1]: Started Journal Service.1884client # [ 1.925014] systemd[1]: Finished Create Static Device Nodes in /dev.1885client # [ 1.925204] systemd[1]: Reached target Preparation for Local File Systems.1886client # [ 1.925230] systemd[1]: Reached target Local File Systems.1887worker1 # [ 1.976384] systemd-modules-load[74]: Inserted module 'dm_mod'1888lb # [ 1.958829] Console: switching to colour frame buffer device 160x501889lb # [ 1.965372] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1890worker1 # [ 1.978812] systemd-modules-load[74]: Module 'virtio_balloon' is built in1891client # [ 1.931343] systemd[1]: Starting Rule-based Manager for Device Events and Files...1892client # [ 1.933662] Console: switching to colour frame buffer device 160x501893worker1 # [ 1.991789] systemd-modules-load[74]: Module 'virtio_console' is built in1894worker1 # [ 1.993031] systemd-modules-load[74]: Inserted module 'virtio_gpu'1895worker1 # [ 1.994012] systemd-modules-load[74]: Module 'virtio_rng' is built in1896worker2 # [ 1.953588] systemd[1]: Finished Load Kernel Modules.1897lb # [ 1.982090] systemd[1]: Finished Load Kernel Modules.1898worker1 # [ 1.994963] systemd[1]: Finished Load Kernel Modules.1899worker1 # [ 1.995798] systemd[1]: Starting Apply Kernel Variables...1900lb # [ 1.985000] systemd[1]: Starting Apply Kernel Variables...1901worker2 # [ 1.956727] systemd[1]: Starting Apply Kernel Variables...1902client # [ 1.961167] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1903worker1 # [ 2.016152] systemd[1]: Starting Create System Files and Directories...1904lb # [ 1.996173] systemd-modules-load[74]: Inserted module 'dm_mod'1905lb # [ 1.997291] systemd-modules-load[74]: Module 'virtio_balloon' is built in1906lb # [ 1.998352] systemd-modules-load[74]: Module 'virtio_console' is built in1907lb # [ 1.999390] systemd-modules-load[74]: Inserted module 'virtio_gpu'1908client # [ 1.977657] systemd[1]: Finished Load Kernel Modules.1909lb # [ 2.013934] systemd[1]: Started Journal Service.1910client # [ 1.984855] systemd[1]: Starting Apply Kernel Variables...1911worker1 # [ 2.033647] systemd-udevd[78]: Using default interface naming scheme 'v261'.1912lb # [ 2.016439] systemd-modules-load[74]: Module 'virtio_rng' is built in1913lb # [ 2.017507] systemd[1]: Starting Create System Files and Directories...1914worker2 # [ 2.009951] systemd[1]: Finished Apply Kernel Variables.1915worker1 # [ 2.052373] systemd[1]: Finished Create System Files and Directories.1916worker1 # [ 2.054048] systemd[1]: Finished Apply Kernel Variables.1917lb # [ 2.031061] systemd[1]: Finished Apply Kernel Variables.1918client # [ 1.996396] systemd-modules-load[74]: Inserted module 'dm_mod'1919client # [ 2.013917] systemd[1]: Started Journal Service.1920client # [ 2.000432] systemd-modules-load[74]: Module 'virtio_balloon' is built in1921client # [ 2.001555] systemd-modules-load[74]: Module 'virtio_console' is built in1922client # [ 2.002596] systemd-modules-load[74]: Inserted module 'virtio_gpu'1923client # [ 2.003580] systemd-modules-load[74]: Module 'virtio_rng' is built in1924worker2 # [ 2.028750] systemd[1]: Started Journal Service.1925lb # [ 2.049443] systemd[1]: Finished Create System Files and Directories.1926worker1 # [ 2.076628] systemd[1]: Started Rule-based Manager for Device Events and Files.1927worker2 # [ 2.026357] systemd-modules-load[73]: Inserted module 'dm_mod'1928worker2 # [ 2.027586] systemd-modules-load[73]: Module 'virtio_balloon' is built in1929lb # [ 2.058156] systemd-udevd[78]: Using default interface naming scheme 'v261'.1930worker2 # [ 2.032672] systemd-modules-load[73]: Module 'virtio_console' is built in1931worker2 # [ 2.035687] systemd-modules-load[73]: Inserted module 'virtio_gpu'1932client # [ 2.028932] systemd[1]: Starting Create System Files and Directories...1933client # [ 2.030260] systemd-udevd[79]: Using default interface naming scheme 'v261'.1934worker2 # [ 2.044362] systemd-modules-load[73]: Module 'virtio_rng' is built in1935worker2 # [ 2.045646] systemd-udevd[78]: Using default interface naming scheme 'v261'.1936worker2 # [ 2.046709] systemd[1]: Starting Create System Files and Directories...1937client # [ 2.037335] systemd[1]: Finished Apply Kernel Variables.1938lb # [ 2.086917] systemd[1]: Started Rule-based Manager for Device Events and Files.1939client # [ 2.049031] systemd[1]: Finished Create System Files and Directories.1940worker2 # [ 2.061832] systemd[1]: Finished Create System Files and Directories.1941worker1 # [ 2.123794] systemd[1]: Starting Virtual Console Setup...1942client # [ 2.064106] systemd[1]: Started Rule-based Manager for Device Events and Files.1943worker2 # [ 2.077155] systemd[1]: Started Rule-based Manager for Device Events and Files.1944lb # [ 2.144113] systemd[1]: Starting Virtual Console Setup...1945worker1 # [ 2.173227] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1946worker2 # [ 2.123586] systemd[1]: Starting Virtual Console Setup...1947worker1 # [ 2.180193] systemd[1]: Finished Virtual Console Setup.1948client # [ 2.120231] systemd[1]: Starting Virtual Console Setup...1949lb # [ 2.196465] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1950lb # [ 2.199716] systemd[1]: Finished Virtual Console Setup.1951worker2 # [ 2.177742] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1952worker2 # [ 2.179702] systemd[1]: Finished Virtual Console Setup.1953client # [ 2.168449] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954client # [ 2.171681] systemd[1]: Finished Virtual Console Setup.1955worker1 # [ 2.796703] systemd[1]: Finished Coldplug All udev Devices.1956worker1 # [ 2.798514] systemd[1]: Reached target System Initialization.1957worker1 # [ 2.804483] systemd[1]: Reached target Basic System.1958lb # [ 2.791263] systemd[1]: Finished Coldplug All udev Devices.1959lb # [ 2.792459] systemd[1]: Reached target System Initialization.1960lb # [ 2.793272] systemd[1]: Reached target Basic System.1961client # [ 2.761908] systemd[1]: Finished Coldplug All udev Devices.1962client # [ 2.762901] systemd[1]: Reached target System Initialization.1963client # [ 2.763730] systemd[1]: Reached target Basic System.1964worker2 # [ 2.782667] systemd[1]: Finished Coldplug All udev Devices.1965worker2 # [ 2.788367] systemd[1]: Reached target System Initialization.1966worker2 # [ 2.789246] systemd[1]: Reached target Basic System.1967worker1 # [ 2.935856] (udev-worker)[98]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1968worker1 # [ 2.966016] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1969client # [ 2.912181] (udev-worker)[100]: Network interface NamePolicy= disabled on kernel command line.1970worker1 # [ 2.977931] (udev-worker)[98]: Network interface NamePolicy= disabled on kernel command line.1971worker2 # [ 2.946526] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 2.980377] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1973client # [ 2.939970] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1974client # [ 2.945872] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1975lb # [ 2.992976] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976worker2 # [ 2.970751] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1977lb # [ 3.000949] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1978worker2 # [ 2.975363] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1979worker1 # [ 3.029865] systemd[1]: Found device /dev/disk/by-label/nixos.1980worker1 # [ 3.032155] systemd[1]: Reached target Initrd Root Device.1981worker1 # [ 3.035167] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982lb # [ 3.048808] systemd[1]: Found device /dev/disk/by-label/nixos.1983lb # [ 3.051982] systemd[1]: Reached target Initrd Root Device.1984lb # [ 3.055609] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985client # [ 3.024187] systemd[1]: Found device /dev/disk/by-label/nixos.1986worker2 # [ 3.036846] systemd[1]: Found device /dev/disk/by-label/nixos.1987client # [ 3.025155] systemd[1]: Reached target Initrd Root Device.1988worker2 # [ 3.041026] systemd[1]: Reached target Initrd Root Device.1989worker1 # [ 3.093271] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1990client # [ 3.029270] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1991worker2 # [ 3.045461] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992worker1 # [ 3.100155] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993worker1 # [ 3.103815] systemd[1]: Mounting /sysroot...1994lb # [ 3.109895] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1995worker1 # [ 3.152821] EXT4-fs (vda): mounted filesystem 4e060669-9984-4396-8f73-f143f0318f33 r/w with ordered data mode. Quota mode: none.1996client # [ 3.077059] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1997lb # [ 3.117551] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998worker1 # [ 3.146247] systemd[1]: Mounted /sysroot.1999client # [ 3.083663] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.2000worker1 # [ 3.149203] systemd[1]: Reached target Initrd Root File System.
loading 2,241 hidden lines…
4242worker2 # [ 50.642716] nix-daemon[843]: reaped child process 926, status = succeeded4243lb # [ 50.678893] niks3-server[1016]: 2026/09/22 16:54:31 INFO Received uploads request method=POST path=/api/pending_closures4244worker2 # [ 50.665660] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.311Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4245worker2 # [ 50.667524] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.311Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4246worker2 # [ 50.709105] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.356Z level=INFO msg="Uploading 1 narinfos"4247lb # [ 50.727553] niks3-server[1016]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4248lb # [ 50.740612] niks3-server[1016]: 2026/09/22 16:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4249lb # [ 50.752326] niks3-server[1016]: 2026/09/22 16:54:31 INFO Signed narinfos id=4 count=14250lb # [ 50.755023] niks3-server[1016]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4251lb # [ 50.763838] niks3-server[1016]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4252worker2 # [ 50.745502] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.392Z level=INFO msg="Upload complete. (294ms)"4253lb # [ 50.773905] niks3-server[1016]: 2026/09/22 16:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4254lb # [ 50.778017] niks3-server[1016]: 2026/09/22 16:54:31 INFO Completed upload id=44255lb # [ 50.781485] niks3-server[1016]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4256worker2 # [ 50.757166] nix-grpc-daemon[774]: ts=2026-09-22T16:54:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=2 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14257worker1 # [ 50.818829] nix-grpc-daemon[767]: ts=2026-09-22T16:54:31Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04258worker1 # [ 50.824616] nix-grpc-daemon[767]: ts=2026-09-22T16:54:31Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.4:50051 clients=14259worker2 # [ 50.777591] nix-daemon[843]: reaped child process 905, status = succeeded4260worker2 # [ 50.812880] nix-grpc-daemon[774]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304261worker2 # [ 50.829046] nix-grpc-daemon[774]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304262worker2 # [ 50.844527] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.491Z level=INFO msg="All 2 paths already cached"4263client # copying 1 paths to 'lb:50051'...4264client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4265worker2 # [ 50.855114] nix-grpc-daemon[774]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406304266worker2 # [ 50.868882] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4267worker2 # [ 51.024225] nix-daemon[843]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4268worker2 # [ 51.044721] nix-daemon[843]: reaped child process 933, status = succeeded4269lb # [ 51.077848] niks3-server[1016]: 2026/09/22 16:54:31 INFO Received uploads request method=POST path=/api/pending_closures4270worker2 # [ 51.059831] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.706Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4271worker2 # [ 51.061753] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.706Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4272lb # [ 51.114243] niks3-server[1016]: 2026/09/22 16:54:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4273worker2 # [ 51.093362] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.740Z level=INFO msg="Uploading 1 narinfos"4274lb # [ 51.128368] niks3-server[1016]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4275lb # [ 51.145953] niks3-server[1016]: 2026/09/22 16:54:31 INFO Signed narinfos id=5 count=14276lb # [ 51.152704] niks3-server[1016]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4277worker2 # [ 51.133460] nix-grpc-daemon[857]: time=2026-09-22T16:54:31.780Z level=INFO msg="Upload complete. (247ms)"4278lb # [ 51.162244] niks3-server[1016]: 2026/09/22 16:54:31 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4279lb # [ 51.169264] niks3-server[1016]: 2026/09/22 16:54:31 INFO Completed upload id=54280worker2 # [ 51.143260] nix-grpc-daemon[774]: ts=2026-09-22T16:54:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_in=7524281lb # [ 51.173817] niks3-server[1016]: 2026/09/22 16:54:31 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4282worker2 # [ 51.156615] nix-grpc-daemon[774]: ts=2026-09-22T16:54:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304283worker2 # [ 51.170179] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4284worker2 # [ 51.175433] nix-daemon[843]: reaped child process 928, status = succeeded4285client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4286client # farm-top> worker2: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4287worker2 # [ 51.189998] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4288client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4289client # farm-top> LOG-farm-top-t14290worker2 # [ 52.660104] nix-daemon[843]: reaped child process 937, status = succeeded4291worker2 # [ 52.770585] nix-daemon[843]: accepted connection from pid 953, user nix-grpc-daemon (trusted)4292worker2 # [ 52.792051] nix-daemon[843]: reaped child process 956, status = succeeded4293lb # [ 52.818737] niks3-server[1016]: 2026/09/22 16:54:33 INFO Received uploads request method=POST path=/api/pending_closures4294worker2 # [ 52.803191] nix-grpc-daemon[857]: time=2026-09-22T16:54:33.448Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4295worker2 # [ 52.804895] nix-grpc-daemon[857]: time=2026-09-22T16:54:33.448Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4296lb # [ 52.856231] niks3-server[1016]: 2026/09/22 16:54:33 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4297lb # [ 52.876253] niks3-server[1016]: 2026/09/22 16:54:33 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4298worker2 # [ 52.857396] nix-grpc-daemon[857]: time=2026-09-22T16:54:33.504Z level=INFO msg="Uploading 1 narinfos"4299lb # [ 52.885652] niks3-server[1016]: 2026/09/22 16:54:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4300lb # [ 52.893336] niks3-server[1016]: 2026/09/22 16:54:33 INFO Signed narinfos id=6 count=14301lb # [ 52.898306] niks3-server[1016]: 2026/09/22 16:54:33 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4302lb # [ 52.909294] niks3-server[1016]: 2026/09/22 16:54:33 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4303lb # [ 52.916852] niks3-server[1016]: 2026/09/22 16:54:33 INFO Completed upload id=64304worker2 # [ 52.887226] nix-grpc-daemon[857]: time=2026-09-22T16:54:33.534Z level=INFO msg="Upload complete. (285ms)"4305lb # [ 52.918653] niks3-server[1016]: 2026/09/22 16:54:33 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4306worker2 # [ 52.901230] nix-grpc-daemon[774]: ts=2026-09-22T16:54:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14307worker2 # [ 52.919103] nix-daemon[843]: reaped child process 935, status = succeeded4308client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.94 seconds)4309client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4310client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.19 seconds)4311client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4312client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4313client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4314client # copying 3 paths...4315client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4316client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4317client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4318client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds)4319(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.49 seconds)4320subtest: repeat is answered Cached by the scheduler without building4321worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4322worker1 # finding garbage collector roots...4323worker1 # deleting specified paths...4324worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4325worker1 # deleting unused links...4326worker1 # note: hard linking is currently saving 0.0 KiB4327worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.24 seconds)4328worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4329worker2 # finding garbage collector roots...4330worker2 # deleting specified paths...4331worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4332worker2 # deleting unused links...4333worker2 # note: hard linking is currently saving 0.0 KiB4334worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.22 seconds)4335client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24336worker2 # [ 54.179764] nix-grpc-daemon[774]: ts=2026-09-22T16:54:34Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406304337client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4338client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4339worker2 # [ 54.199795] nix-grpc-daemon[774]: ts=2026-09-22T16:54:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304340worker2 # [ 54.210943] nix-grpc-daemon[857]: time=2026-09-22T16:54:34.857Z level=INFO msg="All 2 paths already cached"4341worker1 # [ 54.282395] nix-grpc-daemon[767]: ts=2026-09-22T16:54:34Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14342client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.30 seconds)4343client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4344client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.16 seconds)4345lb: must succeed: curl -sf localhost:9901/clusters4346lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4347lb: must succeed: curl -sf localhost:9901/clusters4348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4349worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4350worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4351(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.20 seconds)4352subtest: inputs already in the cache are not sent to the scheduler4353lb: must succeed: curl -sf localhost:9901/clusters4354lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4355lb: must succeed: curl -sf localhost:9901/clusters4356lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4357worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4358worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4359client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24360worker1 # [ 55.138860] nix-grpc-daemon[767]: ts=2026-09-22T16:54:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366964361client # don't know how to build these paths:4362client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4363worker1 # [ 55.149712] nix-grpc-daemon[767]: ts=2026-09-22T16:54:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366964364worker1 # [ 55.175318] nix-grpc-daemon[767]: ts=2026-09-22T16:54:35Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04365worker1 # [ 55.181074] nix-grpc-daemon[767]: ts=2026-09-22T16:54:35Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.4:50051 clients=14366worker2 # [ 55.171310] nix-grpc-daemon[774]: ts=2026-09-22T16:54:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304367worker2 # [ 55.187668] nix-grpc-daemon[774]: ts=2026-09-22T16:54:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304368worker2 # [ 55.202421] nix-grpc-daemon[857]: time=2026-09-22T16:54:35.849Z level=INFO msg="All 2 paths already cached"4369client # copying 1 paths to 'lb:50051'...4370client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4371worker2 # [ 55.213170] nix-grpc-daemon[774]: ts=2026-09-22T16:54:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406304372worker2 # [ 55.228145] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4373worker2 # [ 55.392150] nix-daemon[843]: accepted connection from pid 973, user nix-grpc-daemon (trusted)4374worker2 # [ 55.410105] nix-daemon[843]: reaped child process 976, status = succeeded4375lb # [ 55.443630] niks3-server[1016]: 2026/09/22 16:54:36 INFO Received uploads request method=POST path=/api/pending_closures4376worker2 # [ 55.425891] nix-grpc-daemon[857]: time=2026-09-22T16:54:36.072Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4377worker2 # [ 55.427452] nix-grpc-daemon[857]: time=2026-09-22T16:54:36.072Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4378lb # [ 55.475224] niks3-server[1016]: 2026/09/22 16:54:36 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4379worker2 # [ 55.459523] nix-grpc-daemon[857]: time=2026-09-22T16:54:36.106Z level=INFO msg="Uploading 1 narinfos"4380lb # [ 55.490507] niks3-server[1016]: 2026/09/22 16:54:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4381lb # [ 55.500961] niks3-server[1016]: 2026/09/22 16:54:36 INFO Signed narinfos id=7 count=14382lb # [ 55.503952] niks3-server[1016]: 2026/09/22 16:54:36 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4383worker2 # [ 55.490955] nix-grpc-daemon[857]: time=2026-09-22T16:54:36.138Z level=INFO msg="Upload complete. (245ms)"4384lb # [ 55.518601] niks3-server[1016]: 2026/09/22 16:54:36 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4385lb # [ 55.527949] niks3-server[1016]: 2026/09/22 16:54:36 INFO Completed upload id=74386worker2 # [ 55.500725] nix-grpc-daemon[774]: ts=2026-09-22T16:54:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_in=7524387lb # [ 55.534435] niks3-server[1016]: 2026/09/22 16:54:36 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4388worker2 # [ 55.516935] nix-grpc-daemon[774]: ts=2026-09-22T16:54:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304389worker2 # [ 55.526515] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4390worker2 # [ 55.533678] nix-daemon[843]: reaped child process 971, status = succeeded4391client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4392worker2 # [ 55.547498] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4393client # farm-top> worker2: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4394client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4395client # farm-top> LOG-farm-top-t1b4396worker2 # [ 56.811373] nix-daemon[843]: reaped child process 980, status = succeeded4397worker2 # [ 56.934529] nix-daemon[843]: accepted connection from pid 996, user nix-grpc-daemon (trusted)4398worker2 # [ 56.952989] nix-daemon[843]: reaped child process 999, status = succeeded4399lb # [ 56.987413] niks3-server[1016]: 2026/09/22 16:54:37 INFO Received uploads request method=POST path=/api/pending_closures4400worker2 # [ 56.972593] nix-grpc-daemon[857]: time=2026-09-22T16:54:37.616Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4401worker2 # [ 56.975222] nix-grpc-daemon[857]: time=2026-09-22T16:54:37.616Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4402lb # [ 57.032911] niks3-server[1016]: 2026/09/22 16:54:37 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4403worker2 # [ 57.017930] nix-grpc-daemon[857]: time=2026-09-22T16:54:37.664Z level=INFO msg="Uploading 1 narinfos"4404lb # [ 57.050198] niks3-server[1016]: 2026/09/22 16:54:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4405lb # [ 57.058469] niks3-server[1016]: 2026/09/22 16:54:37 INFO Signed narinfos id=8 count=14406lb # [ 57.063452] niks3-server[1016]: 2026/09/22 16:54:37 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4407lb # [ 57.076508] niks3-server[1016]: 2026/09/22 16:54:37 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4408lb # [ 57.079472] niks3-server[1016]: 2026/09/22 16:54:37 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4409lb # [ 57.084861] niks3-server[1016]: 2026/09/22 16:54:37 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4410lb # [ 57.091105] niks3-server[1016]: 2026/09/22 16:54:37 INFO Completed upload id=84411worker2 # [ 57.063947] nix-grpc-daemon[857]: time=2026-09-22T16:54:37.711Z level=INFO msg="Upload complete. (302ms)"4412worker2 # [ 57.077066] nix-grpc-daemon[774]: ts=2026-09-22T16:54:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14413worker2 # [ 57.095466] nix-daemon[843]: reaped child process 978, status = succeeded4414client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.24 seconds)4415client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4416client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.18 seconds)4417lb: must succeed: curl -sf localhost:9901/clusters4418lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4419lb: must succeed: curl -sf localhost:9901/clusters4420lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4421worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4422worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4423(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.78 seconds)4424subtest: outputs read back through any worker carry the cache signature4425client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24426client # don't know how to build these paths:4427client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4428worker1 # [ 57.747444] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366964429worker2 # [ 57.698054] nix-grpc-daemon[774]: ts=2026-09-22T16:54:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304430worker1 # [ 57.777483] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04431worker1 # [ 57.785009] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.3:50051 clients=14432worker1 # [ 57.794632] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964433worker1 # [ 57.806142] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366964434client # copying 1 paths to 'lb:50051'...4435client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4436worker1 # [ 57.821816] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366964437worker1 # [ 57.837640] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4438worker1 # [ 57.865431] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=info event=niks3_push_started pid=9024439worker1 # [ 58.034767] nix-grpc-daemon[902]: time=2026-09-22T16:54:38.669Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""4440worker1 # [ 58.143724] nix-daemon[851]: accepted connection from pid 915, user nix-grpc-daemon (trusted)4441worker1 # [ 58.166446] nix-daemon[851]: reaped child process 918, status = succeeded4442lb # [ 58.146664] niks3-server[1016]: 2026/09/22 16:54:38 INFO Received uploads request method=POST path=/api/pending_closures4443worker1 # [ 58.183684] nix-grpc-daemon[902]: time=2026-09-22T16:54:38.817Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4444worker1 # [ 58.187701] nix-grpc-daemon[902]: time=2026-09-22T16:54:38.817Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4445lb # [ 58.188000] niks3-server[1016]: 2026/09/22 16:54:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4446worker1 # [ 58.223579] nix-grpc-daemon[902]: time=2026-09-22T16:54:38.858Z level=INFO msg="Uploading 1 narinfos"4447lb # [ 58.199397] niks3-server[1016]: 2026/09/22 16:54:38 INFO Signed narinfos id=9 count=14448lb # [ 58.208485] niks3-server[1016]: 2026/09/22 16:54:38 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4449lb # [ 58.221863] niks3-server[1016]: 2026/09/22 16:54:38 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4450worker1 # [ 58.257364] nix-grpc-daemon[902]: time=2026-09-22T16:54:38.892Z level=INFO msg="Upload complete. (218ms)"4451lb # [ 58.234257] niks3-server[1016]: 2026/09/22 16:54:38 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4452lb # [ 58.242589] niks3-server[1016]: 2026/09/22 16:54:38 INFO Completed upload id=94453worker1 # [ 58.268079] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_in=5124454lb # [ 58.245194] niks3-server[1016]: 2026/09/22 16:54:38 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4455worker1 # [ 58.281050] nix-grpc-daemon[767]: ts=2026-09-22T16:54:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964456worker1 # [ 58.287261] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4457worker1 # [ 58.291994] nix-daemon[851]: reaped child process 900, status = succeeded4458client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4459worker1 # [ 58.303458] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4460client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4461lb # [ 59.146420] envoy[895]: conn peer=[2001:db8:1::1]:41144 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9575 tx=4392 ms=67224462lb # [ 59.151257] envoy[895]: conn peer=[2001:db8:1::1]:44388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=744463lb # [ 59.156073] envoy[895]: conn peer=[2001:db8:1::1]:44392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4185 tx=1897 ms=20014464client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4465client # farm-a> LOG-farm-a-sig4466worker1 # [ 59.792663] nix-daemon[851]: reaped child process 922, status = succeeded4467worker1 # [ 59.893378] nix-daemon[851]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4468worker1 # [ 59.909380] nix-daemon[851]: reaped child process 941, status = succeeded4469lb # [ 59.894260] niks3-server[1016]: 2026/09/22 16:54:40 INFO Received uploads request method=POST path=/api/pending_closures4470worker1 # [ 59.927833] nix-grpc-daemon[902]: time=2026-09-22T16:54:40.560Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4471worker1 # [ 59.931019] nix-grpc-daemon[902]: time=2026-09-22T16:54:40.560Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4472lb # [ 59.928369] niks3-server[1016]: 2026/09/22 16:54:40 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4473lb # [ 59.939842] niks3-server[1016]: 2026/09/22 16:54:40 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4474worker1 # [ 59.975351] nix-grpc-daemon[902]: time=2026-09-22T16:54:40.610Z level=INFO msg="Uploading 1 narinfos"4475lb # [ 59.953276] niks3-server[1016]: 2026/09/22 16:54:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4476lb # [ 59.962392] niks3-server[1016]: 2026/09/22 16:54:40 INFO Signed narinfos id=10 count=14477lb # [ 59.971241] niks3-server[1016]: 2026/09/22 16:54:40 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4478lb # [ 59.979536] niks3-server[1016]: 2026/09/22 16:54:40 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4479lb # [ 59.984077] niks3-server[1016]: 2026/09/22 16:54:40 INFO Completed upload id=104480lb # [ 59.987305] niks3-server[1016]: 2026/09/22 16:54:40 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4481worker1 # [ 60.003563] nix-grpc-daemon[902]: time=2026-09-22T16:54:40.638Z level=INFO msg="Upload complete. (285ms)"4482worker1 # [ 60.021577] nix-grpc-daemon[767]: ts=2026-09-22T16:54:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14483worker1 # [ 60.042009] nix-grpc-daemon[767]: ts=2026-09-22T16:54:40Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04484worker1 # [ 60.046807] nix-grpc-daemon[767]: ts=2026-09-22T16:54:40Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14485worker1 # [ 60.051793] nix-daemon[851]: reaped child process 920, status = succeeded4486worker1 # [ 60.081404] nix-grpc-daemon[767]: ts=2026-09-22T16:54:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964487worker1 # [ 60.096781] nix-grpc-daemon[767]: ts=2026-09-22T16:54:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366964488worker1 # [ 60.110753] nix-grpc-daemon[902]: time=2026-09-22T16:54:40.746Z level=INFO msg="All 1 paths already cached"4489client # copying 1 paths to 'lb:50051'...4490client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4491worker1 # [ 60.121477] nix-grpc-daemon[767]: ts=2026-09-22T16:54:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366964492worker1 # [ 60.140465] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4493worker1 # [ 60.284831] nix-daemon[851]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4494worker1 # [ 60.306150] nix-daemon[851]: reaped child process 950, status = succeeded4495lb # [ 60.284946] niks3-server[1016]: 2026/09/22 16:54:40 INFO Received uploads request method=POST path=/api/pending_closures4496worker1 # [ 60.317592] nix-grpc-daemon[902]: time=2026-09-22T16:54:40.952Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4497worker1 # [ 60.319198] nix-grpc-daemon[902]: time=2026-09-22T16:54:40.952Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4498lb # [ 60.317896] niks3-server[1016]: 2026/09/22 16:54:40 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4499lb # [ 60.330401] niks3-server[1016]: 2026/09/22 16:54:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4500worker1 # [ 60.361845] nix-grpc-daemon[902]: time=2026-09-22T16:54:40.996Z level=INFO msg="Uploading 1 narinfos"4501lb # [ 60.342805] niks3-server[1016]: 2026/09/22 16:54:40 INFO Signed narinfos id=11 count=14502lb # [ 60.348384] niks3-server[1016]: 2026/09/22 16:54:40 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4503worker1 # [ 60.382182] nix-grpc-daemon[902]: time=2026-09-22T16:54:41.017Z level=INFO msg="Upload complete. (231ms)"4504lb # [ 60.359546] niks3-server[1016]: 2026/09/22 16:54:41 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4505lb # [ 60.364626] niks3-server[1016]: 2026/09/22 16:54:41 INFO Completed upload id=114506lb # [ 60.367983] niks3-server[1016]: 2026/09/22 16:54:41 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4507worker1 # [ 60.393365] nix-grpc-daemon[767]: ts=2026-09-22T16:54:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_in=6244508worker1 # [ 60.407870] nix-grpc-daemon[767]: ts=2026-09-22T16:54:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964509worker1 # [ 60.414541] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4510worker1 # [ 60.419846] nix-daemon[851]: reaped child process 945, status = succeeded4511client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4512client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4513worker1 # [ 60.435235] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4514client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4515client # farm-b> LOG-farm-b-sig4516worker1 # [ 62.144316] nix-daemon[851]: reaped child process 954, status = succeeded4517worker1 # [ 62.216793] nix-daemon[851]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4518worker1 # [ 62.237372] nix-daemon[851]: reaped child process 973, status = succeeded4519lb # [ 62.217365] niks3-server[1016]: 2026/09/22 16:54:42 INFO Received uploads request method=POST path=/api/pending_closures4520worker1 # [ 62.251675] nix-grpc-daemon[902]: time=2026-09-22T16:54:42.885Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4521worker1 # [ 62.253646] nix-grpc-daemon[902]: time=2026-09-22T16:54:42.885Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4522lb # [ 62.254930] niks3-server[1016]: 2026/09/22 16:54:42 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4523worker1 # [ 62.294740] nix-grpc-daemon[902]: time=2026-09-22T16:54:42.929Z level=INFO msg="Uploading 1 narinfos"4524lb # [ 62.271670] niks3-server[1016]: 2026/09/22 16:54:42 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4525lb # [ 62.281675] niks3-server[1016]: 2026/09/22 16:54:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4526lb # [ 62.290623] niks3-server[1016]: 2026/09/22 16:54:42 INFO Signed narinfos id=12 count=14527worker1 # [ 62.318992] nix-grpc-daemon[902]: time=2026-09-22T16:54:42.954Z level=INFO msg="Upload complete. (271ms)"4528lb # [ 62.295506] niks3-server[1016]: 2026/09/22 16:54:42 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4529lb # [ 62.302347] niks3-server[1016]: 2026/09/22 16:54:42 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4530lb # [ 62.306264] niks3-server[1016]: 2026/09/22 16:54:42 INFO Completed upload id=124531lb # [ 62.308941] niks3-server[1016]: 2026/09/22 16:54:42 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4532worker1 # [ 62.334315] nix-grpc-daemon[767]: ts=2026-09-22T16:54:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14533worker1 # [ 62.355233] nix-grpc-daemon[767]: ts=2026-09-22T16:54:42Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04534worker1 # [ 62.360377] nix-grpc-daemon[767]: ts=2026-09-22T16:54:42Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14535worker1 # [ 62.364947] nix-daemon[851]: reaped child process 952, status = succeeded4536worker1 # [ 62.393682] nix-grpc-daemon[767]: ts=2026-09-22T16:54:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964537worker1 # [ 62.409227] nix-grpc-daemon[767]: ts=2026-09-22T16:54:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366964538worker1 # [ 62.424402] nix-grpc-daemon[902]: time=2026-09-22T16:54:43.059Z level=INFO msg="All 2 paths already cached"4539client # copying 1 paths to 'lb:50051'...4540client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4541worker1 # [ 62.435849] nix-grpc-daemon[767]: ts=2026-09-22T16:54:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366964542worker1 # [ 62.453213] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4543worker1 # [ 62.608114] nix-daemon[851]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4544worker1 # [ 62.627800] nix-daemon[851]: reaped child process 982, status = succeeded4545lb # [ 62.601731] niks3-server[1016]: 2026/09/22 16:54:43 INFO Received uploads request method=POST path=/api/pending_closures4546worker1 # [ 62.636162] nix-grpc-daemon[902]: time=2026-09-22T16:54:43.271Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4547worker1 # [ 62.637597] nix-grpc-daemon[902]: time=2026-09-22T16:54:43.271Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4548lb # [ 62.635743] niks3-server[1016]: 2026/09/22 16:54:43 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4549worker1 # [ 62.668801] nix-grpc-daemon[902]: time=2026-09-22T16:54:43.303Z level=INFO msg="Uploading 1 narinfos"4550lb # [ 62.651360] niks3-server[1016]: 2026/09/22 16:54:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4551lb # [ 62.660441] niks3-server[1016]: 2026/09/22 16:54:43 INFO Signed narinfos id=13 count=14552lb # [ 62.662884] niks3-server[1016]: 2026/09/22 16:54:43 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4553worker1 # [ 62.693187] nix-grpc-daemon[902]: time=2026-09-22T16:54:43.328Z level=INFO msg="Upload complete. (222ms)"4554lb # [ 62.674851] niks3-server[1016]: 2026/09/22 16:54:43 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4555lb # [ 62.679671] niks3-server[1016]: 2026/09/22 16:54:43 INFO Completed upload id=134556worker1 # [ 62.704870] nix-grpc-daemon[767]: ts=2026-09-22T16:54:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_in=7524557lb # [ 62.682258] niks3-server[1016]: 2026/09/22 16:54:43 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4558worker1 # [ 62.715657] nix-grpc-daemon[767]: ts=2026-09-22T16:54:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964559worker1 # [ 62.722464] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4560worker1 # [ 62.727099] nix-daemon[851]: reaped child process 977, status = succeeded4561client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4562client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4563worker1 # [ 62.740853] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4564client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4565client # farm-top> LOG-farm-top-sig4566worker1 # [ 64.424531] nix-daemon[851]: reaped child process 986, status = succeeded4567worker1 # [ 64.502775] nix-daemon[851]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4568worker1 # [ 64.522517] nix-daemon[851]: reaped child process 1005, status = succeeded4569lb # [ 64.503265] niks3-server[1016]: 2026/09/22 16:54:45 INFO Received uploads request method=POST path=/api/pending_closures4570worker1 # [ 64.538366] nix-grpc-daemon[902]: time=2026-09-22T16:54:45.171Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4571worker1 # [ 64.539967] nix-grpc-daemon[902]: time=2026-09-22T16:54:45.171Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4572lb # [ 64.546044] niks3-server[1016]: 2026/09/22 16:54:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4573worker1 # [ 64.581683] nix-grpc-daemon[902]: time=2026-09-22T16:54:45.216Z level=INFO msg="Uploading 1 narinfos"4574lb # [ 64.562227] niks3-server[1016]: 2026/09/22 16:54:45 INFO Signed narinfos id=14 count=14575lb # [ 64.573203] niks3-server[1016]: 2026/09/22 16:54:45 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4576lb # [ 64.584406] niks3-server[1016]: 2026/09/22 16:54:45 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4577lb # [ 64.595198] niks3-server[1016]: 2026/09/22 16:54:45 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4578lb # [ 64.604409] niks3-server[1016]: 2026/09/22 16:54:45 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4579lb # [ 64.610055] niks3-server[1016]: 2026/09/22 16:54:45 INFO Completed upload id=144580worker1 # [ 64.634528] nix-grpc-daemon[902]: time=2026-09-22T16:54:45.269Z level=INFO msg="Upload complete. (282ms)"4581lb # [ 64.613639] niks3-server[1016]: 2026/09/22 16:54:45 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4582worker1 # [ 64.649149] nix-grpc-daemon[767]: ts=2026-09-22T16:54:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14583worker1 # [ 64.666554] nix-daemon[851]: reaped child process 984, status = succeeded4584client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 7.16 seconds)4585client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4586client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.18 seconds)4587client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4588worker1 # [ 65.001383] nix-grpc-daemon[767]: ts=2026-09-22T16:54:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:454384589worker1 # [ 65.042788] nix-grpc-daemon[767]: ts=2026-09-22T16:54:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:454384590client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.24 seconds)4591client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4592worker2 # [ 65.243996] nix-grpc-daemon[774]: ts=2026-09-22T16:54:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:374584593client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4594client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4595client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4596client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4597worker2 # [ 65.270389] nix-grpc-daemon[774]: ts=2026-09-22T16:54:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:374584598worker2 # [ 65.279672] nix-grpc-daemon[774]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4599worker2 # [ 65.297174] nix-grpc-daemon[774]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4600worker2 # [ 65.311277] nix-grpc-daemon[774]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4601client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.29 seconds)4602client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4603client # finding garbage collector roots...4604client # removing stale temporary roots file "/nix/var/nix/temproots/870"4605client # removing stale temporary roots file "/nix/var/nix/temproots/891"4606client # removing stale temporary roots file "/nix/var/nix/temproots/932"4607client # removing stale temporary roots file "/nix/var/nix/temproots/911"4608client # removing stale temporary roots file "/nix/var/nix/temproots/903"4609client # removing stale temporary roots file "/nix/var/nix/temproots/947"4610client # removing stale temporary roots file "/nix/var/nix/temproots/924"4611client # removing stale temporary roots file "/nix/var/nix/temproots/855"4612client # deleting specified paths...4613client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4614client # deleting unused links...4615client # note: hard linking is currently saving 0.0 KiB4616worker1 # [ 65.746551] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366964617worker1 # [ 65.755343] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366964618worker1 # [ 65.766028] nix-grpc-daemon[902]: time=2026-09-22T16:54:46.400Z level=INFO msg="All 1 paths already cached"4619worker1 # [ 65.774196] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:366964620worker2 # [ 65.731340] nix-grpc-daemon[774]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406304621client # copying 3 paths...4622worker2 # [ 65.746388] nix-grpc-daemon[774]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406304623client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4624worker1 # [ 65.825371] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:366964625worker1 # [ 65.832905] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_out=2324626worker2 # [ 65.788496] nix-grpc-daemon[774]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406304627worker2 # [ 65.795760] nix-grpc-daemon[774]: ts=2026-09-22T16:54:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_out=1764628worker1 # [ 65.854843] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:366964629worker1 # [ 65.861693] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_out=1284630client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4631client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4632client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.55 seconds)4633(finished: subtest: outputs read back through any worker carry the cache signature, in 8.43 seconds)4634subtest: two clients wanting the same drv build it once4635client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4636client # [ 65.943902] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4637client # Running as unit: dedup1.service; invocation ID: 8434597c6e8244c18f6fd6eeb0ca4d204638client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4639client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4640worker2 # [ 66.145903] nix-grpc-daemon[774]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406304641client # [ 66.140848] nix[1000]: don't know how to build these paths:4642client # [ 66.142569] nix[1000]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4643worker1 # [ 66.215560] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04644worker1 # [ 66.221635] nix-grpc-daemon[767]: ts=2026-09-22T16:54:46Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14645worker2 # [ 66.209177] nix-grpc-daemon[774]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304646worker2 # [ 66.223977] nix-grpc-daemon[774]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304647client # [ 66.224230] nix[1000]: copying 1 paths to 'lb:50051'...4648client # [ 66.229524] nix[1000]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4649worker2 # [ 66.241118] nix-grpc-daemon[774]: ts=2026-09-22T16:54:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406304650worker2 # [ 66.257975] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4651worker2 # [ 66.417634] nix-daemon[843]: accepted connection from pid 1006, user nix-grpc-daemon (trusted)4652worker2 # [ 66.435605] nix-daemon[843]: reaped child process 1009, status = succeeded4653lb # [ 66.471104] niks3-server[1016]: 2026/09/22 16:54:47 INFO Received uploads request method=POST path=/api/pending_closures4654worker2 # [ 66.453243] nix-grpc-daemon[857]: time=2026-09-22T16:54:47.099Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4655worker2 # [ 66.454804] nix-grpc-daemon[857]: time=2026-09-22T16:54:47.099Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4656lb # [ 66.503212] niks3-server[1016]: 2026/09/22 16:54:47 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4657worker2 # [ 66.486138] nix-grpc-daemon[857]: time=2026-09-22T16:54:47.133Z level=INFO msg="Uploading 1 narinfos"4658lb # [ 66.515788] niks3-server[1016]: 2026/09/22 16:54:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4659lb # [ 66.527637] niks3-server[1016]: 2026/09/22 16:54:47 INFO Signed narinfos id=15 count=14660lb # [ 66.533486] niks3-server[1016]: 2026/09/22 16:54:47 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4661lb # [ 66.544389] niks3-server[1016]: 2026/09/22 16:54:47 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4662lb # [ 66.549968] niks3-server[1016]: 2026/09/22 16:54:47 INFO Completed upload id=154663worker2 # [ 66.522435] nix-grpc-daemon[857]: time=2026-09-22T16:54:47.169Z level=INFO msg="Upload complete. (249ms)"4664lb # [ 66.553160] niks3-server[1016]: 2026/09/22 16:54:47 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4665worker2 # [ 66.534570] nix-grpc-daemon[774]: ts=2026-09-22T16:54:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_in=4324666worker2 # [ 66.553279] nix-grpc-daemon[774]: ts=2026-09-22T16:54:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304667worker2 # [ 66.563480] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4668worker2 # [ 66.572385] nix-daemon[843]: reaped child process 1004, status = succeeded4669client # [ 66.566334] nix[1000]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4670worker2 # [ 66.582398] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4671client # [ 67.038290] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4672client # Running as unit: dedup2.service; invocation ID: a76b0024c53b4515956c56ad9a8838c94673client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.11 seconds)4674client: must succeed: sleep 24675worker1 # [ 67.358653] nix-grpc-daemon[767]: ts=2026-09-22T16:54:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366964676client # [ 67.301986] nix[1016]: don't know how to build these paths:4677client # [ 67.305209] nix[1016]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4678worker1 # [ 67.381183] nix-grpc-daemon[767]: ts=2026-09-22T16:54:48Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04679worker2 # [ 67.333310] nix-grpc-daemon[774]: ts=2026-09-22T16:54:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304680client # [ 67.324508] nix[1016]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4681lb # [ 69.147493] envoy[895]: conn peer=[2001:db8:1::1]:44398 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9571 tx=4388 ms=69154682lb # [ 69.152497] envoy[895]: conn peer=[2001:db8:1::1]:49176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2887 ms=1944683lb # [ 69.157254] envoy[895]: conn peer=[2001:db8:1::1]:49216 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=874684lb # [ 69.161876] envoy[895]: conn peer=[2001:db8:1::1]:49200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=375 ms=1024685lb # [ 69.167211] envoy[895]: conn peer=[2001:db8:1::1]:49188 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=1164686client: (finished: must succeed: sleep 2, in 2.07 seconds)4687client: waiting for success: ! systemctl is-active dedup1 dedup24688worker2 # [ 69.647920] nix-daemon[843]: reaped child process 1013, status = succeeded4689worker2 # [ 69.775846] nix-daemon[843]: accepted connection from pid 1039, user nix-grpc-daemon (trusted)4690worker2 # [ 69.797033] nix-daemon[843]: reaped child process 1042, status = succeeded4691lb # [ 69.823999] niks3-server[1016]: 2026/09/22 16:54:50 INFO Received uploads request method=POST path=/api/pending_closures4692worker2 # [ 69.807817] nix-grpc-daemon[857]: time=2026-09-22T16:54:50.453Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4693worker2 # [ 69.809854] nix-grpc-daemon[857]: time=2026-09-22T16:54:50.453Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4694lb # [ 69.864140] niks3-server[1016]: 2026/09/22 16:54:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4695worker2 # [ 69.844333] nix-grpc-daemon[857]: time=2026-09-22T16:54:50.491Z level=INFO msg="Uploading 1 narinfos"4696lb # [ 69.880700] niks3-server[1016]: 2026/09/22 16:54:50 INFO Signed narinfos id=16 count=14697lb # [ 69.884007] niks3-server[1016]: 2026/09/22 16:54:50 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4698lb # [ 69.901785] niks3-server[1016]: 2026/09/22 16:54:50 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4699lb # [ 69.915401] niks3-server[1016]: 2026/09/22 16:54:50 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4700worker2 # [ 69.887998] nix-grpc-daemon[857]: time=2026-09-22T16:54:50.535Z level=INFO msg="Upload complete. (291ms)"4701lb # [ 69.921257] niks3-server[1016]: 2026/09/22 16:54:50 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4702lb # [ 69.927681] niks3-server[1016]: 2026/09/22 16:54:50 INFO Completed upload id=164703lb # [ 69.931348] niks3-server[1016]: 2026/09/22 16:54:50 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4704worker2 # [ 69.901078] nix-grpc-daemon[774]: ts=2026-09-22T16:54:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14705worker2 # [ 69.919921] nix-grpc-daemon[774]: ts=2026-09-22T16:54:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=2 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14706worker2 # [ 69.927458] nix-daemon[843]: reaped child process 1011, status = succeeded4707client # [ 69.930040] systemd[1]: dedup2.service: Deactivated successfully.4708client # [ 69.935382] systemd[1]: dedup2.service: Consumed 212ms CPU time over 2.893s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4709client # [ 69.976324] systemd[1]: dedup1.service: Deactivated successfully.4710client # [ 69.979693] systemd[1]: dedup1.service: Consumed 171ms CPU time over 4.035s wall clock time, 43.5M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.4711client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4712client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4713client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4714worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4715worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4716worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4717worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4718worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4719worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4720worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4721worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4722(finished: subtest: two clients wanting the same drv build it once, in 4.98 seconds)4723subtest: --timeout and --max-silent-time reach the builder4724worker1 # [ 71.173233] nix-grpc-daemon[767]: ts=2026-09-22T16:54:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366964725worker1 # [ 71.189558] nix-grpc-daemon[767]: ts=2026-09-22T16:54:51Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04726worker1 # [ 71.196762] nix-grpc-daemon[767]: ts=2026-09-22T16:54:51Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14727worker1 # [ 71.205323] nix-grpc-daemon[767]: ts=2026-09-22T16:54:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964728worker1 # [ 71.216309] nix-grpc-daemon[767]: ts=2026-09-22T16:54:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366964729worker1 # [ 71.230022] nix-grpc-daemon[767]: ts=2026-09-22T16:54:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366964730worker1 # [ 71.245872] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4731worker1 # [ 71.398995] nix-daemon[851]: accepted connection from pid 1032, user nix-grpc-daemon (trusted)4732worker1 # [ 71.419426] nix-daemon[851]: reaped child process 1035, status = succeeded4733lb # [ 71.399556] niks3-server[1016]: 2026/09/22 16:54:52 INFO Received uploads request method=POST path=/api/pending_closures4734worker1 # [ 71.433430] nix-grpc-daemon[902]: time=2026-09-22T16:54:52.068Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4735worker1 # [ 71.434976] nix-grpc-daemon[902]: time=2026-09-22T16:54:52.068Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4736lb # [ 71.432927] niks3-server[1016]: 2026/09/22 16:54:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4737worker1 # [ 71.463859] nix-grpc-daemon[902]: time=2026-09-22T16:54:52.099Z level=INFO msg="Uploading 1 narinfos"4738lb # [ 71.444995] niks3-server[1016]: 2026/09/22 16:54:52 INFO Signed narinfos id=17 count=14739lb # [ 71.453861] niks3-server[1016]: 2026/09/22 16:54:52 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4740lb # [ 71.464783] niks3-server[1016]: 2026/09/22 16:54:52 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4741lb # [ 71.479211] niks3-server[1016]: 2026/09/22 16:54:52 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4742worker1 # [ 71.509364] nix-grpc-daemon[902]: time=2026-09-22T16:54:52.144Z level=INFO msg="Upload complete. (248ms)"4743lb # [ 71.486550] niks3-server[1016]: 2026/09/22 16:54:52 INFO Completed upload id=174744lb # [ 71.493853] niks3-server[1016]: 2026/09/22 16:54:52 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4745worker1 # [ 71.520371] nix-grpc-daemon[767]: ts=2026-09-22T16:54:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_in=4244746worker1 # [ 71.540150] nix-grpc-daemon[767]: ts=2026-09-22T16:54:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964747worker1 # [ 71.548383] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4748worker1 # [ 71.555136] nix-daemon[851]: reaped child process 1030, status = succeeded4749worker1 # [ 71.569732] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4750worker1 # [ 77.993794] nix-grpc-daemon[767]: ts=2026-09-22T16:54:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04751worker1 # [ 78.037000] nix-daemon[851]: reaped child process 1037, status = succeeded4752worker1 # [ 78.039660] nix-daemon[851]: reaped child process 1039, status = succeeded4753worker1 # [ 78.238497] nix-grpc-daemon[767]: ts=2026-09-22T16:54:58Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366964754worker1 # [ 78.251478] nix-grpc-daemon[767]: ts=2026-09-22T16:54:58Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04755worker2 # [ 78.205439] nix-grpc-daemon[774]: ts=2026-09-22T16:54:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304756worker1 # [ 78.257260] nix-grpc-daemon[767]: ts=2026-09-22T16:54:58Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14757worker2 # [ 78.221270] nix-grpc-daemon[774]: ts=2026-09-22T16:54:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304758worker2 # [ 78.236116] nix-grpc-daemon[774]: ts=2026-09-22T16:54:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406304759worker2 # [ 78.251284] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4760worker2 # [ 78.415717] nix-daemon[843]: accepted connection from pid 1055, user nix-grpc-daemon (trusted)4761worker2 # [ 78.439407] nix-daemon[843]: reaped child process 1058, status = succeeded4762lb # [ 78.466004] niks3-server[1016]: 2026/09/22 16:54:59 INFO Received uploads request method=POST path=/api/pending_closures4763worker2 # [ 78.447809] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.094Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4764worker2 # [ 78.450016] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.096Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4765lb # [ 78.502524] niks3-server[1016]: 2026/09/22 16:54:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4766worker2 # [ 78.484082] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.131Z level=INFO msg="Uploading 1 narinfos"4767lb # [ 78.514052] niks3-server[1016]: 2026/09/22 16:54:59 INFO Signed narinfos id=18 count=14768lb # [ 78.524454] niks3-server[1016]: 2026/09/22 16:54:59 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4769lb # [ 78.536515] niks3-server[1016]: 2026/09/22 16:54:59 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4770worker2 # [ 78.521750] nix-grpc-daemon[857]: time=2026-09-22T16:54:59.169Z level=INFO msg="Upload complete. (253ms)"4771lb # [ 78.549564] niks3-server[1016]: 2026/09/22 16:54:59 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4772lb # [ 78.559740] niks3-server[1016]: 2026/09/22 16:54:59 INFO Completed upload id=184773worker2 # [ 78.532766] nix-grpc-daemon[774]: ts=2026-09-22T16:54:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_in=4244774lb # [ 78.562613] niks3-server[1016]: 2026/09/22 16:54:59 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4775worker2 # [ 78.547986] nix-grpc-daemon[774]: ts=2026-09-22T16:54:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304776worker2 # [ 78.556472] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4777worker2 # [ 78.559298] nix-daemon[843]: reaped child process 1053, status = succeeded4778worker2 # [ 78.569637] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4779lb # [ 79.148398] envoy[895]: conn peer=[2001:db8:1::1]:49236 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=26184780lb # [ 79.148727] envoy[895]: conn peer=[2001:db8:1::1]:49226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3031 tx=1541 ms=38244781lb # [ 79.148846] envoy[895]: conn peer=[2001:db8:1::1]:33462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3014 tx=952 ms=68494782worker2 # [ 84.987722] nix-grpc-daemon[774]: ts=2026-09-22T16:55:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04783worker2 # [ 85.032892] nix-daemon[843]: reaped child process 1060, status = succeeded4784worker2 # [ 85.035751] nix-daemon[843]: reaped child process 1062, status = succeeded4785(finished: subtest: --timeout and --max-silent-time reach the builder, in 14.33 seconds)4786subtest: the first client leaving does not fail the second4787client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4788client # [ 85.276849] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4789client # Running as unit: share1.service; invocation ID: 25c8a1141ba34e2fa8474454c4a820204790client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.09 seconds)4791client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4792worker2 # [ 85.488154] nix-grpc-daemon[774]: ts=2026-09-22T16:55:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:406304793client # [ 85.483005] nix[1088]: don't know how to build these paths:4794client # [ 85.483974] nix[1088]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4795worker1 # [ 85.562022] nix-grpc-daemon[767]: ts=2026-09-22T16:55:06Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04796worker1 # [ 85.569903] nix-grpc-daemon[767]: ts=2026-09-22T16:55:06Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14797worker1 # [ 85.577859] nix-grpc-daemon[767]: ts=2026-09-22T16:55:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964798worker1 # [ 85.589204] nix-grpc-daemon[767]: ts=2026-09-22T16:55:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366964799client # [ 85.536470] nix[1088]: copying 1 paths to 'lb:50051'...4800client # [ 85.541146] nix[1088]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4801worker1 # [ 85.605067] nix-grpc-daemon[767]: ts=2026-09-22T16:55:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366964802worker1 # [ 85.620253] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4803worker1 # [ 85.796116] nix-daemon[851]: accepted connection from pid 1066, user nix-grpc-daemon (trusted)4804worker1 # [ 85.814152] nix-daemon[851]: reaped child process 1069, status = succeeded4805lb # [ 85.796686] niks3-server[1016]: 2026/09/22 16:55:06 INFO Received uploads request method=POST path=/api/pending_closures4806worker1 # [ 85.832430] nix-grpc-daemon[902]: time=2026-09-22T16:55:06.466Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4807worker1 # [ 85.833977] nix-grpc-daemon[902]: time=2026-09-22T16:55:06.466Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4808lb # [ 85.838281] niks3-server[1016]: 2026/09/22 16:55:06 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4809worker1 # [ 85.876910] nix-grpc-daemon[902]: time=2026-09-22T16:55:06.511Z level=INFO msg="Uploading 1 narinfos"4810lb # [ 85.852371] niks3-server[1016]: 2026/09/22 16:55:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4811lb # [ 85.862698] niks3-server[1016]: 2026/09/22 16:55:06 INFO Signed narinfos id=19 count=14812lb # [ 85.869140] niks3-server[1016]: 2026/09/22 16:55:06 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4813lb # [ 85.879454] niks3-server[1016]: 2026/09/22 16:55:06 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4814lb # [ 85.891695] niks3-server[1016]: 2026/09/22 16:55:06 INFO Completed upload id=194815worker1 # [ 85.914728] nix-grpc-daemon[902]: time=2026-09-22T16:55:06.550Z level=INFO msg="Upload complete. (274ms)"4816lb # [ 85.897618] niks3-server[1016]: 2026/09/22 16:55:06 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4817worker1 # [ 85.929120] nix-grpc-daemon[767]: ts=2026-09-22T16:55:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_in=4324818worker1 # [ 85.944460] nix-grpc-daemon[767]: ts=2026-09-22T16:55:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964819worker1 # [ 85.956561] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4820worker1 # [ 85.959971] nix-daemon[851]: reaped child process 1064, status = succeeded4821client # [ 85.906859] nix[1088]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4822worker1 # [ 85.974463] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)4823client # [ 86.374644] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4824client # Running as unit: share2.service; invocation ID: cdb015f6f89c400b9816512bbd6905cb4825client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4826worker1 # [ 86.634532] nix-grpc-daemon[767]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366964827client # [ 86.584227] nix[1104]: this derivation will be built:4828worker2 # [ 86.600808] nix-grpc-daemon[774]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406304829worker2 # [ 86.615470] nix-grpc-daemon[774]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' from 'http://lb:5751'...4830client # [ 86.626403] nix[1104]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4831worker1 # [ 86.702490] nix-grpc-daemon[767]: ts=2026-09-22T16:55:07Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04832worker1 # [ 86.710561] nix-grpc-daemon[767]: ts=2026-09-22T16:55:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366964833client # [ 86.653671] nix[1104]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4834client: must succeed: sleep 2; systemctl kill -s INT share14835lb # [ 89.149308] envoy[895]: conn peer=[2001:db8:1::1]:33468 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3028 tx=953 ms=68604836client # [ 89.748959] systemd[1]: share1.service: Sent signal SIGINT to main process 1088 (nix) on client request.4837client # [ 89.758549] nix[1088]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4838client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4839client: waiting for success: ! systemctl is-active share14840client # [ 89.773743] nix[1088]: error: interrupted by the user4841client # [ 89.837339] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4842client # [ 89.840156] systemd[1]: share1.service: Failed with result 'exit-code'.4843client # [ 89.843212] systemd[1]: share1.service: Consumed 181ms CPU time over 4.564s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8.3K outgoing IP traffic.4844client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4845worker1 # [ 91.199277] nix-daemon[851]: reaped child process 1073, status = succeeded4846client: waiting for success: ! systemctl is-active share24847worker1 # [ 91.301729] nix-daemon[851]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)4848worker1 # [ 91.318797] nix-daemon[851]: reaped child process 1112, status = succeeded4849lb # [ 91.302964] niks3-server[1016]: 2026/09/22 16:55:11 INFO Received uploads request method=POST path=/api/pending_closures4850worker1 # [ 91.340401] nix-grpc-daemon[902]: time=2026-09-22T16:55:11.974Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4851worker1 # [ 91.341846] nix-grpc-daemon[902]: time=2026-09-22T16:55:11.974Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4852lb # [ 91.354784] niks3-server[1016]: 2026/09/22 16:55:12 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4853worker1 # [ 91.391445] nix-grpc-daemon[902]: time=2026-09-22T16:55:12.026Z level=INFO msg="Uploading 1 narinfos"4854lb # [ 91.369095] niks3-server[1016]: 2026/09/22 16:55:12 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4855lb # [ 91.378969] niks3-server[1016]: 2026/09/22 16:55:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4856lb # [ 91.388372] niks3-server[1016]: 2026/09/22 16:55:12 INFO Signed narinfos id=20 count=14857worker1 # [ 91.412763] nix-grpc-daemon[902]: time=2026-09-22T16:55:12.048Z level=INFO msg="Upload complete. (224ms)"4858lb # [ 91.390411] niks3-server[1016]: 2026/09/22 16:55:12 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4859worker1 # [ 91.424300] nix-grpc-daemon[767]: ts=2026-09-22T16:55:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14860lb # [ 91.401560] niks3-server[1016]: 2026/09/22 16:55:12 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4861lb # [ 91.408409] niks3-server[1016]: 2026/09/22 16:55:12 INFO Completed upload id=204862lb # [ 91.411038] niks3-server[1016]: 2026/09/22 16:55:12 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4863worker1 # [ 91.440543] nix-grpc-daemon[767]: ts=2026-09-22T16:55:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14864worker1 # [ 91.447789] nix-daemon[851]: reaped child process 1071, status = succeeded4865client # [ 91.435522] systemd[1]: share2.service: Deactivated successfully.4866client # [ 91.440120] systemd[1]: share2.service: Consumed 187ms CPU time over 5.062s wall clock time, 33.3M memory peak, 6.5K incoming IP traffic, 5.9K outgoing IP traffic.4867client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4868client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4869client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.06 seconds)4870(finished: subtest: the first client leaving does not fail the second, in 7.11 seconds)4871subtest: build hook: nix-daemon with builders = grpc://lb4872client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4873client # [ 92.462137] systemd[1]: Started Nix Daemon.4874client # [ 92.583285] nix-daemon[1156]: accepted connection from pid 1153, user root (trusted)4875worker2 # [ 92.742261] nix-grpc-daemon[774]: ts=2026-09-22T16:55:13Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:406304876worker2 # [ 92.749280] nix-grpc-daemon[774]: ts=2026-09-22T16:55:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304877worker1 # [ 92.813318] nix-grpc-daemon[767]: ts=2026-09-22T16:55:13Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04878worker1 # [ 92.819600] nix-grpc-daemon[767]: ts=2026-09-22T16:55:13Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14879worker2 # [ 92.770447] nix-grpc-daemon[774]: ts=2026-09-22T16:55:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304880worker2 # [ 92.785663] nix-grpc-daemon[774]: ts=2026-09-22T16:55:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304881worker2 # [ 92.799203] nix-grpc-daemon[774]: ts=2026-09-22T16:55:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406304882worker2 # [ 92.814356] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4883worker2 # [ 92.984188] nix-daemon[843]: accepted connection from pid 1102, user nix-grpc-daemon (trusted)4884worker2 # [ 93.001361] nix-daemon[843]: reaped child process 1105, status = succeeded4885lb # [ 93.037590] niks3-server[1016]: 2026/09/22 16:55:13 INFO Received uploads request method=POST path=/api/pending_closures4886worker2 # [ 93.020284] nix-grpc-daemon[857]: time=2026-09-22T16:55:13.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4887worker2 # [ 93.021760] nix-grpc-daemon[857]: time=2026-09-22T16:55:13.666Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4888lb # [ 93.069897] niks3-server[1016]: 2026/09/22 16:55:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4889worker2 # [ 93.050249] nix-grpc-daemon[857]: time=2026-09-22T16:55:13.696Z level=INFO msg="Uploading 1 narinfos"4890lb # [ 93.082732] niks3-server[1016]: 2026/09/22 16:55:13 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4891lb # [ 93.092970] niks3-server[1016]: 2026/09/22 16:55:13 INFO Signed narinfos id=21 count=14892lb # [ 93.101070] niks3-server[1016]: 2026/09/22 16:55:13 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4893worker2 # [ 93.081671] nix-grpc-daemon[857]: time=2026-09-22T16:55:13.728Z level=INFO msg="Upload complete. (247ms)"4894lb # [ 93.111467] niks3-server[1016]: 2026/09/22 16:55:13 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4895lb # [ 93.117835] niks3-server[1016]: 2026/09/22 16:55:13 INFO Completed upload id=214896worker2 # [ 93.092348] nix-grpc-daemon[774]: ts=2026-09-22T16:55:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_in=5124897lb # [ 93.121757] niks3-server[1016]: 2026/09/22 16:55:13 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4898worker2 # [ 93.103716] nix-grpc-daemon[774]: ts=2026-09-22T16:55:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304899worker2 # [ 93.113587] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4900worker2 # [ 93.116369] nix-daemon[843]: reaped child process 1100, status = succeeded4901worker2 # [ 93.130561] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4902worker2 # [ 94.590603] nix-daemon[843]: reaped child process 1109, status = succeeded4903worker2 # [ 94.721437] nix-daemon[843]: accepted connection from pid 1125, user nix-grpc-daemon (trusted)4904worker2 # [ 94.742580] nix-daemon[843]: reaped child process 1128, status = succeeded4905lb # [ 94.771160] niks3-server[1016]: 2026/09/22 16:55:15 INFO Received uploads request method=POST path=/api/pending_closures4906worker2 # [ 94.756257] nix-grpc-daemon[857]: time=2026-09-22T16:55:15.400Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4907worker2 # [ 94.757952] nix-grpc-daemon[857]: time=2026-09-22T16:55:15.400Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4908lb # [ 94.818491] niks3-server[1016]: 2026/09/22 16:55:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4909worker2 # [ 94.799126] nix-grpc-daemon[857]: time=2026-09-22T16:55:15.446Z level=INFO msg="Uploading 1 narinfos"4910lb # [ 94.830467] niks3-server[1016]: 2026/09/22 16:55:15 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4911lb # [ 94.841299] niks3-server[1016]: 2026/09/22 16:55:15 INFO Signed narinfos id=22 count=14912lb # [ 94.843403] niks3-server[1016]: 2026/09/22 16:55:15 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4913lb # [ 94.849313] niks3-server[1016]: 2026/09/22 16:55:15 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4914lb # [ 94.857634] niks3-server[1016]: 2026/09/22 16:55:15 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4915lb # [ 94.861137] niks3-server[1016]: 2026/09/22 16:55:15 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4916worker2 # [ 94.833943] nix-grpc-daemon[857]: time=2026-09-22T16:55:15.481Z level=INFO msg="Upload complete. (292ms)"4917lb # [ 94.864212] niks3-server[1016]: 2026/09/22 16:55:15 INFO Completed upload id=224918worker2 # [ 94.845245] nix-grpc-daemon[774]: ts=2026-09-22T16:55:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14919worker2 # [ 94.863699] nix-daemon[843]: reaped child process 1107, status = succeeded4920worker2 # [ 94.888937] nix-grpc-daemon[774]: ts=2026-09-22T16:55:15Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406304921worker2 # [ 94.918628] nix-grpc-daemon[774]: ts=2026-09-22T16:55:15Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406304922worker2 # [ 94.925563] nix-grpc-daemon[774]: ts=2026-09-22T16:55:15Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_out=1284923worker1 # [ 95.265212] nix-grpc-daemon[767]: ts=2026-09-22T16:55:15Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:366964924worker2 # [ 95.216615] nix-grpc-daemon[774]: ts=2026-09-22T16:55:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304925worker2 # [ 95.226434] nix-grpc-daemon[857]: time=2026-09-22T16:55:15.873Z level=INFO msg="All 1 paths already cached"4926worker1 # [ 95.297532] nix-grpc-daemon[767]: ts=2026-09-22T16:55:15Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04927worker1 # [ 95.303239] nix-grpc-daemon[767]: ts=2026-09-22T16:55:15Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14928worker2 # [ 95.293809] nix-grpc-daemon[774]: ts=2026-09-22T16:55:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304929worker2 # [ 95.312746] nix-grpc-daemon[774]: ts=2026-09-22T16:55:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304930worker2 # [ 95.330264] nix-grpc-daemon[857]: time=2026-09-22T16:55:15.977Z level=INFO msg="All 1 paths already cached"4931worker2 # [ 95.342402] nix-grpc-daemon[774]: ts=2026-09-22T16:55:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406304932worker2 # [ 95.356235] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4933worker2 # [ 95.520881] nix-daemon[843]: accepted connection from pid 1132, user nix-grpc-daemon (trusted)4934worker2 # [ 95.541088] nix-daemon[843]: reaped child process 1135, status = succeeded4935lb # [ 95.567377] niks3-server[1016]: 2026/09/22 16:55:16 INFO Received uploads request method=POST path=/api/pending_closures4936worker2 # [ 95.550181] nix-grpc-daemon[857]: time=2026-09-22T16:55:16.197Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4937worker2 # [ 95.551803] nix-grpc-daemon[857]: time=2026-09-22T16:55:16.197Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4938lb # [ 95.599594] niks3-server[1016]: 2026/09/22 16:55:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4939worker2 # [ 95.580831] nix-grpc-daemon[857]: time=2026-09-22T16:55:16.227Z level=INFO msg="Uploading 1 narinfos"4940lb # [ 95.616131] niks3-server[1016]: 2026/09/22 16:55:16 INFO Signed narinfos id=23 count=14941lb # [ 95.622803] niks3-server[1016]: 2026/09/22 16:55:16 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4942lb # [ 95.634806] niks3-server[1016]: 2026/09/22 16:55:16 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4943worker2 # [ 95.614931] nix-grpc-daemon[857]: time=2026-09-22T16:55:16.262Z level=INFO msg="Upload complete. (241ms)"4944lb # [ 95.643588] niks3-server[1016]: 2026/09/22 16:55:16 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4945lb # [ 95.652374] niks3-server[1016]: 2026/09/22 16:55:16 INFO Completed upload id=234946worker2 # [ 95.628381] nix-grpc-daemon[774]: ts=2026-09-22T16:55:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_in=6244947lb # [ 95.655035] niks3-server[1016]: 2026/09/22 16:55:16 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4948worker2 # [ 95.642938] nix-grpc-daemon[774]: ts=2026-09-22T16:55:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304949worker2 # [ 95.650807] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4950worker2 # [ 95.655770] nix-daemon[843]: reaped child process 1130, status = succeeded4951worker2 # [ 95.668598] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4952worker2 # [ 96.507535] nix-daemon[843]: reaped child process 1139, status = succeeded4953worker2 # [ 96.625495] nix-daemon[843]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)4954worker2 # [ 96.641581] nix-daemon[843]: reaped child process 1158, status = succeeded4955lb # [ 96.677181] niks3-server[1016]: 2026/09/22 16:55:17 INFO Received uploads request method=POST path=/api/pending_closures4956worker2 # [ 96.660670] nix-grpc-daemon[857]: time=2026-09-22T16:55:17.306Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4957worker2 # [ 96.662117] nix-grpc-daemon[857]: time=2026-09-22T16:55:17.306Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4958lb # [ 96.720555] niks3-server[1016]: 2026/09/22 16:55:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4959worker2 # [ 96.702846] nix-grpc-daemon[857]: time=2026-09-22T16:55:17.349Z level=INFO msg="Uploading 1 narinfos"4960lb # [ 96.736366] niks3-server[1016]: 2026/09/22 16:55:17 INFO Signed narinfos id=24 count=14961lb # [ 96.744463] niks3-server[1016]: 2026/09/22 16:55:17 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4962lb # [ 96.753702] niks3-server[1016]: 2026/09/22 16:55:17 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4963lb # [ 96.764889] niks3-server[1016]: 2026/09/22 16:55:17 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4964lb # [ 96.769994] niks3-server[1016]: 2026/09/22 16:55:17 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4965worker2 # [ 96.742648] nix-grpc-daemon[857]: time=2026-09-22T16:55:17.389Z level=INFO msg="Upload complete. (261ms)"4966lb # [ 96.773508] niks3-server[1016]: 2026/09/22 16:55:17 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4967lb # [ 96.777609] niks3-server[1016]: 2026/09/22 16:55:17 INFO Completed upload id=244968worker2 # [ 96.754516] nix-grpc-daemon[774]: ts=2026-09-22T16:55:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14969worker2 # [ 96.773817] nix-daemon[843]: reaped child process 1137, status = succeeded4970worker1 # [ 96.853273] nix-grpc-daemon[767]: ts=2026-09-22T16:55:17Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:366964971worker1 # [ 96.875245] nix-grpc-daemon[767]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4972worker1 # [ 96.893018] nix-grpc-daemon[767]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4973worker2 # [ 96.878410] nix-grpc-daemon[774]: ts=2026-09-22T16:55:17Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406304974worker2 # [ 96.885317] nix-grpc-daemon[774]: ts=2026-09-22T16:55:17Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_out=1764975worker1 # [ 97.173252] nix-grpc-daemon[767]: ts=2026-09-22T16:55:17Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:366964976worker1 # [ 97.180888] nix-grpc-daemon[767]: ts=2026-09-22T16:55:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366964977worker1 # [ 97.188375] nix-grpc-daemon[902]: time=2026-09-22T16:55:17.821Z level=INFO msg="All 2 paths already cached"4978worker1 # [ 97.200150] nix-grpc-daemon[767]: ts=2026-09-22T16:55:17Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04979worker1 # [ 97.205902] nix-grpc-daemon[767]: ts=2026-09-22T16:55:17Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14980worker2 # [ 97.194004] nix-grpc-daemon[774]: ts=2026-09-22T16:55:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406304981worker2 # [ 97.209907] nix-grpc-daemon[774]: ts=2026-09-22T16:55:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406304982worker2 # [ 97.226477] nix-grpc-daemon[857]: time=2026-09-22T16:55:17.873Z level=INFO msg="All 2 paths already cached"4983worker2 # [ 97.236758] nix-grpc-daemon[774]: ts=2026-09-22T16:55:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406304984worker2 # [ 97.250891] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)4985worker2 # [ 97.413263] nix-daemon[843]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)4986worker2 # [ 97.433409] nix-daemon[843]: reaped child process 1165, status = succeeded4987lb # [ 97.459136] niks3-server[1016]: 2026/09/22 16:55:18 INFO Received uploads request method=POST path=/api/pending_closures4988worker2 # [ 97.441499] nix-grpc-daemon[857]: time=2026-09-22T16:55:18.088Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4989worker2 # [ 97.442955] nix-grpc-daemon[857]: time=2026-09-22T16:55:18.088Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4990worker2 # [ 97.460262] nix-grpc-daemon[857]: time=2026-09-22T16:55:18.107Z level=INFO msg="Uploading 1 narinfos"4991lb # [ 97.487805] niks3-server[1016]: 2026/09/22 16:55:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4992lb # [ 97.500291] niks3-server[1016]: 2026/09/22 16:55:18 INFO Signed narinfos id=25 count=14993lb # [ 97.503314] niks3-server[1016]: 2026/09/22 16:55:18 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4994lb # [ 97.517582] niks3-server[1016]: 2026/09/22 16:55:18 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4995worker2 # [ 97.501936] nix-grpc-daemon[857]: time=2026-09-22T16:55:18.149Z level=INFO msg="Upload complete. (233ms)"4996lb # [ 97.530021] niks3-server[1016]: 2026/09/22 16:55:18 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4997lb # [ 97.535832] niks3-server[1016]: 2026/09/22 16:55:18 INFO Completed upload id=254998worker2 # [ 97.512279] nix-grpc-daemon[774]: ts=2026-09-22T16:55:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_in=7524999lb # [ 97.541371] niks3-server[1016]: 2026/09/22 16:55:18 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo5000worker2 # [ 97.528148] nix-grpc-daemon[774]: ts=2026-09-22T16:55:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406305001worker2 # [ 97.534425] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5002worker2 # [ 97.539464] nix-daemon[843]: reaped child process 1160, status = succeeded5003worker2 # [ 97.550999] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5004worker2 # [ 98.790016] nix-daemon[843]: reaped child process 1169, status = succeeded5005worker2 # [ 98.903199] nix-daemon[843]: accepted connection from pid 1185, user nix-grpc-daemon (trusted)5006worker2 # [ 98.920764] nix-daemon[843]: reaped child process 1188, status = succeeded5007lb # [ 98.955700] niks3-server[1016]: 2026/09/22 16:55:19 INFO Received uploads request method=POST path=/api/pending_closures5008worker2 # [ 98.938461] nix-grpc-daemon[857]: time=2026-09-22T16:55:19.584Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5009worker2 # [ 98.940292] nix-grpc-daemon[857]: time=2026-09-22T16:55:19.584Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"5010lb # [ 98.995942] niks3-server[1016]: 2026/09/22 16:55:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5011worker2 # [ 98.976472] nix-grpc-daemon[857]: time=2026-09-22T16:55:19.623Z level=INFO msg="Uploading 1 narinfos"5012lb # [ 99.011262] niks3-server[1016]: 2026/09/22 16:55:19 INFO Signed narinfos id=26 count=15013lb # [ 99.017480] niks3-server[1016]: 2026/09/22 16:55:19 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv5014worker2 # [ 99.001530] nix-grpc-daemon[857]: time=2026-09-22T16:55:19.648Z level=INFO msg="Upload complete. (238ms)"5015lb # [ 99.028356] niks3-server[1016]: 2026/09/22 16:55:19 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst5016lb # [ 99.036606] niks3-server[1016]: 2026/09/22 16:55:19 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls5017lb # [ 99.040270] niks3-server[1016]: 2026/09/22 16:55:19 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5018lb # [ 99.043818] niks3-server[1016]: 2026/09/22 16:55:19 INFO Completed upload id=265019worker2 # [ 99.012975] nix-grpc-daemon[774]: ts=2026-09-22T16:55:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=15020lb # [ 99.046400] niks3-server[1016]: 2026/09/22 16:55:19 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo5021worker2 # [ 99.029059] nix-daemon[843]: reaped child process 1167, status = succeeded5022worker1 # [ 99.078653] nix-grpc-daemon[767]: ts=2026-09-22T16:55:19Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:366965023worker1 # [ 99.086569] nix-grpc-daemon[767]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...5024worker1 # [ 99.122618] nix-grpc-daemon[767]: ts=2026-09-22T16:55:19Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:366965025worker1 # [ 99.129771] nix-grpc-daemon[767]: ts=2026-09-22T16:55:19Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_out=2325026lb # [ 99.147436] envoy[895]: conn peer=[2001:db8:1::1]:36442 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3003 tx=811 ms=43045027lb # [ 99.152718] envoy[895]: conn peer=[2001:db8:1::1]:36446 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1605 tx=1599 ms=48495028lb # [ 99.158387] envoy[895]: conn peer=[2001:db8:1::1]:55588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3445 tx=1943 ms=22435029lb # [ 99.164654] envoy[895]: conn peer=[2001:db8:1::1]:55590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=725030lb # [ 99.169522] envoy[895]: conn peer=[2001:db8:1::1]:55620 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=275031lb # [ 99.174790] envoy[895]: conn peer=[2001:db8:1::1]:55604 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3900 tx=2155 ms=16905032client # [ 99.202791] nix-daemon[1156]: reaped child process 1158, status = succeeded5033client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.90 seconds)5034client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top5035client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)5036client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5037client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)5038client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5039client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)5040(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.03 seconds)5041subtest: an input only one worker has still reaches the builder5042client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only5043client # this derivation will be built:5044client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv5045client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...5046client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.30 seconds)5047client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure5048client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)5049worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5050worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)5051worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only5052worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)5053client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&25054client # [ 100.163985] nix-daemon[1156]: accepted connection from pid 1239, user root (trusted)5055client # this derivation will be built:5056client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5057client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5058worker2 # [ 100.324875] nix-grpc-daemon[774]: ts=2026-09-22T16:55:20Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:406305059worker1 # [ 100.381537] nix-grpc-daemon[767]: ts=2026-09-22T16:55:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366965060worker1 # [ 100.546411] nix-daemon[851]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)5061worker1 # [ 100.565336] nix-daemon[851]: reaped child process 1131, status = succeeded5062lb # [ 100.546509] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received uploads request method=POST path=/api/pending_closures5063worker1 # [ 100.581662] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.216Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5064worker1 # [ 100.583118] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.216Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"5065lb # [ 100.579646] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls5066worker1 # [ 100.615315] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.250Z level=INFO msg="Uploading 1 narinfos"5067lb # [ 100.592574] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5068lb # [ 100.603428] niks3-server[1016]: 2026/09/22 16:55:21 INFO Signed narinfos id=27 count=15069lb # [ 100.609629] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5070lb # [ 100.622009] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5071lb # [ 100.628283] niks3-server[1016]: 2026/09/22 16:55:21 INFO Completed upload id=275072worker1 # [ 100.651475] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.286Z level=INFO msg="Upload complete. (270ms)"5073lb # [ 100.630488] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo5074worker1 # [ 100.676001] nix-grpc-daemon[767]: ts=2026-09-22T16:55:21Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=05075worker1 # [ 100.685869] nix-grpc-daemon[767]: ts=2026-09-22T16:55:21Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=15076worker1 # [ 100.693615] nix-grpc-daemon[767]: ts=2026-09-22T16:55:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366965077worker1 # [ 100.703713] nix-grpc-daemon[767]: ts=2026-09-22T16:55:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366965078client # copying 2 paths to 'lb:50051'...5079client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...5080worker1 # [ 100.721819] nix-grpc-daemon[767]: ts=2026-09-22T16:55:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366965081client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...5082worker1 # [ 100.735564] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5083worker1 # [ 100.883962] nix-daemon[851]: accepted connection from pid 1137, user nix-grpc-daemon (trusted)5084worker1 # [ 100.906185] nix-daemon[851]: reaped child process 1140, status = succeeded5085lb # [ 100.885941] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received uploads request method=POST path=/api/pending_closures5086lb # [ 100.895933] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received uploads request method=POST path=/api/pending_closures5087worker1 # [ 100.930146] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.563Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5088worker1 # [ 100.931638] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.563Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"5089worker1 # [ 100.933926] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.564Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"5090lb # [ 100.927833] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst5091lb # [ 100.949418] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst5092lb # [ 100.959940] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls5093worker1 # [ 100.994481] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.628Z level=INFO msg="Uploading 3 narinfos"5094lb # [ 100.971048] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5095lb # [ 100.979904] niks3-server[1016]: 2026/09/22 16:55:21 INFO Signed narinfos id=28 count=15096lb # [ 100.984535] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5097lb # [ 100.987698] niks3-server[1016]: 2026/09/22 16:55:21 INFO Signed narinfos id=29 count=25098lb # [ 100.997376] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls5099lb # [ 101.009427] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5100lb # [ 101.017455] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5101lb # [ 101.023320] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo5102lb # [ 101.029523] niks3-server[1016]: 2026/09/22 16:55:21 INFO Completed upload id=285103worker1 # [ 101.054042] nix-grpc-daemon[902]: time=2026-09-22T16:55:21.689Z level=INFO msg="Upload complete. (301ms)"5104lb # [ 101.031438] niks3-server[1016]: 2026/09/22 16:55:21 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5105lb # [ 101.034739] niks3-server[1016]: 2026/09/22 16:55:21 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo5106lb # [ 101.038732] niks3-server[1016]: 2026/09/22 16:55:21 INFO Completed upload id=295107worker1 # [ 101.064197] nix-grpc-daemon[767]: ts=2026-09-22T16:55:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=2 nar_bytes_in=9845108worker1 # [ 101.081486] nix-grpc-daemon[767]: ts=2026-09-22T16:55:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366965109worker1 # [ 101.095243] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5110worker1 # [ 101.103928] nix-daemon[851]: reaped child process 1135, status = succeeded5111client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...5112client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5113worker1 # [ 101.121462] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5114client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5115worker1 # [ 102.361169] nix-daemon[851]: reaped child process 1144, status = succeeded5116worker1 # [ 102.468220] nix-daemon[851]: accepted connection from pid 1160, user nix-grpc-daemon (trusted)5117worker1 # [ 102.483371] nix-daemon[851]: reaped child process 1163, status = succeeded5118lb # [ 102.468539] niks3-server[1016]: 2026/09/22 16:55:23 INFO Received uploads request method=POST path=/api/pending_closures5119worker1 # [ 102.502366] nix-grpc-daemon[902]: time=2026-09-22T16:55:23.135Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5120lb # [ 102.507746] niks3-server[1016]: 2026/09/22 16:55:23 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5121worker1 # [ 102.541526] nix-grpc-daemon[902]: time=2026-09-22T16:55:23.176Z level=INFO msg="Uploading 1 narinfos"5122lb # [ 102.522584] niks3-server[1016]: 2026/09/22 16:55:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5123lb # [ 102.531009] niks3-server[1016]: 2026/09/22 16:55:23 INFO Signed narinfos id=30 count=15124lb # [ 102.536964] niks3-server[1016]: 2026/09/22 16:55:23 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5125worker1 # [ 102.573374] nix-grpc-daemon[902]: time=2026-09-22T16:55:23.208Z level=INFO msg="Upload complete. (238ms)"5126lb # [ 102.549688] niks3-server[1016]: 2026/09/22 16:55:23 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5127lb # [ 102.554456] niks3-server[1016]: 2026/09/22 16:55:23 INFO Completed upload id=305128lb # [ 102.562655] niks3-server[1016]: 2026/09/22 16:55:23 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5129worker1 # [ 102.589081] nix-grpc-daemon[767]: ts=2026-09-22T16:55:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15130worker1 # [ 102.603845] nix-daemon[851]: reaped child process 1142, status = succeeded5131client # copying 1 paths...5132worker2 # [ 102.583875] nix-grpc-daemon[774]: ts=2026-09-22T16:55:23Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406305133worker2 # [ 102.599274] nix-grpc-daemon[774]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...5134client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5135worker2 # [ 102.644149] nix-grpc-daemon[774]: ts=2026-09-22T16:55:23Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:406305136worker2 # [ 102.651845] nix-grpc-daemon[774]: ts=2026-09-22T16:55:23Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=0 paths=1 nar_bytes_out=1205137client # [ 102.768295] nix-daemon[1156]: reaped child process 1242, status = succeeded5138client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.80 seconds)5139client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25140client # [ 102.942573] nix-daemon[1156]: accepted connection from pid 1261, user root (trusted)5141client # this derivation will be built:5142client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5143worker2 # [ 103.102370] nix-grpc-daemon[774]: ts=2026-09-22T16:55:23Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:406305144client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5145worker2 # [ 103.110266] nix-grpc-daemon[774]: ts=2026-09-22T16:55:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406305146worker1 # [ 103.181555] nix-grpc-daemon[767]: ts=2026-09-22T16:55:23Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05147worker2 # [ 103.133260] nix-grpc-daemon[774]: ts=2026-09-22T16:55:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406305148worker1 # [ 103.187935] nix-grpc-daemon[767]: ts=2026-09-22T16:55:23Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15149worker2 # [ 103.147402] nix-grpc-daemon[774]: ts=2026-09-22T16:55:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:406305150client # copying 1 paths to 'lb:50051'...5151client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5152worker2 # [ 103.168413] nix-grpc-daemon[774]: ts=2026-09-22T16:55:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:406305153worker2 # [ 103.183535] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5154worker2 # [ 103.205228] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5)5155worker2 # [ 103.311241] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 176 ms (attempt 2/5)5156worker2 # [ 103.489429] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 219 ms (attempt 3/5)5157worker2 # [ 103.719342] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 788 ms (attempt 4/5)5158worker2 # [ 104.509108] nix-grpc-daemon[774]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5159worker2 # [ 104.509518] nix-grpc-daemon[774]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5160worker2 # [ 104.521461] nix-grpc-daemon[774]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5161worker2 # [ 104.709733] nix-daemon[843]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5162worker2 # [ 104.730366] nix-daemon[843]: reaped child process 1204, status = succeeded5163lb # [ 104.763870] niks3-server[1016]: 2026/09/22 16:55:25 INFO Received uploads request method=POST path=/api/pending_closures5164worker2 # [ 104.746982] nix-grpc-daemon[857]: time=2026-09-22T16:55:25.393Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5165worker2 # [ 104.748783] nix-grpc-daemon[857]: time=2026-09-22T16:55:25.393Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5166lb # [ 104.794291] niks3-server[1016]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5167worker2 # [ 104.778190] nix-grpc-daemon[857]: time=2026-09-22T16:55:25.425Z level=INFO msg="Uploading 1 narinfos"5168lb # [ 104.807742] niks3-server[1016]: 2026/09/22 16:55:25 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5169lb # [ 104.818795] niks3-server[1016]: 2026/09/22 16:55:25 INFO Signed narinfos id=31 count=15170worker2 # [ 104.801142] nix-grpc-daemon[857]: time=2026-09-22T16:55:25.448Z level=INFO msg="Upload complete. (255ms)"5171lb # [ 104.829872] niks3-server[1016]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5172lb # [ 104.836462] niks3-server[1016]: 2026/09/22 16:55:25 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5173lb # [ 104.842718] niks3-server[1016]: 2026/09/22 16:55:25 INFO Completed upload id=315174lb # [ 104.845387] niks3-server[1016]: 2026/09/22 16:55:25 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5175worker2 # [ 104.812987] nix-grpc-daemon[774]: ts=2026-09-22T16:55:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=1 paths=1 nar_bytes_in=5685176worker2 # [ 104.826971] nix-grpc-daemon[774]: ts=2026-09-22T16:55:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:406305177worker2 # [ 104.832711] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5178worker2 # [ 104.837559] nix-daemon[843]: reaped child process 1194, status = succeeded5179worker2 # [ 104.845978] nix-grpc-daemon[774]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...5180worker2 # [ 104.860243] nix-daemon[843]: accepted connection from pid 774, user nix-grpc-daemon (trusted)5181client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5182client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5183client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5184worker2 # [ 105.927331] nix-daemon[843]: reaped child process 1209, status = succeeded5185worker2 # [ 106.060204] nix-daemon[843]: accepted connection from pid 1225, user nix-grpc-daemon (trusted)5186worker2 # [ 106.075521] nix-daemon[843]: reaped child process 1228, status = succeeded5187lb # [ 106.111871] niks3-server[1016]: 2026/09/22 16:55:26 INFO Received uploads request method=POST path=/api/pending_closures5188worker2 # [ 106.096470] nix-grpc-daemon[857]: time=2026-09-22T16:55:26.740Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5189lb # [ 106.142109] niks3-server[1016]: 2026/09/22 16:55:26 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5190lb # [ 106.154044] niks3-server[1016]: 2026/09/22 16:55:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5191worker2 # [ 106.132452] nix-grpc-daemon[857]: time=2026-09-22T16:55:26.779Z level=INFO msg="Uploading 1 narinfos"5192lb # [ 106.167250] niks3-server[1016]: 2026/09/22 16:55:26 INFO Signed narinfos id=32 count=15193lb # [ 106.172803] niks3-server[1016]: 2026/09/22 16:55:26 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5194lb # [ 106.184855] niks3-server[1016]: 2026/09/22 16:55:26 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5195lb # [ 106.187902] niks3-server[1016]: 2026/09/22 16:55:26 INFO Completed upload id=325196lb # [ 106.191816] niks3-server[1016]: 2026/09/22 16:55:26 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5197worker2 # [ 106.155707] nix-grpc-daemon[857]: time=2026-09-22T16:55:26.802Z level=INFO msg="Upload complete. (278ms)"5198client # copying 1 paths...5199worker2 # [ 106.169356] nix-grpc-daemon[774]: ts=2026-09-22T16:55:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40630 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15200worker2 # [ 106.187163] nix-grpc-daemon[774]: ts=2026-09-22T16:55:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:406305201worker2 # [ 106.193060] nix-daemon[843]: reaped child process 1206, status = succeeded5202client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5203worker1 # [ 106.272950] nix-grpc-daemon[767]: ts=2026-09-22T16:55:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:366965204worker1 # [ 106.286945] nix-grpc-daemon[767]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5205worker1 # [ 106.304142] nix-grpc-daemon[767]: ts=2026-09-22T16:55:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_out=1205206client # [ 106.363283] nix-daemon[1156]: reaped child process 1264, status = succeeded5207client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.59 seconds)5208client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25209client # [ 106.534941] nix-daemon[1156]: accepted connection from pid 1283, user root (trusted)5210client # this derivation will be built:5211client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5212worker1 # [ 106.747980] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:366965213client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5214worker1 # [ 106.755637] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366965215worker1 # [ 106.763931] nix-grpc-daemon[902]: time=2026-09-22T16:55:27.396Z level=INFO msg="All 1 paths already cached"5216worker1 # [ 106.775185] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05217worker1 # [ 106.784269] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15218worker1 # [ 106.792850] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366965219worker1 # [ 106.800263] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:366965220worker1 # [ 106.810353] nix-grpc-daemon[902]: time=2026-09-22T16:55:27.445Z level=INFO msg="All 1 paths already cached"5221client # copying 1 paths to 'lb:50051'...5222client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5223worker1 # [ 106.820782] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:366965224worker1 # [ 106.837068] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5225worker1 # [ 106.982959] nix-daemon[851]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)5226worker1 # [ 107.001594] nix-daemon[851]: reaped child process 1175, status = succeeded5227lb # [ 106.981976] niks3-server[1016]: 2026/09/22 16:55:27 INFO Received uploads request method=POST path=/api/pending_closures5228worker1 # [ 107.015603] nix-grpc-daemon[902]: time=2026-09-22T16:55:27.650Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5229worker1 # [ 107.017289] nix-grpc-daemon[902]: time=2026-09-22T16:55:27.650Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5230lb # [ 107.011184] niks3-server[1016]: 2026/09/22 16:55:27 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5231worker1 # [ 107.043988] nix-grpc-daemon[902]: time=2026-09-22T16:55:27.679Z level=INFO msg="Uploading 1 narinfos"5232lb # [ 107.024613] niks3-server[1016]: 2026/09/22 16:55:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5233lb # [ 107.036406] niks3-server[1016]: 2026/09/22 16:55:27 INFO Signed narinfos id=33 count=15234worker1 # [ 107.072505] nix-grpc-daemon[902]: time=2026-09-22T16:55:27.707Z level=INFO msg="Upload complete. (221ms)"5235lb # [ 107.049255] niks3-server[1016]: 2026/09/22 16:55:27 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5236lb # [ 107.058123] niks3-server[1016]: 2026/09/22 16:55:27 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5237worker1 # [ 107.080234] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_in=5685238lb # [ 107.065713] niks3-server[1016]: 2026/09/22 16:55:27 INFO Completed upload id=335239lb # [ 107.068700] niks3-server[1016]: 2026/09/22 16:55:27 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5240worker1 # [ 107.095644] nix-grpc-daemon[767]: ts=2026-09-22T16:55:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:366965241worker1 # [ 107.102459] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5242worker1 # [ 107.107384] nix-daemon[851]: reaped child process 1170, status = succeeded5243client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5244worker1 # [ 107.122086] nix-daemon[851]: accepted connection from pid 767, user nix-grpc-daemon (trusted)5245client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5246client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5247worker1 # [ 108.192290] nix-daemon[851]: reaped child process 1179, status = succeeded5248worker1 # [ 108.284144] nix-daemon[851]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)5249worker1 # [ 108.305155] nix-daemon[851]: reaped child process 1198, status = succeeded5250lb # [ 108.279599] niks3-server[1016]: 2026/09/22 16:55:28 INFO Received uploads request method=POST path=/api/pending_closures5251worker1 # [ 108.316405] nix-grpc-daemon[902]: time=2026-09-22T16:55:28.950Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5252lb # [ 108.313750] niks3-server[1016]: 2026/09/22 16:55:28 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5253worker1 # [ 108.349292] nix-grpc-daemon[902]: time=2026-09-22T16:55:28.984Z level=INFO msg="Uploading 1 narinfos"5254lb # [ 108.326085] niks3-server[1016]: 2026/09/22 16:55:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5255lb # [ 108.335023] niks3-server[1016]: 2026/09/22 16:55:28 INFO Signed narinfos id=34 count=15256lb # [ 108.342461] niks3-server[1016]: 2026/09/22 16:55:28 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5257worker1 # [ 108.375249] nix-grpc-daemon[902]: time=2026-09-22T16:55:29.010Z level=INFO msg="Upload complete. (243ms)"5258lb # [ 108.352703] niks3-server[1016]: 2026/09/22 16:55:29 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5259lb # [ 108.360205] niks3-server[1016]: 2026/09/22 16:55:29 INFO Completed upload id=345260lb # [ 108.362327] niks3-server[1016]: 2026/09/22 16:55:29 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5261worker1 # [ 108.384710] nix-grpc-daemon[767]: ts=2026-09-22T16:55:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15262worker1 # [ 108.400264] nix-daemon[851]: reaped child process 1177, status = succeeded5263client # copying 1 paths...5264worker1 # [ 108.433736] nix-grpc-daemon[767]: ts=2026-09-22T16:55:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:366965265client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5266worker1 # [ 108.464840] nix-grpc-daemon[767]: ts=2026-09-22T16:55:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:366965267worker1 # [ 108.471266] nix-grpc-daemon[767]: ts=2026-09-22T16:55:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:36696 duration_s=0 paths=1 nar_bytes_out=1205268client # [ 108.535605] nix-daemon[1156]: reaped child process 1286, status = succeeded5269client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.17 seconds)5270(finished: subtest: an input only one worker has still reaches the builder, in 9.20 seconds)5271subtest: upload whose reference the worker lacks is completed from the cache5272client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5273client # this derivation will be built:5274client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5275client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5276client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5277client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5278client # this derivation will be built:5279client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5280lb # [ 109.146220] envoy[895]: conn peer=[2001:db8:1::1]:55632 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4338 tx=2392 ms=20255281lb # [ 109.151288] envoy[895]: conn peer=[2001:db8:1::1]:55636 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=795282lb # [ 109.156351] envoy[895]: conn peer=[2001:db8:1::1]:55646 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3874 tx=2068 ms=23875283lb # [ 109.161057] envoy[895]: conn peer=[2001:db8:1::1]:37512 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=695284lb # [ 109.165501] envoy[895]: conn peer=[2001:db8:1::1]:37514 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3744 tx=2127 ms=32075285lb # [ 109.170249] envoy[895]: conn peer=[2001:db8:1::1]:37530 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=945286lb # [ 109.175651] envoy[895]: conn peer=[2001:db8:1::1]:37540 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3783 tx=2099 ms=17815287client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5288lb # [ 109.180510] envoy[895]: conn peer=[2001:db8:1::1]:37554 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=685289client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)5290client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5291client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5292worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5293worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.20 seconds)5294worker2 # [ 109.551013] systemd[1]: Stopping Nix worker-protocol over gRPC...5295worker2 # [ 110.490617] nix-grpc-daemon[774]: ts=2026-09-22T16:55:31Z level=info event=signal_exit5296worker1 # [ 112.339315] nix-grpc-daemon[767]: ts=2026-09-22T16:55:32Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453865297worker2 # [ 112.290142] nix-grpc-daemon[774]: ts=2026-09-22T16:55:32Z level=info event=scheduler_restarting5298worker2 # [ 112.319695] nix-grpc-daemon[774]: ts=2026-09-22T16:55:32Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5299worker2 # [ 112.345689] nix-daemon[843]: reaped child process 845, status = succeeded5300worker2 # [ 112.666742] nix-grpc-daemon[774]: E0000 00:00:1790096133.313697 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5301worker2 # [ 112.673349] nix-grpc-daemon[774]: E0000 00:00:1790096133.313996 819 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5302worker1 # [ 112.344419] nix-grpc-daemon[767]: ts=2026-09-22T16:55:32Z level=info event=worker_gone addr=192.168.1.4:500515303worker2 # [ 112.965714] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5304worker2 # [ 112.970204] systemd[1]: Stopped Nix worker-protocol over gRPC.5305worker2 # [ 112.976443] systemd[1]: nix-grpc-daemon.service: Consumed 3.720s CPU time over 1min 33.722s wall clock time, 62M memory peak, 104.8K incoming IP traffic, 254K outgoing IP traffic.5306worker2 # [ 112.992473] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5307worker2 # [ 113.004815] systemd[1]: Closed Nix worker-protocol over gRPC.5308worker2 # [ 113.006697] systemd[1]: nix-grpc-daemon.socket: Consumed 763us CPU time over 1min 43.507s wall clock time, 256K memory peak, 106.3K incoming IP traffic, 90.1K outgoing IP traffic.5309lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5310lb: must succeed: curl -sf localhost:9901/clusters5311lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5312lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5313client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25314worker1 # [ 113.350786] nix-grpc-daemon[767]: ts=2026-09-22T16:55:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:366965315worker1 # [ 113.359669] nix-grpc-daemon[767]: ts=2026-09-22T16:55:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:366965316client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5317client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5318worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5319worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5320worker2 # [ 113.436328] systemd[1]: Listening on Nix worker-protocol over gRPC.5321worker1 # [ 113.566521] systemd[1]: Stopping Nix worker-protocol over gRPC...5322worker2 # [ 113.963359] systemd[1]: Starting scheduler-token.service...5323worker2 # [ 114.049883] systemd[1]: scheduler-token.service: Deactivated successfully.5324worker2 # [ 114.056711] systemd[1]: Finished scheduler-token.service.5325worker2 # [ 114.058704] systemd[1]: scheduler-token.service: Consumed 56ms CPU time over 89ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5326worker2 # [ 114.069770] systemd[1]: Starting Nix worker-protocol over gRPC...5327worker2 # [ 114.203594] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:34Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5328worker2 # [ 114.206064] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:34Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5329worker2 # [ 114.209868] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:34Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15330worker2 # [ 114.215445] nix-grpc-daemon[1252]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5331worker2 # [ 114.233896] nix-grpc-daemon[1252]: E0000 00:00:1790096134.861939 1252 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5332worker2 # [ 114.263787] nix-grpc-daemon[1252]: E0000 00:00:1790096134.865588 1252 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5333worker2 # [ 114.268254] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:34Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5334worker2 # [ 114.270380] nix-grpc-daemon[1252]: E0000 00:00:1790096134.877650 1292 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5335worker2 # [ 114.272996] nix-grpc-daemon[1252]: E0000 00:00:1790096134.897270 1292 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5336worker2 # [ 114.275287] nix-grpc-daemon[1252]: E0000 00:00:1790096134.897417 1292 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5337worker2 # [ 114.280490] systemd[1]: Started Nix worker-protocol over gRPC.5338worker2 # [ 114.281375] nix-grpc-daemon[1252]: E0000 00:00:1790096134.914658 1292 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5339worker2 # [ 114.285660] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:34Z level=info event=scheduler_connected addr=lb:500515340worker1 # [ 114.337268] nix-grpc-daemon[767]: ts=2026-09-22T16:55:34Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:453865341worker1 # [ 114.343233] nix-grpc-daemon[767]: ts=2026-09-22T16:55:34Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05342worker1 # [ 114.459528] nix-grpc-daemon[767]: ts=2026-09-22T16:55:35Z level=info event=signal_exit5343lb # [ 115.314259] niks3-server[1016]: 2026/09/22 16:55:35 INFO lead: released remote=192.168.1.3:525625344worker2 # [ 116.281689] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:36Z level=info event=scheduler_restarting addr=lb:500515345worker1 # [ 116.335432] nix-grpc-daemon[767]: ts=2026-09-22T16:55:36Z level=info event=scheduler_restarting5346worker1 # [ 116.346014] nix-grpc-daemon[767]: ts=2026-09-22T16:55:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453865347worker1 # [ 116.351262] nix-grpc-daemon[767]: ts=2026-09-22T16:55:36Z level=info event=worker_gone addr=192.168.1.4:500515348worker1 # [ 116.355931] nix-grpc-daemon[767]: ts=2026-09-22T16:55:36Z level=info event=scheduler_restarting addr=lb:500515349worker1 # [ 116.360663] nix-grpc-daemon[767]: ts=2026-09-22T16:55:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:453865350worker1 # [ 116.366292] nix-grpc-daemon[767]: ts=2026-09-22T16:55:36Z level=info event=worker_gone addr=192.168.1.3:500515351worker1 # [ 116.392397] nix-daemon[851]: reaped child process 853, status = succeeded5352worker2 # [ 116.481308] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_connected addr=lb:500515353worker2 # [ 116.486537] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5354worker1 # [ 116.568671] nix-grpc-daemon[767]: ts=2026-09-22T16:55:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5355worker1 # [ 116.610475] nix-grpc-daemon[767]: E0000 00:00:1790096137.245376 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5356worker1 # [ 116.617089] nix-grpc-daemon[767]: E0000 00:00:1790096137.245714 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5357worker2 # [ 116.687667] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_connected addr=lb:500515358worker2 # [ 116.692913] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5359worker2 # [ 116.894092] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_connected addr=lb:500515360worker2 # [ 116.899384] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5361worker1 # [ 116.964159] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5362worker1 # [ 116.971478] systemd[1]: Stopped Nix worker-protocol over gRPC.5363worker1 # [ 116.977825] systemd[1]: nix-grpc-daemon.service: Consumed 3.000s CPU time over 1min 37.788s wall clock time, 65.9M memory peak, 100.6K incoming IP traffic, 198.2K outgoing IP traffic.5364worker1 # [ 116.997056] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5365worker1 # [ 116.999140] systemd[1]: Closed Nix worker-protocol over gRPC.5366lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5367lb: must succeed: curl -sf localhost:9901/clusters5368worker1 # [ 117.004902] systemd[1]: nix-grpc-daemon.socket: Consumed 565us CPU time over 1min 46.741s wall clock time, 256K memory peak, 122.9K incoming IP traffic, 105.4K outgoing IP traffic.5369lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5370lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5371client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25372worker2 # [ 117.100472] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_connected addr=lb:500515373worker2 # [ 117.105770] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374worker2 # [ 117.258853] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374965375worker2 # [ 117.273685] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5376worker2 # [ 117.303029] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)5377worker2 # [ 117.310549] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_connected addr=lb:500515378worker2 # [ 117.314530] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5379worker2 # [ 117.321722] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 69 ms (attempt 2/5)5380worker2 # [ 117.398216] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 89 ms (attempt 3/5)5381worker2 # [ 117.488835] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 757 ms (attempt 4/5)5382worker2 # [ 117.515601] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_connected addr=lb:500515383worker2 # [ 117.520935] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5384worker2 # [ 117.722042] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_connected addr=lb:500515385worker2 # [ 117.727071] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5386worker2 # [ 117.928299] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_connected addr=lb:500515387worker2 # [ 117.933345] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5388worker2 # [ 118.134314] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_connected addr=lb:500515389worker2 # [ 118.139403] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5390worker2 # [ 118.249454] nix-grpc-daemon[1252]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5391worker2 # [ 118.256461] nix-grpc-daemon[1252]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5392client # copying 2 paths...5393client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5394worker2 # [ 118.279759] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374965395client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5396worker2 # [ 118.298433] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5397worker2 # [ 118.329067] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=niks3_push_started pid=13125398worker2 # [ 118.341332] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_connected addr=lb:500515399worker2 # [ 118.356650] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:38Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5400worker2 # [ 118.495884] nix-grpc-daemon[1312]: time=2026-09-22T16:55:39.140Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5401worker2 # [ 118.548998] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=scheduler_connected addr=lb:500515402worker2 # [ 118.551781] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5403worker2 # [ 118.604078] nix-daemon[843]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)5404worker2 # [ 118.626351] nix-daemon[843]: reaped child process 1328, status = succeeded5405lb # [ 118.652988] niks3-server[1016]: 2026/09/22 16:55:39 INFO Received uploads request method=POST path=/api/pending_closures5406lb # [ 118.663687] niks3-server[1016]: 2026/09/22 16:55:39 INFO Received uploads request method=POST path=/api/pending_closures5407worker2 # [ 118.647055] nix-grpc-daemon[1312]: time=2026-09-22T16:55:39.292Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5408worker2 # [ 118.650944] nix-grpc-daemon[1312]: time=2026-09-22T16:55:39.292Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5409worker2 # [ 118.653330] nix-grpc-daemon[1312]: time=2026-09-22T16:55:39.292Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5410lb # [ 118.722436] niks3-server[1016]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5411lb # [ 118.739083] niks3-server[1016]: 2026/09/22 16:55:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5412lb # [ 118.747699] niks3-server[1016]: 2026/09/22 16:55:39 INFO Signed narinfos id=35 count=15413worker2 # [ 118.719355] nix-grpc-daemon[1312]: time=2026-09-22T16:55:39.364Z level=INFO msg="Uploading 3 narinfos"5414lb # [ 118.752711] niks3-server[1016]: 2026/09/22 16:55:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5415lb # [ 118.755159] niks3-server[1016]: 2026/09/22 16:55:39 INFO Signed narinfos id=36 count=25416lb # [ 118.763175] niks3-server[1016]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5417lb # [ 118.768511] niks3-server[1016]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5418lb # [ 118.770548] niks3-server[1016]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5419worker2 # [ 118.752388] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=scheduler_connected addr=lb:500515420lb # [ 118.783441] niks3-server[1016]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5421worker2 # [ 118.758040] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5422lb # [ 118.790794] niks3-server[1016]: 2026/09/22 16:55:39 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5423lb # [ 118.795788] niks3-server[1016]: 2026/09/22 16:55:39 INFO Completed upload id=355424lb # [ 118.798560] niks3-server[1016]: 2026/09/22 16:55:39 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5425worker2 # [ 118.771233] nix-grpc-daemon[1312]: time=2026-09-22T16:55:39.418Z level=INFO msg="Upload complete. (276ms)"5426lb # [ 118.800727] niks3-server[1016]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5427lb # [ 118.802881] niks3-server[1016]: 2026/09/22 16:55:39 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5428lb # [ 118.804926] niks3-server[1016]: 2026/09/22 16:55:39 INFO Completed upload id=365429worker2 # [ 118.784603] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=0 paths=2 nar_bytes_in=3125430worker2 # [ 118.800645] nix-daemon[843]: reaped child process 1310, status = succeeded5431client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.79 seconds)5432worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5433worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5434worker1 # [ 118.975125] systemd[1]: Listening on Nix worker-protocol over gRPC.5435worker1 # [ 118.986275] systemd[1]: Starting Nix worker-protocol over gRPC...5436worker2 # [ 118.959200] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=scheduler_connected addr=lb:500515437worker2 # [ 118.964671] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5438worker1 # [ 119.135317] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5439worker1 # [ 119.137771] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5440worker1 # [ 119.141778] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15441worker1 # [ 119.147236] nix-grpc-daemon[1221]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5442lb # [ 119.132563] niks3-server[1016]: 2026/09/22 16:55:39 INFO lead: acquired remote=192.168.1.3:606525443worker1 # [ 119.168208] nix-grpc-daemon[1221]: E0000 00:00:1790096139.781952 1221 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5444worker1 # [ 119.170582] nix-grpc-daemon[1221]: E0000 00:00:1790096139.785689 1221 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5445lb # [ 119.147730] envoy[895]: conn peer=[2001:db8:1::4]:57320 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=76823 tx=4257 ms=927905446lb # [ 119.153115] envoy[895]: conn peer=[2001:db8:1::1]:40944 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=315447lb # [ 119.157783] envoy[895]: conn peer=[2001:db8:1::3]:41134 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=15122 tx=4082 ms=970725448lb # [ 119.163114] envoy[895]: conn peer=[2001:db8:1::1]:40960 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1004 tx=305 ms=15405449lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5450lb: must succeed: curl -sf localhost:9901/clusters5451worker1 # [ 119.196981] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5452worker1 # [ 119.199132] nix-grpc-daemon[1221]: E0000 00:00:1790096139.792545 1263 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5453worker1 # [ 119.201864] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:39Z level=info event=scheduler_take_over5454worker1 # [ 119.202971] systemd[1]: Started Nix worker-protocol over gRPC.5455worker1 # [ 119.203743] nix-grpc-daemon[1221]: E0000 00:00:1790096139.814687 1263 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5456worker1 # [ 119.206198] nix-grpc-daemon[1221]: E0000 00:00:1790096139.814892 1263 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5457worker1 # [ 119.211664] nix-grpc-daemon[1221]: E0000 00:00:1790096139.831440 1263 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5458worker1 # [ 119.216053] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:39Z level=info event=scheduler_connected addr=lb:500515459worker1 # [ 119.217787] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5460worker2 # [ 119.165932] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=scheduler_connected addr=lb:500515461worker2 # [ 119.171116] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:39Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5462lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5463worker1 # [ 119.418891] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=scheduler_connected addr=lb:500515464worker1 # [ 119.423887] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5465worker2 # [ 119.373790] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:40Z level=info event=scheduler_connected addr=lb:500515466worker2 # [ 119.377994] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5467worker1 # [ 119.624565] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=scheduler_connected addr=lb:500515468worker2 # [ 119.574551] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:40Z level=info event=scheduler_connected addr=lb:500515469worker1 # [ 119.629870] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5470worker2 # [ 119.579812] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5471worker1 # [ 119.830728] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=scheduler_connected addr=lb:500515472worker2 # [ 119.780843] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:40Z level=info event=scheduler_connected addr=lb:500515473worker1 # [ 119.836151] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5474worker2 # [ 119.785950] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5475worker1 # [ 120.036754] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=scheduler_connected addr=lb:500515476worker2 # [ 119.987108] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:40Z level=info event=scheduler_connected addr=lb:500515477worker1 # [ 120.045579] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:528985478worker1 # [ 120.052480] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05479worker1 # [ 120.064124] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=oidc_jwks provider=mock keys=15480worker1 # [ 120.068726] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:528985481worker1 # [ 120.073764] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:40Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05482lb: must succeed: curl -sf localhost:9901/clusters5483lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5484lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5485(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.73 seconds)5486subtest: a niks3 restart does not cost the leader its role5487lb: must succeed: curl -sf localhost:9901/clusters5488lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5489lb: must succeed: curl -sf localhost:9901/clusters5490lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5491lb # [ 120.438654] systemd[1]: Stopping niks3 server...5492lb # [ 120.441814] niks3-server[1016]: 2026/09/22 16:55:41 INFO Shutdown signal received, draining in-flight requests timeout=10s5493worker1 # [ 120.471472] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:41Z level=info event=scheduler_lock_lost status=2005494worker2 # [ 120.419472] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:41Z level=info event=scheduler_lock_lost status=2005495lb # [ 120.465004] niks3-server[1016]: 2026/09/22 16:55:41 INFO lead: released remote=192.168.1.3:606525496lb # [ 120.470081] systemd[1]: niks3.service: Deactivated successfully.5497lb # [ 120.472363] systemd[1]: Stopped niks3 server.5498lb # [ 120.473149] systemd[1]: niks3.service: Consumed 658ms CPU time over 1min 31.396s wall clock time, 27.8M memory peak, 100.1K incoming IP traffic, 143.1K outgoing IP traffic.5499lb # [ 120.483586] systemd[1]: Starting niks3 server...5500lb # [ 120.549411] niks3-server[1154]: 2026/09/22 16:55:41 goose: no migrations to run. current version: 202609200000005501lb # [ 120.557077] niks3-server[1154]: 2026/09/22 16:55:41 OK 1_commit_pending_closure.sql (5.73ms)5502lb # [ 120.560660] niks3-server[1154]: 2026/09/22 16:55:41 OK 2_object_stats_trigger.sql (3.21ms)5503lb # [ 120.561813] niks3-server[1154]: 2026/09/22 16:55:41 goose: up to current file version: 25504lb # [ 120.563449] niks3-server[1154]: 2026/09/22 16:55:41 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5505lb # [ 120.574246] niks3-server[1154]: 2026/09/22 16:55:41 INFO Read proxy enabled — serving cache objects from S35506lb # [ 120.578892] niks3-server[1154]: 2026/09/22 16:55:41 INFO Using socket-activated listener address=0.0.0.0:57515507lb # [ 120.581874] niks3-server[1154]: 2026/09/22 16:55:41 INFO systemd watchdog enabled interval=15s5508lb # [ 120.584415] niks3-server[1154]: 2026/09/22 16:55:41 INFO Starting HTTP server address=0.0.0.0:57515509lb # [ 120.585805] systemd[1]: Started niks3 server.5510lb: waiting for unit niks3.service5511lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5512lb: must succeed: curl -sf localhost:9901/clusters5513lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5514worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5515worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5516lb: must succeed: curl -sf localhost:9901/clusters5517lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5518lb: must succeed: curl -sf localhost:9901/clusters5519lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5520worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5521worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.07 seconds)5522client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25523worker1 # [ 121.090168] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:432445524worker1 # [ 121.101744] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5525client # don't know how to build these paths:5526client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5527worker2 # [ 121.077912] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374965528worker1 # [ 121.163159] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:41Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05529worker1 # [ 121.170355] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:41Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=15530worker1 # [ 121.216984] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432445531worker1 # [ 121.236976] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 1/5)5532worker1 # [ 121.317954] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 2/5)5533worker1 # [ 121.340955] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 258 ms (attempt 3/5)5534lb # [ 121.452155] niks3-server[1154]: 2026/09/22 16:55:42 INFO lead: acquired remote=192.168.1.3:379905535worker1 # [ 121.609529] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 504 ms (attempt 4/5)5536worker1 # [ 122.115408] nix-grpc-daemon[1221]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5537worker1 # [ 122.121410] nix-grpc-daemon[1221]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5538worker1 # [ 122.140275] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432445539client # copying 1 paths to 'lb:50051'...5540client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5541worker1 # [ 122.158996] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432445542worker1 # [ 122.174364] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5543worker1 # [ 122.209791] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:42Z level=info event=niks3_push_started pid=12935544worker1 # [ 122.358882] nix-grpc-daemon[1293]: time=2026-09-22T16:55:42.993Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5545worker1 # [ 122.484234] nix-daemon[851]: accepted connection from pid 1305, user nix-grpc-daemon (trusted)5546worker1 # [ 122.503763] nix-daemon[851]: reaped child process 1308, status = succeeded5547lb # [ 122.485998] niks3-server[1154]: 2026/09/22 16:55:43 INFO Received uploads request method=POST path=/api/pending_closures5548worker1 # [ 122.523751] nix-grpc-daemon[1293]: time=2026-09-22T16:55:43.157Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5549worker1 # [ 122.527719] nix-grpc-daemon[1293]: time=2026-09-22T16:55:43.157Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5550lb # [ 122.543807] niks3-server[1154]: 2026/09/22 16:55:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5551lb # [ 122.558942] niks3-server[1154]: 2026/09/22 16:55:43 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5552lb # [ 122.581003] niks3-server[1154]: 2026/09/22 16:55:43 INFO Signed narinfos id=37 count=15553worker1 # [ 122.609952] nix-grpc-daemon[1293]: time=2026-09-22T16:55:43.243Z level=INFO msg="Uploading 1 narinfos"5554lb # [ 122.600177] niks3-server[1154]: 2026/09/22 16:55:43 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5555lb # [ 122.603238] niks3-server[1154]: 2026/09/22 16:55:43 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5556lb # [ 122.612810] niks3-server[1154]: 2026/09/22 16:55:43 INFO Completed upload id=375557worker1 # [ 122.637747] nix-grpc-daemon[1293]: time=2026-09-22T16:55:43.273Z level=INFO msg="Upload complete. (275ms)"5558lb # [ 122.618484] niks3-server[1154]: 2026/09/22 16:55:43 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5559worker1 # [ 122.649147] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43244 duration_s=0 paths=1 nar_bytes_in=5445560worker1 # [ 122.668987] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432445561worker1 # [ 122.677902] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5562worker1 # [ 122.685170] nix-daemon[851]: reaped child process 1291, status = succeeded5563client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5564client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5565worker1 # [ 122.701860] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5566client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5567client # farm-a> LOG-farm-a-after-niks3-restart5568worker1 # [ 123.611716] nix-daemon[851]: reaped child process 1312, status = succeeded5569worker1 # [ 123.682812] nix-daemon[851]: accepted connection from pid 1328, user nix-grpc-daemon (trusted)5570worker1 # [ 123.705204] nix-daemon[851]: reaped child process 1331, status = succeeded5571lb # [ 123.682064] niks3-server[1154]: 2026/09/22 16:55:44 INFO Received uploads request method=POST path=/api/pending_closures5572worker1 # [ 123.719626] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.354Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5573worker1 # [ 123.724613] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.356Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5574lb # [ 123.721862] niks3-server[1154]: 2026/09/22 16:55:44 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5575lb # [ 123.742049] niks3-server[1154]: 2026/09/22 16:55:44 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5576worker1 # [ 123.779152] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.413Z level=INFO msg="Uploading 1 narinfos"5577lb # [ 123.759096] niks3-server[1154]: 2026/09/22 16:55:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5578lb # [ 123.768776] niks3-server[1154]: 2026/09/22 16:55:44 INFO Signed narinfos id=38 count=15579lb # [ 123.770887] niks3-server[1154]: 2026/09/22 16:55:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5580worker1 # [ 123.802908] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.438Z level=INFO msg="Upload complete. (288ms)"5581lb # [ 123.778574] niks3-server[1154]: 2026/09/22 16:55:44 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5582lb # [ 123.783452] niks3-server[1154]: 2026/09/22 16:55:44 INFO Completed upload id=385583lb # [ 123.788476] niks3-server[1154]: 2026/09/22 16:55:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5584worker1 # [ 123.824992] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43244 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15585worker1 # [ 123.840644] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:44Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05586worker1 # [ 123.846742] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:44Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=15587worker1 # [ 123.851035] nix-daemon[851]: reaped child process 1310, status = succeeded5588worker1 # [ 123.870010] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432445589worker1 # [ 123.886428] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432445590worker1 # [ 123.901312] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.536Z level=INFO msg="All 1 paths already cached"5591client # copying 1 paths to 'lb:50051'...5592client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5593worker1 # [ 123.912454] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432445594worker1 # [ 123.926344] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5595worker1 # [ 124.084505] nix-daemon[851]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5596worker1 # [ 124.103386] nix-daemon[851]: reaped child process 1340, status = succeeded5597lb # [ 124.083190] niks3-server[1154]: 2026/09/22 16:55:44 INFO Received uploads request method=POST path=/api/pending_closures5598worker1 # [ 124.119935] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.754Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5599worker1 # [ 124.122251] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.754Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5600lb # [ 124.116251] niks3-server[1154]: 2026/09/22 16:55:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5601worker1 # [ 124.150412] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.785Z level=INFO msg="Uploading 1 narinfos"5602lb # [ 124.133890] niks3-server[1154]: 2026/09/22 16:55:44 INFO Signed narinfos id=39 count=15603lb # [ 124.142846] niks3-server[1154]: 2026/09/22 16:55:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5604lb # [ 124.154340] niks3-server[1154]: 2026/09/22 16:55:44 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5605worker1 # [ 124.191951] nix-grpc-daemon[1293]: time=2026-09-22T16:55:44.827Z level=INFO msg="Upload complete. (249ms)"5606lb # [ 124.168131] niks3-server[1154]: 2026/09/22 16:55:44 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5607lb # [ 124.171795] niks3-server[1154]: 2026/09/22 16:55:44 INFO Completed upload id=395608lb # [ 124.175836] niks3-server[1154]: 2026/09/22 16:55:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5609worker1 # [ 124.204151] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43244 duration_s=0 paths=1 nar_bytes_in=6565610worker1 # [ 124.220707] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432445611worker1 # [ 124.227965] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5612worker1 # [ 124.234108] nix-daemon[851]: reaped child process 1335, status = succeeded5613client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5614worker1 # [ 124.246158] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5615client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5616client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5617client # farm-b> LOG-farm-b-after-niks3-restart5618worker1 # [ 125.482636] nix-daemon[851]: reaped child process 1344, status = succeeded5619worker1 # [ 125.591079] nix-daemon[851]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5620worker1 # [ 125.610723] nix-daemon[851]: reaped child process 1363, status = succeeded5621lb # [ 125.591994] niks3-server[1154]: 2026/09/22 16:55:46 INFO Received uploads request method=POST path=/api/pending_closures5622worker1 # [ 125.627416] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.261Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5623worker1 # [ 125.629149] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.261Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5624lb # [ 125.636455] niks3-server[1154]: 2026/09/22 16:55:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5625worker1 # [ 125.671827] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.306Z level=INFO msg="Uploading 1 narinfos"5626lb # [ 125.650710] niks3-server[1154]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5627lb # [ 125.662837] niks3-server[1154]: 2026/09/22 16:55:46 INFO Signed narinfos id=40 count=15628lb # [ 125.672319] niks3-server[1154]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5629worker1 # [ 125.702507] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.337Z level=INFO msg="Upload complete. (244ms)"5630lb # [ 125.675986] niks3-server[1154]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5631lb # [ 125.683850] niks3-server[1154]: 2026/09/22 16:55:46 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5632lb # [ 125.690065] niks3-server[1154]: 2026/09/22 16:55:46 INFO Completed upload id=405633lb # [ 125.692551] niks3-server[1154]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5634worker1 # [ 125.720386] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43244 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15635worker1 # [ 125.736970] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:46Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05636worker1 # [ 125.742548] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:46Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=15637worker1 # [ 125.747225] nix-daemon[851]: reaped child process 1342, status = succeeded5638worker1 # [ 125.777631] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432445639worker1 # [ 125.793264] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432445640worker1 # [ 125.807001] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.442Z level=INFO msg="All 2 paths already cached"5641client # copying 1 paths to 'lb:50051'...5642client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5643worker1 # [ 125.818719] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432445644worker1 # [ 125.833106] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5645worker1 # [ 125.989689] nix-daemon[851]: accepted connection from pid 1369, user nix-grpc-daemon (trusted)5646worker1 # [ 126.008315] nix-daemon[851]: reaped child process 1372, status = succeeded5647lb # [ 125.990545] niks3-server[1154]: 2026/09/22 16:55:46 INFO Received uploads request method=POST path=/api/pending_closures5648worker1 # [ 126.024801] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.659Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5649worker1 # [ 126.026654] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.659Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5650lb # [ 126.021886] niks3-server[1154]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5651lb # [ 126.032724] niks3-server[1154]: 2026/09/22 16:55:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5652worker1 # [ 126.065214] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.700Z level=INFO msg="Uploading 1 narinfos"5653lb # [ 126.045941] niks3-server[1154]: 2026/09/22 16:55:46 INFO Signed narinfos id=41 count=15654lb # [ 126.051688] niks3-server[1154]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5655worker1 # [ 126.087202] nix-grpc-daemon[1293]: time=2026-09-22T16:55:46.722Z level=INFO msg="Upload complete. (238ms)"5656lb # [ 126.064748] niks3-server[1154]: 2026/09/22 16:55:46 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5657lb # [ 126.069853] niks3-server[1154]: 2026/09/22 16:55:46 INFO Completed upload id=415658worker1 # [ 126.096257] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43244 duration_s=0 paths=1 nar_bytes_in=7845659lb # [ 126.073763] niks3-server[1154]: 2026/09/22 16:55:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5660worker1 # [ 126.108496] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432445661worker1 # [ 126.118471] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5662worker1 # [ 126.124773] nix-daemon[851]: reaped child process 1367, status = succeeded5663client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5664worker1 # [ 126.137782] nix-daemon[851]: accepted connection from pid 1221, user nix-grpc-daemon (trusted)5665client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5666client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5667client # farm-top> LOG-farm-top-after-niks3-restart5668worker1 # [ 127.242301] nix-daemon[851]: reaped child process 1376, status = succeeded5669worker1 # [ 127.312987] nix-daemon[851]: accepted connection from pid 1392, user nix-grpc-daemon (trusted)5670worker1 # [ 127.330158] nix-daemon[851]: reaped child process 1395, status = succeeded5671lb # [ 127.313922] niks3-server[1154]: 2026/09/22 16:55:47 INFO Received uploads request method=POST path=/api/pending_closures5672worker1 # [ 127.348428] nix-grpc-daemon[1293]: time=2026-09-22T16:55:47.983Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5673worker1 # [ 127.350001] nix-grpc-daemon[1293]: time=2026-09-22T16:55:47.983Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5674lb # [ 127.355266] niks3-server[1154]: 2026/09/22 16:55:48 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5675worker1 # [ 127.392720] nix-grpc-daemon[1293]: time=2026-09-22T16:55:48.027Z level=INFO msg="Uploading 1 narinfos"5676lb # [ 127.367706] niks3-server[1154]: 2026/09/22 16:55:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5677lb # [ 127.380895] niks3-server[1154]: 2026/09/22 16:55:48 INFO Signed narinfos id=42 count=15678lb # [ 127.383643] niks3-server[1154]: 2026/09/22 16:55:48 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5679lb # [ 127.399663] niks3-server[1154]: 2026/09/22 16:55:48 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5680worker1 # [ 127.429756] nix-grpc-daemon[1293]: time=2026-09-22T16:55:48.065Z level=INFO msg="Upload complete. (281ms)"5681lb # [ 127.410044] niks3-server[1154]: 2026/09/22 16:55:48 INFO Completed upload id=425682lb # [ 127.414409] niks3-server[1154]: 2026/09/22 16:55:48 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5683lb # [ 127.421822] niks3-server[1154]: 2026/09/22 16:55:48 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5684worker1 # [ 127.445360] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43244 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15685worker1 # [ 127.463440] nix-daemon[851]: reaped child process 1374, status = succeeded5686client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.62 seconds)5687client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5688client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds)5689(finished: subtest: a niks3 restart does not cost the leader its role, in 7.34 seconds)5690subtest: scheduler down: the other node takes the lock and keeps it5691lb: must succeed: curl -sf localhost:9901/clusters5692lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5693lb: must succeed: curl -sf localhost:9901/clusters5694lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5695lb: must succeed: curl -sf localhost:9901/clusters5696lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5697lb: must succeed: curl -sf localhost:9901/clusters5698lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5699worker1 # [ 127.927929] systemd[1]: Stopping Nix worker-protocol over gRPC...5700worker1 # [ 128.159189] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:48Z level=info event=signal_exit5701lb # [ 128.456580] niks3-server[1154]: 2026/09/22 16:55:49 INFO lead: released remote=192.168.1.3:379905702lb # [ 129.148939] envoy[895]: conn peer=[2001:db8:1::1]:57674 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9723 tx=4419 ms=63805703worker1 # [ 129.482215] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:50Z level=info event=scheduler_restarting5704worker2 # [ 129.432913] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_restarting addr=lb:500515705worker1 # [ 129.495403] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:528985706worker1 # [ 129.500564] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:50Z level=info event=worker_gone addr=192.168.1.4:500515707worker1 # [ 129.505142] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:50Z level=info event=scheduler_restarting addr=lb:500515708worker1 # [ 129.510291] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:50Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:528985709worker1 # [ 129.520562] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:50Z level=info event=worker_gone addr=192.168.1.3:500515710worker1 # [ 129.538146] nix-daemon[851]: reaped child process 1281, status = succeeded5711worker2 # [ 129.632972] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_connected addr=lb:500515712worker2 # [ 129.638175] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5713worker1 # [ 129.711978] nix-grpc-daemon[1221]: ts=2026-09-22T16:55:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5714worker2 # [ 129.839314] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_connected addr=lb:500515715worker2 # [ 129.845015] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5716worker1 # [ 129.997971] nix-grpc-daemon[1221]: E0000 00:00:1790096150.633176 1266 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 10, started true, shutdown true5717worker1 # [ 130.006104] nix-grpc-daemon[1221]: E0000 00:00:1790096150.641340 1266 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5718worker2 # [ 130.046056] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_connected addr=lb:500515719worker2 # [ 130.051564] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5720worker2 # [ 130.252838] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_connected addr=lb:500515721worker2 # [ 130.257937] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5722worker2 # [ 130.461098] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_connected addr=lb:500515723worker2 # [ 130.465335] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5724worker2 # [ 130.661855] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_connected addr=lb:500515725worker2 # [ 130.667229] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5726worker1 # [ 130.761450] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5727worker1 # [ 130.768839] systemd[1]: Stopped Nix worker-protocol over gRPC.5728worker1 # [ 130.771416] systemd[1]: nix-grpc-daemon.service: Consumed 1.255s CPU time over 11.778s wall clock time, 38.1M memory peak, 36.4K incoming IP traffic, 59.1K outgoing IP traffic.5729worker1 # [ 130.788430] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5730worker1 # [ 130.790640] systemd[1]: Closed Nix worker-protocol over gRPC.5731worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5732worker1 # [ 130.799804] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 11.823s wall clock time, 256K memory peak, 33.4K incoming IP traffic, 24.6K outgoing IP traffic.5733worker2 # [ 130.868454] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_connected addr=lb:500515734worker2 # [ 130.873620] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5735worker2 # [ 131.074775] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_connected addr=lb:500515736worker2 # [ 131.080384] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5737worker2 # [ 131.281081] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_connected addr=lb:500515738worker2 # [ 131.287280] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5739worker2 # [ 131.425925] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_take_over5740lb # [ 131.454237] niks3-server[1154]: 2026/09/22 16:55:52 INFO lead: acquired remote=192.168.1.4:602345741worker2 # [ 131.490277] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_connected addr=lb:500515742worker2 # [ 131.494521] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5743worker2 # [ 131.691626] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_connected addr=lb:500515744worker2 # [ 131.697089] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5745worker2 # [ 131.898626] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_connected addr=lb:500515746worker2 # [ 131.905112] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5747worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.17 seconds)5748worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5749worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5750worker2 # [ 132.104255] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_connected addr=lb:500515751worker2 # [ 132.109846] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5752worker2 # [ 132.317443] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_connected addr=lb:500515753worker2 # [ 132.330554] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5754worker2 # [ 132.531351] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=info event=scheduler_connected addr=lb:500515755worker2 # [ 132.560175] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=info event=oidc_jwks provider=mock keys=15756worker2 # [ 132.565322] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:470285757worker2 # [ 132.570723] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05758worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5759worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5760client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25761client # don't know how to build these paths:5762client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5763worker2 # [ 133.241746] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374965764worker2 # [ 133.249891] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374965765worker2 # [ 133.276138] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05766worker2 # [ 133.282357] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15767worker2 # [ 133.329093] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374965768worker2 # [ 133.345280] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374965769client # copying 1 paths to 'lb:50051'...5770client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5771worker2 # [ 133.361053] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374965772worker2 # [ 133.375927] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5773worker2 # [ 133.533304] nix-daemon[843]: accepted connection from pid 1357, user nix-grpc-daemon (trusted)5774worker2 # [ 133.553392] nix-daemon[843]: reaped child process 1360, status = succeeded5775lb # [ 133.586093] niks3-server[1154]: 2026/09/22 16:55:54 INFO Received uploads request method=POST path=/api/pending_closures5776worker2 # [ 133.571744] nix-grpc-daemon[1312]: time=2026-09-22T16:55:54.218Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5777worker2 # [ 133.573674] nix-grpc-daemon[1312]: time=2026-09-22T16:55:54.218Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5778lb # [ 133.618107] niks3-server[1154]: 2026/09/22 16:55:54 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5779worker2 # [ 133.600186] nix-grpc-daemon[1312]: time=2026-09-22T16:55:54.247Z level=INFO msg="Uploading 1 narinfos"5780lb # [ 133.628800] niks3-server[1154]: 2026/09/22 16:55:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5781lb # [ 133.637138] niks3-server[1154]: 2026/09/22 16:55:54 INFO Signed narinfos id=43 count=15782lb # [ 133.647699] niks3-server[1154]: 2026/09/22 16:55:54 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5783lb # [ 133.661821] niks3-server[1154]: 2026/09/22 16:55:54 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5784worker2 # [ 133.643385] nix-grpc-daemon[1312]: time=2026-09-22T16:55:54.290Z level=INFO msg="Upload complete. (250ms)"5785lb # [ 133.673774] niks3-server[1154]: 2026/09/22 16:55:54 INFO Completed upload id=435786lb # [ 133.676308] niks3-server[1154]: 2026/09/22 16:55:54 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5787worker2 # [ 133.657381] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=0 paths=1 nar_bytes_in=5205788worker2 # [ 133.672881] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374965789worker2 # [ 133.681596] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5790worker2 # [ 133.687085] nix-daemon[843]: reaped child process 1355, status = succeeded5791client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5792worker2 # [ 133.698242] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5793client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5794client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5795client # farm-a> LOG-farm-a-on-standby5796worker2 # [ 135.158909] nix-daemon[843]: reaped child process 1364, status = succeeded5797worker2 # [ 135.277793] nix-daemon[843]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5798worker2 # [ 135.293598] nix-daemon[843]: reaped child process 1383, status = succeeded5799lb # [ 135.330767] niks3-server[1154]: 2026/09/22 16:55:55 INFO Received uploads request method=POST path=/api/pending_closures5800worker2 # [ 135.316468] nix-grpc-daemon[1312]: time=2026-09-22T16:55:55.961Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5801worker2 # [ 135.318260] nix-grpc-daemon[1312]: time=2026-09-22T16:55:55.961Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5802lb # [ 135.381898] niks3-server[1154]: 2026/09/22 16:55:56 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5803lb # [ 135.392195] niks3-server[1154]: 2026/09/22 16:55:56 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5804worker2 # [ 135.372849] nix-grpc-daemon[1312]: time=2026-09-22T16:55:56.019Z level=INFO msg="Uploading 1 narinfos"5805lb # [ 135.404852] niks3-server[1154]: 2026/09/22 16:55:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5806lb # [ 135.413284] niks3-server[1154]: 2026/09/22 16:55:56 INFO Signed narinfos id=44 count=15807lb # [ 135.415685] niks3-server[1154]: 2026/09/22 16:55:56 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5808lb # [ 135.428489] niks3-server[1154]: 2026/09/22 16:55:56 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5809worker2 # [ 135.400421] nix-grpc-daemon[1312]: time=2026-09-22T16:55:56.047Z level=INFO msg="Upload complete. (289ms)"5810lb # [ 135.432439] niks3-server[1154]: 2026/09/22 16:55:56 INFO Completed upload id=445811lb # [ 135.435632] niks3-server[1154]: 2026/09/22 16:55:56 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5812worker2 # [ 135.422273] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15813worker2 # [ 135.439269] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:56Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05814worker2 # [ 135.443510] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:56Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15815worker2 # [ 135.447786] nix-daemon[843]: reaped child process 1362, status = succeeded5816worker2 # [ 135.465370] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374965817worker2 # [ 135.480657] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374965818worker2 # [ 135.495709] nix-grpc-daemon[1312]: time=2026-09-22T16:55:56.142Z level=INFO msg="All 1 paths already cached"5819client # copying 1 paths to 'lb:50051'...5820client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5821worker2 # [ 135.506644] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374965822worker2 # [ 135.520918] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5823worker2 # [ 135.671047] nix-daemon[843]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)5824worker2 # [ 135.689690] nix-daemon[843]: reaped child process 1392, status = succeeded5825lb # [ 135.720991] niks3-server[1154]: 2026/09/22 16:55:56 INFO Received uploads request method=POST path=/api/pending_closures5826worker2 # [ 135.703480] nix-grpc-daemon[1312]: time=2026-09-22T16:55:56.350Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5827worker2 # [ 135.705276] nix-grpc-daemon[1312]: time=2026-09-22T16:55:56.350Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5828lb # [ 135.751140] niks3-server[1154]: 2026/09/22 16:55:56 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5829worker2 # [ 135.735720] nix-grpc-daemon[1312]: time=2026-09-22T16:55:56.382Z level=INFO msg="Uploading 1 narinfos"5830lb # [ 135.766556] niks3-server[1154]: 2026/09/22 16:55:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5831lb # [ 135.776463] niks3-server[1154]: 2026/09/22 16:55:56 INFO Signed narinfos id=45 count=15832lb # [ 135.781571] niks3-server[1154]: 2026/09/22 16:55:56 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5833lb # [ 135.790666] niks3-server[1154]: 2026/09/22 16:55:56 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5834worker2 # [ 135.769337] nix-grpc-daemon[1312]: time=2026-09-22T16:55:56.416Z level=INFO msg="Upload complete. (232ms)"5835lb # [ 135.799665] niks3-server[1154]: 2026/09/22 16:55:56 INFO Completed upload id=455836lb # [ 135.802138] niks3-server[1154]: 2026/09/22 16:55:56 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5837worker2 # [ 135.779818] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=0 paths=1 nar_bytes_in=6405838worker2 # [ 135.796840] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374965839worker2 # [ 135.804992] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5840worker2 # [ 135.810573] nix-daemon[843]: reaped child process 1387, status = succeeded5841client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5842worker2 # [ 135.823801] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5843client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5844client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5845client # farm-b> LOG-farm-b-on-standby5846worker2 # [ 136.935596] nix-daemon[843]: reaped child process 1396, status = succeeded5847worker2 # [ 137.009800] nix-daemon[843]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5848worker2 # [ 137.027451] nix-daemon[843]: reaped child process 1415, status = succeeded5849lb # [ 137.061891] niks3-server[1154]: 2026/09/22 16:55:57 INFO Received uploads request method=POST path=/api/pending_closures5850worker2 # [ 137.045501] nix-grpc-daemon[1312]: time=2026-09-22T16:55:57.691Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5851worker2 # [ 137.047754] nix-grpc-daemon[1312]: time=2026-09-22T16:55:57.691Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5852lb # [ 137.101005] niks3-server[1154]: 2026/09/22 16:55:57 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5853worker2 # [ 137.086191] nix-grpc-daemon[1312]: time=2026-09-22T16:55:57.732Z level=INFO msg="Uploading 1 narinfos"5854lb # [ 137.116534] niks3-server[1154]: 2026/09/22 16:55:57 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5855lb # [ 137.125633] niks3-server[1154]: 2026/09/22 16:55:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5856lb # [ 137.144454] niks3-server[1154]: 2026/09/22 16:55:57 INFO Signed narinfos id=46 count=15857lb # [ 137.153310] niks3-server[1154]: 2026/09/22 16:55:57 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5858worker2 # [ 137.125885] nix-grpc-daemon[1312]: time=2026-09-22T16:55:57.773Z level=INFO msg="Upload complete. (287ms)"5859lb # [ 137.161607] niks3-server[1154]: 2026/09/22 16:55:57 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5860lb # [ 137.170289] niks3-server[1154]: 2026/09/22 16:55:57 INFO Completed upload id=465861worker2 # [ 137.142150] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15862worker2 # [ 137.164320] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:57Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05863worker2 # [ 137.169947] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:57Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15864worker2 # [ 137.175215] nix-daemon[843]: reaped child process 1394, status = succeeded5865worker2 # [ 137.205003] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374965866worker2 # [ 137.220333] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374965867worker2 # [ 137.235897] nix-grpc-daemon[1312]: time=2026-09-22T16:55:57.883Z level=INFO msg="All 2 paths already cached"5868client # copying 1 paths to 'lb:50051'...5869client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5870worker2 # [ 137.247074] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374965871worker2 # [ 137.261170] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5872worker2 # [ 137.412969] nix-daemon[843]: accepted connection from pid 1421, user nix-grpc-daemon (trusted)5873worker2 # [ 137.432322] nix-daemon[843]: reaped child process 1424, status = succeeded5874lb # [ 137.465726] niks3-server[1154]: 2026/09/22 16:55:58 INFO Received uploads request method=POST path=/api/pending_closures5875worker2 # [ 137.450208] nix-grpc-daemon[1312]: time=2026-09-22T16:55:58.097Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5876worker2 # [ 137.451972] nix-grpc-daemon[1312]: time=2026-09-22T16:55:58.097Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5877lb # [ 137.500481] niks3-server[1154]: 2026/09/22 16:55:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5878worker2 # [ 137.481393] nix-grpc-daemon[1312]: time=2026-09-22T16:55:58.128Z level=INFO msg="Uploading 1 narinfos"5879lb # [ 137.514579] niks3-server[1154]: 2026/09/22 16:55:58 INFO Signed narinfos id=47 count=15880lb # [ 137.521444] niks3-server[1154]: 2026/09/22 16:55:58 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5881lb # [ 137.534817] niks3-server[1154]: 2026/09/22 16:55:58 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5882lb # [ 137.539878] niks3-server[1154]: 2026/09/22 16:55:58 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5883worker2 # [ 137.512209] nix-grpc-daemon[1312]: time=2026-09-22T16:55:58.159Z level=INFO msg="Upload complete. (234ms)"5884lb # [ 137.543231] niks3-server[1154]: 2026/09/22 16:55:58 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5885lb # [ 137.547683] niks3-server[1154]: 2026/09/22 16:55:58 INFO Completed upload id=475886worker2 # [ 137.524049] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=0 paths=1 nar_bytes_in=7685887worker2 # [ 137.532977] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374965888worker2 # [ 137.544757] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5889worker2 # [ 137.549799] nix-daemon[843]: reaped child process 1419, status = succeeded5890client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5891client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5892worker2 # [ 137.564122] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5893client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5894client # farm-top> LOG-farm-top-on-standby5895worker2 # [ 138.395132] nix-daemon[843]: reaped child process 1428, status = succeeded5896worker2 # [ 138.497518] nix-daemon[843]: accepted connection from pid 1444, user nix-grpc-daemon (trusted)5897worker2 # [ 138.514621] nix-daemon[843]: reaped child process 1447, status = succeeded5898lb # [ 138.548768] niks3-server[1154]: 2026/09/22 16:55:59 INFO Received uploads request method=POST path=/api/pending_closures5899worker2 # [ 138.534560] nix-grpc-daemon[1312]: time=2026-09-22T16:55:59.180Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5900worker2 # [ 138.536101] nix-grpc-daemon[1312]: time=2026-09-22T16:55:59.180Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5901lb # [ 138.592734] niks3-server[1154]: 2026/09/22 16:55:59 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5902worker2 # [ 138.574274] nix-grpc-daemon[1312]: time=2026-09-22T16:55:59.221Z level=INFO msg="Uploading 1 narinfos"5903lb # [ 138.604386] niks3-server[1154]: 2026/09/22 16:55:59 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5904lb # [ 138.613952] niks3-server[1154]: 2026/09/22 16:55:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5905lb # [ 138.624778] niks3-server[1154]: 2026/09/22 16:55:59 INFO Signed narinfos id=48 count=15906worker2 # [ 138.596247] nix-grpc-daemon[1312]: time=2026-09-22T16:55:59.243Z level=INFO msg="Upload complete. (222ms)"5907lb # [ 138.627119] niks3-server[1154]: 2026/09/22 16:55:59 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5908lb # [ 138.635474] niks3-server[1154]: 2026/09/22 16:55:59 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5909worker2 # [ 138.608842] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15910lb # [ 138.643338] niks3-server[1154]: 2026/09/22 16:55:59 INFO Completed upload id=485911lb # [ 138.646935] niks3-server[1154]: 2026/09/22 16:55:59 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5912worker2 # [ 138.623765] nix-daemon[843]: reaped child process 1426, status = succeeded5913client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.63 seconds)5914client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5915client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5916worker1 # [ 138.964895] systemd[1]: Listening on Nix worker-protocol over gRPC.5917worker1 # [ 138.971494] systemd[1]: Starting Nix worker-protocol over gRPC...5918worker1 # [ 139.121358] nix-grpc-daemon[1413]: ts=2026-09-22T16:55:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5919worker1 # [ 139.123646] nix-grpc-daemon[1413]: ts=2026-09-22T16:55:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5920worker1 # [ 139.127437] nix-grpc-daemon[1413]: ts=2026-09-22T16:55:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15921worker1 # [ 139.132048] nix-grpc-daemon[1413]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5922worker1 # [ 139.133478] nix-grpc-daemon[1413]: E0000 00:00:1790096159.767371 1413 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5923lb: must succeed: curl -sf localhost:9901/clusters5924worker1 # [ 139.146170] nix-grpc-daemon[1413]: E0000 00:00:1790096159.773374 1413 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5925worker1 # [ 139.162356] nix-grpc-daemon[1413]: ts=2026-09-22T16:55:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5926lb # [ 139.147631] envoy[895]: conn peer=[2001:db8:1::3]:41382 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1605 tx=620 ms=104935927lb # [ 139.155879] envoy[895]: conn peer=[2001:db8:1::1]:33162 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9589 tx=4400 ms=53845928worker1 # [ 139.183457] systemd[1]: Started Nix worker-protocol over gRPC.5929worker1 # [ 139.184916] nix-grpc-daemon[1413]: E0000 00:00:1790096159.782723 1453 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5930worker1 # [ 139.187403] nix-grpc-daemon[1413]: E0000 00:00:1790096159.802935 1453 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5931worker1 # [ 139.192782] nix-grpc-daemon[1413]: E0000 00:00:1790096159.803089 1453 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5932worker1 # [ 139.197129] nix-grpc-daemon[1413]: E0000 00:00:1790096159.817794 1453 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5933worker1 # [ 139.199430] nix-grpc-daemon[1413]: ts=2026-09-22T16:55:59Z level=info event=scheduler_connected addr=lb:500515934worker2 # [ 139.145835] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:59Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:470285935worker2 # [ 139.151685] nix-grpc-daemon[1252]: ts=2026-09-22T16:55:59Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05936lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5937worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5938worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5939lb: must succeed: curl -sf localhost:9901/clusters5940lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5941lb: must succeed: curl -sf localhost:9901/clusters5942lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5943worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5944worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5945(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.78 seconds)5946subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5947client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5948client # [ 139.459054] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5949client # Running as unit: rs.service; invocation ID: bde0feca499e45dd9e20a99a185452ba5950client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5951client # [ 139.668156] nix[1410]: don't know how to build these paths:5952client # [ 139.669225] nix[1410]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5953worker2 # [ 139.679194] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:374965954worker2 # [ 139.693090] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:00Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05955worker2 # [ 139.700455] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:00Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15956worker2 # [ 139.740902] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374965957worker2 # [ 139.756728] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:374965958client # [ 139.759395] nix[1410]: copying 1 paths to 'lb:50051'...5959worker2 # [ 139.771312] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:374965960client # [ 139.762024] nix[1410]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5961worker2 # [ 139.786784] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5962worker2 # [ 139.938975] nix-daemon[843]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)5963worker2 # [ 139.961464] nix-daemon[843]: reaped child process 1463, status = succeeded5964lb # [ 139.987573] niks3-server[1154]: 2026/09/22 16:56:00 INFO Received uploads request method=POST path=/api/pending_closures5965worker2 # [ 139.969943] nix-grpc-daemon[1312]: time=2026-09-22T16:56:00.616Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5966worker2 # [ 139.971509] nix-grpc-daemon[1312]: time=2026-09-22T16:56:00.616Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5967lb # [ 140.013873] niks3-server[1154]: 2026/09/22 16:56:00 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5968lb # [ 140.028999] niks3-server[1154]: 2026/09/22 16:56:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5969worker2 # [ 140.007122] nix-grpc-daemon[1312]: time=2026-09-22T16:56:00.654Z level=INFO msg="Uploading 1 narinfos"5970lb # [ 140.040991] niks3-server[1154]: 2026/09/22 16:56:00 INFO Signed narinfos id=49 count=15971lb # [ 140.046845] niks3-server[1154]: 2026/09/22 16:56:00 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5972worker2 # [ 140.033020] nix-grpc-daemon[1312]: time=2026-09-22T16:56:00.680Z level=INFO msg="Upload complete. (229ms)"5973lb # [ 140.061067] niks3-server[1154]: 2026/09/22 16:56:00 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5974lb # [ 140.068537] niks3-server[1154]: 2026/09/22 16:56:00 INFO Completed upload id=495975worker2 # [ 140.044263] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=0 paths=1 nar_bytes_in=4165976lb # [ 140.073697] niks3-server[1154]: 2026/09/22 16:56:00 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5977worker2 # [ 140.053022] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:374965978worker2 # [ 140.063865] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5979worker2 # [ 140.069919] nix-daemon[843]: reaped child process 1458, status = succeeded5980worker2 # [ 140.079779] nix-daemon[843]: accepted connection from pid 1252, user nix-grpc-daemon (trusted)5981client # [ 140.066955] nix[1410]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5982lb: must succeed: curl -sf localhost:9901/clusters5983lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5984lb: must succeed: curl -sf localhost:9901/clusters5985lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5986lb: must succeed: curl -sf localhost:9901/clusters5987lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5988lb: must succeed: curl -sf localhost:9901/clusters5989lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5990worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5991worker1 # [ 142.319556] systemd-journald[271]: Received client request to rotate journal, rotating.5992worker1 # [ 142.342297] systemd-journald[271]: Vacuuming done, freed 0B of archived journals from /var/log/journal/9afbec48643448bf9b5cdf96fc447472.5993worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5994worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5995worker2 # [ 142.337771] systemd[1]: Stopping Nix worker-protocol over gRPC...5996worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5997client: must succeed: sleep 5.05998worker2 # [ 143.236842] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:03Z level=info event=draining5999client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6000worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6001worker2 # [ 147.718273] nix-daemon[843]: reaped child process 1467, status = succeeded6002worker2 # [ 147.816774] nix-daemon[843]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)6003worker2 # [ 147.837199] nix-daemon[843]: reaped child process 1511, status = succeeded6004lb # [ 147.869804] niks3-server[1154]: 2026/09/22 16:56:08 INFO Received uploads request method=POST path=/api/pending_closures6005worker2 # [ 147.852575] nix-grpc-daemon[1312]: time=2026-09-22T16:56:08.499Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6006worker2 # [ 147.855874] nix-grpc-daemon[1312]: time=2026-09-22T16:56:08.499Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"6007lb # [ 147.915003] niks3-server[1154]: 2026/09/22 16:56:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6008worker2 # [ 147.897368] nix-grpc-daemon[1312]: time=2026-09-22T16:56:08.544Z level=INFO msg="Uploading 1 narinfos"6009lb # [ 147.930381] niks3-server[1154]: 2026/09/22 16:56:08 INFO Signed narinfos id=50 count=16010lb # [ 147.944484] niks3-server[1154]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6011lb # [ 147.961393] niks3-server[1154]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls6012lb # [ 147.968479] niks3-server[1154]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv6013worker2 # [ 147.940398] nix-grpc-daemon[1312]: time=2026-09-22T16:56:08.587Z level=INFO msg="Upload complete. (244ms)"6014lb # [ 147.971612] niks3-server[1154]: 2026/09/22 16:56:08 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6015lb # [ 147.976668] niks3-server[1154]: 2026/09/22 16:56:08 INFO Completed upload id=506016lb # [ 147.978674] niks3-server[1154]: 2026/09/22 16:56:08 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo6017worker2 # [ 147.953098] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37496 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=16018worker2 # [ 147.972697] nix-daemon[843]: reaped child process 1465, status = succeeded6019client # [ 147.988206] systemd[1]: rs.service: Deactivated successfully.6020client # [ 147.993253] systemd[1]: rs.service: Consumed 198ms CPU time over 8.532s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6021worker2 # [ 148.237594] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:08Z level=info event=signal_exit6022lb # [ 148.456744] niks3-server[1154]: 2026/09/22 16:56:09 INFO lead: released remote=192.168.1.4:602346023lb # [ 149.121145] niks3-server[1154]: 2026/09/22 16:56:09 INFO lead: acquired remote=192.168.1.3:544906024worker1 # [ 149.146054] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:09Z level=info event=scheduler_take_over6025lb # [ 149.146688] envoy[895]: conn peer=[2001:db8:1::1]:33174 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1542 ms=83026026worker1 # [ 149.482531] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:10Z level=info event=scheduler_restarting addr=lb:500516027worker2 # [ 149.431148] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:10Z level=info event=scheduler_restarting6028worker2 # [ 149.440533] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:10Z level=info event=scheduler_restarting addr=lb:500516029worker2 # [ 149.445024] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:470286030worker2 # [ 149.449789] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:10Z level=info event=worker_gone addr=192.168.1.4:500516031worker2 # [ 149.454048] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:10Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:470286032worker2 # [ 149.458948] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:10Z level=info event=worker_gone addr=192.168.1.3:500516033worker2 # [ 149.484316] nix-daemon[843]: reaped child process 1304, status = succeeded6034worker1 # [ 149.683332] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:10Z level=info event=scheduler_connected addr=lb:500516035worker1 # [ 149.689082] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:10Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6036worker2 # [ 149.651437] nix-grpc-daemon[1252]: ts=2026-09-22T16:56:10Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6037worker1 # [ 149.889713] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:10Z level=info event=scheduler_connected addr=lb:500516038worker1 # [ 149.901495] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:10Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:462946039worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.21 seconds)6040client: waiting for success: ! systemctl is-active rs6041worker1 # [ 149.907142] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:10Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06042client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)6043client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6044client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)6045client: must fail: journalctl -u rs | grep -q 'reconnecting'6046client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)6047lb: must succeed: curl -sf localhost:9901/clusters6048lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6049worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6050worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6051worker2 # [ 150.355468] nix-grpc-daemon[1252]: E0000 00:00:1790096171.002558 1297 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6052worker2 # [ 150.388734] nix-grpc-daemon[1252]: E0000 00:00:1790096171.035867 1297 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6053lb: must succeed: curl -sf localhost:9901/clusters6054lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6055worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6056worker2 # [ 151.266456] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6057worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6058worker2 # [ 151.272160] systemd[1]: Stopped Nix worker-protocol over gRPC.6059worker2 # [ 151.274725] systemd[1]: nix-grpc-daemon.service: Consumed 1.760s CPU time over 37.198s wall clock time, 39.7M memory peak, 61.7K incoming IP traffic, 144.5K outgoing IP traffic.6060worker2 # [ 151.295974] systemd[1]: Starting scheduler-token.service...6061worker2 # [ 151.370441] systemd[1]: scheduler-token.service: Deactivated successfully.6062worker2 # [ 151.374960] systemd[1]: Finished scheduler-token.service.6063worker2 # [ 151.380834] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 77ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6064worker2 # [ 151.387400] systemd[1]: Starting Nix worker-protocol over gRPC...6065worker2 # [ 151.518971] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:12Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6066worker2 # [ 151.524103] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:12Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6067worker2 # [ 151.526033] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:12Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16068worker2 # [ 151.552158] nix-grpc-daemon[1521]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6069worker2 # [ 151.553564] nix-grpc-daemon[1521]: E0000 00:00:1790096172.176995 1521 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6070worker2 # [ 151.555803] nix-grpc-daemon[1521]: E0000 00:00:1790096172.179017 1521 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6071worker2 # [ 151.575104] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:12Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6072worker2 # [ 151.577323] nix-grpc-daemon[1521]: E0000 00:00:1790096172.185391 1560 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6073worker2 # [ 151.579819] systemd[1]: Started Nix worker-protocol over gRPC.6074worker2 # [ 151.580712] nix-grpc-daemon[1521]: E0000 00:00:1790096172.207027 1560 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6075worker2 # [ 151.586000] nix-grpc-daemon[1521]: E0000 00:00:1790096172.207176 1560 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6076worker2 # [ 151.590347] nix-grpc-daemon[1521]: E0000 00:00:1790096172.221584 1560 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6077worker2 # [ 151.592995] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:12Z level=info event=scheduler_connected addr=lb:500516078worker1 # [ 151.651987] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:12Z level=info event=oidc_jwks provider=mock keys=16079worker1 # [ 151.654101] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:12Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:462946080worker1 # [ 151.654234] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:12Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06081lb: must succeed: curl -sf localhost:9901/clusters6082lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6083worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6084worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6085(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.99 seconds)6086subtest: scheduler restart mid-build: build finishes, no double build6087client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid6088client # [ 152.453569] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.6089client # Running as unit: mid.service; invocation ID: ca709b7867f348d4b51242b4dce26c5a6090client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)6091worker2 # [ 152.675423] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:439106092worker2 # [ 152.689584] nix-daemon[843]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)6093client # [ 152.695128] nix[1446]: don't know how to build these paths:6094client # [ 152.697034] nix[1446]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6095worker1 # [ 152.778264] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:13Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06096worker1 # [ 152.778567] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:13Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16097worker1 # [ 152.836043] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500186098worker1 # [ 152.851005] nix-daemon[851]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6099worker1 # [ 152.875332] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)6100worker1 # [ 152.909830] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 2/5)6101worker1 # [ 152.984722] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 138 ms (attempt 3/5)6102worker1 # [ 153.132100] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 142 ms (attempt 4/5)6103worker1 # [ 153.276192] nix-grpc-daemon[1413]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6104worker1 # [ 153.282454] nix-grpc-daemon[1413]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6105worker1 # [ 153.301469] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500186106client # [ 153.250498] nix[1446]: copying 1 paths to 'lb:50051'...6107client # [ 153.255907] nix[1446]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...6108worker1 # [ 153.321103] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500186109worker1 # [ 153.335101] nix-daemon[851]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6110worker1 # [ 153.372237] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:13Z level=info event=niks3_push_started pid=15386111worker1 # [ 153.519222] nix-grpc-daemon[1538]: time=2026-09-22T16:56:14.154Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6112worker1 # [ 153.626174] nix-daemon[851]: accepted connection from pid 1551, user nix-grpc-daemon (trusted)6113worker1 # [ 153.641639] nix-daemon[851]: reaped child process 1554, status = succeeded6114lb # [ 153.625984] niks3-server[1154]: 2026/09/22 16:56:14 INFO Received uploads request method=POST path=/api/pending_closures6115worker1 # [ 153.660240] nix-grpc-daemon[1538]: time=2026-09-22T16:56:14.295Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6116worker1 # [ 153.666316] nix-grpc-daemon[1538]: time=2026-09-22T16:56:14.298Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"6117lb # [ 153.665769] niks3-server[1154]: 2026/09/22 16:56:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6118worker1 # [ 153.703042] nix-grpc-daemon[1538]: time=2026-09-22T16:56:14.337Z level=INFO msg="Uploading 1 narinfos"6119lb # [ 153.681464] niks3-server[1154]: 2026/09/22 16:56:14 INFO Signed narinfos id=51 count=16120lb # [ 153.691510] niks3-server[1154]: 2026/09/22 16:56:14 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst6121lb # [ 153.709527] niks3-server[1154]: 2026/09/22 16:56:14 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6122worker1 # [ 153.741416] nix-grpc-daemon[1538]: time=2026-09-22T16:56:14.376Z level=INFO msg="Upload complete. (218ms)"6123worker1 # [ 153.748102] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50018 duration_s=0 paths=1 nar_bytes_in=4246124lb # [ 153.725662] niks3-server[1154]: 2026/09/22 16:56:14 INFO Completed upload id=516125lb # [ 153.729073] niks3-server[1154]: 2026/09/22 16:56:14 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo6126worker1 # [ 153.757870] nix-grpc-daemon[1413]: ts=2026-09-22T16:56:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500186127worker1 # [ 153.762235] nix-daemon[851]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6128lb # [ 153.736972] niks3-server[1154]: 2026/09/22 16:56:14 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls6129worker1 # [ 153.766063] nix-daemon[851]: reaped child process 1536, status = succeeded6130worker1 # [ 153.774105] nix-daemon[851]: accepted connection from pid 1413, user nix-grpc-daemon (trusted)6131client # [ 153.710637] nix[1446]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6132lb: must succeed: curl -sf localhost:9901/clusters6133lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6134lb: must succeed: curl -sf localhost:9901/clusters6135lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6136worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6137worker1 # [ 155.207104] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1413 (nix-grpc-daemon) on client request.6138worker1 # [ 155.209352] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6139lb # [ 155.186206] niks3-server[1154]: 2026/09/22 16:56:15 INFO lead: released remote=192.168.1.3:544906140client # [ 155.153993] nix[1446]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6141worker2 # [ 155.166198] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:15Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6142worker1 # [ 155.227200] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6143worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.06 seconds)6144client: must succeed: sleep 5.06145worker1 # [ 155.232511] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6146worker1 # [ 155.233744] systemd[1]: nix-grpc-daemon.service: Consumed 519ms CPU time over 16.255s wall clock time, 34.5M memory peak, 15.5K incoming IP traffic, 26.6K outgoing IP traffic.6147worker1 # [ 155.258333] nix-daemon[851]: reaped child process 1560, status = succeeded6148worker1 # [ 155.259762] nix-daemon[851]: reaped child process 1530, status = succeeded6149worker1 # [ 155.268054] nix-daemon[1562]: unexpected Nix daemon error: error: interrupted by the user6150worker1 # [ 155.271855] nix-daemon[851]: reaped child process 1562, status = failed with exit code 16151worker2 # [ 155.366651] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:16Z level=info event=scheduler_connected addr=lb:500516152worker1 # [ 155.422795] systemd[1]: nix-grpc-daemon.service: Scheduled restart job, restart counter is at 1.6153worker1 # [ 155.434765] systemd[1]: Starting Nix worker-protocol over gRPC...6154worker1 # [ 155.596469] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6155worker1 # [ 155.599226] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6156worker1 # [ 155.603256] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16157worker1 # [ 155.608052] nix-grpc-daemon[1595]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6158worker1 # [ 155.609535] nix-grpc-daemon[1595]: E0000 00:00:1790096176.243354 1595 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6159lb # [ 155.601225] niks3-server[1154]: 2026/09/22 16:56:16 INFO lead: acquired remote=192.168.1.3:542686160worker1 # [ 155.652055] nix-grpc-daemon[1595]: E0000 00:00:1790096176.249530 1595 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6161worker1 # [ 155.654331] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6162worker1 # [ 155.655941] nix-grpc-daemon[1595]: E0000 00:00:1790096176.257895 1634 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6163worker1 # [ 155.663167] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=scheduler_take_over6164worker1 # [ 155.664784] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=oidc_jwks provider=mock keys=16165worker1 # [ 155.666165] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:500266166worker1 # [ 155.667840] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06167worker1 # [ 155.672390] nix-grpc-daemon[1595]: E0000 00:00:1790096176.276916 1634 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6168worker1 # [ 155.676704] nix-grpc-daemon[1595]: E0000 00:00:1790096176.277033 1634 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6169worker1 # [ 155.679552] systemd[1]: Started Nix worker-protocol over gRPC.6170worker1 # [ 155.680435] nix-grpc-daemon[1595]: E0000 00:00:1790096176.297840 1634 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6171worker1 # [ 155.682665] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=scheduler_connected addr=lb:500516172worker1 # [ 155.683910] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:500266173worker1 # [ 155.685536] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06174worker1 # [ 155.726961] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06175worker1 # [ 155.730463] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:16Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16176worker2 # [ 155.721508] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439106177worker2 # [ 155.738264] nix-grpc-daemon[1521]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 34 ms (attempt 1/5)6178worker2 # [ 155.781683] nix-grpc-daemon[1521]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 2/5)6179client # [ 155.160429] nix[1446]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6180worker2 # [ 155.953025] nix-grpc-daemon[1521]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 194 ms (attempt 3/5)6181worker2 # [ 156.156313] nix-grpc-daemon[1521]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 57 ms (attempt 4/5)6182worker2 # [ 156.217344] nix-grpc-daemon[1521]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6183worker2 # [ 156.222898] nix-grpc-daemon[1521]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6184worker2 # [ 156.239747] nix-grpc-daemon[1521]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' from 'http://lb:5751'...6185worker2 # [ 156.268473] nix-daemon[843]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)6186client # [ 156.274522] nix[1446]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6187worker2 # [ 156.289072] nix-daemon[843]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)6188lb # [ 159.148226] envoy[895]: conn peer=[2001:db8:1::4]:39572 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=27145 tx=1735 ms=353616189lb # [ 159.153391] envoy[895]: conn peer=[2001:db8:1::3]:41878 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=801 tx=376 ms=160306190client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6191client: waiting for success: ! systemctl is-active mid6192worker2 # [ 160.513735] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:21Z level=info event=niks3_push_started pid=16236193worker2 # [ 160.538722] nix-daemon[843]: reaped child process 1598, status = succeeded6194worker2 # [ 160.632592] nix-grpc-daemon[1623]: time=2026-09-22T16:56:21.279Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6195worker2 # [ 160.737921] nix-daemon[843]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6196worker2 # [ 160.753510] nix-daemon[843]: reaped child process 1638, status = succeeded6197lb # [ 160.789731] niks3-server[1154]: 2026/09/22 16:56:21 INFO Received uploads request method=POST path=/api/pending_closures6198worker2 # [ 160.775253] nix-grpc-daemon[1623]: time=2026-09-22T16:56:21.420Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6199worker2 # [ 160.779291] nix-grpc-daemon[1623]: time=2026-09-22T16:56:21.420Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6200lb # [ 160.843615] niks3-server[1154]: 2026/09/22 16:56:21 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6201worker2 # [ 160.826348] nix-grpc-daemon[1623]: time=2026-09-22T16:56:21.472Z level=INFO msg="Uploading 1 narinfos"6202lb # [ 160.859841] niks3-server[1154]: 2026/09/22 16:56:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6203lb # [ 160.870727] niks3-server[1154]: 2026/09/22 16:56:21 INFO Signed narinfos id=52 count=16204lb # [ 160.876347] niks3-server[1154]: 2026/09/22 16:56:21 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6205lb # [ 160.889363] niks3-server[1154]: 2026/09/22 16:56:21 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6206worker2 # [ 160.866567] nix-grpc-daemon[1623]: time=2026-09-22T16:56:21.513Z level=INFO msg="Upload complete. (230ms)"6207lb # [ 160.893278] niks3-server[1154]: 2026/09/22 16:56:21 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6208lb # [ 160.902134] niks3-server[1154]: 2026/09/22 16:56:21 INFO Completed upload id=526209worker2 # [ 160.876295] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43910 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16210lb # [ 160.905427] niks3-server[1154]: 2026/09/22 16:56:21 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6211worker2 # [ 160.892212] nix-daemon[843]: reaped child process 1596, status = succeeded6212client # [ 160.904768] systemd[1]: mid.service: Deactivated successfully.6213client # [ 160.909094] systemd[1]: mid.service: Consumed 210ms CPU time over 8.454s wall clock time, 41.5M memory peak, 8.6K incoming IP traffic, 10.3K outgoing IP traffic.6214client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6215client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6216client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.05 seconds)6217(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.25 seconds)6218subtest: interrupting the client stops the build on the worker6219client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6220client # [ 161.679273] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6221client # Running as unit: intr.service; invocation ID: 59b5bee707734241bd2a0267e1b84d496222client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.06 seconds)6223client # [ 161.856770] nix[1482]: don't know how to build these paths:6224client # [ 161.857753] nix[1482]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6225worker2 # [ 161.867556] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:439106226worker1 # [ 161.935958] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:22Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06227worker1 # [ 161.936377] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:22Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=16228worker1 # [ 161.985396] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626229worker1 # [ 162.001724] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6230worker1 # [ 162.023612] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)6231worker1 # [ 162.035854] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 140 ms (attempt 2/5)6232worker1 # [ 162.185516] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 3/5)6233worker1 # [ 162.246257] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 4/5)6234worker1 # [ 162.269368] nix-grpc-daemon[1595]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6235worker1 # [ 162.275731] nix-grpc-daemon[1595]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6236worker1 # [ 162.293396] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626237client # [ 162.244180] nix[1482]: copying 1 paths to 'lb:50051'...6238worker1 # [ 162.308820] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626239client # [ 162.246471] nix[1482]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6240worker1 # [ 162.324633] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6241worker1 # [ 162.351718] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:22Z level=info event=niks3_push_started pid=16716242worker1 # [ 162.500474] nix-grpc-daemon[1671]: time=2026-09-22T16:56:23.135Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6243worker1 # [ 162.605356] nix-daemon[851]: accepted connection from pid 1683, user nix-grpc-daemon (trusted)6244worker1 # [ 162.625623] nix-daemon[851]: reaped child process 1686, status = succeeded6245lb # [ 162.600813] niks3-server[1154]: 2026/09/22 16:56:23 INFO Received uploads request method=POST path=/api/pending_closures6246worker1 # [ 162.636080] nix-grpc-daemon[1671]: time=2026-09-22T16:56:23.271Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6247worker1 # [ 162.637534] nix-grpc-daemon[1671]: time=2026-09-22T16:56:23.271Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6248lb # [ 162.647514] niks3-server[1154]: 2026/09/22 16:56:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6249worker1 # [ 162.681683] nix-grpc-daemon[1671]: time=2026-09-22T16:56:23.316Z level=INFO msg="Uploading 1 narinfos"6250lb # [ 162.659604] niks3-server[1154]: 2026/09/22 16:56:23 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6251lb # [ 162.670493] niks3-server[1154]: 2026/09/22 16:56:23 INFO Signed narinfos id=53 count=16252worker1 # [ 162.703407] nix-grpc-daemon[1671]: time=2026-09-22T16:56:23.338Z level=INFO msg="Upload complete. (199ms)"6253lb # [ 162.680291] niks3-server[1154]: 2026/09/22 16:56:23 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6254lb # [ 162.685122] niks3-server[1154]: 2026/09/22 16:56:23 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6255lb # [ 162.689537] niks3-server[1154]: 2026/09/22 16:56:23 INFO Completed upload id=536256worker1 # [ 162.712213] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=4246257lb # [ 162.691452] niks3-server[1154]: 2026/09/22 16:56:23 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6258worker1 # [ 162.723730] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626259worker1 # [ 162.729573] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6260worker1 # [ 162.733941] nix-daemon[851]: reaped child process 1669, status = succeeded6261client # [ 162.679178] nix[1482]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6262worker1 # [ 162.746196] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6263client: must succeed: systemctl kill -s INT intr6264worker1 # [ 164.231869] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:24Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6265client # [ 164.166688] systemd[1]: intr.service: Sent signal SIGINT to main process 1482 (nix) on client request.6266client # [ 164.175721] nix[1482]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6267client # [ 164.188696] nix[1482]: error: interrupted by the user6268client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6269client # [ 164.213877] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6270client # [ 164.218628] systemd[1]: intr.service: Failed with result 'exit-code'.6271client # [ 164.221500] systemd[1]: intr.service: Consumed 170ms CPU time over 2.537s wall clock time, 41.5M memory peak, 6.4K incoming IP traffic, 8.4K outgoing IP traffic.6272worker1 # [ 164.368512] nix-daemon[1690]: unexpected Nix daemon error: error: interrupted by the user6273worker1 # [ 164.377335] nix-daemon[851]: reaped child process 1688, status = succeeded6274worker1 # [ 164.381685] nix-daemon[851]: reaped child process 1690, status = failed with exit code 16275(finished: subtest: interrupting the client stops the build on the worker, in 3.83 seconds)6276subtest: a deploy mid-build drains: the build finishes, the new generation takes over6277client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6278client # [ 165.533185] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6279client # Running as unit: sw.service; invocation ID: 9eec3e288831490aa0950ddef08f944c6280client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.09 seconds)6281client # [ 165.745203] nix[1503]: don't know how to build these paths:6282client # [ 165.746295] nix[1503]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6283worker2 # [ 165.757043] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:26Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:439106284worker1 # [ 165.822340] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:26Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06285worker1 # [ 165.827927] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:26Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=16286worker2 # [ 165.816432] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439106287worker2 # [ 165.832342] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:439106288client # [ 165.834867] nix[1503]: copying 1 paths to 'lb:50051'...6289worker2 # [ 165.846977] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:439106290client # [ 165.837670] nix[1503]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6291worker2 # [ 165.862012] nix-daemon[843]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)6292worker2 # [ 166.019797] nix-daemon[843]: accepted connection from pid 1666, user nix-grpc-daemon (trusted)6293worker2 # [ 166.039140] nix-daemon[843]: reaped child process 1669, status = succeeded6294lb # [ 166.072768] niks3-server[1154]: 2026/09/22 16:56:26 INFO Received uploads request method=POST path=/api/pending_closures6295worker2 # [ 166.055182] nix-grpc-daemon[1623]: time=2026-09-22T16:56:26.702Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6296worker2 # [ 166.057747] nix-grpc-daemon[1623]: time=2026-09-22T16:56:26.704Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6297lb # [ 166.104734] niks3-server[1154]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6298worker2 # [ 166.088873] nix-grpc-daemon[1623]: time=2026-09-22T16:56:26.734Z level=INFO msg="Uploading 1 narinfos"6299lb # [ 166.118366] niks3-server[1154]: 2026/09/22 16:56:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6300lb # [ 166.130330] niks3-server[1154]: 2026/09/22 16:56:26 INFO Signed narinfos id=54 count=16301worker2 # [ 166.112549] nix-grpc-daemon[1623]: time=2026-09-22T16:56:26.759Z level=INFO msg="Upload complete. (233ms)"6302lb # [ 166.139926] niks3-server[1154]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6303lb # [ 166.145824] niks3-server[1154]: 2026/09/22 16:56:26 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6304lb # [ 166.151321] niks3-server[1154]: 2026/09/22 16:56:26 INFO Completed upload id=546305worker2 # [ 166.120171] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43910 duration_s=0 paths=1 nar_bytes_in=4166306lb # [ 166.155577] niks3-server[1154]: 2026/09/22 16:56:26 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6307worker2 # [ 166.136683] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:439106308worker2 # [ 166.145882] nix-daemon[843]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)6309worker2 # [ 166.151217] nix-daemon[843]: reaped child process 1664, status = succeeded6310worker2 # [ 166.164157] nix-daemon[843]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)6311client # [ 166.150677] nix[1503]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6312worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26313worker2 # Checking switch inhibitors... done6314worker2 # [ 168.546174] nixos[1705]: switching to system configuration /nix/store/v5mnmfs27v4kkcfdrk24lffriqdnfny8-nixos-system-worker2-test6315worker2 # activating the configuration...6316worker2 # [ 168.556174] systemd[1]: Stopped target Local File Systems.6317worker2 # [ 168.557071] systemd[1]: Stopped target Remote File Systems.6318worker2 # setting up /etc...6319worker2 # [ 169.109204] systemd[1]: Reload requested from client PID 1705 ('.switch-to-conf') (unit backdoor.service)...6320worker2 # [ 169.110837] systemd[1]: Reloading...6321lb # [ 169.149493] envoy[895]: conn peer=[2001:db8:1::1]:55172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3940 tx=1941 ms=82276322lb # [ 169.154575] envoy[895]: conn peer=[2001:db8:1::1]:35698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=802 ms=23416323worker2 # [ 173.956428] systemd[1]: Reloading finished in 4839 ms.6324worker2 # restarting sysinit-reactivation.target6325worker2 # [ 174.088466] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6326worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6327worker2 # [ 174.222397] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6328worker2 # [ 174.223628] systemd[1]: Reached target Reactivate sysinit units.6329worker2 # [ 174.237103] systemd[1]: Reloading D-Bus System Message Bus...6330worker2 # [ 174.238303] dbus-broker-launch[488]: Caught SIGHUP, trigger reload.6331worker2 # [ 174.239237] systemd[1]: Reloading Nix worker-protocol over gRPC...6332worker2 # [ 174.248136] dbus-broker-launch[488]: Looking up NSS user entry for 'systemd-timesync'...6333worker2 # [ 174.249417] dbus-broker-launch[488]: NSS returned no entry for 'systemd-timesync'6334worker2 # [ 174.252762] dbus-broker-launch[488]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6335worker2 # [ 174.285302] systemd[1]: Reloaded Nix worker-protocol over gRPC.6336worker2 # [ 174.294231] systemd[1]: Reloaded D-Bus System Message Bus.6337worker2 # [ 174.298057] systemd[1]: Reached target Remote File Systems.6338worker2 # [ 174.309291] systemd[1]: Starting linger-users.service...6339worker2 # [ 174.316784] systemd[1]: Reached target Local File Systems.6340worker2 # [ 174.317666] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6341worker2 # [ 174.334543] systemd[1]: Starting Create SUID/SGID Wrappers...6342worker2 # [ 174.335470] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6343worker2 # [ 174.337046] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676344worker2 # [ 174.339198] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6345worker2 # [ 174.353122] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6346worker2 # [ 174.358531] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6347worker2 # [ 174.366547] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6348worker2 # [ 174.370513] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6349worker2 # [ 174.387862] systemd[1]: linger-users.service: Deactivated successfully.6350worker2 # [ 174.389075] systemd[1]: Finished linger-users.service.6351worker2 # [ 174.542666] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:35Z level=info event=draining6352worker2 # [ 174.706921] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6353worker2 # [ 174.712184] systemd[1]: Finished Create SUID/SGID Wrappers.6354worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6355worker2 # [ 174.963374] nixos[1705]: finished switching to system configuration /nix/store/v5mnmfs27v4kkcfdrk24lffriqdnfny8-nixos-system-worker2-test6356worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.87 seconds)6357worker2: must succeed: pgrep -f 'read -t [9]0 x'6358worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6359client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26360worker2 # [ 175.249743] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:439106361client # don't know how to build these paths:6362client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6363worker1 # [ 175.313446] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626364worker1 # [ 175.341525] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:35Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06365worker1 # [ 175.349005] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:35Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16366worker1 # [ 175.390230] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626367worker1 # [ 175.422566] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626368client # copying 1 paths to 'lb:50051'...6369client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6370worker1 # [ 175.439619] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626371worker1 # [ 175.457958] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6372worker1 # [ 175.629029] nix-daemon[851]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6373worker1 # [ 175.649867] nix-daemon[851]: reaped child process 1748, status = succeeded6374lb # [ 175.632835] niks3-server[1154]: 2026/09/22 16:56:36 INFO Received uploads request method=POST path=/api/pending_closures6375worker1 # [ 175.668367] nix-grpc-daemon[1671]: time=2026-09-22T16:56:36.302Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6376worker1 # [ 175.670011] nix-grpc-daemon[1671]: time=2026-09-22T16:56:36.302Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6377lb # [ 175.668454] niks3-server[1154]: 2026/09/22 16:56:36 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6378worker1 # [ 175.706581] nix-grpc-daemon[1671]: time=2026-09-22T16:56:36.341Z level=INFO msg="Uploading 1 narinfos"6379lb # [ 175.684461] niks3-server[1154]: 2026/09/22 16:56:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6380lb # [ 175.693194] niks3-server[1154]: 2026/09/22 16:56:36 INFO Signed narinfos id=55 count=16381lb # [ 175.698885] niks3-server[1154]: 2026/09/22 16:56:36 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6382worker1 # [ 175.732950] nix-grpc-daemon[1671]: time=2026-09-22T16:56:36.368Z level=INFO msg="Upload complete. (257ms)"6383lb # [ 175.709477] niks3-server[1154]: 2026/09/22 16:56:36 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6384lb # [ 175.714352] niks3-server[1154]: 2026/09/22 16:56:36 INFO Completed upload id=556385lb # [ 175.717580] niks3-server[1154]: 2026/09/22 16:56:36 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6386worker1 # [ 175.742430] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=5286387worker1 # [ 175.753229] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626388worker1 # [ 175.765576] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6389worker1 # [ 175.770878] nix-daemon[851]: reaped child process 1743, status = succeeded6390client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6391worker1 # [ 175.782111] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6392client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6393client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6394client # farm-a> LOG-farm-a-during-drain6395worker1 # [ 176.889419] nix-daemon[851]: reaped child process 1752, status = succeeded6396worker1 # [ 176.958681] nix-daemon[851]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)6397worker1 # [ 176.982108] nix-daemon[851]: reaped child process 1771, status = succeeded6398lb # [ 176.961384] niks3-server[1154]: 2026/09/22 16:56:37 INFO Received uploads request method=POST path=/api/pending_closures6399worker1 # [ 176.996406] nix-grpc-daemon[1671]: time=2026-09-22T16:56:37.630Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6400worker1 # [ 176.997998] nix-grpc-daemon[1671]: time=2026-09-22T16:56:37.630Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6401lb # [ 177.019171] niks3-server[1154]: 2026/09/22 16:56:37 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6402lb # [ 177.033592] niks3-server[1154]: 2026/09/22 16:56:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6403lb # [ 177.047852] niks3-server[1154]: 2026/09/22 16:56:37 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6404lb # [ 177.056951] niks3-server[1154]: 2026/09/22 16:56:37 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6405lb # [ 177.070367] niks3-server[1154]: 2026/09/22 16:56:37 INFO Signed narinfos id=56 count=16406worker1 # [ 177.095707] nix-grpc-daemon[1671]: time=2026-09-22T16:56:37.730Z level=INFO msg="Uploading 1 narinfos"6407lb # [ 177.081102] niks3-server[1154]: 2026/09/22 16:56:37 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6408lb # [ 177.093673] niks3-server[1154]: 2026/09/22 16:56:37 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6409lb # [ 177.100167] niks3-server[1154]: 2026/09/22 16:56:37 INFO Completed upload id=566410worker1 # [ 177.125843] nix-grpc-daemon[1671]: time=2026-09-22T16:56:37.761Z level=INFO msg="Upload complete. (330ms)"6411worker1 # [ 177.144574] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16412worker1 # [ 177.164470] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:37Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06413worker1 # [ 177.169321] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:37Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16414worker1 # [ 177.174046] nix-daemon[851]: reaped child process 1750, status = succeeded6415worker1 # [ 177.189513] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626416worker1 # [ 177.205160] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626417worker1 # [ 177.221038] nix-grpc-daemon[1671]: time=2026-09-22T16:56:37.856Z level=INFO msg="All 1 paths already cached"6418client # copying 1 paths to 'lb:50051'...6419worker1 # [ 177.232108] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626420client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6421worker1 # [ 177.254888] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6422worker1 # [ 177.426373] nix-daemon[851]: accepted connection from pid 1777, user nix-grpc-daemon (trusted)6423worker1 # [ 177.450445] nix-daemon[851]: reaped child process 1780, status = succeeded6424lb # [ 177.423611] niks3-server[1154]: 2026/09/22 16:56:38 INFO Received uploads request method=POST path=/api/pending_closures6425worker1 # [ 177.460184] nix-grpc-daemon[1671]: time=2026-09-22T16:56:38.095Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6426worker1 # [ 177.462876] nix-grpc-daemon[1671]: time=2026-09-22T16:56:38.097Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6427lb # [ 177.466239] niks3-server[1154]: 2026/09/22 16:56:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6428worker1 # [ 177.497042] nix-grpc-daemon[1671]: time=2026-09-22T16:56:38.131Z level=INFO msg="Uploading 1 narinfos"6429lb # [ 177.480813] niks3-server[1154]: 2026/09/22 16:56:38 INFO Signed narinfos id=57 count=16430lb # [ 177.483528] niks3-server[1154]: 2026/09/22 16:56:38 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6431lb # [ 177.494786] niks3-server[1154]: 2026/09/22 16:56:38 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6432worker1 # [ 177.527672] nix-grpc-daemon[1671]: time=2026-09-22T16:56:38.162Z level=INFO msg="Upload complete. (254ms)"6433lb # [ 177.504970] niks3-server[1154]: 2026/09/22 16:56:38 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6434lb # [ 177.508890] niks3-server[1154]: 2026/09/22 16:56:38 INFO Completed upload id=576435lb # [ 177.511796] niks3-server[1154]: 2026/09/22 16:56:38 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6436worker1 # [ 177.540313] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=6406437worker1 # [ 177.555724] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626438worker1 # [ 177.562373] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6439worker1 # [ 177.567472] nix-daemon[851]: reaped child process 1775, status = succeeded6440client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6441worker1 # [ 177.579395] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6442client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6443client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6444client # farm-b> LOG-farm-b-during-drain6445worker1 # [ 178.418260] nix-daemon[851]: reaped child process 1784, status = succeeded6446worker1 # [ 178.523480] nix-daemon[851]: accepted connection from pid 1800, user nix-grpc-daemon (trusted)6447worker1 # [ 178.543301] nix-daemon[851]: reaped child process 1803, status = succeeded6448lb # [ 178.523955] niks3-server[1154]: 2026/09/22 16:56:39 INFO Received uploads request method=POST path=/api/pending_closures6449worker1 # [ 178.559532] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.193Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6450worker1 # [ 178.563488] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.194Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6451lb # [ 178.568215] niks3-server[1154]: 2026/09/22 16:56:39 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6452lb # [ 178.582183] niks3-server[1154]: 2026/09/22 16:56:39 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6453worker1 # [ 178.622156] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.257Z level=INFO msg="Uploading 1 narinfos"6454lb # [ 178.600281] niks3-server[1154]: 2026/09/22 16:56:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6455lb # [ 178.607946] niks3-server[1154]: 2026/09/22 16:56:39 INFO Signed narinfos id=58 count=16456lb # [ 178.610341] niks3-server[1154]: 2026/09/22 16:56:39 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6457worker1 # [ 178.646130] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.281Z level=INFO msg="Upload complete. (254ms)"6458lb # [ 178.622208] niks3-server[1154]: 2026/09/22 16:56:39 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6459lb # [ 178.627027] niks3-server[1154]: 2026/09/22 16:56:39 INFO Completed upload id=586460lb # [ 178.632540] niks3-server[1154]: 2026/09/22 16:56:39 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6461worker1 # [ 178.661187] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16462worker1 # [ 178.682359] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:39Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06463worker1 # [ 178.689328] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:39Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16464worker1 # [ 178.694363] nix-daemon[851]: reaped child process 1782, status = succeeded6465worker1 # [ 178.729728] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626466worker1 # [ 178.746620] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626467worker1 # [ 178.762960] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.398Z level=INFO msg="All 2 paths already cached"6468client # copying 1 paths to 'lb:50051'...6469client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6470worker1 # [ 178.775126] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626471worker1 # [ 178.793504] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6472worker1 # [ 178.955647] nix-daemon[851]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6473worker1 # [ 178.978507] nix-daemon[851]: reaped child process 1812, status = succeeded6474lb # [ 178.957566] niks3-server[1154]: 2026/09/22 16:56:39 INFO Received uploads request method=POST path=/api/pending_closures6475worker1 # [ 178.992042] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.627Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6476worker1 # [ 178.993642] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.627Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6477lb # [ 178.992436] niks3-server[1154]: 2026/09/22 16:56:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6478worker1 # [ 179.023786] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.658Z level=INFO msg="Uploading 1 narinfos"6479lb # [ 179.009270] niks3-server[1154]: 2026/09/22 16:56:39 INFO Signed narinfos id=59 count=16480lb # [ 179.015883] niks3-server[1154]: 2026/09/22 16:56:39 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6481lb # [ 179.029148] niks3-server[1154]: 2026/09/22 16:56:39 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6482worker1 # [ 179.060275] nix-grpc-daemon[1671]: time=2026-09-22T16:56:39.695Z level=INFO msg="Upload complete. (249ms)"6483lb # [ 179.036377] niks3-server[1154]: 2026/09/22 16:56:39 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6484lb # [ 179.042633] niks3-server[1154]: 2026/09/22 16:56:39 INFO Completed upload id=596485worker1 # [ 179.070398] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=7686486lb # [ 179.048537] niks3-server[1154]: 2026/09/22 16:56:39 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6487worker1 # [ 179.088363] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626488worker1 # [ 179.097257] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6489worker1 # [ 179.103346] nix-daemon[851]: reaped child process 1807, status = succeeded6490client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6491client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6492worker1 # [ 179.120353] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6493client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6494client # farm-top> LOG-farm-top-during-drain6495worker1 # [ 179.751776] nix-daemon[851]: reaped child process 1816, status = succeeded6496worker1 # [ 179.859971] nix-daemon[851]: accepted connection from pid 1832, user nix-grpc-daemon (trusted)6497worker1 # [ 179.882255] nix-daemon[851]: reaped child process 1835, status = succeeded6498lb # [ 179.865986] niks3-server[1154]: 2026/09/22 16:56:40 INFO Received uploads request method=POST path=/api/pending_closures6499worker1 # [ 179.904467] nix-grpc-daemon[1671]: time=2026-09-22T16:56:40.537Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6500worker1 # [ 179.906084] nix-grpc-daemon[1671]: time=2026-09-22T16:56:40.537Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6501lb # [ 179.911514] niks3-server[1154]: 2026/09/22 16:56:40 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6502lb # [ 179.925916] niks3-server[1154]: 2026/09/22 16:56:40 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6503worker1 # [ 179.958572] nix-grpc-daemon[1671]: time=2026-09-22T16:56:40.593Z level=INFO msg="Uploading 1 narinfos"6504lb # [ 179.937506] niks3-server[1154]: 2026/09/22 16:56:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6505lb # [ 179.944467] niks3-server[1154]: 2026/09/22 16:56:40 INFO Signed narinfos id=60 count=16506lb # [ 179.946389] niks3-server[1154]: 2026/09/22 16:56:40 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6507lb # [ 179.960101] niks3-server[1154]: 2026/09/22 16:56:40 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6508lb # [ 179.968239] niks3-server[1154]: 2026/09/22 16:56:40 INFO Completed upload id=606509worker1 # [ 179.990991] nix-grpc-daemon[1671]: time=2026-09-22T16:56:40.626Z level=INFO msg="Upload complete. (259ms)"6510lb # [ 179.972702] niks3-server[1154]: 2026/09/22 16:56:40 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6511worker1 # [ 179.997311] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16512worker1 # [ 180.008222] nix-daemon[851]: reaped child process 1814, status = succeeded6513client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 4.97 seconds)6514client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6515client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.19 seconds)6516worker2: must succeed: pkill -f 'read -t [9]0 x'6517worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.06 seconds)6518client: waiting for success: ! systemctl is-active sw6519worker2 # [ 180.453507] nix-daemon[843]: reaped child process 1673, status = succeeded6520worker2 # [ 180.590312] nix-daemon[843]: accepted connection from pid 1846, user nix-grpc-daemon (trusted)6521worker2 # [ 180.617387] nix-daemon[843]: reaped child process 1849, status = succeeded6522lb # [ 180.648731] niks3-server[1154]: 2026/09/22 16:56:41 INFO Received uploads request method=POST path=/api/pending_closures6523worker2 # [ 180.632605] nix-grpc-daemon[1623]: time=2026-09-22T16:56:41.278Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6524worker2 # [ 180.634360] nix-grpc-daemon[1623]: time=2026-09-22T16:56:41.278Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6525lb # [ 180.694463] niks3-server[1154]: 2026/09/22 16:56:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6526worker2 # [ 180.673169] nix-grpc-daemon[1623]: time=2026-09-22T16:56:41.319Z level=INFO msg="Uploading 1 narinfos"6527lb # [ 180.712303] niks3-server[1154]: 2026/09/22 16:56:41 INFO Signed narinfos id=61 count=16528lb # [ 180.714750] niks3-server[1154]: 2026/09/22 16:56:41 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6529lb # [ 180.726787] niks3-server[1154]: 2026/09/22 16:56:41 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6530lb # [ 180.736944] niks3-server[1154]: 2026/09/22 16:56:41 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6531worker2 # [ 180.714879] nix-grpc-daemon[1623]: time=2026-09-22T16:56:41.361Z level=INFO msg="Upload complete. (320ms)"6532lb # [ 180.743834] niks3-server[1154]: 2026/09/22 16:56:41 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6533lb # [ 180.746733] niks3-server[1154]: 2026/09/22 16:56:41 INFO Completed upload id=616534lb # [ 180.751959] niks3-server[1154]: 2026/09/22 16:56:41 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6535worker2 # [ 180.729436] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43910 duration_s=14 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16536worker2 # [ 180.751196] nix-daemon[843]: reaped child process 1671, status = succeeded6537client # [ 180.752556] systemd[1]: sw.service: Deactivated successfully.6538client # [ 180.758140] systemd[1]: sw.service: Consumed 207ms CPU time over 15.223s wall clock time, 39.4M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.6539client: (finished: waiting for success: ! systemctl is-active sw, in 1.11 seconds)6540client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6541client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)6542worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6543worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.04 seconds)6544lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6545lb: must succeed: curl -sf localhost:9901/clusters6546lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6547worker2 # [ 181.550437] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:42Z level=info event=signal_exit6548lb: must succeed: curl -sf localhost:9901/clusters6549lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6550worker1 # [ 183.604161] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:500266551worker1 # [ 183.609073] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:44Z level=info event=worker_gone addr=192.168.1.4:500516552worker2 # [ 183.560827] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:44Z level=info event=scheduler_restarting6553worker2 # [ 183.581122] nix-grpc-daemon[1521]: ts=2026-09-22T16:56:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6554lb: must succeed: curl -sf localhost:9901/clusters6555worker2 # [ 183.612864] nix-daemon[843]: reaped child process 1577, status = succeeded6556lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6557worker2 # [ 184.360235] nix-grpc-daemon[1521]: E0000 00:00:1790096205.007085 1566 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6558worker2 # [ 184.367595] nix-grpc-daemon[1521]: E0000 00:00:1790096205.007339 1566 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6559lb: must succeed: curl -sf localhost:9901/clusters6560lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6561worker2 # [ 185.597633] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6562worker2 # [ 185.604189] systemd[1]: nix-grpc-daemon.service: Consumed 849ms CPU time over 34.211s wall clock time, 35.9M memory peak, 21.4K incoming IP traffic, 29.6K outgoing IP traffic.6563lb: must succeed: curl -sf localhost:9901/clusters6564lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6565worker2 # [ 186.595206] systemd[1]: Starting scheduler-token.service...6566worker2 # [ 186.688172] systemd[1]: scheduler-token.service: Deactivated successfully.6567worker2 # [ 186.696234] systemd[1]: Finished scheduler-token.service.6568worker2 # [ 186.698484] systemd[1]: scheduler-token.service: Consumed 62ms CPU time over 97ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6569worker2 # [ 186.720704] systemd[1]: Starting Nix worker-protocol over gRPC...6570lb: must succeed: curl -sf localhost:9901/clusters6571worker2 # [ 186.858541] nix-grpc-daemon[1864]: ts=2026-09-22T16:56:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6572worker2 # [ 186.861277] nix-grpc-daemon[1864]: ts=2026-09-22T16:56:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6573worker2 # [ 186.865622] nix-grpc-daemon[1864]: ts=2026-09-22T16:56:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16574worker2 # [ 186.872105] nix-grpc-daemon[1864]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6575worker2 # [ 186.873765] nix-grpc-daemon[1864]: E0000 00:00:1790096207.517910 1864 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6576lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6577lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 5.46 seconds)6578lb: must succeed: curl -sf localhost:9901/clusters6579worker2 # [ 186.924112] nix-grpc-daemon[1864]: E0000 00:00:1790096207.525341 1864 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6580worker2 # [ 186.932437] nix-grpc-daemon[1864]: ts=2026-09-22T16:56:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6581worker2 # [ 186.935084] nix-grpc-daemon[1864]: E0000 00:00:1790096207.541074 1904 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6582worker2 # [ 186.941355] nix-grpc-daemon[1864]: E0000 00:00:1790096207.559558 1904 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6583worker2 # [ 186.946721] nix-grpc-daemon[1864]: E0000 00:00:1790096207.559813 1904 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6584worker2 # [ 186.949723] systemd[1]: Started Nix worker-protocol over gRPC.6585worker2 # [ 186.950582] nix-grpc-daemon[1864]: E0000 00:00:1790096207.578303 1904 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6586worker2 # [ 186.953103] nix-grpc-daemon[1864]: ts=2026-09-22T16:56:47Z level=info event=scheduler_connected addr=lb:500516587worker1 # [ 187.002030] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:500266588worker1 # [ 187.008733] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06589lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6590worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6591worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6592(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 21.56 seconds)6593subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6594client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16595worker1 # [ 187.334092] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486626596worker2 # [ 187.295068] nix-grpc-daemon[1864]: ts=2026-09-22T16:56:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356526597worker2 # [ 187.311491] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)6598worker2 # [ 187.338051] nix-grpc-daemon[1864]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 90 ms (attempt 1/5)6599worker2 # [ 187.437454] nix-grpc-daemon[1864]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 48 ms (attempt 2/5)6600worker2 # [ 187.487599] nix-grpc-daemon[1864]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 184 ms (attempt 3/5)6601worker2 # [ 187.674022] nix-grpc-daemon[1864]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 170 ms (attempt 4/5)6602worker2 # [ 187.846772] nix-grpc-daemon[1864]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6603worker2 # [ 187.855726] nix-grpc-daemon[1864]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6604worker1 # [ 187.966117] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:48Z level=debug event=want client=5 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06605worker1 # [ 187.974184] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:48Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16606worker1 # [ 187.980527] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:48Z level=debug event=want client=5 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06607worker1 # [ 187.985818] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:48Z level=debug event=want client=5 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06608worker1 # [ 188.015576] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626609worker1 # [ 188.035868] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626610worker1 # [ 188.054438] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626611worker1 # [ 188.071635] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6612worker1 # [ 188.244570] nix-daemon[851]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6613worker1 # [ 188.269855] nix-daemon[851]: reaped child process 1847, status = succeeded6614lb # [ 188.253851] niks3-server[1154]: 2026/09/22 16:56:48 INFO Received uploads request method=POST path=/api/pending_closures6615worker1 # [ 188.292570] nix-grpc-daemon[1671]: time=2026-09-22T16:56:48.926Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6616worker1 # [ 188.294293] nix-grpc-daemon[1671]: time=2026-09-22T16:56:48.926Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6617lb # [ 188.297117] niks3-server[1154]: 2026/09/22 16:56:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6618worker1 # [ 188.331104] nix-grpc-daemon[1671]: time=2026-09-22T16:56:48.966Z level=INFO msg="Uploading 1 narinfos"6619lb # [ 188.310301] niks3-server[1154]: 2026/09/22 16:56:48 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6620lb # [ 188.320950] niks3-server[1154]: 2026/09/22 16:56:48 INFO Signed narinfos id=62 count=16621lb # [ 188.330310] niks3-server[1154]: 2026/09/22 16:56:48 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6622lb # [ 188.341889] niks3-server[1154]: 2026/09/22 16:56:48 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6623worker1 # [ 188.373739] nix-grpc-daemon[1671]: time=2026-09-22T16:56:49.008Z level=INFO msg="Upload complete. (282ms)"6624lb # [ 188.353497] niks3-server[1154]: 2026/09/22 16:56:49 INFO Completed upload id=626625lb # [ 188.357530] niks3-server[1154]: 2026/09/22 16:56:49 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6626worker1 # [ 188.384551] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=5126627worker1 # [ 188.408791] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626628worker1 # [ 188.416912] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6629worker1 # [ 188.423383] nix-daemon[851]: reaped child process 1842, status = succeeded6630worker1 # [ 188.436267] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6631lb # [ 189.152813] envoy[895]: conn peer=[2001:db8:1::1]:53936 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9591 tx=4401 ms=47226632lb # [ 189.157878] envoy[895]: conn peer=[2001:db8:1::1]:35714 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3004 tx=1518 ms=149876633lb # [ 189.162991] envoy[895]: conn peer=[2001:db8:1::4]:57486 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2255 tx=442 ms=319736634worker1 # [ 189.283392] nix-daemon[851]: reaped child process 1851, status = succeeded6635worker1 # [ 189.398864] nix-daemon[851]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)6636worker1 # [ 189.424783] nix-daemon[851]: reaped child process 1870, status = succeeded6637lb # [ 189.398209] niks3-server[1154]: 2026/09/22 16:56:50 INFO Received uploads request method=POST path=/api/pending_closures6638worker1 # [ 189.434566] nix-grpc-daemon[1671]: time=2026-09-22T16:56:50.067Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6639worker1 # [ 189.437690] nix-grpc-daemon[1671]: time=2026-09-22T16:56:50.068Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6640lb # [ 189.453190] niks3-server[1154]: 2026/09/22 16:56:50 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6641worker1 # [ 189.488397] nix-grpc-daemon[1671]: time=2026-09-22T16:56:50.123Z level=INFO msg="Uploading 1 narinfos"6642lb # [ 189.468100] niks3-server[1154]: 2026/09/22 16:56:50 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6643lb # [ 189.471572] niks3-server[1154]: 2026/09/22 16:56:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6644lb # [ 189.483170] niks3-server[1154]: 2026/09/22 16:56:50 INFO Signed narinfos id=63 count=16645lb # [ 189.492514] niks3-server[1154]: 2026/09/22 16:56:50 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6646worker1 # [ 189.516516] nix-grpc-daemon[1671]: time=2026-09-22T16:56:50.151Z level=INFO msg="Upload complete. (266ms)"6647lb # [ 189.495635] niks3-server[1154]: 2026/09/22 16:56:50 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6648lb # [ 189.502885] niks3-server[1154]: 2026/09/22 16:56:50 INFO Completed upload id=636649lb # [ 189.505713] niks3-server[1154]: 2026/09/22 16:56:50 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6650worker1 # [ 189.531206] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:50Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16651worker1 # [ 189.546933] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=7 outputs=16652worker1 # [ 189.552882] nix-daemon[851]: reaped child process 1849, status = succeeded6653worker1 # [ 189.575976] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626654worker1 # [ 189.586673] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:50Z level=debug event=want client=5 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06655worker1 # [ 189.598464] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626656worker1 # [ 189.615358] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626657worker1 # [ 189.629575] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6658worker1 # [ 189.805349] nix-daemon[851]: accepted connection from pid 1876, user nix-grpc-daemon (trusted)6659lb # [ 189.803688] niks3-server[1154]: 2026/09/22 16:56:50 INFO Received uploads request method=POST path=/api/pending_closures6660worker1 # [ 189.832120] nix-daemon[851]: reaped child process 1879, status = succeeded6661worker1 # [ 189.838439] nix-grpc-daemon[1671]: time=2026-09-22T16:56:50.473Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6662worker1 # [ 189.840278] nix-grpc-daemon[1671]: time=2026-09-22T16:56:50.473Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6663worker1 # [ 189.876000] nix-grpc-daemon[1671]: time=2026-09-22T16:56:50.511Z level=INFO msg="Uploading 1 narinfos"6664lb # [ 189.841428] niks3-server[1154]: 2026/09/22 16:56:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6665lb # [ 189.860346] niks3-server[1154]: 2026/09/22 16:56:50 INFO Signed narinfos id=64 count=16666lb # [ 189.867821] niks3-server[1154]: 2026/09/22 16:56:50 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6667lb # [ 189.884844] niks3-server[1154]: 2026/09/22 16:56:50 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6668worker1 # [ 189.915008] nix-grpc-daemon[1671]: time=2026-09-22T16:56:50.550Z level=INFO msg="Upload complete. (269ms)"6669lb # [ 189.894102] niks3-server[1154]: 2026/09/22 16:56:50 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6670lb # [ 189.899260] niks3-server[1154]: 2026/09/22 16:56:50 INFO Completed upload id=646671lb # [ 189.902938] niks3-server[1154]: 2026/09/22 16:56:50 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6672worker1 # [ 189.924420] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=5126673worker1 # [ 189.944158] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626674worker1 # [ 189.951057] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6675worker1 # [ 189.957126] nix-daemon[851]: reaped child process 1874, status = succeeded6676worker1 # [ 189.972506] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6677worker1 # [ 190.805846] nix-daemon[851]: reaped child process 1883, status = succeeded6678worker1 # [ 190.926170] nix-daemon[851]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6679worker1 # [ 190.953122] nix-daemon[851]: reaped child process 1902, status = succeeded6680lb # [ 190.935407] niks3-server[1154]: 2026/09/22 16:56:51 INFO Received uploads request method=POST path=/api/pending_closures6681worker1 # [ 191.012668] nix-grpc-daemon[1671]: time=2026-09-22T16:56:51.645Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6682worker1 # [ 191.014950] nix-grpc-daemon[1671]: time=2026-09-22T16:56:51.646Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6683lb # [ 191.029095] niks3-server[1154]: 2026/09/22 16:56:51 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6684worker1 # [ 191.065302] nix-grpc-daemon[1671]: time=2026-09-22T16:56:51.700Z level=INFO msg="Uploading 1 narinfos"6685lb # [ 191.041272] niks3-server[1154]: 2026/09/22 16:56:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6686lb # [ 191.051112] niks3-server[1154]: 2026/09/22 16:56:51 INFO Signed narinfos id=65 count=16687lb # [ 191.055024] niks3-server[1154]: 2026/09/22 16:56:51 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6688lb # [ 191.065746] niks3-server[1154]: 2026/09/22 16:56:51 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6689lb # [ 191.076465] niks3-server[1154]: 2026/09/22 16:56:51 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6690worker1 # [ 191.106116] nix-grpc-daemon[1671]: time=2026-09-22T16:56:51.741Z level=INFO msg="Upload complete. (321ms)"6691lb # [ 191.084595] niks3-server[1154]: 2026/09/22 16:56:51 INFO Completed upload id=656692lb # [ 191.086709] niks3-server[1154]: 2026/09/22 16:56:51 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6693worker1 # [ 191.126517] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=8 outputs=16694worker1 # [ 191.143989] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:51Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16695worker1 # [ 191.147650] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:51Z level=debug event=want client=5 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06696worker1 # [ 191.152718] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626697worker1 # [ 191.157512] nix-daemon[851]: reaped child process 1881, status = succeeded6698worker1 # [ 191.161736] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626699worker1 # [ 191.176244] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626700worker1 # [ 191.188460] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6701worker1 # [ 191.340229] nix-daemon[851]: accepted connection from pid 1908, user nix-grpc-daemon (trusted)6702worker1 # [ 191.363248] nix-daemon[851]: reaped child process 1911, status = succeeded6703lb # [ 191.343974] niks3-server[1154]: 2026/09/22 16:56:51 INFO Received uploads request method=POST path=/api/pending_closures6704worker1 # [ 191.378436] nix-grpc-daemon[1671]: time=2026-09-22T16:56:52.013Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6705worker1 # [ 191.380195] nix-grpc-daemon[1671]: time=2026-09-22T16:56:52.013Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6706lb # [ 191.380675] niks3-server[1154]: 2026/09/22 16:56:52 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6707worker1 # [ 191.420887] nix-grpc-daemon[1671]: time=2026-09-22T16:56:52.055Z level=INFO msg="Uploading 1 narinfos"6708lb # [ 191.398487] niks3-server[1154]: 2026/09/22 16:56:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6709lb # [ 191.406848] niks3-server[1154]: 2026/09/22 16:56:52 INFO Signed narinfos id=66 count=16710lb # [ 191.413312] niks3-server[1154]: 2026/09/22 16:56:52 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6711lb # [ 191.426052] niks3-server[1154]: 2026/09/22 16:56:52 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6712lb # [ 191.436767] niks3-server[1154]: 2026/09/22 16:56:52 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6713lb # [ 191.441505] niks3-server[1154]: 2026/09/22 16:56:52 INFO Completed upload id=666714worker1 # [ 191.465967] nix-grpc-daemon[1671]: time=2026-09-22T16:56:52.101Z level=INFO msg="Upload complete. (262ms)"6715worker1 # [ 191.477747] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=5126716worker1 # [ 191.493004] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626717worker1 # [ 191.507171] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6718worker1 # [ 191.518895] nix-daemon[851]: reaped child process 1906, status = succeeded6719worker1 # [ 191.528705] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6720worker1 # [ 192.653210] nix-daemon[851]: reaped child process 1915, status = succeeded6721worker1 # [ 192.726506] nix-daemon[851]: accepted connection from pid 1931, user nix-grpc-daemon (trusted)6722worker1 # [ 192.756083] nix-daemon[851]: reaped child process 1934, status = succeeded6723lb # [ 192.729181] niks3-server[1154]: 2026/09/22 16:56:53 INFO Received uploads request method=POST path=/api/pending_closures6724worker1 # [ 192.766573] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.400Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6725worker1 # [ 192.768445] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.400Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6726lb # [ 192.778786] niks3-server[1154]: 2026/09/22 16:56:53 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6727worker1 # [ 192.815459] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.450Z level=INFO msg="Uploading 1 narinfos"6728lb # [ 192.791564] niks3-server[1154]: 2026/09/22 16:56:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6729lb # [ 192.801902] niks3-server[1154]: 2026/09/22 16:56:53 INFO Signed narinfos id=67 count=16730lb # [ 192.806984] niks3-server[1154]: 2026/09/22 16:56:53 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6731lb # [ 192.816578] niks3-server[1154]: 2026/09/22 16:56:53 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6732lb # [ 192.827760] niks3-server[1154]: 2026/09/22 16:56:53 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6733lb # [ 192.835115] niks3-server[1154]: 2026/09/22 16:56:53 INFO Completed upload id=676734worker1 # [ 192.857939] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.493Z level=INFO msg="Upload complete. (313ms)"6735lb # [ 192.838952] niks3-server[1154]: 2026/09/22 16:56:53 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6736worker1 # [ 192.881243] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=9 outputs=16737worker1 # [ 192.898549] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:53Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16738worker1 # [ 192.912494] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626739worker1 # [ 192.921600] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:53Z level=debug event=want client=5 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06740worker1 # [ 192.928688] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626741worker1 # [ 192.942060] nix-daemon[851]: reaped child process 1913, status = succeeded6742worker1 # [ 192.948321] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6743worker1 # [ 192.950765] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.561Z level=INFO msg="All 1 paths already cached"6744worker1 # [ 192.958694] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626745worker1 # [ 193.096555] nix-daemon[851]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)6746worker1 # [ 193.120186] nix-daemon[851]: reaped child process 1943, status = succeeded6747lb # [ 193.101200] niks3-server[1154]: 2026/09/22 16:56:53 INFO Received uploads request method=POST path=/api/pending_closures6748worker1 # [ 193.138931] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.773Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6749worker1 # [ 193.140756] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.773Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6750lb # [ 193.145003] niks3-server[1154]: 2026/09/22 16:56:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6751worker1 # [ 193.177728] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.812Z level=INFO msg="Uploading 1 narinfos"6752lb # [ 193.159525] niks3-server[1154]: 2026/09/22 16:56:53 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6753lb # [ 193.170884] niks3-server[1154]: 2026/09/22 16:56:53 INFO Signed narinfos id=68 count=16754lb # [ 193.179211] niks3-server[1154]: 2026/09/22 16:56:53 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6755worker1 # [ 193.210947] nix-grpc-daemon[1671]: time=2026-09-22T16:56:53.846Z level=INFO msg="Upload complete. (247ms)"6756lb # [ 193.186980] niks3-server[1154]: 2026/09/22 16:56:53 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6757lb # [ 193.194529] niks3-server[1154]: 2026/09/22 16:56:53 INFO Completed upload id=686758worker1 # [ 193.220442] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=6246759lb # [ 193.201204] niks3-server[1154]: 2026/09/22 16:56:53 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6760worker1 # [ 193.232888] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626761worker1 # [ 193.246179] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6762worker1 # [ 193.256578] nix-daemon[851]: reaped child process 1938, status = succeeded6763worker1 # [ 193.265431] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6764worker1 # [ 194.148994] nix-daemon[851]: reaped child process 1947, status = succeeded6765worker1 # [ 194.266272] nix-daemon[851]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)6766lb # [ 194.268859] niks3-server[1154]: 2026/09/22 16:56:54 INFO Received uploads request method=POST path=/api/pending_closures6767worker1 # [ 194.295675] nix-daemon[851]: reaped child process 1966, status = succeeded6768worker1 # [ 194.308559] nix-grpc-daemon[1671]: time=2026-09-22T16:56:54.941Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6769worker1 # [ 194.310193] nix-grpc-daemon[1671]: time=2026-09-22T16:56:54.941Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6770lb # [ 194.306990] niks3-server[1154]: 2026/09/22 16:56:54 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6771lb # [ 194.315690] niks3-server[1154]: 2026/09/22 16:56:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6772worker1 # [ 194.343361] nix-grpc-daemon[1671]: time=2026-09-22T16:56:54.978Z level=INFO msg="Uploading 1 narinfos"6773lb # [ 194.320619] niks3-server[1154]: 2026/09/22 16:56:54 INFO Signed narinfos id=69 count=16774lb # [ 194.330521] niks3-server[1154]: 2026/09/22 16:56:54 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6775lb # [ 194.335548] niks3-server[1154]: 2026/09/22 16:56:54 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6776lb # [ 194.342564] niks3-server[1154]: 2026/09/22 16:56:54 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6777worker1 # [ 194.371234] nix-grpc-daemon[1671]: time=2026-09-22T16:56:55.006Z level=INFO msg="Upload complete. (292ms)"6778lb # [ 194.347331] niks3-server[1154]: 2026/09/22 16:56:54 INFO Completed upload id=696779lb # [ 194.351896] niks3-server[1154]: 2026/09/22 16:56:55 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6780worker1 # [ 194.377022] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=10 outputs=16781worker1 # [ 194.379721] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:55Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16782worker1 # [ 194.389998] nix-daemon[851]: reaped child process 1945, status = succeeded6783worker1 # [ 194.422198] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626784worker1 # [ 194.429847] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:55Z level=debug event=want client=5 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06785worker1 # [ 194.438907] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626786worker1 # [ 194.450921] nix-grpc-daemon[1671]: time=2026-09-22T16:56:55.086Z level=INFO msg="All 1 paths already cached"6787worker1 # [ 194.459311] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626788worker1 # [ 194.470791] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6789worker1 # [ 194.609455] nix-daemon[851]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6790worker1 # [ 194.633795] nix-daemon[851]: reaped child process 1975, status = succeeded6791lb # [ 194.616528] niks3-server[1154]: 2026/09/22 16:56:55 INFO Received uploads request method=POST path=/api/pending_closures6792worker1 # [ 194.653423] nix-grpc-daemon[1671]: time=2026-09-22T16:56:55.288Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6793worker1 # [ 194.655411] nix-grpc-daemon[1671]: time=2026-09-22T16:56:55.288Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6794lb # [ 194.654471] niks3-server[1154]: 2026/09/22 16:56:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6795worker1 # [ 194.689109] nix-grpc-daemon[1671]: time=2026-09-22T16:56:55.324Z level=INFO msg="Uploading 1 narinfos"6796lb # [ 194.671146] niks3-server[1154]: 2026/09/22 16:56:55 INFO Signed narinfos id=70 count=16797lb # [ 194.678036] niks3-server[1154]: 2026/09/22 16:56:55 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6798lb # [ 194.690954] niks3-server[1154]: 2026/09/22 16:56:55 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6799worker1 # [ 194.724090] nix-grpc-daemon[1671]: time=2026-09-22T16:56:55.359Z level=INFO msg="Upload complete. (241ms)"6800lb # [ 194.700407] niks3-server[1154]: 2026/09/22 16:56:55 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6801lb # [ 194.708601] niks3-server[1154]: 2026/09/22 16:56:55 INFO Completed upload id=706802worker1 # [ 194.732230] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=6246803lb # [ 194.714475] niks3-server[1154]: 2026/09/22 16:56:55 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6804worker1 # [ 194.744704] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626805worker1 # [ 194.756669] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6806worker1 # [ 194.762206] nix-daemon[851]: reaped child process 1970, status = succeeded6807worker1 # [ 194.774918] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6808worker1 # [ 196.009956] nix-daemon[851]: reaped child process 1979, status = succeeded6809worker1 # [ 196.122505] nix-daemon[851]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)6810worker1 # [ 196.151655] nix-daemon[851]: reaped child process 1998, status = succeeded6811lb # [ 196.125982] niks3-server[1154]: 2026/09/22 16:56:56 INFO Received uploads request method=POST path=/api/pending_closures6812worker1 # [ 196.166757] nix-grpc-daemon[1671]: time=2026-09-22T16:56:56.800Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6813worker1 # [ 196.168768] nix-grpc-daemon[1671]: time=2026-09-22T16:56:56.800Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6814lb # [ 196.182955] niks3-server[1154]: 2026/09/22 16:56:56 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6815worker1 # [ 196.217806] nix-grpc-daemon[1671]: time=2026-09-22T16:56:56.852Z level=INFO msg="Uploading 1 narinfos"6816lb # [ 196.197176] niks3-server[1154]: 2026/09/22 16:56:56 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6817lb # [ 196.207998] niks3-server[1154]: 2026/09/22 16:56:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6818lb # [ 196.211838] niks3-server[1154]: 2026/09/22 16:56:56 INFO Signed narinfos id=71 count=16819worker1 # [ 196.250309] nix-grpc-daemon[1671]: time=2026-09-22T16:56:56.885Z level=INFO msg="Upload complete. (261ms)"6820lb # [ 196.226072] niks3-server[1154]: 2026/09/22 16:56:56 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6821lb # [ 196.235714] niks3-server[1154]: 2026/09/22 16:56:56 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6822lb # [ 196.242465] niks3-server[1154]: 2026/09/22 16:56:56 INFO Completed upload id=716823lb # [ 196.246624] niks3-server[1154]: 2026/09/22 16:56:56 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6824worker1 # [ 196.268712] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=11 outputs=16825worker1 # [ 196.288376] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:56Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16826worker1 # [ 196.292865] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:56Z level=debug event=want client=5 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06827worker1 # [ 196.298370] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626828worker1 # [ 196.305346] nix-daemon[851]: reaped child process 1977, status = succeeded6829worker1 # [ 196.309804] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626830worker1 # [ 196.321043] nix-grpc-daemon[1671]: time=2026-09-22T16:56:56.956Z level=INFO msg="All 1 paths already cached"6831worker1 # [ 196.330552] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626832worker1 # [ 196.341389] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6833worker1 # [ 196.489454] nix-daemon[851]: accepted connection from pid 2004, user nix-grpc-daemon (trusted)6834worker1 # [ 196.513618] nix-daemon[851]: reaped child process 2007, status = succeeded6835lb # [ 196.495077] niks3-server[1154]: 2026/09/22 16:56:57 INFO Received uploads request method=POST path=/api/pending_closures6836worker1 # [ 196.529650] nix-grpc-daemon[1671]: time=2026-09-22T16:56:57.164Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6837worker1 # [ 196.531283] nix-grpc-daemon[1671]: time=2026-09-22T16:56:57.164Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6838lb # [ 196.530651] niks3-server[1154]: 2026/09/22 16:56:57 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6839worker1 # [ 196.565716] nix-grpc-daemon[1671]: time=2026-09-22T16:56:57.200Z level=INFO msg="Uploading 1 narinfos"6840lb # [ 196.544523] niks3-server[1154]: 2026/09/22 16:56:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6841lb # [ 196.555339] niks3-server[1154]: 2026/09/22 16:56:57 INFO Signed narinfos id=72 count=16842lb # [ 196.562093] niks3-server[1154]: 2026/09/22 16:56:57 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6843lb # [ 196.573148] niks3-server[1154]: 2026/09/22 16:56:57 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6844lb # [ 196.581646] niks3-server[1154]: 2026/09/22 16:56:57 INFO Completed upload id=726845lb # [ 196.583780] niks3-server[1154]: 2026/09/22 16:56:57 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6846worker1 # [ 196.604548] nix-grpc-daemon[1671]: time=2026-09-22T16:56:57.239Z level=INFO msg="Upload complete. (249ms)"6847worker1 # [ 196.617282] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=6246848worker1 # [ 196.632652] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626849worker1 # [ 196.640873] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6850worker1 # [ 196.651817] nix-daemon[851]: reaped child process 2002, status = succeeded6851worker1 # [ 196.662528] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6852worker1 # [ 197.902578] nix-daemon[851]: reaped child process 2011, status = succeeded6853worker1 # [ 198.019360] nix-daemon[851]: accepted connection from pid 2027, user nix-grpc-daemon (trusted)6854worker1 # [ 198.045101] nix-daemon[851]: reaped child process 2030, status = succeeded6855lb # [ 198.026982] niks3-server[1154]: 2026/09/22 16:56:58 INFO Received uploads request method=POST path=/api/pending_closures6856worker1 # [ 198.066639] nix-grpc-daemon[1671]: time=2026-09-22T16:56:58.701Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6857worker1 # [ 198.071487] nix-grpc-daemon[1671]: time=2026-09-22T16:56:58.703Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6858lb # [ 198.078934] niks3-server[1154]: 2026/09/22 16:56:58 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6859worker1 # [ 198.118535] nix-grpc-daemon[1671]: time=2026-09-22T16:56:58.753Z level=INFO msg="Uploading 1 narinfos"6860lb # [ 198.094367] niks3-server[1154]: 2026/09/22 16:56:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6861lb # [ 198.103430] niks3-server[1154]: 2026/09/22 16:56:58 INFO Signed narinfos id=73 count=16862lb # [ 198.110330] niks3-server[1154]: 2026/09/22 16:56:58 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6863lb # [ 198.117569] niks3-server[1154]: 2026/09/22 16:56:58 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6864lb # [ 198.123757] niks3-server[1154]: 2026/09/22 16:56:58 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6865lb # [ 198.130347] niks3-server[1154]: 2026/09/22 16:56:58 INFO Completed upload id=736866worker1 # [ 198.155427] nix-grpc-daemon[1671]: time=2026-09-22T16:56:58.790Z level=INFO msg="Upload complete. (273ms)"6867lb # [ 198.135570] niks3-server[1154]: 2026/09/22 16:56:58 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6868worker1 # [ 198.177272] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=12 outputs=16869worker1 # [ 198.192383] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:58Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16870worker1 # [ 198.204657] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626871worker1 # [ 198.212413] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:58Z level=debug event=want client=5 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06872worker1 # [ 198.219877] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626873worker1 # [ 198.226050] nix-daemon[851]: reaped child process 2009, status = succeeded6874worker1 # [ 198.228453] nix-grpc-daemon[1671]: time=2026-09-22T16:56:58.852Z level=INFO msg="All 2 paths already cached"6875worker1 # [ 198.238614] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626876worker1 # [ 198.247082] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6877worker1 # [ 198.387768] nix-daemon[851]: accepted connection from pid 2036, user nix-grpc-daemon (trusted)6878worker1 # [ 198.414828] nix-daemon[851]: reaped child process 2039, status = succeeded6879lb # [ 198.391420] niks3-server[1154]: 2026/09/22 16:56:59 INFO Received uploads request method=POST path=/api/pending_closures6880worker1 # [ 198.425713] nix-grpc-daemon[1671]: time=2026-09-22T16:56:59.060Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6881worker1 # [ 198.427343] nix-grpc-daemon[1671]: time=2026-09-22T16:56:59.060Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6882lb # [ 198.424925] niks3-server[1154]: 2026/09/22 16:56:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6883worker1 # [ 198.457566] nix-grpc-daemon[1671]: time=2026-09-22T16:56:59.092Z level=INFO msg="Uploading 1 narinfos"6884lb # [ 198.439471] niks3-server[1154]: 2026/09/22 16:56:59 INFO Signed narinfos id=74 count=16885lb # [ 198.447453] niks3-server[1154]: 2026/09/22 16:56:59 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6886lb # [ 198.461500] niks3-server[1154]: 2026/09/22 16:56:59 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6887worker1 # [ 198.493523] nix-grpc-daemon[1671]: time=2026-09-22T16:56:59.128Z level=INFO msg="Upload complete. (238ms)"6888lb # [ 198.469998] niks3-server[1154]: 2026/09/22 16:56:59 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6889worker1 # [ 198.501016] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=7526890lb # [ 198.480289] niks3-server[1154]: 2026/09/22 16:56:59 INFO Completed upload id=746891lb # [ 198.482928] niks3-server[1154]: 2026/09/22 16:56:59 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6892worker1 # [ 198.512959] nix-grpc-daemon[1595]: ts=2026-09-22T16:56:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626893worker1 # [ 198.522902] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6894worker1 # [ 198.536492] nix-daemon[851]: reaped child process 2034, status = succeeded6895worker1 # [ 198.546409] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6896worker1 # [ 199.184650] nix-daemon[851]: reaped child process 2043, status = succeeded6897worker1 # [ 199.295990] nix-daemon[851]: accepted connection from pid 2059, user nix-grpc-daemon (trusted)6898worker1 # [ 199.324369] nix-daemon[851]: reaped child process 2062, status = succeeded6899lb # [ 199.300890] niks3-server[1154]: 2026/09/22 16:56:59 INFO Received uploads request method=POST path=/api/pending_closures6900worker1 # [ 199.335539] nix-grpc-daemon[1671]: time=2026-09-22T16:56:59.969Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6901worker1 # [ 199.337412] nix-grpc-daemon[1671]: time=2026-09-22T16:56:59.969Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6902lb # [ 199.342417] niks3-server[1154]: 2026/09/22 16:56:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6903worker1 # [ 199.372668] nix-grpc-daemon[1671]: time=2026-09-22T16:57:00.007Z level=INFO msg="Uploading 1 narinfos"6904lb # [ 199.358445] niks3-server[1154]: 2026/09/22 16:56:59 INFO Signed narinfos id=75 count=16905lb # [ 199.365792] niks3-server[1154]: 2026/09/22 16:57:00 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6906lb # [ 199.375808] niks3-server[1154]: 2026/09/22 16:57:00 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6907worker1 # [ 199.410544] nix-grpc-daemon[1671]: time=2026-09-22T16:57:00.045Z level=INFO msg="Upload complete. (253ms)"6908lb # [ 199.387585] niks3-server[1154]: 2026/09/22 16:57:00 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6909lb # [ 199.397227] niks3-server[1154]: 2026/09/22 16:57:00 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6910lb # [ 199.400123] niks3-server[1154]: 2026/09/22 16:57:00 INFO Completed upload id=756911worker1 # [ 199.422101] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=13 outputs=16912lb # [ 199.402255] niks3-server[1154]: 2026/09/22 16:57:00 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6913worker1 # [ 199.438642] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:00Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16914worker1 # [ 199.443353] nix-daemon[851]: reaped child process 2041, status = succeeded6915worker1 # [ 199.465403] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626916worker1 # [ 199.481848] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626917worker1 # [ 199.498191] nix-grpc-daemon[1671]: time=2026-09-22T16:57:00.133Z level=INFO msg="All 2 paths already cached"6918worker1 # [ 199.506719] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626919worker1 # [ 199.513380] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6920worker1 # [ 199.648419] nix-daemon[851]: accepted connection from pid 2066, user nix-grpc-daemon (trusted)6921worker1 # [ 199.674802] nix-daemon[851]: reaped child process 2069, status = succeeded6922lb # [ 199.647358] niks3-server[1154]: 2026/09/22 16:57:00 INFO Received uploads request method=POST path=/api/pending_closures6923worker1 # [ 199.685669] nix-grpc-daemon[1671]: time=2026-09-22T16:57:00.320Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6924worker1 # [ 199.687257] nix-grpc-daemon[1671]: time=2026-09-22T16:57:00.320Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6925lb # [ 199.693050] niks3-server[1154]: 2026/09/22 16:57:00 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6926worker1 # [ 199.725289] nix-grpc-daemon[1671]: time=2026-09-22T16:57:00.360Z level=INFO msg="Uploading 1 narinfos"6927lb # [ 199.707227] niks3-server[1154]: 2026/09/22 16:57:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6928lb # [ 199.716361] niks3-server[1154]: 2026/09/22 16:57:00 INFO Signed narinfos id=76 count=16929lb # [ 199.719429] niks3-server[1154]: 2026/09/22 16:57:00 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6930worker1 # [ 199.758059] nix-grpc-daemon[1671]: time=2026-09-22T16:57:00.393Z level=INFO msg="Upload complete. (236ms)"6931lb # [ 199.733994] niks3-server[1154]: 2026/09/22 16:57:00 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6932lb # [ 199.739905] niks3-server[1154]: 2026/09/22 16:57:00 INFO Completed upload id=766933worker1 # [ 199.764399] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=7526934lb # [ 199.743035] niks3-server[1154]: 2026/09/22 16:57:00 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6935worker1 # [ 199.769446] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626936worker1 # [ 199.778929] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6937worker1 # [ 199.782930] nix-daemon[851]: reaped child process 2064, status = succeeded6938worker1 # [ 199.792396] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6939worker1 # [ 200.700992] nix-daemon[851]: reaped child process 2073, status = succeeded6940worker1 # [ 200.771818] nix-daemon[851]: accepted connection from pid 2089, user nix-grpc-daemon (trusted)6941worker1 # [ 200.798398] nix-daemon[851]: reaped child process 2092, status = succeeded6942lb # [ 200.782582] niks3-server[1154]: 2026/09/22 16:57:01 INFO Received uploads request method=POST path=/api/pending_closures6943worker1 # [ 200.820402] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.455Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6944worker1 # [ 200.824693] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.457Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6945lb # [ 200.832901] niks3-server[1154]: 2026/09/22 16:57:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6946worker1 # [ 200.864797] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.499Z level=INFO msg="Uploading 1 narinfos"6947lb # [ 200.841104] niks3-server[1154]: 2026/09/22 16:57:01 INFO Signed narinfos id=77 count=16948lb # [ 200.850376] niks3-server[1154]: 2026/09/22 16:57:01 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6949lb # [ 200.867520] niks3-server[1154]: 2026/09/22 16:57:01 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6950lb # [ 200.874105] niks3-server[1154]: 2026/09/22 16:57:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6951lb # [ 200.883761] niks3-server[1154]: 2026/09/22 16:57:01 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6952worker1 # [ 200.915406] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.550Z level=INFO msg="Upload complete. (311ms)"6953lb # [ 200.893749] niks3-server[1154]: 2026/09/22 16:57:01 INFO Completed upload id=776954lb # [ 200.896334] niks3-server[1154]: 2026/09/22 16:57:01 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6955worker1 # [ 200.925423] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=14 outputs=16956worker1 # [ 200.947777] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:01Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16957worker1 # [ 200.953005] nix-daemon[851]: reaped child process 2071, status = succeeded6958worker1 # [ 200.979305] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626959worker1 # [ 200.999687] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:486626960worker1 # [ 201.018712] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.653Z level=INFO msg="All 2 paths already cached"6961worker1 # [ 201.031829] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:486626962worker1 # [ 201.048078] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6963worker1 # [ 201.215256] nix-daemon[851]: accepted connection from pid 2096, user nix-grpc-daemon (trusted)6964worker1 # [ 201.242571] nix-daemon[851]: reaped child process 2099, status = succeeded6965lb # [ 201.221626] niks3-server[1154]: 2026/09/22 16:57:01 INFO Received uploads request method=POST path=/api/pending_closures6966worker1 # [ 201.256176] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.891Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6967worker1 # [ 201.258804] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.893Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6968lb # [ 201.259520] niks3-server[1154]: 2026/09/22 16:57:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6969worker1 # [ 201.292709] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.927Z level=INFO msg="Uploading 1 narinfos"6970lb # [ 201.275220] niks3-server[1154]: 2026/09/22 16:57:01 INFO Signed narinfos id=78 count=16971lb # [ 201.279547] niks3-server[1154]: 2026/09/22 16:57:01 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6972lb # [ 201.295450] niks3-server[1154]: 2026/09/22 16:57:01 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6973lb # [ 201.305624] niks3-server[1154]: 2026/09/22 16:57:01 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6974worker1 # [ 201.335655] nix-grpc-daemon[1671]: time=2026-09-22T16:57:01.970Z level=INFO msg="Upload complete. (269ms)"6975lb # [ 201.314071] niks3-server[1154]: 2026/09/22 16:57:01 INFO Completed upload id=786976lb # [ 201.318773] niks3-server[1154]: 2026/09/22 16:57:01 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6977worker1 # [ 201.348417] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=0 paths=1 nar_bytes_in=7526978worker1 # [ 201.364424] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:486626979worker1 # [ 201.374369] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6980worker1 # [ 201.380794] nix-daemon[851]: reaped child process 2094, status = succeeded6981worker1 # [ 201.397148] nix-daemon[851]: accepted connection from pid 1595, user nix-grpc-daemon (trusted)6982worker1 # [ 202.681305] nix-daemon[851]: reaped child process 2103, status = succeeded6983worker1 # [ 202.791883] nix-daemon[851]: accepted connection from pid 2119, user nix-grpc-daemon (trusted)6984worker1 # [ 202.814669] nix-daemon[851]: reaped child process 2122, status = succeeded6985lb # [ 202.797579] niks3-server[1154]: 2026/09/22 16:57:03 INFO Received uploads request method=POST path=/api/pending_closures6986worker1 # [ 202.835814] nix-grpc-daemon[1671]: time=2026-09-22T16:57:03.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6987worker1 # [ 202.837749] nix-grpc-daemon[1671]: time=2026-09-22T16:57:03.469Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6988lb # [ 202.846220] niks3-server[1154]: 2026/09/22 16:57:03 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6989worker1 # [ 202.885543] nix-grpc-daemon[1671]: time=2026-09-22T16:57:03.520Z level=INFO msg="Uploading 1 narinfos"6990lb # [ 202.861612] niks3-server[1154]: 2026/09/22 16:57:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6991lb # [ 202.871090] niks3-server[1154]: 2026/09/22 16:57:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6992lb # [ 202.878825] niks3-server[1154]: 2026/09/22 16:57:03 INFO Signed narinfos id=79 count=16993lb # [ 202.886313] niks3-server[1154]: 2026/09/22 16:57:03 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6994worker1 # [ 202.911012] nix-grpc-daemon[1671]: time=2026-09-22T16:57:03.546Z level=INFO msg="Upload complete. (300ms)"6995lb # [ 202.895288] niks3-server[1154]: 2026/09/22 16:57:03 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6996lb # [ 202.900354] niks3-server[1154]: 2026/09/22 16:57:03 INFO Completed upload id=796997lb # [ 202.902533] niks3-server[1154]: 2026/09/22 16:57:03 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6998worker1 # [ 202.926545] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48662 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=15 outputs=16999worker1 # [ 202.948795] nix-daemon[851]: reaped child process 2101, status = succeeded7000client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.91 seconds)7001worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7002worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7003client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7004client # [ 203.075544] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7005client # Running as unit: gpu.service; invocation ID: d9360422809f409595c6f531f58a6aaf7006client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.08 seconds)7007lb: must succeed: curl -sf localhost:9901/clusters7008lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)7009lb: must succeed: curl -sf localhost:9901/clusters7010lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7011worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7012worker1 # [ 203.340369] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:486627013client # [ 203.291176] nix[1583]: don't know how to build these paths:7014client # [ 203.293455] nix[1583]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7015worker2 # [ 203.309489] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356527016worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)7017worker1 # [ 203.388434] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:04Z level=debug event=want client=6 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07018client # [ 203.331790] nix[1583]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7019lb: must succeed: curl -sf localhost:9901/clusters7020lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7021lb: must succeed: curl -sf localhost:9901/clusters7022lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7023worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7024worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7025client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7026lb # [ 209.152787] envoy[895]: conn peer=[2001:db8:1::1]:36064 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27071 tx=12428 ms=156127027client # [ 233.333341] nix[1583]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7028client # [ 233.346195] nix[1583]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7029client # [ 233.390755] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7030client # [ 233.399086] systemd[1]: gpu.service: Failed with result 'exit-code'.7031client # [ 233.403247] systemd[1]: gpu.service: Consumed 202ms CPU time over 30.320s wall clock time, 35.4M memory peak, 5.1K incoming IP traffic, 5K outgoing IP traffic.7032client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.22 seconds)7033client: must succeed: journalctl -u gpu -o cat7034client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7035lb: must succeed: curl -sf localhost:9901/clusters7036lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7037lb: must succeed: curl -sf localhost:9901/clusters7038lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7039worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7040worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7041(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.90 seconds)7042subtest: low disk drains a worker and builds go to the other7043worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7044worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.17 seconds)7045worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7046worker1 # [ 235.667969] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:36Z level=info event=unhealthy reason=min_free7047worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)7048client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27049client # don't know how to build these paths:7050client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7051worker2 # [ 236.433150] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:356527052worker2 # [ 236.442089] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356527053worker1 # [ 236.516934] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:37Z level=debug event=want client=7 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07054worker1 # [ 236.517374] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:37Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17055worker2 # [ 236.513398] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356527056worker2 # [ 236.530418] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356527057client # copying 1 paths to 'lb:50051'...7058client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7059worker2 # [ 236.547553] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:356527060worker2 # [ 236.563890] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7061worker2 # [ 236.594034] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:37Z level=info event=niks3_push_started pid=19247062worker2 # [ 236.743486] nix-grpc-daemon[1924]: time=2026-09-22T16:57:37.390Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""7063worker2 # [ 236.855924] nix-daemon[843]: accepted connection from pid 1936, user nix-grpc-daemon (trusted)7064worker2 # [ 236.879471] nix-daemon[843]: reaped child process 1939, status = succeeded7065lb # [ 236.906023] niks3-server[1154]: 2026/09/22 16:57:37 INFO Received uploads request method=POST path=/api/pending_closures7066worker2 # [ 236.891841] nix-grpc-daemon[1924]: time=2026-09-22T16:57:37.538Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7067worker2 # [ 236.896844] nix-grpc-daemon[1924]: time=2026-09-22T16:57:37.538Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7068lb # [ 236.954175] niks3-server[1154]: 2026/09/22 16:57:37 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7069worker2 # [ 236.938394] nix-grpc-daemon[1924]: time=2026-09-22T16:57:37.584Z level=INFO msg="Uploading 1 narinfos"7070lb # [ 236.970281] niks3-server[1154]: 2026/09/22 16:57:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7071lb # [ 236.979130] niks3-server[1154]: 2026/09/22 16:57:37 INFO Signed narinfos id=80 count=17072worker2 # [ 236.968525] nix-grpc-daemon[1924]: time=2026-09-22T16:57:37.615Z level=INFO msg="Upload complete. (221ms)"7073lb # [ 236.990218] niks3-server[1154]: 2026/09/22 16:57:37 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7074lb # [ 237.001917] niks3-server[1154]: 2026/09/22 16:57:37 INFO Completed upload id=807075worker2 # [ 236.976338] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35652 duration_s=0 paths=1 nar_bytes_in=5127076lb # [ 237.008711] niks3-server[1154]: 2026/09/22 16:57:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7077lb # [ 237.013484] niks3-server[1154]: 2026/09/22 16:57:37 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7078worker2 # [ 236.988449] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356527079worker2 # [ 237.000318] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7080worker2 # [ 237.008432] nix-daemon[843]: reaped child process 1922, status = succeeded7081worker2 # [ 237.013834] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7082client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7083client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7084client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7085client # farm-a> LOG-farm-a-drain7086worker2 # [ 237.916340] nix-daemon[843]: reaped child process 1943, status = succeeded7087worker2 # [ 237.988159] nix-daemon[843]: accepted connection from pid 1959, user nix-grpc-daemon (trusted)7088worker2 # [ 238.006523] nix-daemon[843]: reaped child process 1962, status = succeeded7089lb # [ 238.043116] niks3-server[1154]: 2026/09/22 16:57:38 INFO Received uploads request method=POST path=/api/pending_closures7090worker2 # [ 238.028465] nix-grpc-daemon[1924]: time=2026-09-22T16:57:38.674Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7091worker2 # [ 238.030138] nix-grpc-daemon[1924]: time=2026-09-22T16:57:38.674Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7092lb # [ 238.090875] niks3-server[1154]: 2026/09/22 16:57:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7093worker2 # [ 238.078294] nix-grpc-daemon[1924]: time=2026-09-22T16:57:38.724Z level=INFO msg="Uploading 1 narinfos"7094lb # [ 238.109344] niks3-server[1154]: 2026/09/22 16:57:38 INFO Signed narinfos id=81 count=17095lb # [ 238.121750] niks3-server[1154]: 2026/09/22 16:57:38 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7096lb # [ 238.130836] niks3-server[1154]: 2026/09/22 16:57:38 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7097lb # [ 238.143643] niks3-server[1154]: 2026/09/22 16:57:38 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7098lb # [ 238.147878] niks3-server[1154]: 2026/09/22 16:57:38 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7099lb # [ 238.152711] niks3-server[1154]: 2026/09/22 16:57:38 INFO Completed upload id=817100worker2 # [ 238.125640] nix-grpc-daemon[1924]: time=2026-09-22T16:57:38.772Z level=INFO msg="Upload complete. (299ms)"7101lb # [ 238.155287] niks3-server[1154]: 2026/09/22 16:57:38 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7102worker2 # [ 238.141478] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35652 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=16 outputs=17103worker1 # [ 238.195983] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:38Z level=debug event=want client=7 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07104worker1 # [ 238.196824] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:38Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17105worker2 # [ 238.162842] nix-daemon[843]: reaped child process 1941, status = succeeded7106worker2 # [ 238.194179] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356527107worker2 # [ 238.209395] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356527108worker2 # [ 238.225379] nix-grpc-daemon[1924]: time=2026-09-22T16:57:38.872Z level=INFO msg="All 1 paths already cached"7109client # copying 1 paths to 'lb:50051'...7110client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7111worker2 # [ 238.236721] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:356527112worker2 # [ 238.250884] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7113worker2 # [ 238.408172] nix-daemon[843]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)7114worker2 # [ 238.426718] nix-daemon[843]: reaped child process 1969, status = succeeded7115lb # [ 238.461985] niks3-server[1154]: 2026/09/22 16:57:39 INFO Received uploads request method=POST path=/api/pending_closures7116worker2 # [ 238.444119] nix-grpc-daemon[1924]: time=2026-09-22T16:57:39.090Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7117worker2 # [ 238.446184] nix-grpc-daemon[1924]: time=2026-09-22T16:57:39.090Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7118lb # [ 238.495039] niks3-server[1154]: 2026/09/22 16:57:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7119worker2 # [ 238.476250] nix-grpc-daemon[1924]: time=2026-09-22T16:57:39.123Z level=INFO msg="Uploading 1 narinfos"7120lb # [ 238.510077] niks3-server[1154]: 2026/09/22 16:57:39 INFO Signed narinfos id=82 count=17121lb # [ 238.517864] niks3-server[1154]: 2026/09/22 16:57:39 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7122lb # [ 238.530414] niks3-server[1154]: 2026/09/22 16:57:39 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7123worker2 # [ 238.510133] nix-grpc-daemon[1924]: time=2026-09-22T16:57:39.157Z level=INFO msg="Upload complete. (242ms)"7124lb # [ 238.539438] niks3-server[1154]: 2026/09/22 16:57:39 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7125lb # [ 238.545816] niks3-server[1154]: 2026/09/22 16:57:39 INFO Completed upload id=827126worker2 # [ 238.520324] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35652 duration_s=0 paths=1 nar_bytes_in=6327127lb # [ 238.550017] niks3-server[1154]: 2026/09/22 16:57:39 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7128worker2 # [ 238.536682] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356527129worker2 # [ 238.544556] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7130worker2 # [ 238.550432] nix-daemon[843]: reaped child process 1964, status = succeeded7131client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7132worker2 # [ 238.564724] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7133client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7134lb # [ 239.152931] envoy[895]: conn peer=[2001:db8:1::1]:49684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=897 tx=580 ms=300897135client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7136client # farm-b> LOG-farm-b-drain7137worker2 # [ 239.604669] nix-daemon[843]: reaped child process 1973, status = succeeded7138worker2 # [ 239.720192] nix-daemon[843]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)7139worker2 # [ 239.742656] nix-daemon[843]: reaped child process 1992, status = succeeded7140lb # [ 239.774055] niks3-server[1154]: 2026/09/22 16:57:40 INFO Received uploads request method=POST path=/api/pending_closures7141worker2 # [ 239.760373] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.406Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7142worker2 # [ 239.762249] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.406Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7143lb # [ 239.815962] niks3-server[1154]: 2026/09/22 16:57:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7144worker2 # [ 239.797463] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.444Z level=INFO msg="Uploading 1 narinfos"7145lb # [ 239.833626] niks3-server[1154]: 2026/09/22 16:57:40 INFO Signed narinfos id=83 count=17146lb # [ 239.839821] niks3-server[1154]: 2026/09/22 16:57:40 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7147lb # [ 239.847392] niks3-server[1154]: 2026/09/22 16:57:40 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7148worker2 # [ 239.830954] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.478Z level=INFO msg="Upload complete. (252ms)"7149lb # [ 239.859838] niks3-server[1154]: 2026/09/22 16:57:40 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7150lb # [ 239.866054] niks3-server[1154]: 2026/09/22 16:57:40 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7151lb # [ 239.870469] niks3-server[1154]: 2026/09/22 16:57:40 INFO Completed upload id=837152worker2 # [ 239.841133] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35652 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=17 outputs=17153lb # [ 239.873556] niks3-server[1154]: 2026/09/22 16:57:40 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7154worker1 # [ 239.906218] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:40Z level=debug event=want client=7 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07155worker2 # [ 239.860587] nix-daemon[843]: reaped child process 1971, status = succeeded7156worker1 # [ 239.912683] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:40Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17157worker2 # [ 239.864530] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356527158worker2 # [ 239.877994] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:356527159worker2 # [ 239.890678] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.537Z level=INFO msg="All 2 paths already cached"7160client # copying 1 paths to 'lb:50051'...7161client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7162worker2 # [ 239.900822] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:356527163worker2 # [ 239.913540] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7164worker2 # [ 240.050975] nix-daemon[843]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)7165worker2 # [ 240.074613] nix-daemon[843]: reaped child process 1999, status = succeeded7166lb # [ 240.105984] niks3-server[1154]: 2026/09/22 16:57:40 INFO Received uploads request method=POST path=/api/pending_closures7167worker2 # [ 240.087248] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.734Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7168worker2 # [ 240.088976] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.734Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7169lb # [ 240.136090] niks3-server[1154]: 2026/09/22 16:57:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7170worker2 # [ 240.115227] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.762Z level=INFO msg="Uploading 1 narinfos"7171lb # [ 240.151178] niks3-server[1154]: 2026/09/22 16:57:40 INFO Signed narinfos id=84 count=17172lb # [ 240.158228] niks3-server[1154]: 2026/09/22 16:57:40 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7173lb # [ 240.169407] niks3-server[1154]: 2026/09/22 16:57:40 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7174worker2 # [ 240.147458] nix-grpc-daemon[1924]: time=2026-09-22T16:57:40.794Z level=INFO msg="Upload complete. (220ms)"7175lb # [ 240.175354] niks3-server[1154]: 2026/09/22 16:57:40 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7176lb # [ 240.179243] niks3-server[1154]: 2026/09/22 16:57:40 INFO Completed upload id=847177lb # [ 240.184263] niks3-server[1154]: 2026/09/22 16:57:40 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7178worker2 # [ 240.156302] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35652 duration_s=0 paths=1 nar_bytes_in=7607179worker2 # [ 240.172538] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:356527180worker2 # [ 240.178808] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7181worker2 # [ 240.188351] nix-daemon[843]: reaped child process 1994, status = succeeded7182client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7183worker2 # [ 240.197425] nix-daemon[843]: accepted connection from pid 1864, user nix-grpc-daemon (trusted)7184client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7185client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7186client # farm-top> LOG-farm-top-drain7187worker2 # [ 241.299122] nix-daemon[843]: reaped child process 2003, status = succeeded7188worker2 # [ 241.382384] nix-daemon[843]: accepted connection from pid 2020, user nix-grpc-daemon (trusted)7189worker2 # [ 241.406393] nix-daemon[843]: reaped child process 2023, status = succeeded7190lb # [ 241.433412] niks3-server[1154]: 2026/09/22 16:57:42 INFO Received uploads request method=POST path=/api/pending_closures7191worker2 # [ 241.416856] nix-grpc-daemon[1924]: time=2026-09-22T16:57:42.063Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7192worker2 # [ 241.420730] nix-grpc-daemon[1924]: time=2026-09-22T16:57:42.063Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7193lb # [ 241.479072] niks3-server[1154]: 2026/09/22 16:57:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7194worker2 # [ 241.460288] nix-grpc-daemon[1924]: time=2026-09-22T16:57:42.107Z level=INFO msg="Uploading 1 narinfos"7195lb # [ 241.491621] niks3-server[1154]: 2026/09/22 16:57:42 INFO Signed narinfos id=85 count=17196lb # [ 241.500411] niks3-server[1154]: 2026/09/22 16:57:42 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7197lb # [ 241.503738] niks3-server[1154]: 2026/09/22 16:57:42 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7198lb # [ 241.514945] niks3-server[1154]: 2026/09/22 16:57:42 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7199worker2 # [ 241.494391] nix-grpc-daemon[1924]: time=2026-09-22T16:57:42.141Z level=INFO msg="Upload complete. (285ms)"7200lb # [ 241.523512] niks3-server[1154]: 2026/09/22 16:57:42 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7201lb # [ 241.526894] niks3-server[1154]: 2026/09/22 16:57:42 INFO Completed upload id=857202lb # [ 241.529439] niks3-server[1154]: 2026/09/22 16:57:42 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7203worker2 # [ 241.509218] nix-grpc-daemon[1864]: ts=2026-09-22T16:57:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35652 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=18 outputs=17204worker2 # [ 241.527283] nix-daemon[843]: reaped child process 2001, status = succeeded7205client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.34 seconds)7206client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7207client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7208worker1: must succeed: rm /nix/.rw-store/fill7209worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7210worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7211worker1 # [ 242.672717] nix-grpc-daemon[1595]: ts=2026-09-22T16:57:43Z level=info event=healthy reason=min_free7212worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7213(finished: subtest: low disk drains a worker and builds go to the other, in 9.14 seconds)7214(finished: run the VM test script, in 243.99 seconds)7215test script finished in 244.60s7216cleanup7217kill QemuMachine (pid 49)7218client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7219client # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7220client # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7221client # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7222kill QemuMachine (pid 50)7223lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7224lb # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7225lb # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7226lb # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7227kill QemuMachine (pid 51)7228worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7229worker1 # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7230worker1 # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7231worker1 # [2026-09-22T16:57:44Z INFO virtiofsd] Client disconnected, shutting down7232kill QemuMachine (pid 52)7233worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7234worker2 # [2026-09-22T16:57:45Z INFO virtiofsd] Client disconnected, shutting down7235worker2 # [2026-09-22T16:57:45Z INFO virtiofsd] Client disconnected, shutting down7236worker2 # [2026-09-22T16:57:45Z INFO virtiofsd] Client disconnected, shutting down7237(finished: cleanup, in 1.48 seconds)7238additionally exposed symbols:7239 client, lb, worker1, worker2,7240 vlan1,7241 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh