vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #557
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12lb # Disk image does not exist, creating the virtualisation disk image...13lb: QEMU running (pid 49)14worker1 # Disk image does not exist, creating the virtualisation disk image...15worker1: QEMU running (pid 52)16worker2: QEMU running (pid 51)17client # Disk image does not exist, creating the virtualisation disk image...18worker2 # Disk image does not exist, creating the virtualisation disk image...19lb # Formatting '/build/vm-state-lb/tmp.g0NHmcM0IJ', fmt=raw size=107374182420client # Formatting '/build/vm-state-client/tmp.cHu5Xh8VJ8', fmt=raw size=107374182421worker2 # Formatting '/build/vm-state-worker2/tmp.Pa1gHSgwlP', fmt=raw size=107374182422lb # mke2fs 1.47.4 (6-Mar-2025)23client # mke2fs 1.47.4 (6-Mar-2025)24worker2 # mke2fs 1.47.4 (6-Mar-2025)25lb # Discarding device blocks: 0/262144 done26client # Discarding device blocks: 0/262144 done27worker2 # Discarding device blocks: 0/262144 done28lb # Creating filesystem with 262144 4k blocks and 65536 inodes29client # Creating filesystem with 262144 4k blocks and 65536 inodes30worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes31lb # Filesystem UUID: 88cbc442-c834-4d45-a9ef-88ee54739cc532client # Filesystem UUID: 5bf538c6-46a3-4fdd-93a0-9651152ff58233worker2 # Filesystem UUID: ec749343-9b4b-467c-8ef1-28341be0bda434lb # Superblock backups stored on blocks:35client # Superblock backups stored on blocks:36worker2 # Superblock backups stored on blocks:37lb # 32768, 98304, 163840, 22937638client # 32768, 98304, 163840, 22937639worker2 # 32768, 98304, 163840, 22937640lb # 41client # 42worker2 # 43lb # Allocating group tables: 0/8 done44client # Allocating group tables: 0/8 done45worker2 # Allocating group tables: 0/8 done46lb # Writing inode tables: 0/8 done47client # Writing inode tables: 0/8 done48worker2 # Writing inode tables: 0/8 done49lb # Creating journal (8192 blocks): done50client # Creating journal (8192 blocks): done51worker2 # Creating journal (8192 blocks): done52lb # Writing superblocks and filesystem accounting information: 0/8 done53client # Writing superblocks and filesystem accounting information: 0/8 done54worker2 # Writing superblocks and filesystem accounting information: 0/8 done55lb # 56client # 57worker2 # 58lb # Virtualisation disk image created.59client # Virtualisation disk image created.60worker2 # Virtualisation disk image created.61lb # Starting virtiofs daemons...62client # Starting virtiofs daemons...63worker2 # Starting virtiofs daemons...64lb # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)65client # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)66worker2 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67lb # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether68client # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether69worker2 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70lb # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...71client # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...72worker2 # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...73lb # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74client # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75worker2 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76lb # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77client # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether78worker2 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79lb # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80client # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...81worker2 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)82lb # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83client # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84worker2 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether85lb # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...86client # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87worker2 # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...88lb # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...89client # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...90worker2 # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...91lb # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests92client # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests93worker2 # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests94lb # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests95client # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests96worker2 # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests97lb # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests98client # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests99worker2 # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests100client: QEMU running (pid 50)101worker1 # Formatting '/build/vm-state-worker1/tmp.U00JDtIwEy', fmt=raw size=1073741824102worker1 # mke2fs 1.47.4 (6-Mar-2025)103worker1 # Discarding device blocks: 0/262144 done104worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes105worker1 # Filesystem UUID: 8172407d-5c40-4235-8937-fc8748328c0f106worker1 # Superblock backups stored on blocks:107worker1 # 32768, 98304, 163840, 229376108worker1 # 109(finished: start all VMs, in 0.45 seconds)110worker1 # Allocating group tables: 0/8 done111client: waiting for unit network-addresses-eth1.service112worker1 # Writing inode tables: 0/8 done113client: waiting for the VM to finish booting114worker1 # Creating journal (8192 blocks): done115worker1 # Writing superblocks and filesystem accounting information: 0/8 done116worker1 # 117worker1 # Virtualisation disk image created.118worker1 # Starting virtiofs daemons...119worker1 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120worker1 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121worker1 # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...122worker1 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123worker1 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124worker1 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)125worker1 # [2026-09-25T15:23:45Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether126worker1 # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...127worker1 # [2026-09-25T15:23:45Z INFO virtiofsd] Waiting for vhost-user socket connection...128worker1 # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests129worker1 # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests130worker1 # [2026-09-25T15:23:45Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]133worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026134lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026135worker1 # [ 0.000000] KASLR enabled136lb # [ 0.000000] KASLR enabled137worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]138worker1 # [ 0.000000] random: crng init done139client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]140lb # [ 0.000000] random: crng init done141worker1 # [ 0.000000] Machine model: linux,dummy-virt142lb # [ 0.000000] Machine model: linux,dummy-virt143worker1 # [ 0.000000] efi: UEFI not found.144lb # [ 0.000000] efi: UEFI not found.145worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026146worker2 # [ 0.000000] KASLR enabled147client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026148worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT149lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT150client # [ 0.000000] KASLR enabled151worker2 # [ 0.000000] random: crng init done152client # [ 0.000000] random: crng init done153worker2 # [ 0.000000] Machine model: linux,dummy-virt154worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]155lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]156worker2 # [ 0.000000] efi: UEFI not found.157client # [ 0.000000] Machine model: linux,dummy-virt158worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]159client # [ 0.000000] efi: UEFI not found.160lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]161worker1 # [ 0.000000] Zone ranges:162lb # [ 0.000000] Zone ranges:163worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT164client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT165worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]166lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]167worker1 # [ 0.000000] DMA32 empty168worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]169lb # [ 0.000000] DMA32 empty170worker1 # [ 0.000000] Normal empty171client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]172lb # [ 0.000000] Normal empty173worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]174worker1 # [ 0.000000] Device empty175lb # [ 0.000000] Device empty176worker2 # [ 0.000000] Zone ranges:177client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]178worker1 # [ 0.000000] Movable zone start for each node179lb # [ 0.000000] Movable zone start for each node180client # [ 0.000000] Zone ranges:181worker1 # [ 0.000000] Early memory node ranges182worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]183lb # [ 0.000000] Early memory node ranges184worker2 # [ 0.000000] DMA32 empty185client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]186worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]187worker2 # [ 0.000000] Normal empty188client # [ 0.000000] DMA32 empty189lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]190worker2 # [ 0.000000] Device empty191client # [ 0.000000] Normal empty192worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]193client # [ 0.000000] Device empty194worker2 # [ 0.000000] Movable zone start for each node195lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]196worker2 # [ 0.000000] Early memory node ranges197client # [ 0.000000] Movable zone start for each node198worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000199lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000200client # [ 0.000000] Early memory node ranges201worker1 # [ 0.000000] psci: probing for conduit method from DT.202worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]203lb # [ 0.000000] psci: probing for conduit method from DT.204client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]205worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.206lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.207worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]208worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs209client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]210lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs211worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000212worker1 # [ 0.000000] psci: Trusted OS migration not required213lb # [ 0.000000] psci: Trusted OS migration not required214client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000215worker2 # [ 0.000000] psci: probing for conduit method from DT.216worker1 # [ 0.000000] psci: SMC Calling Convention v1.1217lb # [ 0.000000] psci: SMC Calling Convention v1.1218client # [ 0.000000] psci: probing for conduit method from DT.219worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.220client # [ 0.000000] psci: PSCIv1.3 detected in firmware.221worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs222worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)223lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)224client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs225worker2 # [ 0.000000] psci: Trusted OS migration not required226worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296227client # [ 0.000000] psci: Trusted OS migration not required228lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296229worker2 # [ 0.000000] psci: SMC Calling Convention v1.1230worker1 # [ 0.000000] Detected PIPT I-cache on CPU0231client # [ 0.000000] psci: SMC Calling Convention v1.1232lb # [ 0.000000] Detected PIPT I-cache on CPU0233worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)234worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)235client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)236worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296237lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)238worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface239client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296240worker2 # [ 0.000000] Detected PIPT I-cache on CPU0241worker1 # [ 0.000000] CPU features: detected: Spectre-v4242lb # [ 0.000000] CPU features: detected: GICv3 CPU interface243client # [ 0.000000] Detected PIPT I-cache on CPU0244worker1 # [ 0.000000] CPU features: detected: Spectre-BHB245lb # [ 0.000000] CPU features: detected: Spectre-v4246worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)247lb # [ 0.000000] CPU features: detected: Spectre-BHB248worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38249client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)250worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface251lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38252worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23253client # [ 0.000000] CPU features: detected: GICv3 CPU interface254worker2 # [ 0.000000] CPU features: detected: Spectre-v4255client # [ 0.000000] CPU features: detected: Spectre-v4256worker1 # [ 0.000000] alternatives: applying boot alternatives257worker2 # [ 0.000000] CPU features: detected: Spectre-BHB258lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23259client # [ 0.000000] CPU features: detected: Spectre-BHB260lb # [ 0.000000] alternatives: applying boot alternatives261worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38262client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38263worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23264client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23265worker2 # [ 0.000000] alternatives: applying boot alternatives266client # [ 0.000000] alternatives: applying boot alternatives267worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/cw89fd0zkc3gaxrg6haynbv2xr4pxwgi-nixos-system-worker1-test/init regInfo=/nix/store/zym3rn263nhc2d17g7rlfr13br7bqpcz-closure-info/registration console=ttyAMA0,115200n8 console=tty0268lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/5hmvppg1fs8xkiccvl0m50n4dxjz92fr-nixos-system-lb-test/init regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration console=ttyAMA0,115200n8 console=tty0269worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pv8jv0ma9kiark5g4gx6k5bl6dq0g11a-nixos-system-worker2-test/init regInfo=/nix/store/vl8jijfa43vpyq80f023biwj91a44kaj-closure-info/registration console=ttyAMA0,115200n8 console=tty0270worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/zym3rn263nhc2d17g7rlfr13br7bqpcz-closure-info/registration", will be passed to user space.271lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/4s18yqhkjka9yjskv51fw7r9nw16dgfd-closure-info/registration", will be passed to user space.272client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/nwgxc9xy9a9l9j6z82df6cwp1rq7ly6z-nixos-system-client-test/init regInfo=/nix/store/nnlqyjr31s0109r7gpan7dzgxjg7rrvy-closure-info/registration console=ttyAMA0,115200n8 console=tty0273worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes274lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes275worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)276lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)277client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/nnlqyjr31s0109r7gpan7dzgxjg7rrvy-closure-info/registration", will be passed to user space.278worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/vl8jijfa43vpyq80f023biwj91a44kaj-closure-info/registration", will be passed to user space.279lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)280worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)281client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes282worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes283worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB284lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB285client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)286worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)287worker1 # [ 0.000000] software IO TLB: area num 1.288lb # [ 0.000000] software IO TLB: area num 1.289client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)290worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB291lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB292worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)293client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB294worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB295worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)296lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)297client # [ 0.000000] software IO TLB: area num 1.298worker2 # [ 0.000000] software IO TLB: area num 1.299worker1 # [ 0.000000] Fallback order for Node 0: 0300lb # [ 0.000000] Fallback order for Node 0: 0301client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB302worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB303lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216304worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216305worker1 # [ 0.000000] Policy zone: DMA306lb # [ 0.000000] Policy zone: DMA307client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)308worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)309client # [ 0.000000] Fallback order for Node 0: 0310lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off311worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off312worker2 # [ 0.000000] Fallback order for Node 0: 0313lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1314worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1315client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216316worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216317client # [ 0.000000] Policy zone: DMA318worker2 # [ 0.000000] Policy zone: DMA319lb # [ 0.000000] allocated 3145728 bytes of page_ext320worker1 # [ 0.000000] allocated 3145728 bytes of page_ext321worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages322lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages323client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off324worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off325lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups326worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups327client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1328worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1329lb # [ 0.000000] rcu: Hierarchical RCU implementation.330worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.331client # [ 0.000000] allocated 3145728 bytes of page_ext332worker2 # [ 0.000000] allocated 3145728 bytes of page_ext333worker1 # [ 0.000000] rcu: RCU event tracing is enabled.334lb # [ 0.000000] rcu: RCU event tracing is enabled.335client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages336worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages337lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.338worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.339client # [ 0.000000] ftrace: allocated 294 pages with 4 groups340worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups341worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.342lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.343client # [ 0.000000] rcu: Hierarchical RCU implementation.344worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.345lb # [ 0.000000] Rude variant of Tasks RCU enabled.346worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.347client # [ 0.000000] rcu: RCU event tracing is enabled.348worker2 # [ 0.000000] rcu: RCU event tracing is enabled.349lb # [ 0.000000] Tracing variant of Tasks RCU enabled.350worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.351client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.352worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.353client # [ 0.000000] Trampoline variant of Tasks RCU enabled.354lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.355worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.356worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.357client # [ 0.000000] Rude variant of Tasks RCU enabled.358worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.359lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1360worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1361client # [ 0.000000] Tracing variant of Tasks RCU enabled.362worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.363worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.364lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.365client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.366worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.367client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1368lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.369worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.370worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1371client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.372lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.373worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.374worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.375lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0376worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0377client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.378worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.379lb # [ 0.000000] GICv3: 256 SPIs implemented380worker1 # [ 0.000000] GICv3: 256 SPIs implemented381lb # [ 0.000000] GICv3: 0 Extended SPIs implemented382worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented383client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.384worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.385lb # [ 0.000000] Root IRQ handler: gic_handle_irq386worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq387client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0388lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI389worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI390worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0391client # [ 0.000000] GICv3: 256 SPIs implemented392worker2 # [ 0.000000] GICv3: 256 SPIs implemented393worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0394lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0395client # [ 0.000000] GICv3: 0 Extended SPIs implemented396worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented397lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000398client # [ 0.000000] Root IRQ handler: gic_handle_irq399worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000400worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq401lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]402worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]403client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI404worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI405client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0406worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0407lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)408worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)409client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000410worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000411client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]412worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]413lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)414worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)415lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000416worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000417client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)418worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)419lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000420worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000421client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)422worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)423lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.424worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.425client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000426worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000427lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns428worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns429worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000430client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000431lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).432worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).433client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.434worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.435lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns436worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns437worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns438client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns439worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).440client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).441lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns442worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns443lb # [ 0.000028] arm-pv: using stolen time PV444worker1 # [ 0.000028] arm-pv: using stolen time PV445worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns446client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns447lb # [ 0.000411] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)448worker1 # [ 0.000405] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)449worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns450client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns451lb # [ 0.000583] Console: colour dummy device 80x25452worker1 # [ 0.000592] Console: colour dummy device 80x25453client # [ 0.000030] arm-pv: using stolen time PV454worker2 # [ 0.000028] arm-pv: using stolen time PV455lb # [ 0.000590] printk: legacy console [tty0] enabled456worker1 # [ 0.000599] printk: legacy console [tty0] enabled457worker2 # [ 0.000407] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)458client # [ 0.000411] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)459lb # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)460worker1 # [ 0.000788] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)461client # [ 0.000588] Console: colour dummy device 80x25462worker2 # [ 0.000585] Console: colour dummy device 80x25463lb # [ 0.000781] pid_max: default: 32768 minimum: 301464worker1 # [ 0.000794] pid_max: default: 32768 minimum: 301465worker2 # [ 0.000593] printk: legacy console [tty0] enabled466client # [ 0.000596] printk: legacy console [tty0] enabled467lb # [ 0.000866] LSM: initializing lsm=capability,landlock,yama,bpf,ima468worker1 # [ 0.000864] LSM: initializing lsm=capability,landlock,yama,bpf,ima469lb # [ 0.000996] landlock: Up and running.470worker1 # [ 0.000989] landlock: Up and running.471worker2 # [ 0.000785] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)472lb # [ 0.000998] Yama: becoming mindful.473client # [ 0.000789] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)474worker1 # [ 0.000992] Yama: becoming mindful.475lb # [ 0.001445] LSM support for eBPF active476worker2 # [ 0.000791] pid_max: default: 32768 minimum: 301477client # [ 0.000795] pid_max: default: 32768 minimum: 301478worker1 # [ 0.001422] LSM support for eBPF active479worker2 # [ 0.000877] LSM: initializing lsm=capability,landlock,yama,bpf,ima480lb # [ 0.001588] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)481client # [ 0.000867] LSM: initializing lsm=capability,landlock,yama,bpf,ima482worker1 # [ 0.001612] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)483worker2 # [ 0.001012] landlock: Up and running.484client # [ 0.001012] landlock: Up and running.485worker2 # [ 0.001014] Yama: becoming mindful.486client # [ 0.001015] Yama: becoming mindful.487lb # [ 0.001674] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)488worker1 # [ 0.001652] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)489worker2 # [ 0.001467] LSM support for eBPF active490client # [ 0.001473] LSM support for eBPF active491lb # [ 0.003524] rcu: Hierarchical SRCU implementation.492worker1 # [ 0.003458] rcu: Hierarchical SRCU implementation.493lb # [ 0.003529] rcu: Max phase no-delay instances is 1000.494worker2 # [ 0.001604] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)495worker1 # [ 0.003463] rcu: Max phase no-delay instances is 1000.496client # [ 0.001613] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)497lb # [ 0.004801] fsl-mc MSI: its@8080000 domain created498worker1 # [ 0.004710] fsl-mc MSI: its@8080000 domain created499worker2 # [ 0.001642] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)500client # [ 0.001652] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)501worker1 # [ 0.004795] EFI services will not be available.502worker2 # [ 0.003521] rcu: Hierarchical SRCU implementation.503client # [ 0.003599] rcu: Hierarchical SRCU implementation.504worker1 # [ 0.004866] smp: Bringing up secondary CPUs ...505client # [ 0.003604] rcu: Max phase no-delay instances is 1000.506worker2 # [ 0.003525] rcu: Max phase no-delay instances is 1000.507worker1 # [ 0.004874] smp: Brought up 1 node, 1 CPU508client # [ 0.004861] fsl-mc MSI: its@8080000 domain created509worker2 # [ 0.004770] fsl-mc MSI: its@8080000 domain created510worker1 # [ 0.004877] SMP: Total of 1 processors activated.511client # [ 0.004950] EFI services will not be available.512worker2 # [ 0.004857] EFI services will not be available.513worker1 # [ 0.004882] CPU: All CPU(s) started at EL1514client # [ 0.005034] smp: Bringing up secondary CPUs ...515worker2 # [ 0.004921] smp: Bringing up secondary CPUs ...516worker1 # [ 0.004892] CPU features: detected: Branch Target Identification517worker2 # [ 0.004929] smp: Brought up 1 node, 1 CPU518client # [ 0.005042] smp: Brought up 1 node, 1 CPU519worker1 # [ 0.004897] CPU features: detected: ARMv8.4 Translation Table Level520client # [ 0.005045] SMP: Total of 1 processors activated.521worker2 # [ 0.004932] SMP: Total of 1 processors activated.522client # [ 0.005050] CPU: All CPU(s) started at EL1523worker2 # [ 0.004938] CPU: All CPU(s) started at EL1524worker1 # [ 0.004900] CPU features: detected: Instruction cache invalidation not required for I/D coherence525client # [ 0.005060] CPU features: detected: Branch Target Identification526worker2 # [ 0.004948] CPU features: detected: Branch Target Identification527worker2 # [ 0.004952] CPU features: detected: ARMv8.4 Translation Table Level528worker1 # [ 0.004904] CPU features: detected: Data cache clean to the PoU not required for I/D coherence529client # [ 0.005065] CPU features: detected: ARMv8.4 Translation Table Level530worker1 # [ 0.004907] CPU features: detected: Common not Private translations531worker2 # [ 0.004955] CPU features: detected: Instruction cache invalidation not required for I/D coherence532client # [ 0.005068] CPU features: detected: Instruction cache invalidation not required for I/D coherence533worker1 # [ 0.004910] CPU features: detected: CRC32 instructions534lb # [ 0.004896] EFI services will not be available.535worker2 # [ 0.004959] CPU features: detected: Data cache clean to the PoU not required for I/D coherence536client # [ 0.005072] CPU features: detected: Data cache clean to the PoU not required for I/D coherence537worker1 # [ 0.004913] CPU features: detected: Data cache clean to Point of Deep Persistence538lb # [ 0.004982] smp: Bringing up secondary CPUs ...539worker2 # [ 0.004962] CPU features: detected: Common not Private translations540client # [ 0.005075] CPU features: detected: Common not Private translations541lb # [ 0.004990] smp: Brought up 1 node, 1 CPU542client # [ 0.005079] CPU features: detected: CRC32 instructions543lb # [ 0.004993] SMP: Total of 1 processors activated.544lb # [ 0.004998] CPU: All CPU(s) started at EL1545client # [ 0.005082] CPU features: detected: Data cache clean to Point of Deep Persistence546lb # [ 0.005008] CPU features: detected: Branch Target Identification547client # [ 0.005085] CPU features: detected: Data cache clean to Point of Persistence548lb # [ 0.005012] CPU features: detected: ARMv8.4 Translation Table Level549client # [ 0.005088] CPU features: detected: Data independent timing control (DIT)550lb # [ 0.005015] CPU features: detected: Instruction cache invalidation not required for I/D coherence551client # [ 0.005091] CPU features: detected: E0PD552client # [ 0.005094] CPU features: detected: Enhanced Counter Virtualization553lb # [ 0.005019] CPU features: detected: Data cache clean to the PoU not required for I/D coherence554client # [ 0.005097] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)555lb # [ 0.005023] CPU features: detected: Common not Private translations556lb # [ 0.005026] CPU features: detected: CRC32 instructions557client # [ 0.005101] CPU features: detected: Enhanced Virtualization Traps558client # [ 0.005104] CPU features: detected: Fine Grained Traps559lb # [ 0.005029] CPU features: detected: Data cache clean to Point of Deep Persistence560client # [ 0.005107] CPU features: detected: Generic authentication (architected QARMA5 algorithm)561lb # [ 0.005033] CPU features: detected: Data cache clean to Point of Persistence562worker1 # [ 0.004916] CPU features: detected: Data cache clean to Point of Persistence563worker2 # [ 0.004966] CPU features: detected: CRC32 instructions564client # [ 0.005112] CPU features: detected: RCpc load-acquire (LDAPR)565lb # [ 0.005036] CPU features: detected: Data independent timing control (DIT)566worker1 # [ 0.004919] CPU features: detected: Data independent timing control (DIT)567client # [ 0.005115] CPU features: detected: LSE atomic instructions568lb # [ 0.005039] CPU features: detected: E0PD569worker2 # [ 0.004968] CPU features: detected: Data cache clean to Point of Deep Persistence570worker1 # [ 0.004922] CPU features: detected: E0PD571client # [ 0.005118] CPU features: detected: Privileged Access Never572lb # [ 0.005041] CPU features: detected: Enhanced Counter Virtualization573worker2 # [ 0.004972] CPU features: detected: Data cache clean to Point of Persistence574worker1 # [ 0.004925] CPU features: detected: Enhanced Counter Virtualization575client # [ 0.005120] CPU features: detected: PMUv3576lb # [ 0.005044] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)577worker2 # [ 0.004975] CPU features: detected: Data independent timing control (DIT)578worker1 # [ 0.004928] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)579client # [ 0.005123] CPU features: detected: RAS Extension Support580worker2 # [ 0.004978] CPU features: detected: E0PD581lb # [ 0.005048] CPU features: detected: Enhanced Virtualization Traps582client # [ 0.005126] CPU features: detected: RASv1p1 Extension Support583worker1 # [ 0.004932] CPU features: detected: Enhanced Virtualization Traps584worker2 # [ 0.004981] CPU features: detected: Enhanced Counter Virtualization585lb # [ 0.005051] CPU features: detected: Fine Grained Traps586client # [ 0.005129] CPU features: detected: Random Number Generator587worker1 # [ 0.004934] CPU features: detected: Fine Grained Traps588client # [ 0.005131] CPU features: detected: Speculation barrier (SB)589worker2 # [ 0.004984] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)590lb # [ 0.005054] CPU features: detected: Generic authentication (architected QARMA5 algorithm)591worker1 # [ 0.004938] CPU features: detected: Generic authentication (architected QARMA5 algorithm)592worker2 # [ 0.004988] CPU features: detected: Enhanced Virtualization Traps593client # [ 0.005134] CPU features: detected: Stage-2 Force Write-Back594lb # [ 0.005059] CPU features: detected: RCpc load-acquire (LDAPR)595worker1 # [ 0.004943] CPU features: detected: RCpc load-acquire (LDAPR)596worker2 # [ 0.004991] CPU features: detected: Fine Grained Traps597lb # [ 0.005062] CPU features: detected: LSE atomic instructions598client # [ 0.005137] CPU features: detected: TLB range maintenance instructions599worker1 # [ 0.004945] CPU features: detected: LSE atomic instructions600lb # [ 0.005065] CPU features: detected: Privileged Access Never601worker2 # [ 0.004994] CPU features: detected: Generic authentication (architected QARMA5 algorithm)602client # [ 0.005141] CPU features: detected: Speculative Store Bypassing Safe (SSBS)603worker1 # [ 0.004948] CPU features: detected: Privileged Access Never604lb # [ 0.005067] CPU features: detected: PMUv3605worker1 # [ 0.004951] CPU features: detected: PMUv3606worker2 # [ 0.004999] CPU features: detected: RCpc load-acquire (LDAPR)607client # [ 0.005180] alternatives: applying system-wide alternatives608lb # [ 0.005071] CPU features: detected: RAS Extension Support609worker1 # [ 0.004954] CPU features: detected: RAS Extension Support610worker2 # [ 0.005002] CPU features: detected: LSE atomic instructions611client # [ 0.008084] CPU features: detected: BBM Level 2 without TLB conflict abort612lb # [ 0.005073] CPU features: detected: RASv1p1 Extension Support613worker1 # [ 0.004957] CPU features: detected: RASv1p1 Extension Support614worker2 # [ 0.005005] CPU features: detected: Privileged Access Never615lb # [ 0.005076] CPU features: detected: Random Number Generator616worker1 # [ 0.004959] CPU features: detected: Random Number Generator617worker2 # [ 0.005007] CPU features: detected: PMUv3618lb # [ 0.005079] CPU features: detected: Speculation barrier (SB)619worker2 # [ 0.005010] CPU features: detected: RAS Extension Support620worker1 # [ 0.004962] CPU features: detected: Speculation barrier (SB)621client # [ 0.008239] Memory: 1405644K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)622lb # [ 0.005081] CPU features: detected: Stage-2 Force Write-Back623worker1 # [ 0.004964] CPU features: detected: Stage-2 Force Write-Back624worker2 # [ 0.005013] CPU features: detected: RASv1p1 Extension Support625client # [ 0.008585] devtmpfs: initialized626worker2 # [ 0.005016] CPU features: detected: Random Number Generator627lb # [ 0.005084] CPU features: detected: TLB range maintenance instructions628worker1 # [ 0.004967] CPU features: detected: TLB range maintenance instructions629client # [ 0.010427] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)630worker2 # [ 0.005019] CPU features: detected: Speculation barrier (SB)631lb # [ 0.005089] CPU features: detected: Speculative Store Bypassing Safe (SSBS)632worker1 # [ 0.004972] CPU features: detected: Speculative Store Bypassing Safe (SSBS)633worker2 # [ 0.005021] CPU features: detected: Stage-2 Force Write-Back634lb # [ 0.005125] alternatives: applying system-wide alternatives635client # [ 0.010451] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).636worker1 # [ 0.005009] alternatives: applying system-wide alternatives637worker2 # [ 0.005024] CPU features: detected: TLB range maintenance instructions638client # [ 0.010651] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL639lb # [ 0.007929] CPU features: detected: BBM Level 2 without TLB conflict abort640worker1 # [ 0.007806] CPU features: detected: BBM Level 2 without TLB conflict abort641client # [ 0.010655] 0 pages in range for non-PLT usage642worker2 # [ 0.005029] CPU features: detected: Speculative Store Bypassing Safe (SSBS)643client # [ 0.010656] 508272 pages in range for PLT usage644worker2 # [ 0.005067] alternatives: applying system-wide alternatives645lb # [ 0.008092] Memory: 1405660K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)646client # [ 0.010752] pinctrl core: initialized pinctrl subsystem647worker1 # [ 0.007985] Memory: 1405620K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)648lb # [ 0.008434] devtmpfs: initialized649client # [ 0.011518] DMI not present or invalid.650worker2 # [ 0.007892] CPU features: detected: BBM Level 2 without TLB conflict abort651worker1 # [ 0.008374] devtmpfs: initialized652client # [ 0.014973] NET: Registered PF_NETLINK/PF_ROUTE protocol family653lb # [ 0.010288] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)654worker1 # [ 0.010198] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)655client # [ 0.017348] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations656worker2 # [ 0.008073] Memory: 1405668K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)657lb # [ 0.010311] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).658worker1 # [ 0.010221] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).659worker2 # [ 0.008477] devtmpfs: initialized660client # [ 0.017513] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations661lb # [ 0.010474] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL662worker1 # [ 0.010397] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL663worker2 # [ 0.010302] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)664lb # [ 0.010478] 0 pages in range for non-PLT usage665worker1 # [ 0.010401] 0 pages in range for non-PLT usage666client # [ 0.017706] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations667lb # [ 0.010479] 508272 pages in range for PLT usage668worker1 # [ 0.010403] 508272 pages in range for PLT usage669worker2 # [ 0.010324] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).670client # [ 0.017726] audit: initializing netlink subsys (disabled)671lb # [ 0.010591] pinctrl core: initialized pinctrl subsystem672worker1 # [ 0.010506] pinctrl core: initialized pinctrl subsystem673lb # [ 0.011368] DMI not present or invalid.674worker2 # [ 0.010497] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL675client # [ 0.018309] thermal_sys: Registered thermal governor 'fair_share'676worker1 # [ 0.011238] DMI not present or invalid.677worker2 # [ 0.010501] 0 pages in range for non-PLT usage678lb # [ 0.014908] NET: Registered PF_NETLINK/PF_ROUTE protocol family679client # [ 0.018311] thermal_sys: Registered thermal governor 'bang_bang'680worker1 # [ 0.014804] NET: Registered PF_NETLINK/PF_ROUTE protocol family681worker2 # [ 0.010502] 508272 pages in range for PLT usage682lb # [ 0.017211] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations683client # [ 0.018314] thermal_sys: Registered thermal governor 'step_wise'684worker1 # [ 0.017068] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations685worker2 # [ 0.010608] pinctrl core: initialized pinctrl subsystem686worker2 # [ 0.011411] DMI not present or invalid.687client # [ 0.018317] thermal_sys: Registered thermal governor 'user_space'688lb # [ 0.017378] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations689worker1 # [ 0.017238] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations690worker2 # [ 0.014872] NET: Registered PF_NETLINK/PF_ROUTE protocol family691client # [ 0.018323] thermal_sys: Registered thermal governor 'power_allocator'692lb # [ 0.017572] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations693worker1 # [ 0.017430] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations694worker2 # [ 0.017120] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations695client # [ 0.018347] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1696lb # [ 0.017592] audit: initializing netlink subsys (disabled)697worker1 # [ 0.017449] audit: initializing netlink subsys (disabled)698client # [ 0.018355] cpuidle: using governor ladder699worker2 # [ 0.017290] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations700lb # [ 0.018204] thermal_sys: Registered thermal governor 'fair_share'701worker1 # [ 0.018073] thermal_sys: Registered thermal governor 'fair_share'702client # [ 0.018361] cpuidle: using governor menu703lb # [ 0.018206] thermal_sys: Registered thermal governor 'bang_bang'704worker1 # [ 0.018075] thermal_sys: Registered thermal governor 'bang_bang'705worker2 # [ 0.017483] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations706client # [ 0.018553] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.707lb # [ 0.018209] thermal_sys: Registered thermal governor 'step_wise'708worker1 # [ 0.018079] thermal_sys: Registered thermal governor 'step_wise'709worker2 # [ 0.017504] audit: initializing netlink subsys (disabled)710client # [ 0.018567] ASID allocator initialised with 65536 entries711lb # [ 0.018212] thermal_sys: Registered thermal governor 'user_space'712client # [ 0.019783] Serial: AMBA PL011 UART driver713worker1 # [ 0.018082] thermal_sys: Registered thermal governor 'user_space'714worker2 # [ 0.018039] thermal_sys: Registered thermal governor 'fair_share'715lb # [ 0.018217] thermal_sys: Registered thermal governor 'power_allocator'716worker2 # [ 0.018041] thermal_sys: Registered thermal governor 'bang_bang'717worker1 # [ 0.018087] thermal_sys: Registered thermal governor 'power_allocator'718client # [ 0.024937] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1719client # [ 0.025058] printk: console [ttyAMA0] enabled720worker2 # [ 0.018044] thermal_sys: Registered thermal governor 'step_wise'721lb # [ 0.018240] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1722worker1 # [ 0.018109] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1723lb # [ 0.018250] cpuidle: using governor ladder724worker2 # [ 0.018047] thermal_sys: Registered thermal governor 'user_space'725worker1 # [ 0.018118] cpuidle: using governor ladder726lb # [ 0.018255] cpuidle: using governor menu727worker1 # [ 0.018124] cpuidle: using governor menu728worker2 # [ 0.018052] thermal_sys: Registered thermal governor 'power_allocator'729lb # [ 0.018433] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.730worker1 # [ 0.018309] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.731worker2 # [ 0.018081] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1732lb # [ 0.018447] ASID allocator initialised with 65536 entries733worker1 # [ 0.018323] ASID allocator initialised with 65536 entries734worker2 # [ 0.018092] cpuidle: using governor ladder735lb # [ 0.019655] Serial: AMBA PL011 UART driver736worker1 # [ 0.019510] Serial: AMBA PL011 UART driver737worker2 # [ 0.018097] cpuidle: using governor menu738lb # [ 0.024872] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1739worker2 # [ 0.018282] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.740worker1 # [ 0.024621] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1741lb # [ 0.025029] printk: console [ttyAMA0] enabled742worker1 # [ 0.024765] printk: console [ttyAMA0] enabled743worker2 # [ 0.018298] ASID allocator initialised with 65536 entries744worker2 # [ 0.019544] Serial: AMBA PL011 UART driver745lb # [ 0.149291] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages746worker1 # [ 0.149398] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages747worker2 # [ 0.024722] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1748lb # [ 0.149307] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page749worker1 # [ 0.149414] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page750worker2 # [ 0.024838] printk: console [ttyAMA0] enabled751client # [ 0.147937] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages752lb # [ 0.149312] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages753worker1 # [ 0.149419] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages754client # [ 0.147952] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page755lb # [ 0.149317] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page756worker2 # [ 0.149439] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages757worker1 # [ 0.149424] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page758client # [ 0.147957] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages759lb # [ 0.149322] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages760worker2 # [ 0.149453] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page761worker1 # [ 0.149429] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages762client # [ 0.147961] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page763worker2 # [ 0.149458] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages764lb # [ 0.149326] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page765worker1 # [ 0.149433] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page766client # [ 0.147966] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages767worker2 # [ 0.149463] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page768lb # [ 0.149330] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages769worker1 # [ 0.149437] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages770client # [ 0.147970] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page771lb # [ 0.149335] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page772worker2 # [ 0.149467] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages773worker1 # [ 0.149442] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page774client # [ 0.147974] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages775worker2 # [ 0.149472] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page776client # [ 0.147979] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page777worker2 # [ 0.149476] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages778lb # [ 0.156760] fbcon: Taking over console779worker2 # [ 0.149480] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page780worker1 # [ 0.156888] fbcon: Taking over console781lb # [ 0.156773] ACPI: Interpreter disabled.782worker1 # [ 0.156900] ACPI: Interpreter disabled.783client # [ 0.155495] fbcon: Taking over console784client # [ 0.155509] ACPI: Interpreter disabled.785worker2 # [ 0.156974] fbcon: Taking over console786worker2 # [ 0.156987] ACPI: Interpreter disabled.787lb # [ 0.164725] iommu: Default domain type: Translated788worker1 # [ 0.164823] iommu: Default domain type: Translated789lb # [ 0.164734] iommu: DMA domain TLB invalidation policy: strict mode790worker1 # [ 0.164833] iommu: DMA domain TLB invalidation policy: strict mode791lb # [ 0.166449] SCSI subsystem initialized792worker1 # [ 0.165278] SCSI subsystem initialized793client # [ 0.164204] iommu: Default domain type: Translated794worker2 # [ 0.164822] iommu: Default domain type: Translated795lb # [ 0.167158] usbcore: registered new interface driver usbfs796client # [ 0.164214] iommu: DMA domain TLB invalidation policy: strict mode797worker2 # [ 0.164830] iommu: DMA domain TLB invalidation policy: strict mode798worker1 # [ 0.167310] usbcore: registered new interface driver usbfs799client # [ 0.164549] SCSI subsystem initialized800lb # [ 0.167196] usbcore: registered new interface driver hub801worker1 # [ 0.167341] usbcore: registered new interface driver hub802worker2 # [ 0.166528] SCSI subsystem initialized803lb # [ 0.167211] usbcore: registered new device driver usb804worker1 # [ 0.167357] usbcore: registered new device driver usb805client # [ 0.166588] usbcore: registered new interface driver usbfs806lb # [ 0.167477] pps_core: LinuxPPS API ver. 1 registered807worker1 # [ 0.167610] pps_core: LinuxPPS API ver. 1 registered808worker2 # [ 0.167204] usbcore: registered new interface driver usbfs809client # [ 0.166633] usbcore: registered new interface driver hub810worker2 # [ 0.167246] usbcore: registered new interface driver hub811lb # [ 0.167484] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>812client # [ 0.166650] usbcore: registered new device driver usb813worker1 # [ 0.167616] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>814lb # [ 0.167494] PTP clock support registered815worker2 # [ 0.167261] usbcore: registered new device driver usb816worker1 # [ 0.167625] PTP clock support registered817client # [ 0.166914] pps_core: LinuxPPS API ver. 1 registered818lb # [ 0.167541] EDAC MC: Ver: 3.0.0819worker1 # [ 0.167673] EDAC MC: Ver: 3.0.0820worker2 # [ 0.167517] pps_core: LinuxPPS API ver. 1 registered821lb # [ 0.172258] scmi_core: SCMI protocol bus registered822client # [ 0.166920] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>823worker1 # [ 0.172389] scmi_core: SCMI protocol bus registered824worker2 # [ 0.167523] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>825client # [ 0.166931] PTP clock support registered826worker2 # [ 0.167533] PTP clock support registered827client # [ 0.166978] EDAC MC: Ver: 3.0.0828lb # [ 0.173244] FPGA manager framework829worker1 # [ 0.173378] FPGA manager framework830worker2 # [ 0.167588] EDAC MC: Ver: 3.0.0831lb # [ 0.174182] vgaarb: loaded832worker1 # [ 0.174287] vgaarb: loaded833client # [ 0.171763] scmi_core: SCMI protocol bus registered834worker2 # [ 0.172315] scmi_core: SCMI protocol bus registered835lb # [ 0.174798] clocksource: Switched to clocksource arch_sys_counter836worker1 # [ 0.174900] clocksource: Switched to clocksource arch_sys_counter837client # [ 0.172760] FPGA manager framework838worker2 # [ 0.173334] FPGA manager framework839client # [ 0.173710] vgaarb: loaded840worker2 # [ 0.174276] vgaarb: loaded841client # [ 0.174325] clocksource: Switched to clocksource arch_sys_counter842lb # [ 0.177067] VFS: Disk quotas dquot_6.6.0843worker1 # [ 0.176995] VFS: Disk quotas dquot_6.6.0844worker2 # [ 0.174905] clocksource: Switched to clocksource arch_sys_counter845lb # [ 0.177101] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)846worker1 # [ 0.177030] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)847worker2 # [ 0.177090] VFS: Disk quotas dquot_6.6.0848worker2 # [ 0.177125] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)849client # [ 0.177989] VFS: Disk quotas dquot_6.6.0850lb # [ 0.180800] netfs: FS-Cache loaded851worker1 # [ 0.180728] netfs: FS-Cache loaded852lb # [ 0.180918] pnp: PnP ACPI: disabled853worker1 # [ 0.180841] pnp: PnP ACPI: disabled854client # [ 0.178023] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)855worker2 # [ 0.180714] netfs: FS-Cache loaded856worker2 # [ 0.180827] pnp: PnP ACPI: disabled857client # [ 0.181641] netfs: FS-Cache loaded858client # [ 0.181764] pnp: PnP ACPI: disabled859lb # [ 0.184838] NET: Registered PF_INET protocol family860worker1 # [ 0.184632] NET: Registered PF_INET protocol family861lb # [ 0.185112] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)862worker1 # [ 0.184899] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)863client # [ 0.185615] NET: Registered PF_INET protocol family864worker2 # [ 0.184711] NET: Registered PF_INET protocol family865client # [ 0.185885] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)866worker2 # [ 0.184980] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)867lb # [ 0.214007] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)868worker1 # [ 0.213530] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)869lb # [ 0.214045] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)870worker1 # [ 0.213567] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)871lb # [ 0.214070] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)872client # [ 0.214548] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)873worker1 # [ 0.213591] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)874worker2 # [ 0.213974] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)875lb # [ 0.214138] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)876client # [ 0.214586] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)877worker1 # [ 0.213661] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)878worker2 # [ 0.214014] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)879lb # [ 0.214280] TCP: Hash tables configured (established 16384 bind 16384)880worker1 # [ 0.213804] TCP: Hash tables configured (established 16384 bind 16384)881client # [ 0.214611] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)882worker2 # [ 0.214040] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)883lb # [ 0.214371] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)884worker1 # [ 0.213875] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)885client # [ 0.214682] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)886worker2 # [ 0.214113] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)887lb # [ 0.214407] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)888worker1 # [ 0.213907] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)889client # [ 0.214825] TCP: Hash tables configured (established 16384 bind 16384)890worker2 # [ 0.214256] TCP: Hash tables configured (established 16384 bind 16384)891lb # [ 0.214443] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)892worker1 # [ 0.213943] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)893client # [ 0.214901] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)894worker2 # [ 0.214328] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)895lb # [ 0.214531] NET: Registered PF_UNIX/PF_LOCAL protocol family896worker1 # [ 0.214027] NET: Registered PF_UNIX/PF_LOCAL protocol family897client # [ 0.214935] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)898lb # [ 0.214557] NET: Registered PF_XDP protocol family899worker2 # [ 0.214361] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker1 # [ 0.214052] NET: Registered PF_XDP protocol family901lb # [ 0.214574] PCI: CLS 0 bytes, default 64902client # [ 0.214971] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)903worker1 # [ 0.214069] PCI: CLS 0 bytes, default 64904worker2 # [ 0.214396] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)905lb # [ 0.214789] Trying to unpack rootfs image as initramfs...906client # [ 0.215069] NET: Registered PF_UNIX/PF_LOCAL protocol family907worker1 # [ 0.214294] Trying to unpack rootfs image as initramfs...908worker2 # [ 0.214497] NET: Registered PF_UNIX/PF_LOCAL protocol family909client # [ 0.215138] NET: Registered PF_XDP protocol family910worker2 # [ 0.214521] NET: Registered PF_XDP protocol family911client # [ 0.215155] PCI: CLS 0 bytes, default 64912worker2 # [ 0.214537] PCI: CLS 0 bytes, default 64913client # [ 0.215378] Trying to unpack rootfs image as initramfs...914worker2 # [ 0.214749] Trying to unpack rootfs image as initramfs...915lb # [ 0.232588] kvm [1]: HYP mode not available916worker1 # [ 0.232745] kvm [1]: HYP mode not available917client # [ 0.232237] kvm [1]: HYP mode not available918worker2 # [ 0.232758] kvm [1]: HYP mode not available919worker1 # [ 0.316009] Initialise system trusted keyrings920lb # [ 0.316336] Initialise system trusted keyrings921worker1 # [ 0.316764] workingset: timestamp_bits=42 max_order=19 bucket_order=0922lb # [ 0.317081] workingset: timestamp_bits=42 max_order=19 bucket_order=0923worker1 # [ 0.318058] squashfs: version 4.0 (2009/01/31) Phillip Lougher924lb # [ 0.318345] squashfs: version 4.0 (2009/01/31) Phillip Lougher925worker1 # [ 0.318855] 9p: Installing v9fs 9p2000 file system support926worker2 # [ 0.316764] Initialise system trusted keyrings927worker2 # [ 0.317520] workingset: timestamp_bits=42 max_order=19 bucket_order=0928worker2 # [ 0.318816] squashfs: version 4.0 (2009/01/31) Phillip Lougher929lb # [ 0.319192] 9p: Installing v9fs 9p2000 file system support930client # [ 0.322861] Initialise system trusted keyrings931client # [ 0.323681] workingset: timestamp_bits=42 max_order=19 bucket_order=0932client # [ 0.325003] squashfs: version 4.0 (2009/01/31) Phillip Lougher933client # [ 0.325805] 9p: Installing v9fs 9p2000 file system support934worker2 # [ 0.326939] 9p: Installing v9fs 9p2000 file system support935lb # [ 0.347307] Key type asymmetric registered936worker1 # [ 0.347641] Key type asymmetric registered937lb # [ 0.347328] Asymmetric key parser 'x509' registered938worker1 # [ 0.347663] Asymmetric key parser 'x509' registered939lb # [ 0.347390] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)940worker1 # [ 0.347724] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)941worker2 # [ 0.346963] Key type asymmetric registered942lb # [ 0.349507] io scheduler mq-deadline registered943worker2 # [ 0.346979] Asymmetric key parser 'x509' registered944worker1 # [ 0.349896] io scheduler mq-deadline registered945lb # [ 0.349518] io scheduler kyber registered946worker1 # [ 0.349907] io scheduler kyber registered947worker2 # [ 0.347033] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)948worker2 # [ 0.349358] io scheduler mq-deadline registered949worker2 # [ 0.349370] io scheduler kyber registered950client # [ 0.354491] Key type asymmetric registered951client # [ 0.354515] Asymmetric key parser 'x509' registered952client # [ 0.354574] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)953client # [ 0.356729] io scheduler mq-deadline registered954client # [ 0.356741] io scheduler kyber registered955lb # [ 0.358942] pl061_gpio 9030000.pl061: PL061 GPIO chip registered956worker1 # [ 0.359036] pl061_gpio 9030000.pl061: PL061 GPIO chip registered957lb # [ 0.360330] ledtrig-cpu: registered to indicate activity on CPUs958lb # [ 0.360702] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:959lb # [ 0.360720] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000960lb # [ 0.360732] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000961worker1 # [ 0.360425] ledtrig-cpu: registered to indicate activity on CPUs962lb # [ 0.360741] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000963worker1 # [ 0.360787] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:964lb # [ 0.360762] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits965worker1 # [ 0.360805] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000966lb # [ 0.360793] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]967client # [ 0.366436] pl061_gpio 9030000.pl061: PL061 GPIO chip registered968worker1 # [ 0.360825] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000969lb # [ 0.361224] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00970lb # [ 0.361235] pci_bus 0000:00: root bus resource [bus 00-ff]971worker1 # [ 0.360835] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000972lb # [ 0.361242] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]973worker1 # [ 0.360855] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits974lb # [ 0.361247] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]975client # [ 0.367843] ledtrig-cpu: registered to indicate activity on CPUs976worker2 # [ 0.363067] pl061_gpio 9030000.pl061: PL061 GPIO chip registered977lb # [ 0.361253] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]978worker1 # [ 0.360878] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]979client # [ 0.368201] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:980worker2 # [ 0.363690] ledtrig-cpu: registered to indicate activity on CPUs981worker1 # [ 0.360960] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00982lb # [ 0.361306] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint983client # [ 0.368218] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000984worker2 # [ 0.364056] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:985worker1 # [ 0.360970] pci_bus 0000:00: root bus resource [bus 00-ff]986lb # [ 0.361814] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint987worker1 # [ 0.360976] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]988worker2 # [ 0.364073] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000989client # [ 0.368230] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000990lb # [ 0.362004] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]991worker1 # [ 0.360981] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]992lb # [ 0.362021] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]993worker2 # [ 0.364086] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000994client # [ 0.368239] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000995worker1 # [ 0.360986] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]996lb # [ 0.362051] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]997client # [ 0.368260] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits998worker2 # [ 0.364094] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000999lb # [ 0.362067] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1000worker1 # [ 0.361041] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1001worker2 # [ 0.364114] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1002client # [ 0.368283] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1003lb # [ 0.362530] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1004worker1 # [ 0.361489] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1005client # [ 0.368389] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001006lb # [ 0.362717] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1007worker1 # [ 0.361763] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1008worker2 # [ 0.364138] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1009client # [ 0.368399] pci_bus 0000:00: root bus resource [bus 00-ff]1010lb # [ 0.362733] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1011worker1 # [ 0.361781] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1012worker2 # [ 0.364209] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001013client # [ 0.368404] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1014lb # [ 0.362763] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1015worker1 # [ 0.361812] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1016worker2 # [ 0.364218] pci_bus 0000:00: root bus resource [bus 00-ff]1017client # [ 0.368409] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1018worker1 # [ 0.361829] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1019worker2 # [ 0.364224] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1020client # [ 0.368414] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1021worker2 # [ 0.364230] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1022worker1 # [ 0.362313] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1023worker1 # [ 0.362503] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1024worker2 # [ 0.364235] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1025client # [ 0.368468] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1026worker1 # [ 0.362519] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1027client # [ 0.368916] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1028worker2 # [ 0.364339] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1029worker1 # [ 0.362549] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1030client # [ 0.369110] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1031worker2 # [ 0.364788] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1032worker1 # [ 0.363037] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1033client # [ 0.369127] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1034worker2 # [ 0.364979] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1035worker1 # [ 0.363228] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1036client # [ 0.369158] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037worker2 # [ 0.364996] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1038worker1 # [ 0.363244] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1039client # [ 0.369174] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1040worker2 # [ 0.365026] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1041worker1 # [ 0.363274] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1042worker2 # [ 0.365042] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1043client # [ 0.369647] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1044worker1 # [ 0.363741] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1045client # [ 0.369837] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1046worker2 # [ 0.365508] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1047worker1 # [ 0.363932] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1048client # [ 0.369853] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1049worker2 # [ 0.365694] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1050worker1 # [ 0.363948] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1051client # [ 0.369883] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052worker2 # [ 0.365710] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1053worker1 # [ 0.363978] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1054worker2 # [ 0.365740] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1055client # [ 0.390128] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1056worker1 # [ 0.363997] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1057client # [ 0.390321] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1058worker2 # [ 0.366204] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1059worker1 # [ 0.364473] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1060worker2 # [ 0.366389] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1061worker1 # [ 0.364667] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1062lb # [ 0.387289] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1063worker2 # [ 0.366405] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1064worker1 # [ 0.364698] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065lb # [ 0.387500] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1066worker2 # [ 0.366435] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1067lb # [ 0.387517] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1068client # [ 0.392033] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker1 # [ 0.365168] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1070worker2 # [ 0.366894] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1071lb # [ 0.387547] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1072worker1 # [ 0.365363] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1073client # [ 0.392063] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1074worker2 # [ 0.367102] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1075worker1 # [ 0.365393] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1076lb # [ 0.388011] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1077worker2 # [ 0.367118] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1078client # [ 0.392555] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1079lb # [ 0.388201] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1080worker2 # [ 0.367148] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1081client # [ 0.392747] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1082worker1 # [ 0.365817] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1083lb # [ 0.388217] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1084worker2 # [ 0.367166] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1085worker1 # [ 0.366006] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1086client # [ 0.392763] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1087lb # [ 0.388247] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1088client # [ 0.392793] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089worker2 # [ 0.367637] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1090worker1 # [ 0.366285] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1091lb # [ 0.388265] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1092client # [ 0.392812] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1093worker2 # [ 0.367829] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1094worker1 # [ 0.366479] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1095lb # [ 0.388730] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1096client # [ 0.393286] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1097worker2 # [ 0.367859] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1098worker1 # [ 0.366510] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1099lb # [ 0.388925] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1100client # [ 0.393480] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1101worker2 # [ 0.368327] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1102lb # [ 0.388955] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103client # [ 0.393511] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1104worker2 # [ 0.368518] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1105lb # [ 0.389419] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1106worker2 # [ 0.368548] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1107client # [ 0.393981] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1108lb # [ 0.389619] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1109client # [ 0.394176] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1110worker2 # [ 0.368944] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1111lb # [ 0.389650] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1112client # [ 0.394206] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1113worker2 # [ 0.369129] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1114lb # [ 0.390084] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1115worker2 # [ 0.369410] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1116lb # [ 0.390271] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1117worker2 # [ 0.369602] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1118lb # [ 0.390533] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1119worker2 # [ 0.369632] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1120lb # [ 0.390725] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1121worker2 # [ 0.370103] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1122lb # [ 0.390756] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123worker2 # [ 0.370293] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1124lb # [ 0.391241] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1125worker2 # [ 0.370323] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1126lb # [ 0.391432] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1127worker2 # [ 0.370800] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1128lb # [ 0.391462] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129lb # [ 0.391921] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1130lb # [ 0.392112] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1131lb # [ 0.392143] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1132lb # [ 0.392608] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1133lb # [ 0.392863] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1134lb # [ 0.392880] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1135lb # [ 0.392910] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1136lb # [ 0.393377] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1137lb # [ 0.393565] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1138client # [ 0.410683] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1139lb # [ 0.393581] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1140client # [ 0.410886] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1141worker1 # [ 0.415084] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1142lb # [ 0.393659] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1143client # [ 0.411199] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1144worker1 # [ 0.415294] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1145lb # [ 0.394228] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1146client # [ 0.411394] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1147worker1 # [ 0.415326] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148lb # [ 0.394240] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1149client # [ 0.411425] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1150worker1 # [ 0.415839] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1151lb # [ 0.394246] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1152client # [ 0.411893] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1153worker1 # [ 0.416034] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1154client # [ 0.412086] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1155lb # [ 0.394290] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1156worker1 # [ 0.416065] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1157client # [ 0.412116] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158worker2 # [ 0.419063] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1159lb # [ 0.394337] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1160worker1 # [ 0.416540] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1161worker2 # [ 0.419106] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1162client # [ 0.412583] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1163worker1 # [ 0.416796] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1164lb # [ 0.394383] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1165client # [ 0.412776] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1166worker1 # [ 0.416813] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1167worker2 # [ 0.419617] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1168lb # [ 0.394430] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1169client # [ 0.412807] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1170worker2 # [ 0.419869] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1171worker1 # [ 0.416844] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1172worker2 # [ 0.419887] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1173lb # [ 0.394477] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1174client # [ 0.413283] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1175worker1 # [ 0.417318] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1176worker2 # [ 0.419917] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1177client # [ 0.413543] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1178lb # [ 0.394524] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1179worker1 # [ 0.417507] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1180client # [ 0.413560] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1181worker1 # [ 0.417574] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker2 # [ 0.420383] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1183lb # [ 0.394570] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1184client # [ 0.413590] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1185worker2 # [ 0.420571] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1186worker1 # [ 0.417607] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1187lb # [ 0.394617] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1188worker2 # [ 0.420587] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1189worker1 # [ 0.418203] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1190client # [ 0.414062] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1191lb # [ 0.394665] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1192client # [ 0.414250] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1193worker2 # [ 0.420617] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1194worker1 # [ 0.418215] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1195client # [ 0.414266] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1196worker2 # [ 0.421209] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1197lb # [ 0.394734] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1198worker1 # [ 0.418221] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1199client # [ 0.414296] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1200lb # [ 0.394780] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1201worker2 # [ 0.421220] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1202worker1 # [ 0.418266] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1203client # [ 0.414894] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1204worker2 # [ 0.421226] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1205worker1 # [ 0.418314] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1206client # [ 0.414906] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1207worker2 # [ 0.421271] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1208worker1 # [ 0.418363] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1209client # [ 0.414911] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1210worker2 # [ 0.421319] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1211worker1 # [ 0.418411] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1212client # [ 0.414958] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1213worker2 # [ 0.421367] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1214worker1 # [ 0.418460] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1215client # [ 0.415006] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1216worker2 # [ 0.421415] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1217worker1 # [ 0.418508] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1218client # [ 0.415054] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1219worker2 # [ 0.421463] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1220worker1 # [ 0.418557] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1221client # [ 0.415103] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1222worker2 # [ 0.421511] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1223worker1 # [ 0.418604] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1224client # [ 0.415151] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1225worker2 # [ 0.421559] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1226worker1 # [ 0.418652] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1227client # [ 0.415205] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1228worker2 # [ 0.421606] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1229worker1 # [ 0.418724] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1230lb # [ 0.442843] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1231client # [ 0.415253] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1232worker2 # [ 0.421653] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1233worker1 # [ 0.418772] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1234lb # [ 0.442885] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1235client # [ 0.415300] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1236worker1 # [ 0.418794] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1237worker2 # [ 0.421721] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1238lb # [ 0.442909] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1239client # [ 0.415347] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1240worker1 # [ 0.418816] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1241worker2 # [ 0.421768] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1242lb # [ 0.442932] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1243client # [ 0.415429] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1244worker1 # [ 0.418838] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1245worker2 # [ 0.421790] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1246lb # [ 0.442954] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1247client # [ 0.415477] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1248worker1 # [ 0.418860] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1249worker2 # [ 0.421811] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1250lb # [ 0.442977] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1251client # [ 0.415499] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1252worker1 # [ 0.418881] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1253worker2 # [ 0.421833] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1254lb # [ 0.443003] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1255client # [ 0.415521] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1256worker1 # [ 0.418928] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1257worker2 # [ 0.421855] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1258lb # [ 0.443025] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1259client # [ 0.415543] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1260worker1 # [ 0.418952] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1261worker2 # [ 0.421876] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1262lb # [ 0.443048] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1263client # [ 0.415565] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1264worker1 # [ 0.418975] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1265worker2 # [ 0.421898] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1266lb # [ 0.443071] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1267worker1 # [ 0.418998] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1268client # [ 0.415586] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1269worker2 # [ 0.421924] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1270lb # [ 0.443094] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1271client # [ 0.415608] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1272worker1 # [ 0.419021] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1273worker2 # [ 0.421946] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1274lb # [ 0.443116] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1275client # [ 0.415634] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1276worker1 # [ 0.419044] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1277worker2 # [ 0.421968] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1278lb # [ 0.443137] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1279worker1 # [ 0.419066] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1280client # [ 0.415656] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1281lb # [ 0.443159] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1282worker2 # [ 0.421990] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1283worker1 # [ 0.419088] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1284client # [ 0.415678] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1285lb # [ 0.443181] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1286worker2 # [ 0.422012] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1287worker1 # [ 0.419109] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1288lb # [ 0.443202] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1289client # [ 0.415701] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1290worker2 # [ 0.422034] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1291worker1 # [ 0.419131] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1292lb # [ 0.443224] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1293worker2 # [ 0.422055] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1294client # [ 0.415723] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1295worker1 # [ 0.419153] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1296lb # [ 0.443251] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1297worker2 # [ 0.422076] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1298client # [ 0.415745] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1299worker1 # [ 0.419175] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1300lb # [ 0.443261] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1301worker2 # [ 0.422097] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1302client # [ 0.415766] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1303worker1 # [ 0.419201] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1304lb # [ 0.443265] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1305worker2 # [ 0.422118] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1306client # [ 0.415788] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1307worker1 # [ 0.419211] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1308lb # [ 0.444092] pci 0000:00:07.0: enabling device (0000 -> 0002)1309worker2 # [ 0.422139] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1310client # [ 0.415809] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1311worker1 # [ 0.419215] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1312worker2 # [ 0.422165] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1313client # [ 0.415830] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1314worker1 # [ 0.420082] pci 0000:00:07.0: enabling device (0000 -> 0002)1315worker2 # [ 0.422174] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1316client # [ 0.415852] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1317worker2 # [ 0.422179] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1318client # [ 0.415878] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1319worker2 # [ 0.423026] pci 0000:00:07.0: enabling device (0000 -> 0002)1320client # [ 0.415888] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1321client # [ 0.415893] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1322client # [ 0.416736] pci 0000:00:07.0: enabling device (0000 -> 0002)1323worker1 # [ 0.481703] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1324worker2 # [ 0.481733] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1325lb # [ 0.485174] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1326client # [ 0.488980] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1327worker2 # [ 0.487898] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1328worker2 # [ 0.490162] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1329worker1 # [ 0.492352] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1330worker1 # [ 0.494572] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1331worker1 # [ 0.498320] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1332client # [ 0.495608] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1333client # [ 0.497550] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1334lb # [ 0.494987] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1335lb # [ 0.497830] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1336lb # [ 0.499695] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1337worker2 # [ 0.500320] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1338lb # [ 0.501630] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1339worker2 # [ 0.502361] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1340worker1 # [ 0.505489] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1341client # [ 0.507663] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1342client # [ 0.509589] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1343lb # [ 0.511613] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1344lb # [ 0.513442] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1345worker2 # [ 0.512850] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1346worker2 # [ 0.514799] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1347worker1 # [ 0.515062] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1348worker1 # [ 0.517201] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1349client # [ 0.520197] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1350worker1 # [ 0.518890] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1351client # [ 0.521928] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1352worker1 # [ 0.520644] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1353worker1 # [ 0.522481] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1354client # [ 0.525373] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1355worker2 # [ 0.524292] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1356worker2 # [ 0.526092] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1357lb # [ 0.523327] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1358lb # [ 0.525116] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1359lb # [ 0.526925] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1360worker2 # [ 0.529636] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1361lb # [ 0.529077] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1362client # [ 0.531810] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1363client # [ 0.534234] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1364worker1 # [ 0.536323] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1365worker2 # [ 0.537164] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1366client # [ 0.544170] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1367lb # [ 0.550084] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1368worker1 # [ 0.552066] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1369lb # [ 0.552710] msm_serial: driver initialized1370worker1 # [ 0.554552] msm_serial: driver initialized1371worker2 # [ 0.552131] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1372lb # [ 0.552858] SuperH (H)SCI(F) driver initialized1373worker1 # [ 0.554697] SuperH (H)SCI(F) driver initialized1374lb # [ 0.552915] STM32 USART driver initialized1375worker1 # [ 0.554748] STM32 USART driver initialized1376worker2 # [ 0.554550] msm_serial: driver initialized1377worker2 # [ 0.554682] SuperH (H)SCI(F) driver initialized1378worker2 # [ 0.554735] STM32 USART driver initialized1379client # [ 0.557146] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1380client # [ 0.559678] msm_serial: driver initialized1381client # [ 0.559841] SuperH (H)SCI(F) driver initialized1382client # [ 0.559895] STM32 USART driver initialized1383lb # [ 0.582064] loop: module loaded1384lb # [ 0.582261] virtio_blk virtio2: 1/0/0 default/read/poll queues1385worker1 # [ 0.585695] loop: module loaded1386worker1 # [ 0.585867] virtio_blk virtio2: 1/0/0 default/read/poll queues1387worker1 # [ 0.586626] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1388worker2 # [ 0.585710] loop: module loaded1389worker2 # [ 0.585873] virtio_blk virtio2: 1/0/0 default/read/poll queues1390worker2 # [ 0.586622] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1391lb # [ 0.591106] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1392client # [ 0.590915] loop: module loaded1393worker1 # [ 0.591415] megasas: 07.734.00.00-rc11394client # [ 0.591087] virtio_blk virtio2: 1/0/0 default/read/poll queues1395worker1 # [ 0.592059] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1396client # [ 0.591814] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1397worker2 # [ 0.591460] megasas: 07.734.00.00-rc11398worker2 # [ 0.592157] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1399lb # [ 0.595366] megasas: 07.734.00.00-rc11400lb # [ 0.596092] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1401lb # [ 0.599154] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001402lb # [ 0.599177] Intel/Sharp Extended Query Table at 0x00311403lb # [ 0.600699] Using buffer write method1404lb # [ 0.600760] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1405client # [ 0.602876] megasas: 07.734.00.00-rc11406lb # [ 0.602531] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001407worker1 # [ 0.602940] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001408client # [ 0.603648] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1409worker2 # [ 0.604317] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001410lb # [ 0.602551] Intel/Sharp Extended Query Table at 0x00311411worker1 # [ 0.602977] Intel/Sharp Extended Query Table at 0x00311412worker2 # [ 0.604348] Intel/Sharp Extended Query Table at 0x00311413worker1 # [ 0.605167] Using buffer write method1414client # [ 0.605586] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001415worker2 # [ 0.606012] Using buffer write method1416client # [ 0.605610] Intel/Sharp Extended Query Table at 0x00311417worker1 # [ 0.605242] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1418worker2 # [ 0.606071] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1419worker1 # [ 0.607055] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001420worker1 # [ 0.607075] Intel/Sharp Extended Query Table at 0x00311421worker1 # [ 0.608840] Using buffer write method1422worker1 # [ 0.608863] Concatenating MTD devices:1423worker1 # [ 0.608867] (0): "0.flash"1424lb # [ 0.615610] Using buffer write method1425worker1 # [ 0.608871] (1): "0.flash"1426lb # [ 0.615641] Concatenating MTD devices:1427worker1 # [ 0.608882] into device "0.flash"1428lb # [ 0.615645] (0): "0.flash"1429lb # [ 0.615649] (1): "0.flash"1430client # [ 0.610784] Using buffer write method1431lb # [ 0.615657] into device "0.flash"1432client # [ 0.610842] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1433client # [ 0.612840] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001434client # [ 0.612860] Intel/Sharp Extended Query Table at 0x00311435worker2 # [ 0.616911] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001436worker2 # [ 0.616938] Intel/Sharp Extended Query Table at 0x00311437worker2 # [ 0.618539] Using buffer write method1438worker2 # [ 0.618563] Concatenating MTD devices:1439worker2 # [ 0.618567] (0): "0.flash"1440worker2 # [ 0.618571] (1): "0.flash"1441worker2 # [ 0.618579] into device "0.flash"1442client # [ 0.630872] Using buffer write method1443client # [ 0.630905] Concatenating MTD devices:1444client # [ 0.630909] (0): "0.flash"1445client # [ 0.630913] (1): "0.flash"1446client # [ 0.630921] into device "0.flash"1447worker1 # [ 0.858111] Freeing initrd memory: 26976K1448lb # [ 0.860773] Freeing initrd memory: 26948K1449worker2 # [ 0.861254] Freeing initrd memory: 26944K1450worker1 # [ 0.864022] tun: Universal TUN/TAP device driver, 1.61451client # [ 0.860240] Freeing initrd memory: 26972K1452lb # [ 0.866767] tun: Universal TUN/TAP device driver, 1.61453worker2 # [ 0.867262] tun: Universal TUN/TAP device driver, 1.61454lb # [ 0.870421] thunder_xcv, ver 1.01455lb # [ 0.870464] thunder_bgx, ver 1.01456lb # [ 0.870487] nicpf, ver 1.01457worker1 # [ 0.867880] thunder_xcv, ver 1.01458worker1 # [ 0.867921] thunder_bgx, ver 1.01459worker1 # [ 0.867943] nicpf, ver 1.01460client # [ 0.866269] tun: Universal TUN/TAP device driver, 1.61461worker1 # [ 0.868487] e1000: Intel(R) PRO/1000 Network Driver1462client # [ 0.869219] thunder_xcv, ver 1.01463client # [ 0.869257] thunder_bgx, ver 1.01464worker1 # [ 0.868495] e1000: Copyright (c) 1999-2006 Intel Corporation.1465client # [ 0.869279] nicpf, ver 1.01466worker1 # [ 0.868517] e1000e: Intel(R) PRO/1000 Network Driver1467client # [ 0.869816] e1000: Intel(R) PRO/1000 Network Driver1468worker1 # [ 0.868525] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1469client # [ 0.869824] e1000: Copyright (c) 1999-2006 Intel Corporation.1470lb # [ 0.872294] e1000: Intel(R) PRO/1000 Network Driver1471worker1 # [ 0.868553] igb: Intel(R) Gigabit Ethernet Network Driver1472client # [ 0.869846] e1000e: Intel(R) PRO/1000 Network Driver1473lb # [ 0.872306] e1000: Copyright (c) 1999-2006 Intel Corporation.1474worker1 # [ 0.868559] igb: Copyright (c) 2007-2014 Intel Corporation.1475client # [ 0.869855] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1476lb # [ 0.872330] e1000e: Intel(R) PRO/1000 Network Driver1477worker1 # [ 0.868580] igbvf: Intel(R) Gigabit Virtual Function Network Driver1478client # [ 0.869884] igb: Intel(R) Gigabit Ethernet Network Driver1479worker2 # [ 0.870992] thunder_xcv, ver 1.01480lb # [ 0.872340] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1481worker1 # [ 0.868586] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1482worker2 # [ 0.871027] thunder_bgx, ver 1.01483client # [ 0.869890] igb: Copyright (c) 2007-2014 Intel Corporation.1484worker2 # [ 0.871049] nicpf, ver 1.01485lb # [ 0.872369] igb: Intel(R) Gigabit Ethernet Network Driver1486worker1 # [ 0.868715] sky2: driver version 1.301487client # [ 0.869927] igbvf: Intel(R) Gigabit Virtual Function Network Driver1488worker2 # [ 0.871631] e1000: Intel(R) PRO/1000 Network Driver1489lb # [ 0.872375] igb: Copyright (c) 2007-2014 Intel Corporation.1490worker1 # [ 0.870249] usbcore: registered new interface driver usb-storage1491worker2 # [ 0.871638] e1000: Copyright (c) 1999-2006 Intel Corporation.1492client # [ 0.869933] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1493lb # [ 0.872398] igbvf: Intel(R) Gigabit Virtual Function Network Driver1494worker1 # [ 0.870294] usbcore: registered new interface driver usbserial_generic1495client # [ 0.870065] sky2: driver version 1.301496worker2 # [ 0.871661] e1000e: Intel(R) PRO/1000 Network Driver1497lb # [ 0.872404] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1498worker1 # [ 0.870307] usbserial: USB Serial support registered for generic1499worker2 # [ 0.871669] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1500lb # [ 0.872538] sky2: driver version 1.301501worker1 # [ 0.870875] hv_vmbus: registering driver hyperv_keyboard1502worker2 # [ 0.871698] igb: Intel(R) Gigabit Ethernet Network Driver1503lb # [ 0.874240] usbcore: registered new interface driver usb-storage1504worker1 # [ 0.871545] ehci-pci 0000:00:07.0: EHCI Host Controller1505worker2 # [ 0.871704] igb: Copyright (c) 2007-2014 Intel Corporation.1506lb # [ 0.874367] usbcore: registered new interface driver usbserial_generic1507worker1 # [ 0.871567] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11508worker2 # [ 0.871727] igbvf: Intel(R) Gigabit Virtual Function Network Driver1509lb # [ 0.874382] usbserial: USB Serial support registered for generic1510worker2 # [ 0.871732] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1511lb # [ 0.875472] ehci-pci 0000:00:07.0: EHCI Host Controller1512client # [ 0.879811] usbcore: registered new interface driver usb-storage1513worker2 # [ 0.871863] sky2: driver version 1.301514worker1 # [ 0.871732] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001515lb # [ 0.875495] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11516worker1 # [ 0.883803] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001517worker2 # [ 0.873396] usbcore: registered new interface driver usb-storage1518client # [ 0.879863] usbcore: registered new interface driver usbserial_generic1519worker1 # [ 0.884089] hub 1-0:1.0: USB hub found1520lb # [ 0.875735] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001521client # [ 0.879877] usbserial: USB Serial support registered for generic1522worker2 # [ 0.873477] usbcore: registered new interface driver usbserial_generic1523worker1 # [ 0.884103] hub 1-0:1.0: 6 ports detected1524client # [ 0.880463] hv_vmbus: registering driver hyperv_keyboard1525lb # [ 0.886674] hv_vmbus: registering driver hyperv_keyboard1526worker2 # [ 0.873492] usbserial: USB Serial support registered for generic1527client # [ 0.881657] ehci-pci 0000:00:07.0: EHCI Host Controller1528worker2 # [ 0.874066] hv_vmbus: registering driver hyperv_keyboard1529lb # [ 0.887501] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001530worker2 # [ 0.875394] ehci-pci 0000:00:07.0: EHCI Host Controller1531lb # [ 0.888530] hub 1-0:1.0: USB hub found1532client # [ 0.881680] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11533worker1 # [ 0.887673] rtc-pl031 9010000.pl031: registered as rtc01534lb # [ 0.889032] hub 1-0:1.0: 6 ports detected1535client # [ 0.881841] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001536worker2 # [ 0.875424] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11537worker1 # [ 0.887701] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:23:47 UTC (1790349827)1538worker1 # [ 0.887994] i2c_dev: i2c /dev entries driver1539worker2 # [ 0.875633] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001540lb # [ 0.890725] rtc-pl031 9010000.pl031: registered as rtc01541client # [ 0.887545] rtc-pl031 9010000.pl031: registered as rtc01542worker2 # [ 0.887643] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001543worker2 # [ 0.887941] hub 1-0:1.0: USB hub found1544lb # [ 0.890753] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:23:47 UTC (1790349827)1545client # [ 0.887574] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:23:47 UTC (1790349827)1546worker2 # [ 0.887956] hub 1-0:1.0: 6 ports detected1547client # [ 0.887899] i2c_dev: i2c /dev entries driver1548lb # [ 0.892851] i2c_dev: i2c /dev entries driver1549client # [ 0.891006] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001550worker2 # [ 0.890979] rtc-pl031 9010000.pl031: registered as rtc01551client # [ 0.891367] hub 1-0:1.0: USB hub found1552worker1 # [ 0.893329] sdhci: Secure Digital Host Controller Interface driver1553client # [ 0.891388] hub 1-0:1.0: 6 ports detected1554worker2 # [ 0.891005] rtc-pl031 9010000.pl031: setting system clock to 2026-09-25T15:23:47 UTC (1790349827)1555worker1 # [ 0.893341] sdhci: Copyright(c) Pierre Ossman1556worker2 # [ 0.891331] i2c_dev: i2c /dev entries driver1557worker1 # [ 0.893606] Synopsys Designware Multimedia Card Interface Driver1558worker1 # [ 0.893975] sdhci-pltfm: SDHCI platform and OF driver helper1559lb # [ 0.896432] sdhci: Secure Digital Host Controller Interface driver1560worker1 # [ 0.898302] hid: raw HID events driver (C) Jiri Kosina1561lb # [ 0.896445] sdhci: Copyright(c) Pierre Ossman1562worker1 # [ 0.898540] usbcore: registered new interface driver usbhid1563worker1 # [ 0.898546] usbhid: USB HID core driver1564lb # [ 0.896714] Synopsys Designware Multimedia Card Interface Driver1565client # [ 0.895860] sdhci: Secure Digital Host Controller Interface driver1566lb # [ 0.897084] sdhci-pltfm: SDHCI platform and OF driver helper1567worker2 # [ 0.896793] sdhci: Secure Digital Host Controller Interface driver1568client # [ 0.895871] sdhci: Copyright(c) Pierre Ossman1569lb # [ 0.898642] hid: raw HID events driver (C) Jiri Kosina1570worker2 # [ 0.896804] sdhci: Copyright(c) Pierre Ossman1571client # [ 0.896129] Synopsys Designware Multimedia Card Interface Driver1572worker2 # [ 0.897067] Synopsys Designware Multimedia Card Interface Driver1573lb # [ 0.902339] usbcore: registered new interface driver usbhid1574client # [ 0.896501] sdhci-pltfm: SDHCI platform and OF driver helper1575lb # [ 0.902349] usbhid: USB HID core driver1576worker2 # [ 0.897432] sdhci-pltfm: SDHCI platform and OF driver helper1577client # [ 0.897988] hid: raw HID events driver (C) Jiri Kosina1578client # [ 0.898221] usbcore: registered new interface driver usbhid1579client # [ 0.898228] usbhid: USB HID core driver1580worker2 # [ 0.901807] hid: raw HID events driver (C) Jiri Kosina1581worker2 # [ 0.902059] usbcore: registered new interface driver usbhid1582worker2 # [ 0.902066] usbhid: USB HID core driver1583worker2 # [ 0.923749] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1584client # [ 0.923551] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1585worker1 # [ 0.925907] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1586lb # [ 0.925977] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1587worker1 # [ 0.927408] drop_monitor: Initializing network drop monitor service1588client # [ 0.925070] drop_monitor: Initializing network drop monitor service1589worker2 # [ 0.925248] drop_monitor: Initializing network drop monitor service1590lb # [ 0.927502] drop_monitor: Initializing network drop monitor service1591lb # [ 0.927617] NET: Registered PF_INET6 protocol family1592client # [ 0.925187] NET: Registered PF_INET6 protocol family1593worker2 # [ 0.925366] NET: Registered PF_INET6 protocol family1594worker1 # [ 0.927520] NET: Registered PF_INET6 protocol family1595worker1 # [ 0.930504] Segment Routing with IPv61596client # [ 0.928227] Segment Routing with IPv61597worker2 # [ 0.928446] Segment Routing with IPv61598lb # [ 0.930652] Segment Routing with IPv61599worker1 # [ 0.930535] In-situ OAM (IOAM) with IPv61600client # [ 0.928247] In-situ OAM (IOAM) with IPv61601worker2 # [ 0.928465] In-situ OAM (IOAM) with IPv61602lb # [ 0.930671] In-situ OAM (IOAM) with IPv61603worker1 # [ 0.930562] NET: Registered PF_PACKET protocol family1604lb # [ 0.930697] NET: Registered PF_PACKET protocol family1605worker2 # [ 0.928493] NET: Registered PF_PACKET protocol family1606client # [ 0.928275] NET: Registered PF_PACKET protocol family1607client # [ 0.929922] 9pnet: Installing 9P2000 support1608worker2 # [ 0.930132] 9pnet: Installing 9P2000 support1609client # [ 0.929968] Key type dns_resolver registered1610worker2 # [ 0.930178] Key type dns_resolver registered1611lb # [ 0.932365] 9pnet: Installing 9P2000 support1612lb # [ 0.932410] Key type dns_resolver registered1613worker1 # [ 0.932223] 9pnet: Installing 9P2000 support1614worker1 # [ 0.932270] Key type dns_resolver registered1615worker1 # [ 0.938767] registered taskstats version 11616client # [ 0.936771] registered taskstats version 11617worker2 # [ 0.936930] registered taskstats version 11618client # [ 0.936916] Loading compiled-in X.509 certificates1619worker2 # [ 0.937069] Loading compiled-in X.509 certificates1620lb # [ 0.939199] registered taskstats version 11621worker1 # [ 0.939439] Loading compiled-in X.509 certificates1622lb # [ 0.939334] Loading compiled-in X.509 certificates1623worker1 # [ 0.947398] Demotion targets for Node 0: null1624worker2 # [ 0.945569] Demotion targets for Node 0: null1625client # [ 0.945437] Demotion targets for Node 0: null1626lb # [ 0.947921] Demotion targets for Node 0: null1627worker1 # [ 0.947492] Key type .fscrypt registered1628worker2 # [ 0.945663] Key type .fscrypt registered1629lb # [ 0.948020] Key type .fscrypt registered1630client # [ 0.945532] Key type .fscrypt registered1631worker1 # [ 0.947498] Key type fscrypt-provisioning registered1632worker2 # [ 0.945669] Key type fscrypt-provisioning registered1633lb # [ 0.948027] Key type fscrypt-provisioning registered1634client # [ 0.945539] Key type fscrypt-provisioning registered1635worker1 # [ 0.947585] ima: No TPM chip found, activating TPM-bypass!1636worker2 # [ 0.945758] ima: No TPM chip found, activating TPM-bypass!1637client # [ 0.945629] ima: No TPM chip found, activating TPM-bypass!1638lb # [ 0.948114] ima: No TPM chip found, activating TPM-bypass!1639worker1 # [ 0.947608] ima: Allocated hash algorithm: sha11640worker2 # [ 0.945778] ima: Allocated hash algorithm: sha11641client # [ 0.945648] ima: Allocated hash algorithm: sha11642lb # [ 0.948133] ima: Allocated hash algorithm: sha11643worker1 # [ 0.947628] ima: No architecture policies found1644worker2 # [ 0.945799] ima: No architecture policies found1645client # [ 0.945669] ima: No architecture policies found1646lb # [ 0.948154] ima: No architecture policies found1647worker1 # [ 0.951587] input: gpio-keys as /devices/platform/gpio-keys/input/input01648client # [ 0.949617] input: gpio-keys as /devices/platform/gpio-keys/input/input01649worker2 # [ 0.949847] input: gpio-keys as /devices/platform/gpio-keys/input/input01650lb # [ 0.952090] input: gpio-keys as /devices/platform/gpio-keys/input/input01651lb # [ 0.969197] clk: Disabling unused clocks1652worker1 # [ 0.968662] clk: Disabling unused clocks1653lb # [ 0.969217] PM: genpd: Disabling unused power domains1654worker1 # [ 0.968685] PM: genpd: Disabling unused power domains1655client # [ 0.967159] clk: Disabling unused clocks1656worker2 # [ 0.967524] clk: Disabling unused clocks1657client # [ 0.967235] PM: genpd: Disabling unused power domains1658worker2 # [ 0.967544] PM: genpd: Disabling unused power domains1659worker1 # [ 0.972909] Freeing unused kernel memory: 4736K1660lb # [ 0.973392] Freeing unused kernel memory: 4736K1661worker1 # [ 0.973100] Run /init as init process1662lb # [ 0.973565] Run /init as init process1663client # [ 0.971419] Freeing unused kernel memory: 4736K1664worker2 # [ 0.971702] Freeing unused kernel memory: 4736K1665client # [ 0.971630] Run /init as init process1666worker2 # [ 0.971899] Run /init as init process1667lb # [ 0.988948] systemd[1]: Successfully made /usr/ read-only.1668worker1 # [ 0.988984] systemd[1]: Successfully made /usr/ read-only.1669client # [ 0.988629] systemd[1]: Successfully made /usr/ read-only.1670worker2 # [ 0.989099] systemd[1]: Successfully made /usr/ read-only.1671lb # [ 1.146840] usb 1-1: new high-speed USB device number 2 using ehci-pci1672worker1 # [ 1.146941] usb 1-1: new high-speed USB device number 2 using ehci-pci1673client # [ 1.146362] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker2 # [ 1.146944] usb 1-1: new high-speed USB device number 2 using ehci-pci1675lb # [ 1.294449] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676worker1 # [ 1.294538] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677worker2 # [ 1.294569] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11678client # [ 1.295963] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679lb # [ 1.323600] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1680worker1 # [ 1.323626] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1681lb # [ 1.330201] systemd[1]: Detected virtualization qemu.1682worker1 # [ 1.330121] systemd[1]: Detected virtualization qemu.1683client # [ 1.323321] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1684lb # [ 1.331296] systemd[1]: Detected architecture arm64.1685worker1 # [ 1.331218] systemd[1]: Detected architecture arm64.1686worker2 # [ 1.323729] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1687lb # [ 1.332317] systemd[1]: Running in initrd.1688worker1 # [ 1.332245] systemd[1]: Running in initrd.1689client # [ 1.329850] systemd[1]: Detected virtualization qemu.1690worker2 # [ 1.330287] systemd[1]: Detected virtualization qemu.1691client # [ 1.330990] systemd[1]: Detected architecture arm64.1692worker2 # [ 1.331426] systemd[1]: Detected architecture arm64.1693lb # [ 1.333660] systemd[1]: Initializing machine ID from random generator.1694worker1 # [ 1.333634] systemd[1]: Initializing machine ID from random generator.1695client # [ 1.332020] systemd[1]: Running in initrd.1696worker2 # [ 1.332454] systemd[1]: Running in initrd.1697lb # [ 1.335201] systemd[1]: Hostname set to <lb>.1698worker1 # [ 1.335145] systemd[1]: Hostname set to <worker1>.1699client # [ 1.333389] systemd[1]: Initializing machine ID from random generator.1700worker2 # [ 1.333774] systemd[1]: Initializing machine ID from random generator.1701client # [ 1.334945] systemd[1]: Hostname set to <client>.1702worker2 # [ 1.335333] systemd[1]: Hostname set to <worker2>.1703lb # [ 1.375011] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704worker1 # [ 1.379090] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705worker2 # [ 1.379096] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706client # [ 1.386513] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707lb # [ 1.494841] usb 1-2: new high-speed USB device number 3 using ehci-pci1708worker1 # [ 1.498943] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker2 # [ 1.498946] usb 1-2: new high-speed USB device number 3 using ehci-pci1710client # [ 1.506367] usb 1-2: new high-speed USB device number 3 using ehci-pci1711lb # [ 1.585377] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712worker1 # [ 1.589479] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713client # [ 1.589582] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714worker2 # [ 1.589875] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715lb # [ 1.649473] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21716lb # [ 1.652297] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01717worker1 # [ 1.653656] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21718worker1 # [ 1.656512] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01719client # [ 1.655856] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21720client # [ 1.658405] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01721worker2 # [ 1.662756] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21722worker2 # [ 1.665567] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01723lb # [ 1.670588] systemd[1]: Queued start job for default target Initrd Default Target.1724client # [ 1.671013] systemd[1]: Queued start job for default target Initrd Default Target.1725worker1 # [ 1.674376] systemd[1]: Queued start job for default target Initrd Default Target.1726worker2 # [ 1.677913] systemd[1]: Queued start job for default target Initrd Default Target.1727lb # [ 1.681161] systemd[1]: Created slice Slice /system/modprobe.1728lb # [ 1.682433] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1729client # [ 1.680723] systemd[1]: Created slice Slice /system/modprobe.1730lb # [ 1.683845] systemd[1]: Expecting device /dev/disk/by-label/nixos...1731client # [ 1.681938] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1732lb # [ 1.684971] systemd[1]: Reached target Path Units.1733worker1 # [ 1.685227] systemd[1]: Created slice Slice /system/modprobe.1734lb # [ 1.685809] systemd[1]: Reached target Slice Units.1735client # [ 1.683459] systemd[1]: Expecting device /dev/disk/by-label/nixos...1736lb # [ 1.686642] systemd[1]: Reached target Swaps.1737client # [ 1.684566] systemd[1]: Reached target Path Units.1738worker1 # [ 1.686493] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1739lb # [ 1.687463] systemd[1]: Reached target Timer Units.1740client # [ 1.685422] systemd[1]: Reached target Slice Units.1741worker1 # [ 1.687931] systemd[1]: Expecting device /dev/disk/by-label/nixos...1742client # [ 1.686282] systemd[1]: Reached target Swaps.1743lb # [ 1.688493] systemd[1]: Listening on D-Bus System Message Bus Socket.1744worker1 # [ 1.689065] systemd[1]: Reached target Path Units.1745client # [ 1.687141] systemd[1]: Reached target Timer Units.1746worker1 # [ 1.689913] systemd[1]: Reached target Slice Units.1747lb # [ 1.689895] systemd[1]: Listening on Journal Socket (/dev/log).1748worker1 # [ 1.690787] systemd[1]: Reached target Swaps.1749client # [ 1.688248] systemd[1]: Listening on D-Bus System Message Bus Socket.1750worker1 # [ 1.691662] systemd[1]: Reached target Timer Units.1751worker2 # [ 1.689548] systemd[1]: Created slice Slice /system/modprobe.1752client # [ 1.689514] systemd[1]: Listening on Journal Socket (/dev/log).1753lb # [ 1.691131] systemd[1]: Listening on Journal Sockets.1754worker1 # [ 1.692709] systemd[1]: Listening on D-Bus System Message Bus Socket.1755lb # [ 1.691283] systemd[1]: Listening on udev Control Socket.1756worker2 # [ 1.690797] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1757worker1 # [ 1.694060] systemd[1]: Listening on Journal Socket (/dev/log).1758lb # [ 1.691412] systemd[1]: Listening on udev Kernel Socket.1759client # [ 1.690702] systemd[1]: Listening on Journal Sockets.1760lb # [ 1.691437] systemd[1]: Reached target Socket Units.1761client # [ 1.690838] systemd[1]: Listening on udev Control Socket.1762client # [ 1.690957] systemd[1]: Listening on udev Kernel Socket.1763lb # [ 1.696523] systemd[1]: Starting Create List of Static Device Nodes...1764worker2 # [ 1.692362] systemd[1]: Expecting device /dev/disk/by-label/nixos...1765worker1 # [ 1.695294] systemd[1]: Listening on Journal Sockets.1766client # [ 1.690979] systemd[1]: Reached target Socket Units.1767worker2 # [ 1.692403] systemd[1]: Reached target Path Units.1768worker1 # [ 1.695435] systemd[1]: Listening on udev Control Socket.1769client # [ 1.695993] systemd[1]: Starting Create List of Static Device Nodes...1770worker2 # [ 1.692424] systemd[1]: Reached target Slice Units.1771lb # [ 1.697763] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1772worker1 # [ 1.695555] systemd[1]: Listening on udev Kernel Socket.1773worker2 # [ 1.692445] systemd[1]: Reached target Swaps.1774worker1 # [ 1.695579] systemd[1]: Reached target Socket Units.1775worker2 # [ 1.692465] systemd[1]: Reached target Timer Units.1776client # [ 1.697117] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1777worker1 # [ 1.700668] systemd[1]: Starting Create List of Static Device Nodes...1778worker2 # [ 1.692656] systemd[1]: Listening on D-Bus System Message Bus Socket.1779worker2 # [ 1.692808] systemd[1]: Listening on Journal Socket (/dev/log).1780worker1 # [ 1.701799] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1781worker2 # [ 1.692956] systemd[1]: Listening on Journal Sockets.1782worker2 # [ 1.693089] systemd[1]: Listening on udev Control Socket.1783worker2 # [ 1.693198] systemd[1]: Listening on udev Kernel Socket.1784worker2 # [ 1.693220] systemd[1]: Reached target Socket Units.1785lb # [ 1.706920] systemd[1]: Mounting Kernel Configuration File System...1786worker2 # [ 1.705128] systemd[1]: Starting Create List of Static Device Nodes...1787client # [ 1.706449] systemd[1]: Mounting Kernel Configuration File System...1788worker2 # [ 1.706366] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1789worker1 # [ 1.710781] systemd[1]: Mounting Kernel Configuration File System...1790lb # [ 1.715194] systemd[1]: Starting Journal Service...1791worker2 # [ 1.715069] systemd[1]: Mounting Kernel Configuration File System...1792client # [ 1.714747] systemd[1]: Starting Journal Service...1793worker1 # [ 1.719315] systemd[1]: Starting Journal Service...1794worker2 # [ 1.723189] systemd[1]: Starting Journal Service...1795worker1 # [ 1.735121] systemd[1]: Starting Load Kernel Modules...1796lb # [ 1.734429] systemd[1]: Starting Load Kernel Modules...1797worker1 # [ 1.735972] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1798lb # [ 1.734535] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1799client # [ 1.733946] systemd[1]: Starting Load Kernel Modules...1800client # [ 1.734047] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1801lb # [ 1.744658] systemd[1]: Starting Coldplug All udev Devices...1802client # [ 1.744236] systemd[1]: Starting Coldplug All udev Devices...1803worker2 # [ 1.746352] systemd[1]: Starting Load Kernel Modules...1804worker1 # [ 1.748010] systemd[1]: Starting Coldplug All udev Devices...1805worker2 # [ 1.746455] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1806worker2 # [ 1.766057] systemd-journald[72]: Collecting audit messages is disabled.1807lb # [ 1.770923] systemd[1]: Finished Create List of Static Device Nodes.1808lb # [ 1.771783] systemd[1]: Mounted Kernel Configuration File System.1809client # [ 1.770456] systemd[1]: Finished Create List of Static Device Nodes.1810worker1 # [ 1.771002] systemd[1]: Finished Create List of Static Device Nodes.1811client # [ 1.771362] systemd[1]: Mounted Kernel Configuration File System.1812worker1 # [ 1.771836] systemd[1]: Mounted Kernel Configuration File System.1813lb # [ 1.774696] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1814worker2 # [ 1.775086] systemd[1]: Starting Coldplug All udev Devices...1815worker1 # [ 1.776925] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1816client # [ 1.774411] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1817worker1 # [ 1.790141] systemd-journald[72]: Collecting audit messages is disabled.1818lb # [ 1.796290] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1819worker2 # [ 1.791158] systemd[1]: Finished Create List of Static Device Nodes.1820worker2 # [ 1.792135] systemd[1]: Mounted Kernel Configuration File System.1821client # [ 1.795747] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1822lb # [ 1.799399] systemd-journald[72]: Collecting audit messages is disabled.1823worker1 # [ 1.799767] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1824client # [ 1.798973] systemd-journald[72]: Collecting audit messages is disabled.1825worker2 # [ 1.799237] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1826lb # [ 1.811406] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1827lb # [ 1.813552] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1828client # [ 1.810939] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1829client # [ 1.813347] systemd[1]: Starting Create Static Device Nodes in /dev...1830client # [ 1.817541] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1831worker1 # [ 1.818957] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1832worker1 # [ 1.824930] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01833worker1 # [ 1.825182] [drm] features: -virgl +edid -resource_blob -host_visible1834worker1 # [ 1.825191] [drm] features: -context_init1835worker1 # [ 1.825937] [drm] number of scanouts: 11836worker1 # [ 1.825956] [drm] number of cap sets: 01837worker2 # [ 1.824742] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1838lb # [ 1.831889] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01839lb # [ 1.832145] [drm] features: -virgl +edid -resource_blob -host_visible1840lb # [ 1.832155] [drm] features: -context_init1841lb # [ 1.832849] [drm] number of scanouts: 11842lb # [ 1.832867] [drm] number of cap sets: 01843client # [ 1.831518] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01844client # [ 1.831775] [drm] features: -virgl +edid -resource_blob -host_visible1845client # [ 1.831785] [drm] features: -context_init1846client # [ 1.832484] [drm] number of scanouts: 11847client # [ 1.832502] [drm] number of cap sets: 01848lb # [ 1.838987] systemd[1]: Starting Create Static Device Nodes in /dev...1849worker1 # [ 1.839456] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1850worker1 # [ 1.841876] systemd[1]: Starting Create Static Device Nodes in /dev...1851worker2 # [ 1.839562] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1852worker2 # [ 1.842002] systemd[1]: Starting Create Static Device Nodes in /dev...1853worker2 # [ 1.846012] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1854worker1 # [ 1.847294] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1855worker1 # [ 1.847309] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01856worker2 # [ 1.853198] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01857worker2 # [ 1.853439] [drm] features: -virgl +edid -resource_blob -host_visible1858worker2 # [ 1.853449] [drm] features: -context_init1859worker2 # [ 1.854120] [drm] number of scanouts: 11860worker2 # [ 1.854138] [drm] number of cap sets: 01861client # [ 1.853662] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1862lb # [ 1.855206] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1863client # [ 1.853682] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01864lb # [ 1.855223] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01865client # [ 1.862896] systemd[1]: Finished Create Static Device Nodes in /dev.1866client # [ 1.863057] systemd[1]: Reached target Preparation for Local File Systems.1867client # [ 1.863081] systemd[1]: Reached target Local File Systems.1868lb # [ 1.871389] systemd[1]: Finished Create Static Device Nodes in /dev.1869client # [ 1.870689] systemd[1]: Starting Rule-based Manager for Device Events and Files...1870lb # [ 1.871557] systemd[1]: Reached target Preparation for Local File Systems.1871lb # [ 1.871584] systemd[1]: Reached target Local File Systems.1872lb # [ 1.879139] systemd[1]: Starting Rule-based Manager for Device Events and Files...1873worker1 # [ 1.879151] Console: switching to colour frame buffer device 160x501874worker1 # [ 1.883044] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1875worker1 # [ 1.885904] systemd[1]: Finished Create Static Device Nodes in /dev.1876worker2 # [ 1.879251] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1877worker1 # [ 1.886083] systemd[1]: Reached target Preparation for Local File Systems.1878worker2 # [ 1.879267] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01879lb # [ 1.884089] Console: switching to colour frame buffer device 160x501880worker1 # [ 1.886117] systemd[1]: Reached target Local File Systems.1881lb # [ 1.888187] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1882worker1 # [ 1.891674] systemd[1]: Starting Rule-based Manager for Device Events and Files...1883client # [ 1.886606] Console: switching to colour frame buffer device 160x501884client # [ 1.890870] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1885worker2 # [ 1.899561] systemd[1]: Finished Create Static Device Nodes in /dev.1886worker2 # [ 1.899746] systemd[1]: Reached target Preparation for Local File Systems.1887lb # [ 1.903379] systemd[1]: Finished Load Kernel Modules.1888worker2 # [ 1.899771] systemd[1]: Reached target Local File Systems.1889lb # [ 1.905701] systemd[1]: Starting Apply Kernel Variables...1890worker1 # [ 1.908084] systemd[1]: Finished Load Kernel Modules.1891worker2 # [ 1.903702] systemd[1]: Starting Rule-based Manager for Device Events and Files...1892client # [ 1.907616] systemd[1]: Finished Load Kernel Modules.1893worker2 # [ 1.908355] Console: switching to colour frame buffer device 160x501894worker1 # [ 1.919757] systemd[1]: Starting Apply Kernel Variables...1895client # [ 1.918811] systemd[1]: Starting Apply Kernel Variables...1896worker2 # [ 1.923767] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1897worker2 # [ 1.940248] systemd[1]: Finished Load Kernel Modules.1898worker2 # [ 1.943070] systemd[1]: Starting Apply Kernel Variables...1899worker1 # [ 1.959499] systemd[1]: Finished Apply Kernel Variables.1900lb # [ 1.959480] systemd[1]: Finished Apply Kernel Variables.1901worker2 # [ 1.962471] systemd[1]: Started Journal Service.1902client # [ 1.962932] systemd[1]: Finished Apply Kernel Variables.1903lb # [ 1.971064] systemd[1]: Started Journal Service.1904lb # [ 1.968279] systemd-modules-load[73]: Inserted module 'dm_mod'1905worker2 # [ 1.966541] systemd-modules-load[74]: Inserted module 'dm_mod'1906client # [ 1.979889] systemd[1]: Started Journal Service.1907lb # [ 1.969626] systemd-modules-load[73]: Module 'virtio_balloon' is built in1908worker1 # [ 1.983121] systemd[1]: Started Journal Service.1909worker2 # [ 1.967822] systemd-modules-load[74]: Module 'virtio_balloon' is built in1910lb # [ 1.970717] systemd-modules-load[73]: Module 'virtio_console' is built in1911worker1 # [ 1.971178] systemd-modules-load[73]: Inserted module 'dm_mod'1912worker2 # [ 1.969226] systemd-modules-load[74]: Module 'virtio_console' is built in1913lb # [ 1.971834] systemd-modules-load[73]: Inserted module 'virtio_gpu'1914worker1 # [ 1.972297] systemd-modules-load[73]: Module 'virtio_balloon' is built in1915worker2 # [ 1.970350] systemd-modules-load[74]: Inserted module 'virtio_gpu'1916worker1 # [ 1.973436] systemd-modules-load[73]: Module 'virtio_console' is built in1917worker2 # [ 1.971358] systemd-modules-load[74]: Module 'virtio_rng' is built in1918worker1 # [ 1.974538] systemd-modules-load[73]: Inserted module 'virtio_gpu'1919worker1 # [ 1.975550] systemd-modules-load[73]: Module 'virtio_rng' is built in1920client # [ 1.976116] systemd-modules-load[73]: Inserted module 'dm_mod'1921client # [ 1.978195] systemd-modules-load[73]: Module 'virtio_balloon' is built in1922client # [ 1.979322] systemd-modules-load[73]: Module 'virtio_console' is built in1923lb # [ 1.983517] systemd-modules-load[73]: Module 'virtio_rng' is built in1924client # [ 1.985267] systemd-modules-load[73]: Inserted module 'virtio_gpu'1925client # [ 1.986291] systemd-modules-load[73]: Module 'virtio_rng' is built in1926client # [ 1.987585] systemd-udevd[78]: Using default interface naming scheme 'v261'.1927lb # [ 1.989708] systemd[1]: Starting Create System Files and Directories...1928worker2 # [ 1.989652] systemd[1]: Starting Create System Files and Directories...1929worker1 # [ 1.992413] systemd[1]: Starting Create System Files and Directories...1930worker1 # [ 1.993743] systemd-udevd[78]: Using default interface naming scheme 'v261'.1931lb # [ 1.996399] systemd-udevd[78]: Using default interface naming scheme 'v261'.1932client # [ 2.001486] systemd[1]: Starting Create System Files and Directories...1933lb # [ 2.004337] systemd[1]: Finished Create System Files and Directories.1934worker2 # [ 2.004878] systemd[1]: Finished Apply Kernel Variables.1935worker2 # [ 2.006423] systemd[1]: Finished Create System Files and Directories.1936client # [ 2.010369] systemd[1]: Finished Create System Files and Directories.1937worker1 # [ 2.013859] systemd[1]: Finished Create System Files and Directories.1938worker2 # [ 2.015658] systemd-udevd[78]: Using default interface naming scheme 'v261'.1939lb # [ 2.021061] systemd[1]: Started Rule-based Manager for Device Events and Files.1940client # [ 2.026787] systemd[1]: Started Rule-based Manager for Device Events and Files.1941worker1 # [ 2.030812] systemd[1]: Started Rule-based Manager for Device Events and Files.1942worker2 # [ 2.037455] systemd[1]: Started Rule-based Manager for Device Events and Files.1943lb # [ 2.068283] systemd[1]: Starting Virtual Console Setup...1944client # [ 2.084793] systemd[1]: Starting Virtual Console Setup...1945worker1 # [ 2.088104] systemd[1]: Starting Virtual Console Setup...1946worker2 # [ 2.089647] systemd[1]: Starting Virtual Console Setup...1947lb # [ 2.124505] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1948lb # [ 2.127900] systemd[1]: Finished Virtual Console Setup.1949worker1 # [ 2.136488] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1950client # [ 2.136487] systemd-vconsole-setup[100]: Configuration of first virtual console was skipped, ignoring remaining ones.1951worker1 # [ 2.139964] systemd[1]: Finished Virtual Console Setup.1952client # [ 2.139969] systemd[1]: Finished Virtual Console Setup.1953worker2 # [ 2.145201] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954worker2 # [ 2.147871] systemd[1]: Finished Virtual Console Setup.1955worker1 # [ 2.753221] systemd[1]: Finished Coldplug All udev Devices.1956worker1 # [ 2.754203] systemd[1]: Reached target System Initialization.1957worker1 # [ 2.755111] systemd[1]: Reached target Basic System.1958client # [ 2.753181] systemd[1]: Finished Coldplug All udev Devices.1959client # [ 2.756136] systemd[1]: Reached target System Initialization.1960client # [ 2.757025] systemd[1]: Reached target Basic System.1961worker2 # [ 2.766741] systemd[1]: Finished Coldplug All udev Devices.1962worker2 # [ 2.767765] systemd[1]: Reached target System Initialization.1963worker2 # [ 2.768924] systemd[1]: Reached target Basic System.1964lb # [ 2.789482] systemd[1]: Finished Coldplug All udev Devices.1965lb # [ 2.790458] systemd[1]: Reached target System Initialization.1966lb # [ 2.791350] systemd[1]: Reached target Basic System.1967worker1 # [ 2.899999] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1968client # [ 2.900166] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1969worker2 # [ 2.918598] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1970worker1 # [ 2.933842] (udev-worker)[99]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1971worker1 # [ 2.940183] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1972lb # [ 2.943903] (udev-worker)[96]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973lb # [ 2.956433] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1974client # [ 2.958749] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975worker2 # [ 2.960707] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1976client # [ 2.964258] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1977worker2 # [ 2.965137] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1978lb # [ 2.976510] (udev-worker)[96]: Network interface NamePolicy= disabled on kernel command line.1979worker1 # [ 3.001385] systemd[1]: Found device /dev/disk/by-label/nixos.1980worker1 # [ 3.007833] systemd[1]: Reached target Initrd Root Device.1981worker1 # [ 3.013588] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1982lb # [ 3.027919] systemd[1]: Found device /dev/disk/by-label/nixos.1983lb # [ 3.029748] systemd[1]: Reached target Initrd Root Device.1984lb # [ 3.033352] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1985client # [ 3.037034] systemd[1]: Found device /dev/disk/by-label/nixos.1986worker2 # [ 3.037145] systemd[1]: Found device /dev/disk/by-label/nixos.1987client # [ 3.040462] systemd[1]: Reached target Initrd Root Device.1988worker2 # [ 3.044096] systemd[1]: Reached target Initrd Root Device.1989client # [ 3.045318] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1990worker2 # [ 3.048149] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1991worker1 # [ 3.064202] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1992worker1 # [ 3.072093] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1993worker1 # [ 3.073918] systemd[1]: Mounting /sysroot...1994lb # [ 3.084823] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1995client # [ 3.083039] systemd-fsck[109]: nixos: clean, 12/65536 files, 13019/262144 blocks1996worker2 # [ 3.087805] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1997lb # [ 3.090558] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998client # [ 3.091998] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1999lb # [ 3.096559] systemd[1]: Mounting /sysroot...2000worker2 # [ 3.095057] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.
loading 2,064 hidden lines…
4065lb # [ 58.908882] niks3-server[955]: 2026/09/25 15:24:45 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4066worker1 # [ 58.916666] nix-grpc-daemon[486]: ts=2026-09-25T15:24:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04067worker1 # [ 58.926991] nix-daemon[838]: reaped child process 979, status = succeeded4068client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.33 seconds)4069client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4070client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.14 seconds)4071lb: must succeed: curl -sf localhost:9901/clusters4072lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4073lb: must succeed: curl -sf localhost:9901/clusters4074lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4075worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4076worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.03 seconds)4077(finished: subtest: inputs already in the cache are not sent to the scheduler, in 1.68 seconds)4078subtest: outputs read back through any worker carry the cache signature4079client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24080worker1 # [ 59.357684] nix-grpc-daemon[486]: ts=2026-09-25T15:24:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538824081client # don't know how to build these paths:4082client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4083worker1 # [ 59.364831] nix-grpc-daemon[486]: ts=2026-09-25T15:24:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538824084worker2 # [ 59.378386] nix-grpc-daemon[769]: ts=2026-09-25T15:24:46Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04085worker2 # [ 59.382723] nix-grpc-daemon[769]: ts=2026-09-25T15:24:46Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14086worker2 # [ 59.386939] nix-grpc-daemon[769]: ts=2026-09-25T15:24:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884087worker2 # [ 59.393229] nix-grpc-daemon[769]: ts=2026-09-25T15:24:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884088client # copying 1 paths to 'lb:50051'...4089client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4090worker2 # [ 59.404161] nix-grpc-daemon[769]: ts=2026-09-25T15:24:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884091worker2 # [ 59.413239] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4092worker2 # [ 59.431024] nix-grpc-daemon[769]: ts=2026-09-25T15:24:46Z level=info event=niks3_push_started pid=9164093worker2 # [ 59.577668] nix-grpc-daemon[916]: time=2026-09-25T15:24:46.201Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4094worker2 # [ 59.678999] nix-daemon[857]: accepted connection from pid 928, user nix-grpc-daemon (trusted)4095worker2 # [ 59.699197] nix-daemon[857]: reaped child process 931, status = succeeded4096lb # [ 59.705844] niks3-server[955]: 2026/09/25 15:24:46 INFO Received push request method=POST path=/api/pushes4097worker2 # [ 59.710165] nix-grpc-daemon[916]: time=2026-09-25T15:24:46.332Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4098worker2 # [ 59.711723] nix-grpc-daemon[916]: time=2026-09-25T15:24:46.332Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4099lb # [ 59.731088] niks3-server[955]: 2026/09/25 15:24:46 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4100worker2 # [ 59.737763] nix-grpc-daemon[916]: time=2026-09-25T15:24:46.360Z level=INFO msg="Uploading 1 narinfos"4101lb # [ 59.735422] niks3-server[955]: 2026/09/25 15:24:46 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4102lb # [ 59.742954] niks3-server[955]: 2026/09/25 15:24:46 INFO Signed narinfos id=9 count=14103lb # [ 59.746957] niks3-server[955]: 2026/09/25 15:24:46 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4104lb # [ 59.756498] niks3-server[955]: 2026/09/25 15:24:46 INFO Received complete push request method=POST path=/api/pushes/9/complete4105worker2 # [ 59.759521] nix-grpc-daemon[916]: time=2026-09-25T15:24:46.383Z level=INFO msg="Upload complete. (178ms)"4106lb # [ 59.763347] niks3-server[955]: 2026/09/25 15:24:46 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4107worker2 # [ 59.766792] nix-grpc-daemon[769]: ts=2026-09-25T15:24:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=5124108worker2 # [ 59.776168] nix-grpc-daemon[769]: ts=2026-09-25T15:24:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884109worker2 # [ 59.781737] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4110worker2 # [ 59.783821] nix-daemon[857]: reaped child process 914, status = succeeded4111client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4112worker2 # [ 59.791794] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4113client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4114client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4115client # farm-a> LOG-farm-a-sig4116worker2 # [ 60.640657] nix-daemon[857]: reaped child process 935, status = succeeded4117worker2 # [ 60.714980] nix-daemon[857]: accepted connection from pid 951, user nix-grpc-daemon (trusted)4118worker2 # [ 60.733482] nix-daemon[857]: reaped child process 954, status = succeeded4119lb # [ 60.738894] niks3-server[955]: 2026/09/25 15:24:47 INFO Received push request method=POST path=/api/pushes4120worker2 # [ 60.742932] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.365Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4121worker2 # [ 60.746825] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.365Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4122lb # [ 60.760133] niks3-server[955]: 2026/09/25 15:24:47 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4123worker2 # [ 60.766438] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.390Z level=INFO msg="Uploading 1 narinfos"4124lb # [ 60.768960] niks3-server[955]: 2026/09/25 15:24:47 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4125lb # [ 60.771385] niks3-server[955]: 2026/09/25 15:24:47 INFO Signed narinfos id=10 count=14126lb # [ 60.778644] niks3-server[955]: 2026/09/25 15:24:47 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4127lb # [ 60.788235] niks3-server[955]: 2026/09/25 15:24:47 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4128lb # [ 60.790683] niks3-server[955]: 2026/09/25 15:24:47 INFO Received complete push request method=POST path=/api/pushes/10/complete4129worker2 # [ 60.793218] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.417Z level=INFO msg="Upload complete. (192ms)"4130lb # [ 60.797423] niks3-server[955]: 2026/09/25 15:24:47 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4131worker2 # [ 60.802186] nix-grpc-daemon[769]: ts=2026-09-25T15:24:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04132worker2 # [ 60.817002] nix-grpc-daemon[769]: ts=2026-09-25T15:24:47Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04133worker2 # [ 60.819675] nix-grpc-daemon[769]: ts=2026-09-25T15:24:47Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14134worker2 # [ 60.824378] nix-grpc-daemon[769]: ts=2026-09-25T15:24:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884135worker2 # [ 60.828140] nix-daemon[857]: reaped child process 933, status = succeeded4136worker2 # [ 60.831359] nix-grpc-daemon[769]: ts=2026-09-25T15:24:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884137worker2 # [ 60.839103] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.462Z level=INFO msg="All 1 paths already cached"4138client # copying 1 paths to 'lb:50051'...4139client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4140worker2 # [ 60.845058] nix-grpc-daemon[769]: ts=2026-09-25T15:24:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884141worker2 # [ 60.852090] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4142worker2 # [ 60.967360] nix-daemon[857]: accepted connection from pid 960, user nix-grpc-daemon (trusted)4143worker2 # [ 60.985840] nix-daemon[857]: reaped child process 963, status = succeeded4144lb # [ 60.992302] niks3-server[955]: 2026/09/25 15:24:47 INFO Received push request method=POST path=/api/pushes4145worker2 # [ 60.994899] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.618Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4146worker2 # [ 60.996581] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.618Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4147lb # [ 61.007827] niks3-server[955]: 2026/09/25 15:24:47 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4148worker2 # [ 61.009750] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.633Z level=INFO msg="Uploading 1 narinfos"4149lb # [ 61.015152] niks3-server[955]: 2026/09/25 15:24:47 INFO Signed narinfos id=11 count=14150lb # [ 61.019913] niks3-server[955]: 2026/09/25 15:24:47 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4151lb # [ 61.029318] niks3-server[955]: 2026/09/25 15:24:47 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4152lb # [ 61.033688] niks3-server[955]: 2026/09/25 15:24:47 INFO Received complete push request method=POST path=/api/pushes/11/complete4153worker2 # [ 61.035092] nix-grpc-daemon[916]: time=2026-09-25T15:24:47.658Z level=INFO msg="Upload complete. (174ms)"4154lb # [ 61.037514] niks3-server[955]: 2026/09/25 15:24:47 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4155worker2 # [ 61.040315] nix-grpc-daemon[769]: ts=2026-09-25T15:24:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=6244156worker2 # [ 61.048488] nix-grpc-daemon[769]: ts=2026-09-25T15:24:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884157worker2 # [ 61.056362] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4158worker2 # [ 61.058502] nix-daemon[857]: reaped child process 958, status = succeeded4159client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4160worker2 # [ 61.067113] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4161client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4162client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4163client # farm-b> LOG-farm-b-sig4164worker2 # [ 62.502249] nix-daemon[857]: reaped child process 967, status = succeeded4165worker2 # [ 62.611978] nix-daemon[857]: accepted connection from pid 983, user nix-grpc-daemon (trusted)4166worker2 # [ 62.627865] nix-daemon[857]: reaped child process 986, status = succeeded4167lb # [ 62.636133] niks3-server[955]: 2026/09/25 15:24:49 INFO Received push request method=POST path=/api/pushes4168worker2 # [ 62.639867] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.262Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4169worker2 # [ 62.641691] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.262Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4170lb # [ 62.658399] niks3-server[955]: 2026/09/25 15:24:49 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4171worker2 # [ 62.662946] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.286Z level=INFO msg="Uploading 1 narinfos"4172lb # [ 62.664947] niks3-server[955]: 2026/09/25 15:24:49 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4173lb # [ 62.672542] niks3-server[955]: 2026/09/25 15:24:49 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4174lb # [ 62.674857] niks3-server[955]: 2026/09/25 15:24:49 INFO Signed narinfos id=12 count=14175lb # [ 62.685119] niks3-server[955]: 2026/09/25 15:24:49 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4176worker2 # [ 62.688109] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.311Z level=INFO msg="Upload complete. (211ms)"4177lb # [ 62.690787] niks3-server[955]: 2026/09/25 15:24:49 INFO Received complete push request method=POST path=/api/pushes/12/complete4178lb # [ 62.693995] niks3-server[955]: 2026/09/25 15:24:49 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4179worker2 # [ 62.696079] nix-grpc-daemon[769]: ts=2026-09-25T15:24:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04180worker2 # [ 62.711764] nix-grpc-daemon[769]: ts=2026-09-25T15:24:49Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04181worker2 # [ 62.714870] nix-grpc-daemon[769]: ts=2026-09-25T15:24:49Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14182worker2 # [ 62.719486] nix-grpc-daemon[769]: ts=2026-09-25T15:24:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884183worker2 # [ 62.723902] nix-daemon[857]: reaped child process 965, status = succeeded4184worker2 # [ 62.726812] nix-grpc-daemon[769]: ts=2026-09-25T15:24:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884185worker2 # [ 62.732403] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.356Z level=INFO msg="All 2 paths already cached"4186client # copying 1 paths to 'lb:50051'...4187client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4188worker2 # [ 62.738599] nix-grpc-daemon[769]: ts=2026-09-25T15:24:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884189worker2 # [ 62.746936] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4190worker2 # [ 62.860646] nix-daemon[857]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4191lb # [ 62.881803] niks3-server[955]: 2026/09/25 15:24:49 INFO Received push request method=POST path=/api/pushes4192worker2 # [ 62.881620] nix-daemon[857]: reaped child process 995, status = succeeded4193worker2 # [ 62.884143] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.507Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4194worker2 # [ 62.886204] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.509Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4195lb # [ 62.899696] niks3-server[955]: 2026/09/25 15:24:49 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4196worker2 # [ 62.903312] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.526Z level=INFO msg="Uploading 1 narinfos"4197lb # [ 62.907343] niks3-server[955]: 2026/09/25 15:24:49 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4198lb # [ 62.913799] niks3-server[955]: 2026/09/25 15:24:49 INFO Signed narinfos id=13 count=14199lb # [ 62.921217] niks3-server[955]: 2026/09/25 15:24:49 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4200lb # [ 62.923694] niks3-server[955]: 2026/09/25 15:24:49 INFO Received complete push request method=POST path=/api/pushes/13/complete4201worker2 # [ 62.928940] nix-grpc-daemon[916]: time=2026-09-25T15:24:49.552Z level=INFO msg="Upload complete. (173ms)"4202lb # [ 62.930573] niks3-server[955]: 2026/09/25 15:24:49 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4203worker2 # [ 62.934736] nix-grpc-daemon[769]: ts=2026-09-25T15:24:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=7524204worker2 # [ 62.944919] nix-grpc-daemon[769]: ts=2026-09-25T15:24:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884205worker2 # [ 62.949213] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4206worker2 # [ 62.954316] nix-daemon[857]: reaped child process 990, status = succeeded4207client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4208client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4209worker2 # [ 62.965936] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4210client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4211client # farm-top> LOG-farm-top-sig4212worker2 # [ 64.003744] nix-daemon[857]: reaped child process 999, status = succeeded4213worker2 # [ 64.107103] nix-daemon[857]: accepted connection from pid 1015, user nix-grpc-daemon (trusted)4214worker2 # [ 64.124912] nix-daemon[857]: reaped child process 1018, status = succeeded4215lb # [ 64.130694] niks3-server[955]: 2026/09/25 15:24:50 INFO Received push request method=POST path=/api/pushes4216worker2 # [ 64.135623] nix-grpc-daemon[916]: time=2026-09-25T15:24:50.757Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4217worker2 # [ 64.137471] nix-grpc-daemon[916]: time=2026-09-25T15:24:50.757Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4218lb # [ 64.158207] niks3-server[955]: 2026/09/25 15:24:50 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4219worker2 # [ 64.161655] nix-grpc-daemon[916]: time=2026-09-25T15:24:50.785Z level=INFO msg="Uploading 1 narinfos"4220lb # [ 64.165700] niks3-server[955]: 2026/09/25 15:24:50 INFO Signed narinfos id=14 count=14221lb # [ 64.171028] niks3-server[955]: 2026/09/25 15:24:50 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4222lb # [ 64.178430] niks3-server[955]: 2026/09/25 15:24:50 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4223worker2 # [ 64.186167] nix-grpc-daemon[916]: time=2026-09-25T15:24:50.809Z level=INFO msg="Upload complete. (213ms)"4224lb # [ 64.188567] niks3-server[955]: 2026/09/25 15:24:50 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4225lb # [ 64.191088] niks3-server[955]: 2026/09/25 15:24:50 INFO Received complete push request method=POST path=/api/pushes/14/complete4226worker2 # [ 64.192617] nix-grpc-daemon[769]: ts=2026-09-25T15:24:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04227lb # [ 64.199264] niks3-server[955]: 2026/09/25 15:24:50 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4228worker2 # [ 64.206166] nix-daemon[857]: reaped child process 997, status = succeeded4229client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.04 seconds)4230client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4231client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.14 seconds)4232client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4233worker1 # [ 64.491462] nix-grpc-daemon[486]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:527024234client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4235client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4236client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4237client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4238worker1 # [ 64.514096] nix-grpc-daemon[486]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:527024239worker1 # [ 64.518833] nix-grpc-daemon[486]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4240worker1 # [ 64.528768] nix-grpc-daemon[486]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4241worker1 # [ 64.536943] nix-grpc-daemon[486]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4242client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.19 seconds)4243client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4244worker2 # [ 64.695004] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:367084245worker2 # [ 64.699580] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:367084246client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.16 seconds)4247client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4248client # finding garbage collector roots...4249client # removing stale temporary roots file "/nix/var/nix/temproots/918"4250client # removing stale temporary roots file "/nix/var/nix/temproots/905"4251client # removing stale temporary roots file "/nix/var/nix/temproots/926"4252client # removing stale temporary roots file "/nix/var/nix/temproots/941"4253client # removing stale temporary roots file "/nix/var/nix/temproots/885"4254client # removing stale temporary roots file "/nix/var/nix/temproots/849"4255client # removing stale temporary roots file "/nix/var/nix/temproots/897"4256client # removing stale temporary roots file "/nix/var/nix/temproots/864"4257client # deleting specified paths...4258client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4259client # deleting unused links...4260client # note: hard linking is currently saving 0.0 KiB4261worker2 # [ 64.977711] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564884262worker2 # [ 64.982946] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884263worker2 # [ 64.988657] nix-grpc-daemon[916]: time=2026-09-25T15:24:51.612Z level=INFO msg="All 1 paths already cached"4264worker2 # [ 64.993541] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564884265worker2 # [ 64.998504] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564884266worker2 # [ 65.003461] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564884267client # copying 3 paths...4268worker1 # [ 65.014649] nix-grpc-daemon[486]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:538824269client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4270worker2 # [ 65.028955] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:564884271worker2 # [ 65.033850] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_out=2324272worker2 # [ 65.039270] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:564884273worker2 # [ 65.043327] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_out=1764274worker1 # [ 65.050578] nix-grpc-daemon[486]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:538824275worker1 # [ 65.054612] nix-grpc-daemon[486]: ts=2026-09-25T15:24:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=1 nar_bytes_out=1284276client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4277client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4278client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.37 seconds)4279(finished: subtest: outputs read back through any worker carry the cache signature, in 5.90 seconds)4280subtest: two clients wanting the same drv build it once4281client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4282client # [ 65.118062] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4283client # Running as unit: dedup1.service; invocation ID: 85196cd54fed48c881f208fb773253034284client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.04 seconds)4285client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4286worker2 # [ 65.284337] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564884287client # [ 65.288986] nix[994]: don't know how to build these paths:4288client # [ 65.290672] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4289worker2 # [ 65.296783] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04290worker2 # [ 65.299709] nix-grpc-daemon[769]: ts=2026-09-25T15:24:51Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14291worker1 # [ 65.307464] nix-grpc-daemon[486]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824292worker1 # [ 65.316118] nix-grpc-daemon[486]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538824293client # [ 65.323440] nix[994]: copying 1 paths to 'lb:50051'...4294worker1 # [ 65.324895] nix-grpc-daemon[486]: ts=2026-09-25T15:24:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538824295client # [ 65.324805] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4296worker1 # [ 65.333791] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4297worker1 # [ 65.466415] nix-daemon[838]: accepted connection from pid 1007, user nix-grpc-daemon (trusted)4298lb # [ 65.483255] niks3-server[955]: 2026/09/25 15:24:52 INFO Received push request method=POST path=/api/pushes4299worker1 # [ 65.486707] nix-daemon[838]: reaped child process 1010, status = succeeded4300worker1 # [ 65.488776] nix-grpc-daemon[850]: time=2026-09-25T15:24:52.115Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4301worker1 # [ 65.490375] nix-grpc-daemon[850]: time=2026-09-25T15:24:52.115Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4302lb # [ 65.499001] niks3-server[955]: 2026/09/25 15:24:52 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4303worker1 # [ 65.504141] nix-grpc-daemon[850]: time=2026-09-25T15:24:52.131Z level=INFO msg="Uploading 1 narinfos"4304lb # [ 65.506784] niks3-server[955]: 2026/09/25 15:24:52 INFO Signed narinfos id=15 count=14305lb # [ 65.512131] niks3-server[955]: 2026/09/25 15:24:52 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4306lb # [ 65.514613] niks3-server[955]: 2026/09/25 15:24:52 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4307worker1 # [ 65.525716] nix-grpc-daemon[850]: time=2026-09-25T15:24:52.152Z level=INFO msg="Upload complete. (182ms)"4308lb # [ 65.526288] niks3-server[955]: 2026/09/25 15:24:52 INFO Received complete push request method=POST path=/api/pushes/15/complete4309lb # [ 65.531663] niks3-server[955]: 2026/09/25 15:24:52 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4310worker1 # [ 65.532216] nix-grpc-daemon[486]: ts=2026-09-25T15:24:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=1 nar_bytes_in=4324311worker1 # [ 65.535194] nix-grpc-daemon[486]: ts=2026-09-25T15:24:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824312worker1 # [ 65.546205] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4313worker1 # [ 65.551729] nix-daemon[838]: reaped child process 1005, status = succeeded4314client # [ 65.555654] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4315worker1 # [ 65.558391] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4316client # [ 66.183552] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4317client # Running as unit: dedup2.service; invocation ID: 29c15b6f4fc049178382ffaec7bcde0f4318client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.07 seconds)4319worker2 # [ 66.379451] nix-grpc-daemon[769]: ts=2026-09-25T15:24:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564884320client # [ 66.384544] nix[1010]: don't know how to build these paths:4321client # [ 66.385449] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4322worker2 # [ 66.393435] nix-grpc-daemon[769]: ts=2026-09-25T15:24:53Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04323worker1 # [ 66.397667] nix-grpc-daemon[486]: ts=2026-09-25T15:24:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824324client # [ 66.398716] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4325client: must succeed: sleep 24326lb # [ 68.111864] envoy[902]: conn peer=[2001:db8:1::1]:52812 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4168 tx=1876 ms=11424327lb # [ 68.114697] envoy[902]: conn peer=[2001:db8:1::1]:52820 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9498 tx=4358 ms=48564328lb # [ 68.117293] envoy[902]: conn peer=[2001:db8:1::1]:39790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1091 tx=2908 ms=964329lb # [ 68.119695] envoy[902]: conn peer=[2001:db8:1::1]:39810 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=254330lb # [ 68.122199] envoy[902]: conn peer=[2001:db8:1::1]:39804 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=375 ms=364331lb # [ 68.124685] envoy[902]: conn peer=[2001:db8:1::1]:39796 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=454332client: (finished: must succeed: sleep 2, in 2.03 seconds)4333client: waiting for success: ! systemctl is-active dedup1 dedup24334worker1 # [ 69.623147] nix-daemon[838]: reaped child process 1014, status = succeeded4335worker1 # [ 69.687366] nix-daemon[838]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)4336lb # [ 69.706898] niks3-server[955]: 2026/09/25 15:24:56 INFO Received push request method=POST path=/api/pushes4337worker1 # [ 69.708907] nix-daemon[838]: reaped child process 1047, status = succeeded4338worker1 # [ 69.715486] nix-grpc-daemon[850]: time=2026-09-25T15:24:56.339Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4339worker1 # [ 69.717279] nix-grpc-daemon[850]: time=2026-09-25T15:24:56.339Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4340lb # [ 69.730005] niks3-server[955]: 2026/09/25 15:24:56 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4341worker1 # [ 69.737959] nix-grpc-daemon[850]: time=2026-09-25T15:24:56.364Z level=INFO msg="Uploading 1 narinfos"4342lb # [ 69.737860] niks3-server[955]: 2026/09/25 15:24:56 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4343lb # [ 69.745122] niks3-server[955]: 2026/09/25 15:24:56 INFO Signed narinfos id=16 count=14344lb # [ 69.746922] niks3-server[955]: 2026/09/25 15:24:56 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4345lb # [ 69.757620] niks3-server[955]: 2026/09/25 15:24:56 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4346worker1 # [ 69.765221] nix-grpc-daemon[850]: time=2026-09-25T15:24:56.392Z level=INFO msg="Upload complete. (195ms)"4347lb # [ 69.767163] niks3-server[955]: 2026/09/25 15:24:56 INFO Received complete push request method=POST path=/api/pushes/16/complete4348lb # [ 69.771603] niks3-server[955]: 2026/09/25 15:24:56 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4349worker1 # [ 69.772660] nix-grpc-daemon[486]: ts=2026-09-25T15:24:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04350worker1 # [ 69.783296] nix-grpc-daemon[486]: ts=2026-09-25T15:24:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04351worker1 # [ 69.788466] nix-daemon[838]: reaped child process 1012, status = succeeded4352client # [ 69.811899] systemd[1]: dedup1.service: Deactivated successfully.4353client # [ 69.815333] systemd[1]: dedup1.service: Consumed 150ms CPU time over 4.694s wall clock time, 41.5M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic.4354client # [ 69.820889] systemd[1]: dedup2.service: Deactivated successfully.4355client # [ 69.823695] systemd[1]: dedup2.service: Consumed 183ms CPU time over 3.637s wall clock time, 35.2M memory peak, 5.9K incoming IP traffic, 5.7K outgoing IP traffic.4356client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.07 seconds)4357client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4358client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.05 seconds)4359worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4360worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4361worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4362worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.04 seconds)4363worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4364worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4365worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4366worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4367(finished: subtest: two clients wanting the same drv build it once, in 5.74 seconds)4368subtest: --timeout and --max-silent-time reach the builder4369worker2 # [ 70.994216] nix-grpc-daemon[769]: ts=2026-09-25T15:24:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564884370worker2 # [ 71.003984] nix-grpc-daemon[769]: ts=2026-09-25T15:24:57Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04371worker2 # [ 71.009058] nix-grpc-daemon[769]: ts=2026-09-25T15:24:57Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14372worker2 # [ 71.013657] nix-grpc-daemon[769]: ts=2026-09-25T15:24:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884373worker2 # [ 71.017730] nix-grpc-daemon[769]: ts=2026-09-25T15:24:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884374worker2 # [ 71.023557] nix-grpc-daemon[769]: ts=2026-09-25T15:24:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884375worker2 # [ 71.033453] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4376worker2 # [ 71.157616] nix-daemon[857]: accepted connection from pid 1049, user nix-grpc-daemon (trusted)4377worker2 # [ 71.178300] nix-daemon[857]: reaped child process 1052, status = succeeded4378lb # [ 71.181613] niks3-server[955]: 2026/09/25 15:24:57 INFO Received push request method=POST path=/api/pushes4379worker2 # [ 71.184533] nix-grpc-daemon[916]: time=2026-09-25T15:24:57.808Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4380worker2 # [ 71.186081] nix-grpc-daemon[916]: time=2026-09-25T15:24:57.808Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4381lb # [ 71.203037] niks3-server[955]: 2026/09/25 15:24:57 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4382worker2 # [ 71.206567] nix-grpc-daemon[916]: time=2026-09-25T15:24:57.830Z level=INFO msg="Uploading 1 narinfos"4383lb # [ 71.210541] niks3-server[955]: 2026/09/25 15:24:57 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4384lb # [ 71.214793] niks3-server[955]: 2026/09/25 15:24:57 INFO Signed narinfos id=17 count=14385lb # [ 71.220380] niks3-server[955]: 2026/09/25 15:24:57 INFO Received complete push request method=POST path=/api/pushes/17/complete4386worker2 # [ 71.226225] nix-grpc-daemon[916]: time=2026-09-25T15:24:57.850Z level=INFO msg="Upload complete. (183ms)"4387lb # [ 71.227985] niks3-server[955]: 2026/09/25 15:24:57 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4388lb # [ 71.234616] niks3-server[955]: 2026/09/25 15:24:57 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4389worker2 # [ 71.231747] nix-grpc-daemon[769]: ts=2026-09-25T15:24:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=4244390worker2 # [ 71.242825] nix-grpc-daemon[769]: ts=2026-09-25T15:24:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884391worker2 # [ 71.248090] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4392worker2 # [ 71.251511] nix-daemon[857]: reaped child process 1047, status = succeeded4393worker2 # [ 71.261801] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4394worker2 # [ 77.075472] nix-grpc-daemon[769]: ts=2026-09-25T15:25:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04395worker2 # [ 77.100800] nix-daemon[857]: reaped child process 1054, status = succeeded4396worker2 # [ 77.102425] nix-daemon[857]: reaped child process 1056, status = succeeded4397worker1 # [ 77.276727] nix-grpc-daemon[486]: ts=2026-09-25T15:25:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538824398worker2 # [ 77.284538] nix-grpc-daemon[769]: ts=2026-09-25T15:25:03Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04399worker2 # [ 77.287545] nix-grpc-daemon[769]: ts=2026-09-25T15:25:03Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14400worker1 # [ 77.295346] nix-grpc-daemon[486]: ts=2026-09-25T15:25:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824401worker1 # [ 77.304681] nix-grpc-daemon[486]: ts=2026-09-25T15:25:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538824402worker1 # [ 77.312898] nix-grpc-daemon[486]: ts=2026-09-25T15:25:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538824403worker1 # [ 77.321797] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4404worker1 # [ 77.451546] nix-daemon[838]: accepted connection from pid 1060, user nix-grpc-daemon (trusted)4405worker1 # [ 77.471848] nix-daemon[838]: reaped child process 1063, status = succeeded4406lb # [ 77.470992] niks3-server[955]: 2026/09/25 15:25:04 INFO Received push request method=POST path=/api/pushes4407worker1 # [ 77.477036] nix-grpc-daemon[850]: time=2026-09-25T15:25:04.103Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4408worker1 # [ 77.478609] nix-grpc-daemon[850]: time=2026-09-25T15:25:04.103Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4409lb # [ 77.488268] niks3-server[955]: 2026/09/25 15:25:04 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4410worker1 # [ 77.493724] nix-grpc-daemon[850]: time=2026-09-25T15:25:04.120Z level=INFO msg="Uploading 1 narinfos"4411lb # [ 77.496592] niks3-server[955]: 2026/09/25 15:25:04 INFO Signed narinfos id=18 count=14412lb # [ 77.501336] niks3-server[955]: 2026/09/25 15:25:04 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4413lb # [ 77.508880] niks3-server[955]: 2026/09/25 15:25:04 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4414lb # [ 77.511365] niks3-server[955]: 2026/09/25 15:25:04 INFO Received complete push request method=POST path=/api/pushes/18/complete4415worker1 # [ 77.518295] nix-grpc-daemon[850]: time=2026-09-25T15:25:04.145Z level=INFO msg="Upload complete. (186ms)"4416lb # [ 77.519461] niks3-server[955]: 2026/09/25 15:25:04 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4417worker1 # [ 77.524235] nix-grpc-daemon[486]: ts=2026-09-25T15:25:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=1 nar_bytes_in=4244418worker1 # [ 77.534988] nix-grpc-daemon[486]: ts=2026-09-25T15:25:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824419worker1 # [ 77.540344] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4420worker1 # [ 77.544372] nix-daemon[838]: reaped child process 1058, status = succeeded4421worker1 # [ 77.555360] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4422lb # [ 78.108948] envoy[902]: conn peer=[2001:db8:1::1]:39822 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=1517 ms=44994423lb # [ 78.109086] envoy[902]: conn peer=[2001:db8:1::1]:39830 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=34134424lb # [ 78.109138] envoy[902]: conn peer=[2001:db8:1::1]:56520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2958 tx=934 ms=60984425worker1 # [ 83.767773] nix-grpc-daemon[486]: ts=2026-09-25T15:25:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04426worker1 # [ 83.788921] nix-daemon[838]: reaped child process 1065, status = succeeded4427worker1 # [ 83.790470] nix-daemon[838]: reaped child process 1067, status = succeeded4428worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4429worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4430worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4431worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4432(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.12 seconds)4433subtest: a failing build is counted with its own reason4434worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4435worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4436worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4437worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4438worker1 # [ 84.192822] nix-grpc-daemon[486]: ts=2026-09-25T15:25:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538824439worker2 # [ 84.201889] nix-grpc-daemon[769]: ts=2026-09-25T15:25:10Z level=debug event=want client=9 drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv cached=04440worker2 # [ 84.202132] nix-grpc-daemon[769]: ts=2026-09-25T15:25:10Z level=debug event=assigned drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv worker=192.168.1.4:50051 clients=14441worker2 # [ 84.209354] nix-grpc-daemon[769]: ts=2026-09-25T15:25:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884442worker2 # [ 84.219414] nix-grpc-daemon[769]: ts=2026-09-25T15:25:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884443worker2 # [ 84.227806] nix-grpc-daemon[769]: ts=2026-09-25T15:25:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884444worker2 # [ 84.236593] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4445worker2 # [ 84.366906] nix-daemon[857]: accepted connection from pid 1089, user nix-grpc-daemon (trusted)4446worker2 # [ 84.383826] nix-daemon[857]: reaped child process 1092, status = succeeded4447lb # [ 84.390595] niks3-server[955]: 2026/09/25 15:25:11 INFO Received push request method=POST path=/api/pushes4448worker2 # [ 84.395494] nix-grpc-daemon[916]: time=2026-09-25T15:25:11.018Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4449worker2 # [ 84.397296] nix-grpc-daemon[916]: time=2026-09-25T15:25:11.018Z level=INFO msg="Uploading j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv (400B)"4450lb # [ 84.411259] niks3-server[955]: 2026/09/25 15:25:11 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4451worker2 # [ 84.413887] nix-grpc-daemon[916]: time=2026-09-25T15:25:11.037Z level=INFO msg="Uploading 1 narinfos"4452lb # [ 84.419388] niks3-server[955]: 2026/09/25 15:25:11 INFO Signed narinfos id=19 count=14453lb # [ 84.424159] niks3-server[955]: 2026/09/25 15:25:11 INFO Registered completed upload object_key=nar/0d79vqzwc4s1agk431w3csq2p4brahnzmanwj14fjf383ld797i6.nar.zst4454lb # [ 84.427109] niks3-server[955]: 2026/09/25 15:25:11 INFO Registered completed upload object_key=j4rjl229w5xzbr9614xcipiv7652fdka.ls4455lb # [ 84.438547] niks3-server[955]: 2026/09/25 15:25:11 INFO Received complete push request method=POST path=/api/pushes/19/complete4456worker2 # [ 84.440034] nix-grpc-daemon[916]: time=2026-09-25T15:25:11.063Z level=INFO msg="Upload complete. (194ms)"4457lb # [ 84.444329] niks3-server[955]: 2026/09/25 15:25:11 INFO Registered completed upload object_key=j4rjl229w5xzbr9614xcipiv7652fdka.narinfo4458worker2 # [ 84.447338] nix-grpc-daemon[769]: ts=2026-09-25T15:25:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=4004459worker2 # [ 84.456259] nix-grpc-daemon[769]: ts=2026-09-25T15:25:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884460worker2 # [ 84.463050] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4461worker2 # [ 84.465702] nix-daemon[857]: reaped child process 1087, status = succeeded4462worker2 # [ 84.476449] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4463worker2 # [ 84.884774] nix-grpc-daemon[769]: ts=2026-09-25T15:25:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 drv=j4rjl229w5xzbr9614xcipiv7652fdka-fail-boom.drv assign_id=11 outputs=0 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04464worker2 # [ 84.907377] nix-daemon[857]: reaped child process 1094, status = succeeded4465worker2 # [ 84.909153] nix-daemon[857]: reaped child process 1096, status = succeeded4466worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4467worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4468worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4469worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4470worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4471worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4472worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4473worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4474worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4475worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4476worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4477worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4478worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4479worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4480worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4481worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4482(finished: subtest: a failing build is counted with its own reason, in 1.24 seconds)4483subtest: the first client leaving does not fail the second4484client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4485client # [ 85.225771] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4486client # Running as unit: share1.service; invocation ID: fccd3bc6337240718c93cd914f81a4d34487client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.05 seconds)4488client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4489worker1 # [ 85.413483] nix-grpc-daemon[486]: ts=2026-09-25T15:25:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538824490client # [ 85.416508] nix[1096]: don't know how to build these paths:4491client # [ 85.417418] nix[1096]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4492worker2 # [ 85.424190] nix-grpc-daemon[769]: ts=2026-09-25T15:25:12Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04493worker1 # [ 85.427563] nix-grpc-daemon[486]: ts=2026-09-25T15:25:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824494worker2 # [ 85.427201] nix-grpc-daemon[769]: ts=2026-09-25T15:25:12Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14495worker1 # [ 85.436360] nix-grpc-daemon[486]: ts=2026-09-25T15:25:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538824496client # [ 85.442684] nix[1096]: copying 1 paths to 'lb:50051'...4497worker1 # [ 85.444300] nix-grpc-daemon[486]: ts=2026-09-25T15:25:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538824498client # [ 85.443890] nix[1096]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4499worker1 # [ 85.452874] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4500worker1 # [ 85.579137] nix-daemon[838]: accepted connection from pid 1108, user nix-grpc-daemon (trusted)4501lb # [ 85.595401] niks3-server[955]: 2026/09/25 15:25:12 INFO Received push request method=POST path=/api/pushes4502worker1 # [ 85.597938] nix-daemon[838]: reaped child process 1111, status = succeeded4503worker1 # [ 85.600899] nix-grpc-daemon[850]: time=2026-09-25T15:25:12.227Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4504worker1 # [ 85.602440] nix-grpc-daemon[850]: time=2026-09-25T15:25:12.227Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4505lb # [ 85.611815] niks3-server[955]: 2026/09/25 15:25:12 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4506worker1 # [ 85.617799] nix-grpc-daemon[850]: time=2026-09-25T15:25:12.244Z level=INFO msg="Uploading 1 narinfos"4507lb # [ 85.618992] niks3-server[955]: 2026/09/25 15:25:12 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign4508lb # [ 85.625335] niks3-server[955]: 2026/09/25 15:25:12 INFO Signed narinfos id=20 count=14509lb # [ 85.627527] niks3-server[955]: 2026/09/25 15:25:12 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4510lb # [ 85.637814] niks3-server[955]: 2026/09/25 15:25:12 INFO Received complete push request method=POST path=/api/pushes/20/complete4511worker1 # [ 85.641302] nix-grpc-daemon[850]: time=2026-09-25T15:25:12.268Z level=INFO msg="Upload complete. (179ms)"4512lb # [ 85.642259] niks3-server[955]: 2026/09/25 15:25:12 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4513worker1 # [ 85.647280] nix-grpc-daemon[486]: ts=2026-09-25T15:25:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=1 nar_bytes_in=4324514worker1 # [ 85.657007] nix-grpc-daemon[486]: ts=2026-09-25T15:25:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824515worker1 # [ 85.661204] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4516worker1 # [ 85.665258] nix-daemon[838]: reaped child process 1106, status = succeeded4517client # [ 85.670527] nix[1096]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4518worker1 # [ 85.674960] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4519client # [ 86.305877] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4520client # Running as unit: share2.service; invocation ID: 9d4416d26069475481dea6d2df06f50e4521client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.08 seconds)4522client # [ 86.485293] nix[1112]: don't know how to build these paths:4523worker2 # [ 86.483481] nix-grpc-daemon[769]: ts=2026-09-25T15:25:13Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564884524client # [ 86.486470] nix[1112]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4525worker2 # [ 86.490603] nix-grpc-daemon[769]: ts=2026-09-25T15:25:13Z level=debug event=want client=11 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04526worker1 # [ 86.495279] nix-grpc-daemon[486]: ts=2026-09-25T15:25:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824527client # [ 86.496423] nix[1112]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4528client: must succeed: sleep 2; systemctl kill -s INT share14529lb # [ 88.112842] envoy[902]: conn peer=[2001:db8:1::1]:56524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2972 tx=932 ms=65074530lb # [ 88.115491] envoy[902]: conn peer=[2001:db8:1::1]:41392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2868 tx=1248 ms=7104531client # [ 89.562268] systemd[1]: share1.service: Sent signal SIGINT to main process 1096 (nix) on client request.4532client # [ 89.567639] nix[1096]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4533client # [ 89.569703] nix[1096]: error: interrupted by the user4534client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.05 seconds)4535client: waiting for success: ! systemctl is-active share14536client # [ 89.623445] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4537client # [ 89.625680] systemd[1]: share1.service: Failed with result 'exit-code'.4538client # [ 89.629038] systemd[1]: share1.service: Consumed 157ms CPU time over 4.402s wall clock time, 41.4M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.4539client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)4540client: waiting for success: ! systemctl is-active share24541worker1 # [ 90.904706] nix-daemon[838]: reaped child process 1115, status = succeeded4542worker1 # [ 91.013744] nix-daemon[838]: accepted connection from pid 1149, user nix-grpc-daemon (trusted)4543worker1 # [ 91.031021] nix-daemon[838]: reaped child process 1152, status = succeeded4544lb # [ 91.036509] niks3-server[955]: 2026/09/25 15:25:17 INFO Received push request method=POST path=/api/pushes4545worker1 # [ 91.042837] nix-grpc-daemon[850]: time=2026-09-25T15:25:17.668Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4546worker1 # [ 91.044430] nix-grpc-daemon[850]: time=2026-09-25T15:25:17.668Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4547lb # [ 91.062589] niks3-server[955]: 2026/09/25 15:25:17 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign4548worker1 # [ 91.068273] nix-grpc-daemon[850]: time=2026-09-25T15:25:17.695Z level=INFO msg="Uploading 1 narinfos"4549lb # [ 91.071452] niks3-server[955]: 2026/09/25 15:25:17 INFO Signed narinfos id=21 count=14550lb # [ 91.076172] niks3-server[955]: 2026/09/25 15:25:17 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4551lb # [ 91.078692] niks3-server[955]: 2026/09/25 15:25:17 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4552worker1 # [ 91.089976] nix-grpc-daemon[850]: time=2026-09-25T15:25:17.717Z level=INFO msg="Upload complete. (205ms)"4553lb # [ 91.090772] niks3-server[955]: 2026/09/25 15:25:17 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4554lb # [ 91.097065] niks3-server[955]: 2026/09/25 15:25:17 INFO Received complete push request method=POST path=/api/pushes/21/complete4555lb # [ 91.099938] niks3-server[955]: 2026/09/25 15:25:17 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4556worker1 # [ 91.096557] nix-grpc-daemon[486]: ts=2026-09-25T15:25:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04557worker1 # [ 91.109058] nix-grpc-daemon[486]: ts=2026-09-25T15:25:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=12 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04558worker1 # [ 91.114131] nix-daemon[838]: reaped child process 1113, status = succeeded4559client # [ 91.123765] systemd[1]: share2.service: Deactivated successfully.4560client # [ 91.126169] systemd[1]: share2.service: Consumed 161ms CPU time over 4.820s wall clock time, 35.3M memory peak, 6K incoming IP traffic, 5.6K outgoing IP traffic.4561client: (finished: waiting for success: ! systemctl is-active share2, in 1.07 seconds)4562client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4563client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)4564(finished: subtest: the first client leaving does not fail the second, in 6.75 seconds)4565subtest: build hook: nix-daemon with builders = grpc://lb4566client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4567client # [ 92.062646] systemd[1]: Started Nix Daemon.4568client # [ 92.174572] nix-daemon[1164]: accepted connection from pid 1161, user root (trusted)4569worker2 # [ 92.316398] nix-grpc-daemon[769]: ts=2026-09-25T15:25:18Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:564884570worker2 # [ 92.320186] nix-grpc-daemon[769]: ts=2026-09-25T15:25:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884571worker2 # [ 92.332518] nix-grpc-daemon[769]: ts=2026-09-25T15:25:18Z level=debug event=want client=12 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04572worker2 # [ 92.337844] nix-grpc-daemon[769]: ts=2026-09-25T15:25:18Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14573worker2 # [ 92.342619] nix-grpc-daemon[769]: ts=2026-09-25T15:25:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884574worker2 # [ 92.347162] nix-grpc-daemon[769]: ts=2026-09-25T15:25:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884575worker2 # [ 92.353759] nix-grpc-daemon[769]: ts=2026-09-25T15:25:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884576worker2 # [ 92.362423] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4577worker2 # [ 92.488137] nix-daemon[857]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)4578lb # [ 92.508153] niks3-server[955]: 2026/09/25 15:25:19 INFO Received push request method=POST path=/api/pushes4579worker2 # [ 92.508620] nix-daemon[857]: reaped child process 1151, status = succeeded4580worker2 # [ 92.512162] nix-grpc-daemon[916]: time=2026-09-25T15:25:19.135Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4581worker2 # [ 92.513697] nix-grpc-daemon[916]: time=2026-09-25T15:25:19.135Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4582lb # [ 92.525159] niks3-server[955]: 2026/09/25 15:25:19 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign4583worker2 # [ 92.528440] nix-grpc-daemon[916]: time=2026-09-25T15:25:19.152Z level=INFO msg="Uploading 1 narinfos"4584lb # [ 92.533670] niks3-server[955]: 2026/09/25 15:25:19 INFO Signed narinfos id=22 count=14585lb # [ 92.535483] niks3-server[955]: 2026/09/25 15:25:19 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4586lb # [ 92.545949] niks3-server[955]: 2026/09/25 15:25:19 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4587lb # [ 92.549770] niks3-server[955]: 2026/09/25 15:25:19 INFO Received complete push request method=POST path=/api/pushes/22/complete4588worker2 # [ 92.551729] nix-grpc-daemon[916]: time=2026-09-25T15:25:19.175Z level=INFO msg="Upload complete. (179ms)"4589lb # [ 92.557571] niks3-server[955]: 2026/09/25 15:25:19 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4590worker2 # [ 92.559511] nix-grpc-daemon[769]: ts=2026-09-25T15:25:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=5124591worker2 # [ 92.566606] nix-grpc-daemon[769]: ts=2026-09-25T15:25:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884592worker2 # [ 92.571694] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4593worker2 # [ 92.575385] nix-daemon[857]: reaped child process 1146, status = succeeded4594worker2 # [ 92.584251] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4595worker2 # [ 94.232849] nix-daemon[857]: reaped child process 1155, status = succeeded4596worker2 # [ 94.338426] nix-daemon[857]: accepted connection from pid 1172, user nix-grpc-daemon (trusted)4597lb # [ 94.358579] niks3-server[955]: 2026/09/25 15:25:20 INFO Received push request method=POST path=/api/pushes4598worker2 # [ 94.358993] nix-daemon[857]: reaped child process 1175, status = succeeded4599worker2 # [ 94.364404] nix-grpc-daemon[916]: time=2026-09-25T15:25:20.986Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4600worker2 # [ 94.365919] nix-grpc-daemon[916]: time=2026-09-25T15:25:20.986Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4601lb # [ 94.388345] niks3-server[955]: 2026/09/25 15:25:21 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign4602worker2 # [ 94.391109] nix-grpc-daemon[916]: time=2026-09-25T15:25:21.014Z level=INFO msg="Uploading 1 narinfos"4603lb # [ 94.397536] niks3-server[955]: 2026/09/25 15:25:21 INFO Signed narinfos id=23 count=14604lb # [ 94.402245] niks3-server[955]: 2026/09/25 15:25:21 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4605lb # [ 94.411062] niks3-server[955]: 2026/09/25 15:25:21 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4606lb # [ 94.421593] niks3-server[955]: 2026/09/25 15:25:21 INFO Received complete push request method=POST path=/api/pushes/23/complete4607worker2 # [ 94.423419] nix-grpc-daemon[916]: time=2026-09-25T15:25:21.047Z level=INFO msg="Upload complete. (229ms)"4608lb # [ 94.425748] niks3-server[955]: 2026/09/25 15:25:21 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4609lb # [ 94.428508] niks3-server[955]: 2026/09/25 15:25:21 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4610worker2 # [ 94.430300] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=13 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=04611worker2 # [ 94.442847] nix-daemon[857]: reaped child process 1153, status = succeeded4612worker1 # [ 94.449204] nix-grpc-daemon[486]: ts=2026-09-25T15:25:21Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:538824613worker1 # [ 94.456659] nix-grpc-daemon[486]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4614worker1 # [ 94.478622] nix-grpc-daemon[486]: ts=2026-09-25T15:25:21Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:538824615worker1 # [ 94.482594] nix-grpc-daemon[486]: ts=2026-09-25T15:25:21Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=1 nar_bytes_out=1284616worker2 # [ 94.674783] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:564884617worker2 # [ 94.679149] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884618worker2 # [ 94.683164] nix-grpc-daemon[916]: time=2026-09-25T15:25:21.306Z level=INFO msg="All 1 paths already cached"4619worker2 # [ 94.690800] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=debug event=want client=13 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04620worker2 # [ 94.694547] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14621worker2 # [ 94.699501] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884622worker2 # [ 94.705600] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884623worker2 # [ 94.713436] nix-grpc-daemon[916]: time=2026-09-25T15:25:21.337Z level=INFO msg="All 1 paths already cached"4624worker2 # [ 94.718598] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884625worker2 # [ 94.726676] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4626worker2 # [ 94.845037] nix-daemon[857]: accepted connection from pid 1181, user nix-grpc-daemon (trusted)4627lb # [ 94.865771] niks3-server[955]: 2026/09/25 15:25:21 INFO Received push request method=POST path=/api/pushes4628worker2 # [ 94.865464] nix-daemon[857]: reaped child process 1184, status = succeeded4629worker2 # [ 94.868843] nix-grpc-daemon[916]: time=2026-09-25T15:25:21.492Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4630worker2 # [ 94.870353] nix-grpc-daemon[916]: time=2026-09-25T15:25:21.492Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4631lb # [ 94.883919] niks3-server[955]: 2026/09/25 15:25:21 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4632worker2 # [ 94.888198] nix-grpc-daemon[916]: time=2026-09-25T15:25:21.511Z level=INFO msg="Uploading 1 narinfos"4633lb # [ 94.892395] niks3-server[955]: 2026/09/25 15:25:21 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign4634lb # [ 94.897248] niks3-server[955]: 2026/09/25 15:25:21 INFO Signed narinfos id=24 count=14635lb # [ 94.899020] niks3-server[955]: 2026/09/25 15:25:21 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4636lb # [ 94.911733] niks3-server[955]: 2026/09/25 15:25:21 INFO Received complete push request method=POST path=/api/pushes/24/complete4637lb # [ 94.916138] niks3-server[955]: 2026/09/25 15:25:21 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4638worker2 # [ 94.917566] nix-grpc-daemon[916]: time=2026-09-25T15:25:21.541Z level=INFO msg="Upload complete. (181ms)"4639worker2 # [ 94.923671] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=6244640worker2 # [ 94.933844] nix-grpc-daemon[769]: ts=2026-09-25T15:25:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884641worker2 # [ 94.940754] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4642worker2 # [ 94.943068] nix-daemon[857]: reaped child process 1179, status = succeeded4643worker2 # [ 94.952308] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4644worker2 # [ 95.621695] nix-daemon[857]: reaped child process 1188, status = succeeded4645worker2 # [ 95.675169] nix-daemon[857]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)4646worker2 # [ 95.691259] nix-daemon[857]: reaped child process 1207, status = succeeded4647lb # [ 95.699800] niks3-server[955]: 2026/09/25 15:25:22 INFO Received push request method=POST path=/api/pushes4648worker2 # [ 95.704380] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.326Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4649worker2 # [ 95.705913] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.326Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4650lb # [ 95.718576] niks3-server[955]: 2026/09/25 15:25:22 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4651lb # [ 95.730429] niks3-server[955]: 2026/09/25 15:25:22 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign4652worker2 # [ 95.730908] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.354Z level=INFO msg="Uploading 1 narinfos"4653lb # [ 95.736054] niks3-server[955]: 2026/09/25 15:25:22 INFO Signed narinfos id=25 count=14654lb # [ 95.740109] niks3-server[955]: 2026/09/25 15:25:22 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4655lb # [ 95.747059] niks3-server[955]: 2026/09/25 15:25:22 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4656lb # [ 95.755375] niks3-server[955]: 2026/09/25 15:25:22 INFO Received complete push request method=POST path=/api/pushes/25/complete4657worker2 # [ 95.757151] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.380Z level=INFO msg="Upload complete. (194ms)"4658lb # [ 95.761701] niks3-server[955]: 2026/09/25 15:25:22 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4659worker2 # [ 95.764348] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=14 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04660worker2 # [ 95.776394] nix-daemon[857]: reaped child process 1186, status = succeeded4661worker1 # [ 95.778761] nix-grpc-daemon[486]: ts=2026-09-25T15:25:22Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:538824662worker1 # [ 95.786283] nix-grpc-daemon[486]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4663worker1 # [ 95.807585] nix-grpc-daemon[486]: ts=2026-09-25T15:25:22Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:538824664worker1 # [ 95.811527] nix-grpc-daemon[486]: ts=2026-09-25T15:25:22Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=1 nar_bytes_out=1764665worker2 # [ 96.008263] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:564884666worker2 # [ 96.011996] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884667worker2 # [ 96.016650] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.639Z level=INFO msg="All 2 paths already cached"4668worker2 # [ 96.024725] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=debug event=want client=14 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04669worker2 # [ 96.028563] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14670worker2 # [ 96.033325] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884671worker2 # [ 96.039308] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884672worker2 # [ 96.046844] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.670Z level=INFO msg="All 2 paths already cached"4673worker2 # [ 96.052640] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884674worker2 # [ 96.061420] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4675worker2 # [ 96.184103] nix-daemon[857]: accepted connection from pid 1213, user nix-grpc-daemon (trusted)4676lb # [ 96.204491] niks3-server[955]: 2026/09/25 15:25:22 INFO Received push request method=POST path=/api/pushes4677worker2 # [ 96.204777] nix-daemon[857]: reaped child process 1216, status = succeeded4678worker2 # [ 96.207423] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.830Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4679worker2 # [ 96.210344] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.830Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4680lb # [ 96.253689] niks3-server[955]: 2026/09/25 15:25:22 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign4681worker2 # [ 96.254689] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.878Z level=INFO msg="Uploading 1 narinfos"4682lb # [ 96.258955] niks3-server[955]: 2026/09/25 15:25:22 INFO Signed narinfos id=26 count=14683lb # [ 96.280897] niks3-server[955]: 2026/09/25 15:25:22 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4684lb # [ 96.295044] niks3-server[955]: 2026/09/25 15:25:22 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4685lb # [ 96.312454] niks3-server[955]: 2026/09/25 15:25:22 INFO Received complete push request method=POST path=/api/pushes/26/complete4686worker2 # [ 96.314611] nix-grpc-daemon[916]: time=2026-09-25T15:25:22.938Z level=INFO msg="Upload complete. (244ms)"4687lb # [ 96.319082] niks3-server[955]: 2026/09/25 15:25:22 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4688worker2 # [ 96.323203] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=7524689worker2 # [ 96.331134] nix-grpc-daemon[769]: ts=2026-09-25T15:25:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884690worker2 # [ 96.349976] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4691worker2 # [ 96.366540] nix-daemon[857]: reaped child process 1211, status = succeeded4692worker2 # [ 96.372788] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4693worker2 # [ 97.213791] nix-daemon[857]: reaped child process 1220, status = succeeded4694worker2 # [ 97.303006] nix-daemon[857]: accepted connection from pid 1236, user nix-grpc-daemon (trusted)4695worker2 # [ 97.320665] nix-daemon[857]: reaped child process 1239, status = succeeded4696lb # [ 97.327304] niks3-server[955]: 2026/09/25 15:25:23 INFO Received push request method=POST path=/api/pushes4697worker2 # [ 97.331734] nix-grpc-daemon[916]: time=2026-09-25T15:25:23.954Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4698worker2 # [ 97.333483] nix-grpc-daemon[916]: time=2026-09-25T15:25:23.954Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4699lb # [ 97.350956] niks3-server[955]: 2026/09/25 15:25:23 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign4700worker2 # [ 97.352638] nix-grpc-daemon[916]: time=2026-09-25T15:25:23.976Z level=INFO msg="Uploading 1 narinfos"4701lb # [ 97.358313] niks3-server[955]: 2026/09/25 15:25:23 INFO Signed narinfos id=27 count=14702lb # [ 97.363828] niks3-server[955]: 2026/09/25 15:25:23 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4703lb # [ 97.370968] niks3-server[955]: 2026/09/25 15:25:23 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4704lb # [ 97.376207] niks3-server[955]: 2026/09/25 15:25:23 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4705worker2 # [ 97.383540] nix-grpc-daemon[916]: time=2026-09-25T15:25:24.007Z level=INFO msg="Upload complete. (200ms)"4706lb # [ 97.385924] niks3-server[955]: 2026/09/25 15:25:24 INFO Received complete push request method=POST path=/api/pushes/27/complete4707lb # [ 97.392077] niks3-server[955]: 2026/09/25 15:25:24 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4708worker2 # [ 97.392655] nix-grpc-daemon[769]: ts=2026-09-25T15:25:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=15 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=04709worker1 # [ 97.396936] nix-grpc-daemon[486]: ts=2026-09-25T15:25:24Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:538824710worker2 # [ 97.403019] nix-daemon[857]: reaped child process 1218, status = succeeded4711worker1 # [ 97.405113] nix-grpc-daemon[486]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4712worker2 # [ 97.426254] nix-grpc-daemon[769]: ts=2026-09-25T15:25:24Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:564884713worker2 # [ 97.430248] nix-grpc-daemon[769]: ts=2026-09-25T15:25:24Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_out=2324714client # [ 97.495509] nix-daemon[1164]: reaped child process 1166, status = succeeded4715client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.56 seconds)4716client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4717client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.02 seconds)4718client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4719client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.04 seconds)4720client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4721client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4722(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.65 seconds)4723subtest: an input only one worker has still reaches the builder4724client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4725client # this derivation will be built:4726client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4727client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4728client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)4729client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4730client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4731worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4732lb # [ 98.116336] envoy[902]: conn peer=[2001:db8:1::1]:41408 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2986 tx=792 ms=41714733lb # [ 98.118997] envoy[902]: conn peer=[2001:db8:1::1]:41420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=46274734lb # [ 98.121625] envoy[902]: conn peer=[2001:db8:1::1]:51962 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=204735lb # [ 98.124038] envoy[902]: conn peer=[2001:db8:1::1]:51956 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3397 tx=1924 ms=21804736lb # [ 98.126539] envoy[902]: conn peer=[2001:db8:1::1]:51964 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3845 tx=2136 ms=11454737lb # [ 98.129126] envoy[902]: conn peer=[2001:db8:1::1]:51966 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=154738lb # [ 98.131512] envoy[902]: conn peer=[2001:db8:1::1]:51968 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4378 tx=2390 ms=14334739lb # [ 98.134091] envoy[902]: conn peer=[2001:db8:1::1]:51984 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=406 ms=174740worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.15 seconds)4741worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4742worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.02 seconds)4743client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24744client # [ 98.310617] nix-daemon[1164]: accepted connection from pid 1247, user root (trusted)4745client # this derivation will be built:4746client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4747worker2 # [ 98.450975] nix-grpc-daemon[769]: ts=2026-09-25T15:25:25Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:564884748client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4749worker2 # [ 98.456136] nix-grpc-daemon[769]: ts=2026-09-25T15:25:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884750client # copying 1 paths...4751client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4752worker2 # [ 98.463049] nix-grpc-daemon[769]: ts=2026-09-25T15:25:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884753worker2 # [ 98.471989] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4754worker2 # [ 98.603525] nix-daemon[857]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)4755worker2 # [ 98.620453] nix-daemon[857]: reaped child process 1249, status = succeeded4756lb # [ 98.629077] niks3-server[955]: 2026/09/25 15:25:25 INFO Received push request method=POST path=/api/pushes4757worker2 # [ 98.632484] nix-grpc-daemon[916]: time=2026-09-25T15:25:25.255Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4758worker2 # [ 98.634127] nix-grpc-daemon[916]: time=2026-09-25T15:25:25.255Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4759lb # [ 98.646473] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4760worker2 # [ 98.649919] nix-grpc-daemon[916]: time=2026-09-25T15:25:25.273Z level=INFO msg="Uploading 1 narinfos"4761lb # [ 98.654721] niks3-server[955]: 2026/09/25 15:25:25 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign4762lb # [ 98.660408] niks3-server[955]: 2026/09/25 15:25:25 INFO Signed narinfos id=28 count=14763lb # [ 98.662167] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4764lb # [ 98.671587] niks3-server[955]: 2026/09/25 15:25:25 INFO Received complete push request method=POST path=/api/pushes/28/complete4765worker2 # [ 98.673154] nix-grpc-daemon[916]: time=2026-09-25T15:25:25.296Z level=INFO msg="Upload complete. (192ms)"4766lb # [ 98.676828] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4767worker2 # [ 98.680110] nix-grpc-daemon[769]: ts=2026-09-25T15:25:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_in=1204768worker2 # [ 98.689067] nix-grpc-daemon[769]: ts=2026-09-25T15:25:25Z level=debug event=want client=15 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04769worker1 # [ 98.694051] nix-grpc-daemon[486]: ts=2026-09-25T15:25:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824770worker2 # [ 98.691956] nix-grpc-daemon[769]: ts=2026-09-25T15:25:25Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14771worker2 # [ 98.697515] nix-daemon[857]: reaped child process 1244, status = succeeded4772worker1 # [ 98.703909] nix-grpc-daemon[486]: ts=2026-09-25T15:25:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538824773client # copying 2 paths to 'lb:50051'...4774worker1 # [ 98.714696] nix-grpc-daemon[486]: ts=2026-09-25T15:25:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538824775client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4776client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4777worker1 # [ 98.724061] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4778worker1 # [ 98.855378] nix-daemon[838]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)4779lb # [ 98.872472] niks3-server[955]: 2026/09/25 15:25:25 INFO Received push request method=POST path=/api/pushes4780worker1 # [ 98.875554] nix-daemon[838]: reaped child process 1167, status = succeeded4781worker1 # [ 98.879744] nix-grpc-daemon[850]: time=2026-09-25T15:25:25.504Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4782worker1 # [ 98.882129] nix-grpc-daemon[850]: time=2026-09-25T15:25:25.504Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4783worker1 # [ 98.884723] nix-grpc-daemon[850]: time=2026-09-25T15:25:25.504Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4784lb # [ 98.897422] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4785lb # [ 98.905895] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4786lb # [ 98.916452] niks3-server[955]: 2026/09/25 15:25:25 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign4787worker1 # [ 98.919911] nix-grpc-daemon[850]: time=2026-09-25T15:25:25.546Z level=INFO msg="Uploading 2 narinfos"4788lb # [ 98.923851] niks3-server[955]: 2026/09/25 15:25:25 INFO Signed narinfos id=29 count=24789lb # [ 98.927371] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4790lb # [ 98.932868] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4791lb # [ 98.939463] niks3-server[955]: 2026/09/25 15:25:25 INFO Received complete push request method=POST path=/api/pushes/29/complete4792worker1 # [ 98.945439] nix-grpc-daemon[850]: time=2026-09-25T15:25:25.572Z level=INFO msg="Upload complete. (210ms)"4793lb # [ 98.947467] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4794lb # [ 98.952470] niks3-server[955]: 2026/09/25 15:25:25 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4795worker1 # [ 98.952611] nix-grpc-daemon[486]: ts=2026-09-25T15:25:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=2 nar_bytes_in=9844796worker1 # [ 98.955955] nix-grpc-daemon[486]: ts=2026-09-25T15:25:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824797worker1 # [ 98.967971] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4798worker1 # [ 98.971944] nix-daemon[838]: reaped child process 1162, status = succeeded4799client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4800worker1 # [ 98.983750] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4801client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4802client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4803worker1 # [ 99.815007] nix-daemon[838]: reaped child process 1171, status = succeeded4804worker1 # [ 99.917759] nix-daemon[838]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)4805worker1 # [ 99.934854] nix-daemon[838]: reaped child process 1190, status = succeeded4806lb # [ 99.941128] niks3-server[955]: 2026/09/25 15:25:26 INFO Received push request method=POST path=/api/pushes4807worker1 # [ 99.947735] nix-grpc-daemon[850]: time=2026-09-25T15:25:26.573Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4808lb # [ 99.958427] niks3-server[955]: 2026/09/25 15:25:26 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4809worker1 # [ 99.965492] nix-grpc-daemon[850]: time=2026-09-25T15:25:26.592Z level=INFO msg="Uploading 1 narinfos"4810lb # [ 99.966054] niks3-server[955]: 2026/09/25 15:25:26 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign4811lb # [ 99.972680] niks3-server[955]: 2026/09/25 15:25:26 INFO Signed narinfos id=30 count=14812lb # [ 99.974423] niks3-server[955]: 2026/09/25 15:25:26 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4813lb # [ 99.984242] niks3-server[955]: 2026/09/25 15:25:26 INFO Received complete push request method=POST path=/api/pushes/30/complete4814worker1 # [ 99.988113] nix-grpc-daemon[850]: time=2026-09-25T15:25:26.615Z level=INFO msg="Upload complete. (196ms)"4815lb # [ 99.989686] niks3-server[955]: 2026/09/25 15:25:26 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4816client # copying 1 paths...4817worker1 # [ 99.995191] nix-grpc-daemon[486]: ts=2026-09-25T15:25:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=16 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04818worker1 # [ 100.008465] nix-grpc-daemon[486]: ts=2026-09-25T15:25:26Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:538824819worker1 # [ 100.011882] nix-daemon[838]: reaped child process 1169, status = succeeded4820client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4821worker2 # [ 100.024709] nix-grpc-daemon[769]: ts=2026-09-25T15:25:26Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:564884822worker2 # [ 100.033051] nix-grpc-daemon[769]: copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'http://lb:5751'...4823worker2 # [ 100.043345] nix-grpc-daemon[769]: ts=2026-09-25T15:25:26Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_out=1204824client # [ 100.107395] nix-daemon[1164]: reaped child process 1250, status = succeeded4825client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.94 seconds)4826client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24827client # [ 100.241667] nix-daemon[1164]: accepted connection from pid 1269, user root (trusted)4828client # this derivation will be built:4829client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4830worker2 # [ 100.378533] nix-grpc-daemon[769]: ts=2026-09-25T15:25:27Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:564884831client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4832worker2 # [ 100.383414] nix-grpc-daemon[769]: ts=2026-09-25T15:25:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884833worker2 # [ 100.389817] nix-grpc-daemon[916]: time=2026-09-25T15:25:27.013Z level=INFO msg="All 1 paths already cached"4834worker2 # [ 100.400538] nix-grpc-daemon[769]: ts=2026-09-25T15:25:27Z level=debug event=want client=16 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=04835worker2 # [ 100.404483] nix-grpc-daemon[769]: ts=2026-09-25T15:25:27Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=14836worker2 # [ 100.409223] nix-grpc-daemon[769]: ts=2026-09-25T15:25:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884837worker2 # [ 100.415195] nix-grpc-daemon[769]: ts=2026-09-25T15:25:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564884838client # copying 1 paths to 'lb:50051'...4839client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...4840worker2 # [ 100.426151] nix-grpc-daemon[769]: ts=2026-09-25T15:25:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564884841worker2 # [ 100.434867] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4842worker2 # [ 100.449114] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)4843worker2 # [ 100.506079] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 151 ms (attempt 2/5)4844worker2 # [ 100.662144] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 3/5)4845worker2 # [ 100.868272] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 404 ms (attempt 4/5)4846worker2 # [ 101.274374] nix-grpc-daemon[769]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4847worker2 # [ 101.277735] nix-grpc-daemon[769]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4848worker2 # [ 101.283758] nix-grpc-daemon[769]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...4849worker2 # [ 101.422835] nix-daemon[857]: accepted connection from pid 1263, user nix-grpc-daemon (trusted)4850worker2 # [ 101.445275] nix-daemon[857]: reaped child process 1266, status = succeeded4851lb # [ 101.448704] niks3-server[955]: 2026/09/25 15:25:28 INFO Received push request method=POST path=/api/pushes4852worker2 # [ 101.452561] nix-grpc-daemon[916]: time=2026-09-25T15:25:28.075Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4853worker2 # [ 101.454968] nix-grpc-daemon[916]: time=2026-09-25T15:25:28.075Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"4854lb # [ 101.469408] niks3-server[955]: 2026/09/25 15:25:28 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst4855worker2 # [ 101.473782] nix-grpc-daemon[916]: time=2026-09-25T15:25:28.097Z level=INFO msg="Uploading 1 narinfos"4856lb # [ 101.478525] niks3-server[955]: 2026/09/25 15:25:28 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign4857lb # [ 101.483933] niks3-server[955]: 2026/09/25 15:25:28 INFO Signed narinfos id=31 count=14858lb # [ 101.487910] niks3-server[955]: 2026/09/25 15:25:28 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls4859worker2 # [ 101.494867] nix-grpc-daemon[916]: time=2026-09-25T15:25:28.118Z level=INFO msg="Upload complete. (200ms)"4860lb # [ 101.497183] niks3-server[955]: 2026/09/25 15:25:28 INFO Received complete push request method=POST path=/api/pushes/31/complete4861lb # [ 101.501078] niks3-server[955]: 2026/09/25 15:25:28 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo4862worker2 # [ 101.500186] nix-grpc-daemon[769]: ts=2026-09-25T15:25:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=1 paths=1 nar_bytes_in=5684863worker2 # [ 101.508481] nix-grpc-daemon[769]: ts=2026-09-25T15:25:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564884864worker2 # [ 101.519725] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4865worker2 # [ 101.522199] nix-daemon[857]: reaped child process 1256, status = succeeded4866client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...4867worker2 # [ 101.531063] nix-daemon[857]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4868client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4869client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds4870worker2 # [ 102.988464] nix-daemon[857]: reaped child process 1270, status = succeeded4871worker2 # [ 103.078482] nix-daemon[857]: accepted connection from pid 1286, user nix-grpc-daemon (trusted)4872lb # [ 103.105365] niks3-server[955]: 2026/09/25 15:25:29 INFO Received push request method=POST path=/api/pushes4873worker2 # [ 103.108573] nix-daemon[857]: reaped child process 1289, status = succeeded4874worker2 # [ 103.114544] nix-grpc-daemon[916]: time=2026-09-25T15:25:29.734Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4875lb # [ 103.129835] niks3-server[955]: 2026/09/25 15:25:29 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4876worker2 # [ 103.135845] nix-grpc-daemon[916]: time=2026-09-25T15:25:29.759Z level=INFO msg="Uploading 1 narinfos"4877lb # [ 103.140423] niks3-server[955]: 2026/09/25 15:25:29 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign4878lb # [ 103.146727] niks3-server[955]: 2026/09/25 15:25:29 INFO Signed narinfos id=32 count=14879lb # [ 103.153227] niks3-server[955]: 2026/09/25 15:25:29 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls4880lb # [ 103.161677] niks3-server[955]: 2026/09/25 15:25:29 INFO Received complete push request method=POST path=/api/pushes/32/complete4881worker2 # [ 103.163394] nix-grpc-daemon[916]: time=2026-09-25T15:25:29.787Z level=INFO msg="Upload complete. (219ms)"4882lb # [ 103.168757] niks3-server[955]: 2026/09/25 15:25:29 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo4883worker2 # [ 103.172708] nix-grpc-daemon[769]: ts=2026-09-25T15:25:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=17 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04884client # copying 1 paths...4885worker2 # [ 103.186284] nix-grpc-daemon[769]: ts=2026-09-25T15:25:29Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564884886worker2 # [ 103.189396] nix-daemon[857]: reaped child process 1268, status = succeeded4887client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...4888worker1 # [ 103.213240] nix-grpc-daemon[486]: ts=2026-09-25T15:25:29Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:538824889worker1 # [ 103.217814] nix-grpc-daemon[486]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...4890worker1 # [ 103.229049] nix-grpc-daemon[486]: ts=2026-09-25T15:25:29Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=1 nar_bytes_out=1204891client # [ 103.293679] nix-daemon[1164]: reaped child process 1272, status = succeeded4892client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.19 seconds)4893client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&24894client # [ 103.427699] nix-daemon[1164]: accepted connection from pid 1291, user root (trusted)4895client # this derivation will be built:4896client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4897worker1 # [ 103.575036] nix-grpc-daemon[486]: ts=2026-09-25T15:25:30Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:538824898client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4899worker1 # [ 103.579675] nix-grpc-daemon[486]: ts=2026-09-25T15:25:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538824900worker1 # [ 103.584076] nix-grpc-daemon[850]: time=2026-09-25T15:25:30.210Z level=INFO msg="All 1 paths already cached"4901worker2 # [ 103.590671] nix-grpc-daemon[769]: ts=2026-09-25T15:25:30Z level=debug event=want client=17 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=04902worker1 # [ 103.595019] nix-grpc-daemon[486]: ts=2026-09-25T15:25:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824903worker2 # [ 103.594457] nix-grpc-daemon[769]: ts=2026-09-25T15:25:30Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=14904worker1 # [ 103.604217] nix-grpc-daemon[486]: ts=2026-09-25T15:25:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:538824905worker1 # [ 103.612450] nix-grpc-daemon[850]: time=2026-09-25T15:25:30.239Z level=INFO msg="All 1 paths already cached"4906client # copying 1 paths to 'lb:50051'...4907client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...4908worker1 # [ 103.618855] nix-grpc-daemon[486]: ts=2026-09-25T15:25:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:538824909worker1 # [ 103.628211] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4910worker1 # [ 103.755101] nix-daemon[838]: accepted connection from pid 1195, user nix-grpc-daemon (trusted)4911worker1 # [ 103.772460] nix-daemon[838]: reaped child process 1198, status = succeeded4912lb # [ 103.778293] niks3-server[955]: 2026/09/25 15:25:30 INFO Received push request method=POST path=/api/pushes4913worker1 # [ 103.783573] nix-grpc-daemon[850]: time=2026-09-25T15:25:30.410Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4914worker1 # [ 103.785255] nix-grpc-daemon[850]: time=2026-09-25T15:25:30.410Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"4915lb # [ 103.794553] niks3-server[955]: 2026/09/25 15:25:30 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign4916worker1 # [ 103.797722] nix-grpc-daemon[850]: time=2026-09-25T15:25:30.424Z level=INFO msg="Uploading 1 narinfos"4917lb # [ 103.801788] niks3-server[955]: 2026/09/25 15:25:30 INFO Signed narinfos id=33 count=14918lb # [ 103.806107] niks3-server[955]: 2026/09/25 15:25:30 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls4919lb # [ 103.814411] niks3-server[955]: 2026/09/25 15:25:30 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst4920lb # [ 103.818806] niks3-server[955]: 2026/09/25 15:25:30 INFO Received complete push request method=POST path=/api/pushes/33/complete4921worker1 # [ 103.824147] nix-grpc-daemon[850]: time=2026-09-25T15:25:30.451Z level=INFO msg="Upload complete. (185ms)"4922lb # [ 103.824692] niks3-server[955]: 2026/09/25 15:25:30 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo4923worker1 # [ 103.830548] nix-grpc-daemon[486]: ts=2026-09-25T15:25:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=0 paths=1 nar_bytes_in=5684924worker1 # [ 103.836211] nix-grpc-daemon[486]: ts=2026-09-25T15:25:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:538824925worker1 # [ 103.845954] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4926worker1 # [ 103.847980] nix-daemon[838]: reaped child process 1193, status = succeeded4927client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...4928client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4929worker1 # [ 103.857874] nix-daemon[838]: accepted connection from pid 486, user nix-grpc-daemon (trusted)4930client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds4931worker1 # [ 105.090591] nix-daemon[838]: reaped child process 1202, status = succeeded4932worker1 # [ 105.204088] nix-daemon[838]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)4933lb # [ 105.225192] niks3-server[955]: 2026/09/25 15:25:31 INFO Received push request method=POST path=/api/pushes4934worker1 # [ 105.226722] nix-daemon[838]: reaped child process 1221, status = succeeded4935worker1 # [ 105.231640] nix-grpc-daemon[850]: time=2026-09-25T15:25:31.857Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4936lb # [ 105.244256] niks3-server[955]: 2026/09/25 15:25:31 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign4937worker1 # [ 105.251037] nix-grpc-daemon[850]: time=2026-09-25T15:25:31.878Z level=INFO msg="Uploading 1 narinfos"4938lb # [ 105.252080] niks3-server[955]: 2026/09/25 15:25:31 INFO Signed narinfos id=34 count=14939lb # [ 105.256816] niks3-server[955]: 2026/09/25 15:25:31 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv4940lb # [ 105.265540] niks3-server[955]: 2026/09/25 15:25:31 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls4941worker1 # [ 105.269929] nix-grpc-daemon[850]: time=2026-09-25T15:25:31.897Z level=INFO msg="Upload complete. (205ms)"4942lb # [ 105.270471] niks3-server[955]: 2026/09/25 15:25:31 INFO Received complete push request method=POST path=/api/pushes/34/complete4943lb # [ 105.275075] niks3-server[955]: 2026/09/25 15:25:31 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo4944client # copying 1 paths...4945worker1 # [ 105.276546] nix-grpc-daemon[486]: ts=2026-09-25T15:25:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53882 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=18 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=04946worker2 # [ 105.282148] nix-grpc-daemon[769]: ts=2026-09-25T15:25:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:564884947worker1 # [ 105.289034] nix-daemon[838]: reaped child process 1200, status = succeeded4948worker2 # [ 105.290828] nix-grpc-daemon[769]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...4949client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...4950worker2 # [ 105.317658] nix-grpc-daemon[769]: ts=2026-09-25T15:25:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:564884951worker2 # [ 105.321962] nix-grpc-daemon[769]: ts=2026-09-25T15:25:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56488 duration_s=0 paths=1 nar_bytes_out=1204952client # [ 105.386451] nix-daemon[1164]: reaped child process 1294, status = succeeded4953client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.09 seconds)4954(finished: subtest: an input only one worker has still reaches the builder, in 7.80 seconds)4955subtest: upload whose reference the worker lacks is completed from the cache4956client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache4957client # this derivation will be built:4958client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv4959client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...4960client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.32 seconds)4961client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache4962client # this derivation will be built:4963client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv4964client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...4965client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.28 seconds)4966client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure4967client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.11 seconds)4968worker1: must succeed: nix-store --import < /tmp/shared/ref.closure4969worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.15 seconds)4970worker2 # [ 106.291043] systemd[1]: Stopping Nix worker-protocol over gRPC...4971worker2 # [ 107.054126] nix-grpc-daemon[769]: ts=2026-09-25T15:25:33Z level=info event=signal_exit4972lb # [ 107.588828] niks3-server[955]: 2026/09/25 15:25:34 INFO lead: released remote=192.168.1.4:505124973lb # [ 108.116315] envoy[902]: conn peer=[2001:db8:1::1]:51994 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4223 tx=2051 ms=16044974lb # [ 108.119029] envoy[902]: conn peer=[2001:db8:1::1]:51996 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=304975lb # [ 108.121643] envoy[902]: conn peer=[2001:db8:1::1]:52002 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3752 tx=2122 ms=28604976lb # [ 108.124183] envoy[902]: conn peer=[2001:db8:1::1]:45392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=329 ms=324977lb # [ 108.126572] envoy[902]: conn peer=[2001:db8:1::1]:45398 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3723 tx=2104 ms=17614978lb # [ 108.129107] envoy[902]: conn peer=[2001:db8:1::1]:45400 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=164979worker2 # [ 108.585626] nix-grpc-daemon[769]: ts=2026-09-25T15:25:35Z level=info event=scheduler_restarting4980lb # [ 108.586509] envoy[902]: [2026-09-25 15:25:35.210][902][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}4981worker2 # [ 108.590208] nix-grpc-daemon[769]: ts=2026-09-25T15:25:35Z level=info event=scheduler_restarting addr=lb:500514982worker1 # [ 108.593016] nix-grpc-daemon[486]: ts=2026-09-25T15:25:35Z level=info event=scheduler_restarting addr=lb:500514983worker2 # [ 108.597266] nix-grpc-daemon[769]: ts=2026-09-25T15:25:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:343284984worker2 # [ 108.600410] nix-grpc-daemon[769]: ts=2026-09-25T15:25:35Z level=info event=worker_gone addr=192.168.1.3:500514985worker2 # [ 108.602728] nix-grpc-daemon[769]: ts=2026-09-25T15:25:35Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:343284986worker2 # [ 108.605578] nix-grpc-daemon[769]: ts=2026-09-25T15:25:35Z level=info event=worker_gone addr=192.168.1.4:500514987worker2 # [ 108.622380] nix-daemon[857]: reaped child process 859, status = succeeded4988worker1 # [ 108.793032] nix-grpc-daemon[486]: ts=2026-09-25T15:25:35Z level=info event=scheduler_connected addr=lb:500514989worker1 # [ 108.796109] nix-grpc-daemon[486]: ts=2026-09-25T15:25:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4990worker2 # [ 108.806362] nix-grpc-daemon[769]: ts=2026-09-25T15:25:35Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"4991worker1 # [ 108.996496] nix-grpc-daemon[486]: ts=2026-09-25T15:25:35Z level=info event=scheduler_connected addr=lb:500514992worker1 # [ 108.999304] nix-grpc-daemon[486]: ts=2026-09-25T15:25:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4993worker1 # [ 109.199745] nix-grpc-daemon[486]: ts=2026-09-25T15:25:35Z level=info event=scheduler_connected addr=lb:500514994worker1 # [ 109.202650] nix-grpc-daemon[486]: ts=2026-09-25T15:25:35Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4995worker2 # [ 109.230566] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.4996worker2 # [ 109.234945] systemd[1]: Stopped Nix worker-protocol over gRPC.4997worker2 # [ 109.238129] systemd[1]: nix-grpc-daemon.service: Consumed 2.685s CPU time over 1min 31.364s wall clock time, 64.6M memory peak, 326.5K incoming IP traffic, 395.5K outgoing IP traffic.4998lb: waiting until exactly ['worker1'] are healthy members of aarch64-linux4999lb: must succeed: curl -sf localhost:9901/clusters5000lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5001lb: (finished: waiting until exactly ['worker1'] are healthy members of aarch64-linux, in 0.04 seconds)5002client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25003worker1 # [ 109.403186] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_connected addr=lb:500515004worker1 # [ 109.406173] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5005worker1 # [ 109.462160] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:538825006client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5007worker1 # [ 109.466250] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:538825008client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.19 seconds)5009worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5010worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.02 seconds)5011worker2 # [ 109.541697] systemd[1]: Starting scheduler-token.service...5012worker2 # [ 109.603038] systemd[1]: scheduler-token.service: Deactivated successfully.5013worker1 # [ 109.607817] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_connected addr=lb:500515014worker2 # [ 109.608304] systemd[1]: Finished scheduler-token.service.5015worker1 # [ 109.610235] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5016worker2 # [ 109.609528] systemd[1]: scheduler-token.service: Consumed 35ms CPU time over 64ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5017worker2 # [ 109.622609] systemd[1]: Starting Nix worker-protocol over gRPC...5018worker2 # [ 109.727766] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5019worker2 # [ 109.730304] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5020worker2 # [ 109.734179] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15021worker2 # [ 109.739568] nix-grpc-daemon[1316]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5022worker2 # [ 109.742821] nix-grpc-daemon[1316]: E0000 00:00:1790349936.362686 1316 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5023lb # [ 109.754299] niks3-server[955]: 2026/09/25 15:25:36 INFO lead: acquired remote=192.168.1.4:496705024worker2 # [ 109.764179] nix-grpc-daemon[1316]: E0000 00:00:1790349936.366447 1316 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5025worker2 # [ 109.766597] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5026worker2 # [ 109.791303] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=scheduler_take_over5027worker2 # [ 109.793182] nix-grpc-daemon[1316]: E0000 00:00:1790349936.375729 1357 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5028worker2 # [ 109.798870] systemd[1]: Started Nix worker-protocol over gRPC.5029worker2 # [ 109.801955] nix-grpc-daemon[1316]: E0000 00:00:1790349936.397564 1357 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5030worker2 # [ 109.804437] nix-grpc-daemon[1316]: E0000 00:00:1790349936.397767 1357 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5031worker1 # [ 109.808792] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_connected addr=lb:500515032worker2 # [ 109.806950] nix-grpc-daemon[1316]: E0000 00:00:1790349936.414297 1357 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5033worker2 # [ 109.809619] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=scheduler_connected addr=lb:500515034worker2 # [ 109.810936] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5035worker1 # [ 109.813543] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5036worker1 # [ 109.831263] systemd[1]: Stopping Nix worker-protocol over gRPC...5037worker2 # [ 110.002817] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=scheduler_connected addr=lb:500515038worker2 # [ 110.005614] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5039worker1 # [ 110.012899] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_connected addr=lb:500515040worker1 # [ 110.015742] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5041worker2 # [ 110.206464] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=scheduler_connected addr=lb:500515042worker2 # [ 110.209963] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5043worker1 # [ 110.216298] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_connected addr=lb:500515044worker1 # [ 110.219113] nix-grpc-daemon[486]: ts=2026-09-25T15:25:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5045worker2 # [ 110.410757] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:37Z level=info event=scheduler_connected addr=lb:500515046worker1 # [ 110.419609] nix-grpc-daemon[486]: ts=2026-09-25T15:25:37Z level=info event=scheduler_connected addr=lb:500515047worker2 # [ 110.418695] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:389865048worker2 # [ 110.422483] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05049worker2 # [ 110.429123] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:37Z level=info event=oidc_jwks provider=mock keys=15050worker2 # [ 110.431371] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:389865051worker2 # [ 110.434687] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05052worker1 # [ 110.761402] nix-grpc-daemon[486]: ts=2026-09-25T15:25:37Z level=info event=signal_exit5053worker1 # [ 112.593553] nix-grpc-daemon[486]: ts=2026-09-25T15:25:39Z level=info event=scheduler_restarting5054worker2 # [ 112.590604] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:39Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:389865055worker2 # [ 112.590814] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:39Z level=info event=worker_gone addr=192.168.1.3:500515056worker1 # [ 112.608642] nix-grpc-daemon[486]: ts=2026-09-25T15:25:39Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5057worker1 # [ 112.623878] nix-daemon[838]: reaped child process 840, status = succeeded5058worker2 # [ 113.068269] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:39Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:545305059worker1 # [ 113.204436] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5060worker1 # [ 113.208443] systemd[1]: Stopped Nix worker-protocol over gRPC.5061worker1 # [ 113.209912] systemd[1]: nix-grpc-daemon.service: Consumed 2.633s CPU time over 1min 43.770s wall clock time, 65.2M memory peak, 177.4K incoming IP traffic, 345.8K outgoing IP traffic.5062lb: waiting until exactly ['worker2'] are healthy members of aarch64-linux5063lb: must succeed: curl -sf localhost:9901/clusters5064lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5065lb: (finished: waiting until exactly ['worker2'] are healthy members of aarch64-linux, in 0.04 seconds)5066client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25067worker2 # [ 113.436197] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545425068worker2 # [ 113.445319] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5069worker2 # [ 113.458999] nix-grpc-daemon[1316]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 98 ms (attempt 1/5)5070worker2 # [ 113.562094] nix-grpc-daemon[1316]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 2/5)5071worker2 # [ 113.706759] nix-grpc-daemon[1316]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 354 ms (attempt 3/5)5072worker2 # [ 114.061207] nix-grpc-daemon[1316]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 363 ms (attempt 4/5)5073worker2 # [ 114.425682] nix-grpc-daemon[1316]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5074worker2 # [ 114.427746] nix-grpc-daemon[1316]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5075client # copying 2 paths...5076client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5077worker2 # [ 114.443330] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545425078client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5079worker2 # [ 114.454765] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5080worker2 # [ 114.474528] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:41Z level=info event=niks3_push_started pid=13765081worker2 # [ 114.597747] nix-grpc-daemon[1376]: time=2026-09-25T15:25:41.221Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5082worker2 # [ 114.702474] nix-daemon[857]: accepted connection from pid 1388, user nix-grpc-daemon (trusted)5083worker2 # [ 114.722731] nix-daemon[857]: reaped child process 1391, status = succeeded5084lb # [ 114.724139] niks3-server[955]: 2026/09/25 15:25:41 INFO Received push request method=POST path=/api/pushes5085worker2 # [ 114.728100] nix-grpc-daemon[1376]: time=2026-09-25T15:25:41.350Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5086worker2 # [ 114.729637] nix-grpc-daemon[1376]: time=2026-09-25T15:25:41.350Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5087worker2 # [ 114.735131] nix-grpc-daemon[1376]: time=2026-09-25T15:25:41.351Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5088lb # [ 114.766338] niks3-server[955]: 2026/09/25 15:25:41 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5089lb # [ 114.776485] niks3-server[955]: 2026/09/25 15:25:41 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5090lb # [ 114.785114] niks3-server[955]: 2026/09/25 15:25:41 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5091lb # [ 114.793551] niks3-server[955]: 2026/09/25 15:25:41 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5092lb # [ 114.800381] niks3-server[955]: 2026/09/25 15:25:41 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5093worker2 # [ 114.798795] nix-grpc-daemon[1376]: time=2026-09-25T15:25:41.421Z level=INFO msg="Uploading 2 narinfos"5094lb # [ 114.803455] niks3-server[955]: 2026/09/25 15:25:41 INFO Signed narinfos id=35 count=25095lb # [ 114.821766] niks3-server[955]: 2026/09/25 15:25:41 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5096lb # [ 114.830354] niks3-server[955]: 2026/09/25 15:25:41 INFO Received complete push request method=POST path=/api/pushes/35/complete5097worker2 # [ 114.832565] nix-grpc-daemon[1376]: time=2026-09-25T15:25:41.456Z level=INFO msg="Upload complete. (231ms)"5098lb # [ 114.835425] niks3-server[955]: 2026/09/25 15:25:41 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5099worker2 # [ 114.839793] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54542 duration_s=0 paths=2 nar_bytes_in=3125100worker2 # [ 114.849433] nix-daemon[857]: reaped child process 1374, status = succeeded5101client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.60 seconds)5102worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5103worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.02 seconds)5104worker1 # [ 114.926365] systemd[1]: Starting Nix worker-protocol over gRPC...5105worker1 # [ 115.038160] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5106worker1 # [ 115.040460] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5107worker1 # [ 115.044258] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15108worker1 # [ 115.050631] nix-grpc-daemon[1244]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5109worker1 # [ 115.072321] nix-grpc-daemon[1244]: E0000 00:00:1790349941.675691 1244 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5110worker1 # [ 115.074827] nix-grpc-daemon[1244]: E0000 00:00:1790349941.677638 1244 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5111lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux5112lb: must succeed: curl -sf localhost:9901/clusters5113worker1 # [ 115.097820] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:41Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5114worker1 # [ 115.099700] nix-grpc-daemon[1244]: E0000 00:00:1790349941.685399 1287 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5115worker1 # [ 115.105271] nix-grpc-daemon[1244]: E0000 00:00:1790349941.708453 1287 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5116worker1 # [ 115.109370] nix-grpc-daemon[1244]: E0000 00:00:1790349941.708601 1287 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5117worker2 # [ 115.108650] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:41Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:389865118worker1 # [ 115.112693] nix-grpc-daemon[1244]: E0000 00:00:1790349941.723839 1287 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5119worker2 # [ 115.113463] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:41Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05120worker1 # [ 115.118502] systemd[1]: Started Nix worker-protocol over gRPC.5121worker1 # [ 115.119392] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:41Z level=info event=scheduler_connected addr=lb:500515122lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5123lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 0.06 seconds)5124(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.75 seconds)5125subtest: a niks3 restart does not cost the leader its role5126lb: must succeed: curl -sf localhost:9901/clusters5127lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5128lb: must succeed: curl -sf localhost:9901/clusters5129lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5130lb # [ 115.240774] systemd[1]: Stopping niks3 server...5131worker1 # [ 115.241923] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:41Z level=info event=scheduler_lock_lost status=2005132lb # [ 115.244690] niks3-server[955]: 2026/09/25 15:25:41 INFO Shutdown signal received, draining in-flight requests timeout=10s5133worker2 # [ 115.242509] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:41Z level=info event=scheduler_lock_lost status=2005134lb # [ 115.246389] niks3-server[955]: 2026/09/25 15:25:41 INFO lead: released remote=192.168.1.4:496705135lb # [ 115.268301] systemd[1]: niks3.service: Deactivated successfully.5136lb # [ 115.269234] systemd[1]: Stopped niks3 server.5137lb # [ 115.269903] systemd[1]: niks3.service: Consumed 439ms CPU time over 1min 33.922s wall clock time, 27.9M memory peak, 98K incoming IP traffic, 141.5K outgoing IP traffic.5138lb # [ 115.279310] systemd[1]: Starting niks3 server...5139lb # [ 115.345685] niks3-server[1153]: 2026/09/25 15:25:41 goose: no migrations to run. current version: 202609231200005140lb # [ 115.352124] niks3-server[1153]: 2026/09/25 15:25:41 OK 1_commit_pending_closure.sql (7.16ms)5141lb # [ 115.355482] niks3-server[1153]: 2026/09/25 15:25:41 OK 2_object_stats_trigger.sql (3.31ms)5142lb # [ 115.358525] niks3-server[1153]: 2026/09/25 15:25:41 OK 3_commit_push.sql (2.71ms)5143lb # [ 115.360540] niks3-server[1153]: 2026/09/25 15:25:41 goose: up to current file version: 35144lb # [ 115.362469] niks3-server[1153]: 2026/09/25 15:25:41 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5145lb # [ 115.371979] niks3-server[1153]: 2026/09/25 15:25:41 INFO Read proxy enabled — serving cache objects from S35146lb # [ 115.373894] niks3-server[1153]: 2026/09/25 15:25:41 INFO Using socket-activated listener address=0.0.0.0:57515147lb # [ 115.375179] niks3-server[1153]: 2026/09/25 15:25:41 INFO systemd watchdog enabled interval=15s5148lb # [ 115.380493] niks3-server[1153]: 2026/09/25 15:25:41 INFO Starting HTTP server address=0.0.0.0:57515149lb # [ 115.382066] systemd[1]: Started niks3 server.5150lb: waiting for unit niks3.service5151lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5152lb: must succeed: curl -sf localhost:9901/clusters5153lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5154worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5155worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5156lb: must succeed: curl -sf localhost:9901/clusters5157lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5158lb: must succeed: curl -sf localhost:9901/clusters5159lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5160worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5161worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5162client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25163worker1 # [ 115.733026] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:561885164worker1 # [ 115.741695] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5165client # don't know how to build these paths:5166client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5167worker2 # [ 115.752772] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545425168worker2 # [ 115.771171] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:42Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05169worker2 # [ 115.776901] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:42Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15170worker2 # [ 115.781526] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545425171worker2 # [ 115.785730] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545425172client # copying 1 paths to 'lb:50051'...5173client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5174worker2 # [ 115.792603] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545425175worker2 # [ 115.801449] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5176worker2 # [ 115.927866] nix-daemon[857]: accepted connection from pid 1408, user nix-grpc-daemon (trusted)5177lb # [ 115.947399] niks3-server[1153]: 2026/09/25 15:25:42 INFO Received push request method=POST path=/api/pushes5178worker2 # [ 115.947469] nix-daemon[857]: reaped child process 1411, status = succeeded5179worker2 # [ 115.954291] nix-grpc-daemon[1376]: time=2026-09-25T15:25:42.577Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5180worker2 # [ 115.955902] nix-grpc-daemon[1376]: time=2026-09-25T15:25:42.577Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5181lb # [ 115.970274] niks3-server[1153]: 2026/09/25 15:25:42 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5182lb # [ 115.989214] niks3-server[1153]: 2026/09/25 15:25:42 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5183lb # [ 116.005606] niks3-server[1153]: 2026/09/25 15:25:42 INFO Signed narinfos id=36 count=15184worker2 # [ 116.006368] nix-grpc-daemon[1376]: time=2026-09-25T15:25:42.629Z level=INFO msg="Uploading 1 narinfos"5185lb # [ 116.020131] niks3-server[1153]: 2026/09/25 15:25:42 INFO Received complete push request method=POST path=/api/pushes/36/complete5186lb # [ 116.028262] niks3-server[1153]: 2026/09/25 15:25:42 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5187lb # [ 116.031593] niks3-server[1153]: 2026/09/25 15:25:42 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5188worker2 # [ 116.034411] nix-grpc-daemon[1376]: time=2026-09-25T15:25:42.658Z level=INFO msg="Upload complete. (223ms)"5189worker2 # [ 116.047580] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54542 duration_s=0 paths=1 nar_bytes_in=5445190worker2 # [ 116.051173] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545425191worker2 # [ 116.058699] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5192worker2 # [ 116.061005] nix-daemon[857]: reaped child process 1406, status = succeeded5193client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5194client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5195worker2 # [ 116.071366] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5196lb # [ 116.247825] niks3-server[1153]: 2026/09/25 15:25:42 INFO lead: acquired remote=192.168.1.4:419285197client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5198client # farm-a> LOG-farm-a-after-niks3-restart5199worker2 # [ 117.352899] nix-daemon[857]: reaped child process 1415, status = succeeded5200worker2 # [ 117.408722] nix-daemon[857]: accepted connection from pid 1433, user nix-grpc-daemon (trusted)5201worker2 # [ 117.424103] nix-daemon[857]: reaped child process 1436, status = succeeded5202lb # [ 117.432500] niks3-server[1153]: 2026/09/25 15:25:44 INFO Received push request method=POST path=/api/pushes5203worker2 # [ 117.437728] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.059Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5204worker2 # [ 117.439457] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.059Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5205lb # [ 117.454230] niks3-server[1153]: 2026/09/25 15:25:44 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5206worker2 # [ 117.458763] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.082Z level=INFO msg="Uploading 1 narinfos"5207lb # [ 117.464170] niks3-server[1153]: 2026/09/25 15:25:44 INFO Signed narinfos id=37 count=15208lb # [ 117.471738] niks3-server[1153]: 2026/09/25 15:25:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5209lb # [ 117.484865] niks3-server[1153]: 2026/09/25 15:25:44 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5210lb # [ 117.496731] niks3-server[1153]: 2026/09/25 15:25:44 INFO Received complete push request method=POST path=/api/pushes/37/complete5211lb # [ 117.504114] niks3-server[1153]: 2026/09/25 15:25:44 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5212worker2 # [ 117.506447] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.130Z level=INFO msg="Upload complete. (223ms)"5213lb # [ 117.509414] niks3-server[1153]: 2026/09/25 15:25:44 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5214worker2 # [ 117.520192] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:44Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54542 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05215worker2 # [ 117.531601] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:44Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05216worker2 # [ 117.536161] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:44Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15217worker2 # [ 117.540785] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545425218worker2 # [ 117.545210] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545425219worker2 # [ 117.549217] nix-daemon[857]: reaped child process 1413, status = succeeded5220worker2 # [ 117.551615] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.175Z level=INFO msg="All 1 paths already cached"5221client # copying 1 paths to 'lb:50051'...5222client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5223worker2 # [ 117.557706] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545425224worker2 # [ 117.568190] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5225worker2 # [ 117.686550] nix-daemon[857]: accepted connection from pid 1442, user nix-grpc-daemon (trusted)5226worker2 # [ 117.702861] nix-daemon[857]: reaped child process 1445, status = succeeded5227lb # [ 117.709060] niks3-server[1153]: 2026/09/25 15:25:44 INFO Received push request method=POST path=/api/pushes5228worker2 # [ 117.713522] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.337Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5229worker2 # [ 117.715052] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.337Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5230lb # [ 117.728292] niks3-server[1153]: 2026/09/25 15:25:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5231worker2 # [ 117.731284] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.354Z level=INFO msg="Uploading 1 narinfos"5232lb # [ 117.735905] niks3-server[1153]: 2026/09/25 15:25:44 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5233lb # [ 117.741303] niks3-server[1153]: 2026/09/25 15:25:44 INFO Signed narinfos id=38 count=15234lb # [ 117.743019] niks3-server[1153]: 2026/09/25 15:25:44 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5235lb # [ 117.752673] niks3-server[1153]: 2026/09/25 15:25:44 INFO Received complete push request method=POST path=/api/pushes/38/complete5236lb # [ 117.762538] niks3-server[1153]: 2026/09/25 15:25:44 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5237worker2 # [ 117.758116] nix-grpc-daemon[1376]: time=2026-09-25T15:25:44.381Z level=INFO msg="Upload complete. (183ms)"5238worker2 # [ 117.764367] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54542 duration_s=0 paths=1 nar_bytes_in=6565239worker2 # [ 117.774573] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545425240worker2 # [ 117.782426] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5241worker2 # [ 117.785369] nix-daemon[857]: reaped child process 1440, status = succeeded5242client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5243client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5244worker2 # [ 117.795745] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5245lb # [ 118.114318] envoy[902]: conn peer=[2001:db8:1::4]:39316 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=2958 tx=1121 ms=734085246lb # [ 118.117098] envoy[902]: conn peer=[2001:db8:1::1]:45414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=519 ms=145247lb # [ 118.119524] envoy[902]: conn peer=192.168.1.3:35046 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3624 tx=1380 ms=766695248lb # [ 118.122246] envoy[902]: conn peer=[2001:db8:1::1]:60750 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=305 ms=14145249client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5250client # farm-b> LOG-farm-b-after-niks3-restart5251worker2 # [ 119.044458] nix-daemon[857]: reaped child process 1449, status = succeeded5252worker2 # [ 119.139566] nix-daemon[857]: accepted connection from pid 1465, user nix-grpc-daemon (trusted)5253worker2 # [ 119.156426] nix-daemon[857]: reaped child process 1468, status = succeeded5254lb # [ 119.164971] niks3-server[1153]: 2026/09/25 15:25:45 INFO Received push request method=POST path=/api/pushes5255worker2 # [ 119.169097] nix-grpc-daemon[1376]: time=2026-09-25T15:25:45.791Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5256worker2 # [ 119.170785] nix-grpc-daemon[1376]: time=2026-09-25T15:25:45.792Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5257lb # [ 119.187998] niks3-server[1153]: 2026/09/25 15:25:45 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5258worker2 # [ 119.194063] nix-grpc-daemon[1376]: time=2026-09-25T15:25:45.817Z level=INFO msg="Uploading 1 narinfos"5259lb # [ 119.196135] niks3-server[1153]: 2026/09/25 15:25:45 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5260lb # [ 119.202517] niks3-server[1153]: 2026/09/25 15:25:45 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5261lb # [ 119.208143] niks3-server[1153]: 2026/09/25 15:25:45 INFO Signed narinfos id=39 count=15262lb # [ 119.209953] niks3-server[1153]: 2026/09/25 15:25:45 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5263lb # [ 119.221728] niks3-server[1153]: 2026/09/25 15:25:45 INFO Received complete push request method=POST path=/api/pushes/39/complete5264worker2 # [ 119.224506] nix-grpc-daemon[1376]: time=2026-09-25T15:25:45.848Z level=INFO msg="Upload complete. (220ms)"5265lb # [ 119.226738] niks3-server[1153]: 2026/09/25 15:25:45 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5266worker2 # [ 119.235101] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54542 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05267worker2 # [ 119.248879] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:45Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05268worker2 # [ 119.251725] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:45Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15269worker2 # [ 119.256505] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545425270worker2 # [ 119.260753] nix-daemon[857]: reaped child process 1447, status = succeeded5271worker2 # [ 119.263666] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:45Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:545425272worker2 # [ 119.269421] nix-grpc-daemon[1376]: time=2026-09-25T15:25:45.893Z level=INFO msg="All 2 paths already cached"5273client # copying 1 paths to 'lb:50051'...5274client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5275worker2 # [ 119.275701] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:45Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:545425276worker2 # [ 119.284720] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5277worker2 # [ 119.397317] nix-daemon[857]: accepted connection from pid 1474, user nix-grpc-daemon (trusted)5278lb # [ 119.414998] niks3-server[1153]: 2026/09/25 15:25:46 INFO Received push request method=POST path=/api/pushes5279worker2 # [ 119.416694] nix-daemon[857]: reaped child process 1477, status = succeeded5280worker2 # [ 119.418491] nix-grpc-daemon[1376]: time=2026-09-25T15:25:46.041Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5281worker2 # [ 119.420053] nix-grpc-daemon[1376]: time=2026-09-25T15:25:46.041Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5282lb # [ 119.432310] niks3-server[1153]: 2026/09/25 15:25:46 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5283worker2 # [ 119.435854] nix-grpc-daemon[1376]: time=2026-09-25T15:25:46.059Z level=INFO msg="Uploading 1 narinfos"5284lb # [ 119.440194] niks3-server[1153]: 2026/09/25 15:25:46 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5285lb # [ 119.442638] niks3-server[1153]: 2026/09/25 15:25:46 INFO Signed narinfos id=40 count=15286lb # [ 119.452227] niks3-server[1153]: 2026/09/25 15:25:46 INFO Received complete push request method=POST path=/api/pushes/40/complete5287lb # [ 119.454605] niks3-server[1153]: 2026/09/25 15:25:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5288lb # [ 119.466017] niks3-server[1153]: 2026/09/25 15:25:46 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5289worker2 # [ 119.469015] nix-grpc-daemon[1376]: time=2026-09-25T15:25:46.092Z level=INFO msg="Upload complete. (176ms)"5290worker2 # [ 119.476141] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54542 duration_s=0 paths=1 nar_bytes_in=7845291worker2 # [ 119.479543] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:545425292worker2 # [ 119.490678] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5293worker2 # [ 119.494729] nix-daemon[857]: reaped child process 1472, status = succeeded5294client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5295client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5296worker2 # [ 119.505893] nix-daemon[857]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5297client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5298client # farm-top> LOG-farm-top-after-niks3-restart5299worker2 # [ 120.960798] nix-daemon[857]: reaped child process 1481, status = succeeded5300worker2 # [ 121.058921] nix-daemon[857]: accepted connection from pid 1497, user nix-grpc-daemon (trusted)5301worker2 # [ 121.078132] nix-daemon[857]: reaped child process 1500, status = succeeded5302lb # [ 121.078702] niks3-server[1153]: 2026/09/25 15:25:47 INFO Received push request method=POST path=/api/pushes5303worker2 # [ 121.084407] nix-grpc-daemon[1376]: time=2026-09-25T15:25:47.707Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5304worker2 # [ 121.085942] nix-grpc-daemon[1376]: time=2026-09-25T15:25:47.707Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5305lb # [ 121.108064] niks3-server[1153]: 2026/09/25 15:25:47 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5306worker2 # [ 121.112749] nix-grpc-daemon[1376]: time=2026-09-25T15:25:47.736Z level=INFO msg="Uploading 1 narinfos"5307lb # [ 121.116447] niks3-server[1153]: 2026/09/25 15:25:47 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5308lb # [ 121.122377] niks3-server[1153]: 2026/09/25 15:25:47 INFO Signed narinfos id=41 count=15309lb # [ 121.128644] niks3-server[1153]: 2026/09/25 15:25:47 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5310lb # [ 121.135194] niks3-server[1153]: 2026/09/25 15:25:47 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5311lb # [ 121.140370] niks3-server[1153]: 2026/09/25 15:25:47 INFO Received complete push request method=POST path=/api/pushes/41/complete5312worker2 # [ 121.141345] nix-grpc-daemon[1376]: time=2026-09-25T15:25:47.765Z level=INFO msg="Upload complete. (227ms)"5313lb # [ 121.144743] niks3-server[1153]: 2026/09/25 15:25:47 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5314worker2 # [ 121.155651] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54542 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05315worker2 # [ 121.161062] nix-daemon[857]: reaped child process 1479, status = succeeded5316client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 5.62 seconds)5317client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5318client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.14 seconds)5319(finished: subtest: a niks3 restart does not cost the leader its role, in 6.18 seconds)5320subtest: scheduler down: the other node takes the lock and keeps it5321lb: must succeed: curl -sf localhost:9901/clusters5322lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5323lb: must succeed: curl -sf localhost:9901/clusters5324lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5325lb: must succeed: curl -sf localhost:9901/clusters5326lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5327lb: must succeed: curl -sf localhost:9901/clusters5328lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5329worker2 # [ 121.505625] systemd[1]: Stopping Nix worker-protocol over gRPC...5330worker2 # [ 121.752944] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:48Z level=info event=signal_exit5331lb # [ 122.255065] niks3-server[1153]: 2026/09/25 15:25:48 INFO lead: released remote=192.168.1.4:419285332worker2 # [ 123.251665] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:49Z level=info event=scheduler_restarting5333worker1 # [ 123.255936] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:49Z level=info event=scheduler_restarting addr=lb:500515334lb # [ 123.254502] envoy[902]: [2026-09-25 15:25:49.876][902][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5335worker2 # [ 123.255862] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:49Z level=info event=scheduler_restarting addr=lb:500515336worker2 # [ 123.261602] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:389865337worker2 # [ 123.265129] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:49Z level=info event=worker_gone addr=192.168.1.3:500515338worker2 # [ 123.267886] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:389865339worker2 # [ 123.270629] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:49Z level=info event=worker_gone addr=192.168.1.4:500515340worker2 # [ 123.287766] nix-daemon[857]: reaped child process 1368, status = succeeded5341worker1 # [ 123.455983] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_connected addr=lb:500515342worker1 # [ 123.459146] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5343worker2 # [ 123.469038] nix-grpc-daemon[1316]: ts=2026-09-25T15:25:50Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5344worker2 # [ 123.503943] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5345worker2 # [ 123.508566] systemd[1]: Stopped Nix worker-protocol over gRPC.5346worker2 # [ 123.509969] systemd[1]: nix-grpc-daemon.service: Consumed 1.167s CPU time over 13.890s wall clock time, 39M memory peak, 253K incoming IP traffic, 109.4K outgoing IP traffic.5347worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5348worker1 # [ 123.659841] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_connected addr=lb:500515349worker1 # [ 123.662974] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5350worker1 # [ 123.863638] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_connected addr=lb:500515351worker1 # [ 123.866701] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5352worker1 # [ 124.067351] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_connected addr=lb:500515353worker1 # [ 124.070436] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5354worker1 # [ 124.272182] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_connected addr=lb:500515355worker1 # [ 124.274524] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:50Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5356worker1 # [ 124.472561] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_connected addr=lb:500515357worker1 # [ 124.475544] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5358worker1 # [ 124.676083] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_connected addr=lb:500515359worker1 # [ 124.678976] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5360worker1 # [ 124.879611] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_connected addr=lb:500515361worker1 # [ 124.882596] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5362worker1 # [ 125.083090] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_connected addr=lb:500515363worker1 # [ 125.086000] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5364worker1 # [ 125.287427] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_connected addr=lb:500515365worker1 # [ 125.289758] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:51Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5366worker1 # [ 125.487761] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_connected addr=lb:500515367worker1 # [ 125.490559] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5368worker1 # [ 125.691058] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_connected addr=lb:500515369worker1 # [ 125.693959] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5370worker1 # [ 125.894463] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_connected addr=lb:500515371worker1 # [ 125.897215] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5372worker1 # [ 126.097855] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_connected addr=lb:500515373worker1 # [ 126.100582] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5374lb # [ 126.246604] niks3-server[1153]: 2026/09/25 15:25:52 INFO lead: acquired remote=192.168.1.3:570665375worker1 # [ 126.246960] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_take_over5376worker1 # [ 126.301648] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_connected addr=lb:500515377worker1 # [ 126.304070] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5378worker1 # [ 126.502511] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:53Z level=info event=scheduler_connected addr=lb:500515379worker1 # [ 126.507408] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:440245380worker1 # [ 126.510801] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05381worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 3.17 seconds)5382worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5383worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5384client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25385worker1 # [ 126.897416] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:561885386client # don't know how to build these paths:5387client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5388worker1 # [ 126.903379] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561885389worker1 # [ 126.911383] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 5 ms (attempt 1/5)5390worker1 # [ 126.921154] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 190 ms (attempt 2/5)5391worker1 # [ 127.112318] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 77 ms (attempt 3/5)5392worker1 # [ 127.190586] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 481 ms (attempt 4/5)5393worker1 # [ 127.673845] nix-grpc-daemon[1244]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5394worker1 # [ 127.677719] nix-grpc-daemon[1244]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5395worker1 # [ 127.696085] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:54Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05396worker1 # [ 127.699894] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:54Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15397worker1 # [ 127.705350] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561885398worker1 # [ 127.711986] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561885399client # copying 1 paths to 'lb:50051'...5400client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5401worker1 # [ 127.718578] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561885402worker1 # [ 127.727647] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5403worker1 # [ 127.743829] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:54Z level=info event=niks3_push_started pid=13355404worker1 # [ 127.870552] nix-grpc-daemon[1335]: time=2026-09-25T15:25:54.497Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5405worker1 # [ 127.975934] nix-daemon[838]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5406lb # [ 127.994328] niks3-server[1153]: 2026/09/25 15:25:54 INFO Received push request method=POST path=/api/pushes5407worker1 # [ 127.996879] nix-daemon[838]: reaped child process 1350, status = succeeded5408worker1 # [ 128.000226] nix-grpc-daemon[1335]: time=2026-09-25T15:25:54.626Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5409worker1 # [ 128.005904] nix-grpc-daemon[1335]: time=2026-09-25T15:25:54.627Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5410lb # [ 128.024981] niks3-server[1153]: 2026/09/25 15:25:54 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5411worker1 # [ 128.030333] nix-grpc-daemon[1335]: time=2026-09-25T15:25:54.656Z level=INFO msg="Uploading 1 narinfos"5412lb # [ 128.032088] niks3-server[1153]: 2026/09/25 15:25:54 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5413lb # [ 128.035264] niks3-server[1153]: 2026/09/25 15:25:54 INFO Signed narinfos id=42 count=15414lb # [ 128.046675] niks3-server[1153]: 2026/09/25 15:25:54 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5415worker1 # [ 128.052188] nix-grpc-daemon[1335]: time=2026-09-25T15:25:54.679Z level=INFO msg="Upload complete. (178ms)"5416lb # [ 128.052865] niks3-server[1153]: 2026/09/25 15:25:54 INFO Received complete push request method=POST path=/api/pushes/42/complete5417lb # [ 128.056045] niks3-server[1153]: 2026/09/25 15:25:54 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5418worker1 # [ 128.059970] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56188 duration_s=0 paths=1 nar_bytes_in=5205419worker1 # [ 128.062775] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561885420worker1 # [ 128.074208] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5421worker1 # [ 128.075650] nix-daemon[838]: reaped child process 1333, status = succeeded5422client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5423worker1 # [ 128.083225] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5424client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5425lb # [ 128.113282] envoy[902]: conn peer=[2001:db8:1::1]:60756 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9762 tx=4424 ms=54325426lb # [ 128.115944] envoy[902]: conn peer=[2001:db8:1::4]:45604 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3991 tx=586 ms=136635427client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5428client # farm-a> LOG-farm-a-on-standby5429worker1 # [ 128.932663] nix-daemon[838]: reaped child process 1355, status = succeeded5430worker1 # [ 128.944483] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:55Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:440365431worker1 # [ 129.038313] nix-daemon[838]: accepted connection from pid 1371, user nix-grpc-daemon (trusted)5432lb # [ 129.056784] niks3-server[1153]: 2026/09/25 15:25:55 INFO Received push request method=POST path=/api/pushes5433worker1 # [ 129.059038] nix-daemon[838]: reaped child process 1374, status = succeeded5434worker1 # [ 129.064408] nix-grpc-daemon[1335]: time=2026-09-25T15:25:55.689Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5435worker1 # [ 129.066009] nix-grpc-daemon[1335]: time=2026-09-25T15:25:55.689Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5436lb # [ 129.077403] niks3-server[1153]: 2026/09/25 15:25:55 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5437worker1 # [ 129.089684] nix-grpc-daemon[1335]: time=2026-09-25T15:25:55.716Z level=INFO msg="Uploading 1 narinfos"5438lb # [ 129.089527] niks3-server[1153]: 2026/09/25 15:25:55 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign5439lb # [ 129.095855] niks3-server[1153]: 2026/09/25 15:25:55 INFO Signed narinfos id=43 count=15440lb # [ 129.099784] niks3-server[1153]: 2026/09/25 15:25:55 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5441lb # [ 129.108633] niks3-server[1153]: 2026/09/25 15:25:55 INFO Received complete push request method=POST path=/api/pushes/43/complete5442worker1 # [ 129.116045] nix-grpc-daemon[1335]: time=2026-09-25T15:25:55.743Z level=INFO msg="Upload complete. (224ms)"5443lb # [ 129.117249] niks3-server[1153]: 2026/09/25 15:25:55 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5444worker1 # [ 129.125307] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56188 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05445lb # [ 129.137976] niks3-server[1153]: 2026/09/25 15:25:55 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5446worker1 # [ 129.140140] nix-daemon[838]: reaped child process 1353, status = succeeded5447worker1 # [ 129.144421] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:55Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05448worker1 # [ 129.147888] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:55Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15449worker1 # [ 129.153772] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561885450worker1 # [ 129.159945] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561885451worker1 # [ 129.168353] nix-grpc-daemon[1335]: time=2026-09-25T15:25:55.795Z level=INFO msg="All 1 paths already cached"5452client # copying 1 paths to 'lb:50051'...5453client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5454worker1 # [ 129.174428] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561885455worker1 # [ 129.182192] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5456worker1 # [ 129.299816] nix-daemon[838]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5457lb # [ 129.319905] niks3-server[1153]: 2026/09/25 15:25:55 INFO Received push request method=POST path=/api/pushes5458worker1 # [ 129.323079] nix-daemon[838]: reaped child process 1383, status = succeeded5459worker1 # [ 129.328847] nix-grpc-daemon[1335]: time=2026-09-25T15:25:55.955Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5460worker1 # [ 129.330514] nix-grpc-daemon[1335]: time=2026-09-25T15:25:55.955Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5461lb # [ 129.342391] niks3-server[1153]: 2026/09/25 15:25:55 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign5462worker1 # [ 129.346219] nix-grpc-daemon[1335]: time=2026-09-25T15:25:55.973Z level=INFO msg="Uploading 1 narinfos"5463lb # [ 129.348966] niks3-server[1153]: 2026/09/25 15:25:55 INFO Signed narinfos id=44 count=15464lb # [ 129.354849] niks3-server[1153]: 2026/09/25 15:25:55 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5465lb # [ 129.362856] niks3-server[1153]: 2026/09/25 15:25:55 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5466lb # [ 129.371876] niks3-server[1153]: 2026/09/25 15:25:55 INFO Received complete push request method=POST path=/api/pushes/44/complete5467worker1 # [ 129.376565] nix-grpc-daemon[1335]: time=2026-09-25T15:25:56.003Z level=INFO msg="Upload complete. (185ms)"5468lb # [ 129.381646] niks3-server[1153]: 2026/09/25 15:25:56 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5469worker1 # [ 129.382675] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56188 duration_s=0 paths=1 nar_bytes_in=6405470worker1 # [ 129.394628] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561885471worker1 # [ 129.399189] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5472worker1 # [ 129.403236] nix-daemon[838]: reaped child process 1378, status = succeeded5473client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5474client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5475worker1 # [ 129.415047] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5476client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5477client # farm-b> LOG-farm-b-on-standby5478worker1 # [ 130.695110] nix-daemon[838]: reaped child process 1387, status = succeeded5479worker1 # [ 130.770534] nix-daemon[838]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)5480worker1 # [ 130.786141] nix-daemon[838]: reaped child process 1406, status = succeeded5481lb # [ 130.792152] niks3-server[1153]: 2026/09/25 15:25:57 INFO Received push request method=POST path=/api/pushes5482worker1 # [ 130.798739] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.424Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5483worker1 # [ 130.800294] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.424Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5484lb # [ 130.814433] niks3-server[1153]: 2026/09/25 15:25:57 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign5485worker1 # [ 130.820804] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.447Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 130.823678] niks3-server[1153]: 2026/09/25 15:25:57 INFO Signed narinfos id=45 count=15487lb # [ 130.829801] niks3-server[1153]: 2026/09/25 15:25:57 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5488lb # [ 130.836306] niks3-server[1153]: 2026/09/25 15:25:57 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5489lb # [ 130.845546] niks3-server[1153]: 2026/09/25 15:25:57 INFO Received complete push request method=POST path=/api/pushes/45/complete5490lb # [ 130.853125] niks3-server[1153]: 2026/09/25 15:25:57 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5491lb # [ 130.855783] niks3-server[1153]: 2026/09/25 15:25:57 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5492worker1 # [ 130.859331] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.486Z level=INFO msg="Upload complete. (236ms)"5493worker1 # [ 130.869537] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56188 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=05494worker1 # [ 130.882650] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:57Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05495worker1 # [ 130.886631] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:57Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15496worker1 # [ 130.890563] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561885497worker1 # [ 130.894226] nix-daemon[838]: reaped child process 1385, status = succeeded5498worker1 # [ 130.897062] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561885499worker1 # [ 130.902945] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.530Z level=INFO msg="All 2 paths already cached"5500client # copying 1 paths to 'lb:50051'...5501client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5502worker1 # [ 130.909122] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561885503worker1 # [ 130.916777] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5504worker1 # [ 131.032686] nix-daemon[838]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5505worker1 # [ 131.048160] nix-daemon[838]: reaped child process 1415, status = succeeded5506lb # [ 131.054385] niks3-server[1153]: 2026/09/25 15:25:57 INFO Received push request method=POST path=/api/pushes5507worker1 # [ 131.059683] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.686Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5508worker1 # [ 131.061583] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.686Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5509lb # [ 131.072704] niks3-server[1153]: 2026/09/25 15:25:57 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign5510worker1 # [ 131.077719] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.704Z level=INFO msg="Uploading 1 narinfos"5511lb # [ 131.081068] niks3-server[1153]: 2026/09/25 15:25:57 INFO Signed narinfos id=46 count=15512lb # [ 131.085066] niks3-server[1153]: 2026/09/25 15:25:57 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5513lb # [ 131.094334] niks3-server[1153]: 2026/09/25 15:25:57 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5514worker1 # [ 131.100335] nix-grpc-daemon[1335]: time=2026-09-25T15:25:57.727Z level=INFO msg="Upload complete. (174ms)"5515lb # [ 131.100953] niks3-server[1153]: 2026/09/25 15:25:57 INFO Received complete push request method=POST path=/api/pushes/46/complete5516lb # [ 131.106388] niks3-server[1153]: 2026/09/25 15:25:57 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5517worker1 # [ 131.105417] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56188 duration_s=0 paths=1 nar_bytes_in=7685518worker1 # [ 131.111070] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561885519worker1 # [ 131.121150] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5520worker1 # [ 131.123194] nix-daemon[838]: reaped child process 1410, status = succeeded5521client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5522client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5523worker1 # [ 131.132584] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5524client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5525client # farm-top> LOG-farm-top-on-standby5526worker1 # [ 131.966896] nix-daemon[838]: reaped child process 1419, status = succeeded5527worker1 # [ 132.072873] nix-daemon[838]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5528worker1 # [ 132.091168] nix-daemon[838]: reaped child process 1438, status = succeeded5529lb # [ 132.096483] niks3-server[1153]: 2026/09/25 15:25:58 INFO Received push request method=POST path=/api/pushes5530worker1 # [ 132.103627] nix-grpc-daemon[1335]: time=2026-09-25T15:25:58.729Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5531worker1 # [ 132.105351] nix-grpc-daemon[1335]: time=2026-09-25T15:25:58.729Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5532lb # [ 132.118119] niks3-server[1153]: 2026/09/25 15:25:58 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5533worker1 # [ 132.126281] nix-grpc-daemon[1335]: time=2026-09-25T15:25:58.753Z level=INFO msg="Uploading 1 narinfos"5534lb # [ 132.126623] niks3-server[1153]: 2026/09/25 15:25:58 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign5535lb # [ 132.133267] niks3-server[1153]: 2026/09/25 15:25:58 INFO Signed narinfos id=47 count=15536lb # [ 132.135031] niks3-server[1153]: 2026/09/25 15:25:58 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5537lb # [ 132.147240] niks3-server[1153]: 2026/09/25 15:25:58 INFO Received complete push request method=POST path=/api/pushes/47/complete5538worker1 # [ 132.154442] nix-grpc-daemon[1335]: time=2026-09-25T15:25:58.781Z level=INFO msg="Upload complete. (215ms)"5539lb # [ 132.154515] niks3-server[1153]: 2026/09/25 15:25:58 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5540lb # [ 132.157559] niks3-server[1153]: 2026/09/25 15:25:58 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5541worker1 # [ 132.162553] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56188 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=05542worker1 # [ 132.173840] nix-daemon[838]: reaped child process 1417, status = succeeded5543client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 5.46 seconds)5544client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5545client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.14 seconds)5546worker2 # [ 132.374029] systemd[1]: Starting scheduler-token.service...5547worker2 # [ 132.434898] systemd[1]: scheduler-token.service: Deactivated successfully.5548worker2 # [ 132.440414] systemd[1]: Finished scheduler-token.service.5549worker2 # [ 132.441757] systemd[1]: scheduler-token.service: Consumed 36ms CPU time over 63ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5550worker2 # [ 132.452295] systemd[1]: Starting Nix worker-protocol over gRPC...5551worker2 # [ 132.557391] nix-grpc-daemon[1521]: ts=2026-09-25T15:25:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5552worker2 # [ 132.559874] nix-grpc-daemon[1521]: ts=2026-09-25T15:25:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5553worker2 # [ 132.564064] nix-grpc-daemon[1521]: ts=2026-09-25T15:25:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15554worker2 # [ 132.570843] nix-grpc-daemon[1521]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5555worker2 # [ 132.592112] nix-grpc-daemon[1521]: E0000 00:00:1790349959.192546 1521 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5556worker2 # [ 132.594691] nix-grpc-daemon[1521]: E0000 00:00:1790349959.194548 1521 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5557lb: must succeed: curl -sf localhost:9901/clusters5558worker2 # [ 132.619739] nix-grpc-daemon[1521]: ts=2026-09-25T15:25:59Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5559worker2 # [ 132.621795] nix-grpc-daemon[1521]: E0000 00:00:1790349959.202394 1562 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5560worker2 # [ 132.627499] nix-grpc-daemon[1521]: E0000 00:00:1790349959.226472 1562 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5561worker2 # [ 132.632275] nix-grpc-daemon[1521]: E0000 00:00:1790349959.226625 1562 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5562worker2 # [ 132.635273] nix-grpc-daemon[1521]: E0000 00:00:1790349959.242249 1562 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5563worker2 # [ 132.637889] systemd[1]: Started Nix worker-protocol over gRPC.5564worker1 # [ 132.639363] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=info event=oidc_jwks provider=mock keys=15565worker2 # [ 132.638766] nix-grpc-daemon[1521]: ts=2026-09-25T15:25:59Z level=info event=scheduler_connected addr=lb:500515566worker1 # [ 132.642147] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:440245567worker1 # [ 132.646860] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05568lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5569worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5570worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5571lb: must succeed: curl -sf localhost:9901/clusters5572lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5573lb: must succeed: curl -sf localhost:9901/clusters5574lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5575worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5576worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5577(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.49 seconds)5578subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5579client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5580client # [ 132.841362] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5581client # Running as unit: rs.service; invocation ID: 05b9f34f2cf54e438847431f4e45251c5582client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.06 seconds)5583worker1 # [ 133.023755] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:561885584client # [ 133.026948] nix[1418]: don't know how to build these paths:5585client # [ 133.027971] nix[1418]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5586worker1 # [ 133.033158] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05587worker1 # [ 133.038116] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15588worker1 # [ 133.042882] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561885589worker1 # [ 133.047077] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:561885590client # [ 133.051217] nix[1418]: copying 1 paths to 'lb:50051'...5591worker1 # [ 133.052890] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:561885592client # [ 133.052590] nix[1418]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5593worker1 # [ 133.062104] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5594worker1 # [ 133.181117] nix-daemon[838]: accepted connection from pid 1451, user nix-grpc-daemon (trusted)5595worker1 # [ 133.196624] nix-daemon[838]: reaped child process 1454, status = succeeded5596lb # [ 133.202649] niks3-server[1153]: 2026/09/25 15:25:59 INFO Received push request method=POST path=/api/pushes5597worker1 # [ 133.209201] nix-grpc-daemon[1335]: time=2026-09-25T15:25:59.835Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5598worker1 # [ 133.210784] nix-grpc-daemon[1335]: time=2026-09-25T15:25:59.835Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5599lb # [ 133.222295] niks3-server[1153]: 2026/09/25 15:25:59 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign5600worker1 # [ 133.226893] nix-grpc-daemon[1335]: time=2026-09-25T15:25:59.853Z level=INFO msg="Uploading 1 narinfos"5601lb # [ 133.228784] niks3-server[1153]: 2026/09/25 15:25:59 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5602lb # [ 133.235421] niks3-server[1153]: 2026/09/25 15:25:59 INFO Signed narinfos id=48 count=15603lb # [ 133.243631] niks3-server[1153]: 2026/09/25 15:25:59 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5604worker1 # [ 133.249220] nix-grpc-daemon[1335]: time=2026-09-25T15:25:59.876Z level=INFO msg="Upload complete. (177ms)"5605lb # [ 133.250226] niks3-server[1153]: 2026/09/25 15:25:59 INFO Received complete push request method=POST path=/api/pushes/48/complete5606lb # [ 133.254017] niks3-server[1153]: 2026/09/25 15:25:59 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5607worker1 # [ 133.256622] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56188 duration_s=0 paths=1 nar_bytes_in=4165608worker1 # [ 133.265665] nix-grpc-daemon[1244]: ts=2026-09-25T15:25:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:561885609worker1 # [ 133.273035] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5610worker1 # [ 133.275601] nix-daemon[838]: reaped child process 1449, status = succeeded5611client # [ 133.281869] nix[1418]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5612worker1 # [ 133.284813] nix-daemon[838]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5613lb: must succeed: curl -sf localhost:9901/clusters5614lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5615lb: must succeed: curl -sf localhost:9901/clusters5616lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5617lb: must succeed: curl -sf localhost:9901/clusters5618lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5619lb: must succeed: curl -sf localhost:9901/clusters5620lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5621worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5622worker2 # [ 135.376476] systemd-journald[270]: Received client request to rotate journal, rotating.5623worker2 # [ 135.387786] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/138482321de5455092b057d14e8cf27e.5624worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.05 seconds)5625worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5626worker1 # [ 135.425605] systemd[1]: Stopping Nix worker-protocol over gRPC...5627worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.04 seconds)5628client: must succeed: sleep 5.05629worker1 # [ 136.063251] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:02Z level=info event=draining5630lb # [ 138.114404] envoy[902]: conn peer=[2001:db8:1::1]:54932 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9589 tx=4368 ms=52785631client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5632worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5633worker1 # [ 140.761915] nix-daemon[838]: reaped child process 1458, status = succeeded5634worker1 # [ 140.839678] nix-daemon[838]: accepted connection from pid 1499, user nix-grpc-daemon (trusted)5635worker1 # [ 140.855906] nix-daemon[838]: reaped child process 1502, status = succeeded5636lb # [ 140.862546] niks3-server[1153]: 2026/09/25 15:26:07 INFO Received push request method=POST path=/api/pushes5637worker1 # [ 140.870417] nix-grpc-daemon[1335]: time=2026-09-25T15:26:07.495Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5638worker1 # [ 140.871984] nix-grpc-daemon[1335]: time=2026-09-25T15:26:07.495Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5639lb # [ 140.886998] niks3-server[1153]: 2026/09/25 15:26:07 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign5640worker1 # [ 140.893684] nix-grpc-daemon[1335]: time=2026-09-25T15:26:07.520Z level=INFO msg="Uploading 1 narinfos"5641lb # [ 140.895012] niks3-server[1153]: 2026/09/25 15:26:07 INFO Signed narinfos id=49 count=15642lb # [ 140.899845] niks3-server[1153]: 2026/09/25 15:26:07 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5643lb # [ 140.905201] niks3-server[1153]: 2026/09/25 15:26:07 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5644lb # [ 140.907966] niks3-server[1153]: 2026/09/25 15:26:07 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5645lb # [ 140.919134] niks3-server[1153]: 2026/09/25 15:26:07 INFO Received complete push request method=POST path=/api/pushes/49/complete5646worker1 # [ 140.922384] nix-grpc-daemon[1335]: time=2026-09-25T15:26:07.549Z level=INFO msg="Upload complete. (217ms)"5647lb # [ 140.922539] niks3-server[1153]: 2026/09/25 15:26:07 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5648worker1 # [ 140.930329] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56188 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05649worker1 # [ 140.942523] nix-daemon[838]: reaped child process 1456, status = succeeded5650client # [ 140.951399] systemd[1]: rs.service: Deactivated successfully.5651client # [ 140.954449] systemd[1]: rs.service: Consumed 166ms CPU time over 8.111s wall clock time, 41.4M memory peak, 7K incoming IP traffic, 8.3K outgoing IP traffic.5652worker1 # [ 141.064840] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:07Z level=info event=signal_exit5653lb # [ 141.250635] niks3-server[1153]: 2026/09/25 15:26:07 INFO lead: released remote=192.168.1.3:570665654worker1 # [ 142.247179] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:08Z level=info event=scheduler_restarting5655lb # [ 142.247457] envoy[902]: [2026-09-25 15:26:08.870][902][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5656worker1 # [ 142.254218] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:440245657worker2 # [ 142.252461] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:08Z level=info event=scheduler_restarting addr=lb:500515658worker1 # [ 142.257026] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:08Z level=info event=worker_gone addr=192.168.1.4:500515659worker1 # [ 142.262142] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:08Z level=info event=scheduler_restarting addr=lb:500515660worker1 # [ 142.264910] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:08Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:440245661worker1 # [ 142.267759] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:08Z level=info event=worker_gone addr=192.168.1.3:500515662worker1 # [ 142.284904] nix-daemon[838]: reaped child process 1296, status = succeeded5663worker2 # [ 142.452413] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_connected addr=lb:500515664worker2 # [ 142.455388] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5665worker1 # [ 142.470511] nix-grpc-daemon[1244]: ts=2026-09-25T15:26:09Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5666worker2 # [ 142.578679] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_take_over5667lb # [ 142.581261] niks3-server[1153]: 2026/09/25 15:26:09 INFO lead: acquired remote=192.168.1.4:601825668worker2 # [ 142.656072] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_connected addr=lb:500515669worker2 # [ 142.658897] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5670worker1 # [ 142.732453] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5671worker1 # [ 142.736238] systemd[1]: Stopped Nix worker-protocol over gRPC.5672worker1 # [ 142.737701] systemd[1]: nix-grpc-daemon.service: Consumed 1.336s CPU time over 27.814s wall clock time, 39.8M memory peak, 329.1K incoming IP traffic, 137.7K outgoing IP traffic.5673worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.11 seconds)5674client: waiting for success: ! systemctl is-active rs5675worker1 # [ 142.753288] systemd[1]: Starting Nix worker-protocol over gRPC...5676client: (finished: waiting for success: ! systemctl is-active rs, in 0.03 seconds)5677client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5678client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.04 seconds)5679client: must fail: journalctl -u rs | grep -q 'reconnecting'5680client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5681lb: must succeed: curl -sf localhost:9901/clusters5682worker2 # [ 142.859872] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_connected addr=lb:500515683worker2 # [ 142.863074] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5684worker1 # [ 142.873616] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5685worker1 # [ 142.875869] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5686worker1 # [ 142.879948] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15687lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5688worker1 # [ 142.886463] nix-grpc-daemon[1510]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5689worker1 # [ 142.887924] nix-grpc-daemon[1510]: E0000 00:00:1790349969.511531 1510 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5690worker1 # [ 142.927893] nix-grpc-daemon[1510]: E0000 00:00:1790349969.513473 1510 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5691worker1 # [ 142.933671] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:09Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5692worker1 # [ 142.936959] nix-grpc-daemon[1510]: E0000 00:00:1790349969.523709 1552 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5693worker1 # [ 142.939434] nix-grpc-daemon[1510]: E0000 00:00:1790349969.539403 1552 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5694worker1 # [ 142.942238] nix-grpc-daemon[1510]: E0000 00:00:1790349969.539545 1552 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5695worker1 # [ 142.944690] nix-grpc-daemon[1510]: E0000 00:00:1790349969.553961 1552 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5696worker1 # [ 142.947047] systemd[1]: Started Nix worker-protocol over gRPC.5697worker1 # [ 142.947874] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5698worker2 # [ 143.063903] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_connected addr=lb:500515699worker2 # [ 143.066927] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5700worker1 # [ 143.142860] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:09Z level=info event=scheduler_connected addr=lb:500515701worker1 # [ 143.145788] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5702worker2 # [ 143.268777] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_connected addr=lb:500515703worker2 # [ 143.271017] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5704worker1 # [ 143.346124] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:09Z level=info event=scheduler_connected addr=lb:500515705worker2 # [ 143.347699] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:479485706worker2 # [ 143.350751] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05707worker2 # [ 143.469183] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:10Z level=info event=scheduler_connected addr=lb:500515708worker2 # [ 143.476084] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:10Z level=info event=oidc_jwks provider=mock keys=15709worker2 # [ 143.479260] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:10Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:479485710worker2 # [ 143.482373] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:10Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05711lb: must succeed: curl -sf localhost:9901/clusters5712lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5713worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5714worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5715(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.16 seconds)5716subtest: scheduler restart mid-build: build finishes, no double build5717client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5718client # [ 143.995874] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5719client # Running as unit: mid.service; invocation ID: b671e1ad54e84c90a7ac9b7eed3adfc85720client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.05 seconds)5721worker2 # [ 144.179965] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:479565722worker2 # [ 144.188281] nix-daemon[857]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)5723client # [ 144.198392] nix[1454]: don't know how to build these paths:5724client # [ 144.199484] nix[1454]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5725worker2 # [ 144.206473] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:10Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05726worker2 # [ 144.209741] nix-grpc-daemon[1521]: ts=2026-09-25T15:26:10Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15727worker1 # [ 144.222420] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500945728worker1 # [ 144.230778] nix-daemon[838]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5729worker1 # [ 144.244799] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)5730worker1 # [ 144.275571] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 2/5)5731worker1 # [ 144.348603] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 364 ms (attempt 3/5)5732worker1 # [ 144.713750] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 279 ms (attempt 4/5)5733worker1 # [ 144.994631] nix-grpc-daemon[1510]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5734worker1 # [ 144.997813] nix-grpc-daemon[1510]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5735worker1 # [ 145.008561] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500945736client # [ 145.016127] nix[1454]: copying 1 paths to 'lb:50051'...5737worker1 # [ 145.017665] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500945738client # [ 145.016365] nix[1454]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5739worker1 # [ 145.026150] nix-daemon[838]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5740worker1 # [ 145.042441] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:11Z level=info event=niks3_push_started pid=15745741worker1 # [ 145.188600] nix-grpc-daemon[1574]: time=2026-09-25T15:26:11.813Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5742worker1 # [ 145.303816] nix-daemon[838]: accepted connection from pid 1591, user nix-grpc-daemon (trusted)5743worker1 # [ 145.322371] nix-daemon[838]: reaped child process 1594, status = succeeded5744lb # [ 145.329325] niks3-server[1153]: 2026/09/25 15:26:11 INFO Received push request method=POST path=/api/pushes5745worker1 # [ 145.335187] nix-grpc-daemon[1574]: time=2026-09-25T15:26:11.962Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5746worker1 # [ 145.340516] nix-grpc-daemon[1574]: time=2026-09-25T15:26:11.962Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5747lb # [ 145.357685] niks3-server[1153]: 2026/09/25 15:26:11 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign5748lb # [ 145.367439] niks3-server[1153]: 2026/09/25 15:26:11 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5749lb # [ 145.376932] niks3-server[1153]: 2026/09/25 15:26:11 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5750lb # [ 145.387379] niks3-server[1153]: 2026/09/25 15:26:12 INFO Signed narinfos id=50 count=15751worker1 # [ 145.388070] nix-grpc-daemon[1574]: time=2026-09-25T15:26:12.014Z level=INFO msg="Uploading 1 narinfos"5752lb # [ 145.398576] niks3-server[1153]: 2026/09/25 15:26:12 INFO Received complete push request method=POST path=/api/pushes/50/complete5753worker1 # [ 145.402991] nix-grpc-daemon[1574]: time=2026-09-25T15:26:12.030Z level=INFO msg="Upload complete. (216ms)"5754lb # [ 145.405153] niks3-server[1153]: 2026/09/25 15:26:12 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5755worker1 # [ 145.410172] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50094 duration_s=0 paths=1 nar_bytes_in=4245756worker1 # [ 145.421844] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500945757worker1 # [ 145.427259] nix-daemon[838]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5758worker1 # [ 145.431444] nix-daemon[838]: reaped child process 1572, status = succeeded5759worker1 # [ 145.441126] nix-daemon[838]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5760client # [ 145.438965] nix[1454]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5761lb: must succeed: curl -sf localhost:9901/clusters5762lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5763lb: must succeed: curl -sf localhost:9901/clusters5764lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5765worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5766worker2 # [ 147.671776] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1521 (nix-grpc-daemon) on client request.5767worker2 # [ 147.680342] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5768lb # [ 147.694901] niks3-server[1153]: 2026/09/25 15:26:14 INFO lead: released remote=192.168.1.4:601825769worker1 # [ 147.699687] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"5770worker2 # [ 147.698095] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5771client # [ 147.698496] nix[1454]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5772worker2 # [ 147.711699] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5773worker2 # [ 147.720864] systemd[1]: nix-grpc-daemon.service: Consumed 216ms CPU time over 15.250s wall clock time, 14.6M memory peak, 32.4K incoming IP traffic, 42K outgoing IP traffic.5774worker2 # [ 147.731242] systemd[1]: Starting scheduler-token.service...5775worker2 # [ 147.736516] nix-daemon[857]: reaped child process 1630, status = succeeded5776worker2 # [ 147.777859] systemd[1]: scheduler-token.service: Deactivated successfully.5777worker2 # [ 147.780976] systemd[1]: Finished scheduler-token.service.5778worker2 # [ 147.782061] systemd[1]: scheduler-token.service: Consumed 26ms CPU time over 49ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5779worker2 # [ 147.792177] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5780worker2 # [ 147.794066] systemd[1]: Starting scheduler-token.service...5781worker2 # [ 147.834833] systemd[1]: scheduler-token.service: Deactivated successfully.5782worker2 # [ 147.837694] systemd[1]: Finished scheduler-token.service.5783worker2 # [ 147.838645] systemd[1]: scheduler-token.service: Consumed 25ms CPU time over 48ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5784worker2 # [ 147.844339] systemd[1]: Starting Nix worker-protocol over gRPC...5785lb # [ 147.896714] niks3-server[1153]: 2026/09/25 15:26:14 INFO lead: acquired remote=192.168.1.3:443965786worker1 # [ 147.899041] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:14Z level=info event=scheduler_take_over5787worker1 # [ 147.901130] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:14Z level=info event=scheduler_connected addr=lb:500515788worker1 # [ 147.916065] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. retried and the latest reset reason: remote connection failure"5789worker2 # [ 147.938919] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:14Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5790worker2 # [ 147.941275] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5791worker2 # [ 147.945127] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15792worker2 # [ 147.950413] nix-grpc-daemon[1664]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5793worker2 # [ 147.951982] nix-grpc-daemon[1664]: E0000 00:00:1790349974.573366 1664 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5794worker2 # [ 147.955965] nix-grpc-daemon[1664]: E0000 00:00:1790349974.579591 1664 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5795worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.35 seconds)5796client: must succeed: sleep 5.05797worker2 # [ 147.998407] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:14Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5798worker2 # [ 148.000803] nix-grpc-daemon[1664]: E0000 00:00:1790349974.588130 1704 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5799worker2 # [ 148.006368] nix-grpc-daemon[1664]: E0000 00:00:1790349974.605800 1704 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5800worker2 # [ 148.012870] nix-grpc-daemon[1664]: E0000 00:00:1790349974.605944 1704 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5801worker2 # [ 148.016394] nix-grpc-daemon[1664]: E0000 00:00:1790349974.621356 1704 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5802worker2 # [ 148.019482] systemd[1]: Started Nix worker-protocol over gRPC.5803worker2 # [ 148.020859] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:14Z level=info event=scheduler_connected addr=lb:500515804worker2 # [ 148.022214] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5805lb # [ 148.113926] envoy[902]: conn peer=[2001:db8:1::1]:60340 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2962 tx=1497 ms=79165806worker1 # [ 148.116597] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:14Z level=info event=scheduler_connected addr=lb:500515807lb # [ 148.117418] envoy[902]: conn peer=[2001:db8:1::3]:37082 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3527 tx=1140 ms=273605808worker1 # [ 148.120777] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5809lb # [ 148.120936] envoy[902]: conn peer=[2001:db8:1::4]:39372 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5710 tx=512 ms=150765810client # [ 148.201283] nix[1454]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting5811worker2 # [ 148.221554] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:14Z level=info event=scheduler_connected addr=lb:500515812worker2 # [ 148.225049] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5813worker1 # [ 148.321215] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:14Z level=info event=scheduler_connected addr=lb:500515814worker1 # [ 148.324135] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:14Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5815worker2 # [ 148.425733] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515816worker2 # [ 148.428574] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5817worker1 # [ 148.523535] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515818worker1 # [ 148.525493] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5819worker2 # [ 148.629266] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515820worker2 # [ 148.632232] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5821worker1 # [ 148.726495] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515822worker1 # [ 148.727880] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5823worker2 # [ 148.832687] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515824worker2 # [ 148.835440] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5825worker1 # [ 148.926879] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515826worker1 # [ 148.928720] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5827worker2 # [ 149.036918] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515828worker2 # [ 149.039052] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5829worker1 # [ 149.129020] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515830worker1 # [ 149.130897] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5831client # [ 149.205627] nix[1454]: scheduler at lb:50051: no healthy upstream, reconnecting5832worker2 # [ 149.237333] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515833worker2 # [ 149.240258] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5834worker1 # [ 149.331241] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_connected addr=lb:500515835worker1 # [ 149.333046] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5836worker2 # [ 149.440846] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:16Z level=info event=scheduler_connected addr=lb:500515837worker1 # [ 149.452694] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:16Z level=info event=oidc_jwks provider=mock keys=15838worker1 # [ 149.454062] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:16Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:500965839worker1 # [ 149.455755] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:16Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05840worker1 # [ 149.533324] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:16Z level=info event=scheduler_connected addr=lb:500515841worker1 # [ 149.536140] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:16Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:500965842worker1 # [ 149.537794] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:16Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=15843worker1 # [ 151.586150] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:18Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:365685844client: (finished: must succeed: sleep 5.0, in 5.03 seconds)5845client: waiting for success: ! systemctl is-active mid5846worker1 # [ 153.285051] nix-daemon[838]: reaped child process 1598, status = succeeded5847worker1 # [ 153.405749] nix-daemon[838]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)5848worker1 # [ 153.427687] nix-daemon[838]: reaped child process 1638, status = succeeded5849lb # [ 153.426452] niks3-server[1153]: 2026/09/25 15:26:20 INFO Received push request method=POST path=/api/pushes5850worker1 # [ 153.436511] nix-grpc-daemon[1574]: time=2026-09-25T15:26:20.062Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5851worker1 # [ 153.438108] nix-grpc-daemon[1574]: time=2026-09-25T15:26:20.062Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"5852lb # [ 153.470487] niks3-server[1153]: 2026/09/25 15:26:20 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst5853worker1 # [ 153.476812] nix-grpc-daemon[1574]: time=2026-09-25T15:26:20.103Z level=INFO msg="Uploading 1 narinfos"5854lb # [ 153.478919] niks3-server[1153]: 2026/09/25 15:26:20 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5855lb # [ 153.486057] niks3-server[1153]: 2026/09/25 15:26:20 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign5856lb # [ 153.493241] niks3-server[1153]: 2026/09/25 15:26:20 INFO Signed narinfos id=51 count=15857lb # [ 153.494849] niks3-server[1153]: 2026/09/25 15:26:20 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls5858lb # [ 153.502069] niks3-server[1153]: 2026/09/25 15:26:20 INFO Received complete push request method=POST path=/api/pushes/51/complete5859worker1 # [ 153.505439] nix-grpc-daemon[1574]: time=2026-09-25T15:26:20.132Z level=INFO msg="Upload complete. (249ms)"5860lb # [ 153.506398] niks3-server[1153]: 2026/09/25 15:26:20 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo5861worker1 # [ 153.516827] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50094 duration_s=8 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=05862worker1 # [ 153.528773] nix-daemon[838]: reaped child process 1596, status = succeeded5863client # [ 153.535200] systemd[1]: mid.service: Deactivated successfully.5864client # [ 153.537655] systemd[1]: mid.service: Consumed 174ms CPU time over 9.541s wall clock time, 41.5M memory peak, 8.1K incoming IP traffic, 9.2K outgoing IP traffic.5865client: (finished: waiting for success: ! systemctl is-active mid, in 1.07 seconds)5866client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }5867client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.04 seconds)5868(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.37 seconds)5869subtest: interrupting the client stops the build on the worker5870client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr5871client # [ 154.361240] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.5872client # Running as unit: intr.service; invocation ID: a0a0483c29da436f88430767f61c33655873client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)5874client # [ 154.538234] nix[1490]: don't know how to build these paths:5875worker1 # [ 154.538373] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:500945876client # [ 154.539239] nix[1490]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5877worker1 # [ 154.549504] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:21Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=05878worker1 # [ 154.553362] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:21Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=15879worker2 # [ 154.558169] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437165880worker2 # [ 154.567455] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)5881worker2 # [ 154.583567] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 30 ms (attempt 1/5)5882worker2 # [ 154.619279] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 51 ms (attempt 2/5)5883worker2 # [ 154.672919] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 119 ms (attempt 3/5)5884worker2 # [ 154.794410] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 651 ms (attempt 4/5)5885worker2 # [ 155.447751] nix-grpc-daemon[1664]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5886worker2 # [ 155.451026] nix-grpc-daemon[1664]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5887worker2 # [ 155.465427] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437165888client # [ 155.476142] nix[1490]: copying 1 paths to 'lb:50051'...5889client # [ 155.478888] nix[1490]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...5890worker2 # [ 155.479508] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437165891worker2 # [ 155.489621] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)5892worker2 # [ 155.509282] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:22Z level=info event=niks3_push_started pid=17365893worker2 # [ 155.677244] nix-grpc-daemon[1736]: time=2026-09-25T15:26:22.298Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5894worker2 # [ 155.802202] nix-daemon[857]: accepted connection from pid 1753, user nix-grpc-daemon (trusted)5895worker2 # [ 155.824426] nix-daemon[857]: reaped child process 1756, status = succeeded5896lb # [ 155.834685] niks3-server[1153]: 2026/09/25 15:26:22 INFO Received push request method=POST path=/api/pushes5897worker2 # [ 155.840365] nix-grpc-daemon[1736]: time=2026-09-25T15:26:22.464Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5898worker2 # [ 155.845740] nix-grpc-daemon[1736]: time=2026-09-25T15:26:22.466Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"5899lb # [ 155.868689] niks3-server[1153]: 2026/09/25 15:26:22 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls5900worker2 # [ 155.872095] nix-grpc-daemon[1736]: time=2026-09-25T15:26:22.495Z level=INFO msg="Uploading 1 narinfos"5901lb # [ 155.876370] niks3-server[1153]: 2026/09/25 15:26:22 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign5902lb # [ 155.879395] niks3-server[1153]: 2026/09/25 15:26:22 INFO Signed narinfos id=52 count=15903lb # [ 155.890599] niks3-server[1153]: 2026/09/25 15:26:22 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst5904lb # [ 155.903237] niks3-server[1153]: 2026/09/25 15:26:22 INFO Received complete push request method=POST path=/api/pushes/52/complete5905lb # [ 155.911559] niks3-server[1153]: 2026/09/25 15:26:22 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo5906worker2 # [ 155.913754] nix-grpc-daemon[1736]: time=2026-09-25T15:26:22.537Z level=INFO msg="Upload complete. (238ms)"5907worker2 # [ 155.921788] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=4245908worker2 # [ 155.934426] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437165909worker2 # [ 155.939949] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)5910worker2 # [ 155.944984] nix-daemon[857]: reaped child process 1734, status = succeeded5911client # [ 155.955585] nix[1490]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...5912worker2 # [ 155.958034] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)5913client: must succeed: systemctl kill -s INT intr5914client # [ 157.952463] systemd[1]: intr.service: Sent signal SIGINT to main process 1490 (nix) on client request.5915worker2 # [ 157.956129] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:24Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv5916client # [ 157.957605] nix[1490]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5917client # [ 157.963597] nix[1490]: error: interrupted by the user5918client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)5919client # [ 157.979686] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE5920client # [ 157.983089] systemd[1]: intr.service: Failed with result 'exit-code'.5921client # [ 157.984976] systemd[1]: intr.service: Consumed 164ms CPU time over 3.620s wall clock time, 41.4M memory peak, 6.5K incoming IP traffic, 8.3K outgoing IP traffic.5922worker2 # [ 157.985975] nix-daemon[857]: reaped child process 1758, status = succeeded5923worker2 # [ 157.988490] nix-daemon[1760]: unexpected Nix daemon error: error: interrupted by the user5924worker2 # [ 157.998018] nix-daemon[857]: reaped child process 1760, status = failed with exit code 15925(finished: subtest: interrupting the client stops the build on the worker, in 3.73 seconds)5926subtest: a deploy mid-build drains: the build finishes, the new generation takes over5927client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw5928client # [ 158.100480] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.5929client # Running as unit: sw.service; invocation ID: 1d879ea566b34e6abd8aad404ebf98175930lb # [ 158.114220] envoy[902]: conn peer=[2001:db8:1::1]:39336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3215 tx=1757 ms=93475931lb # [ 158.117010] envoy[902]: conn peer=[2001:db8:1::1]:41058 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3000 tx=850 ms=34355932client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.06 seconds)5933worker2 # [ 158.284420] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:437165934client # [ 158.291227] nix[1511]: don't know how to build these paths:5935client # [ 158.293523] nix[1511]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv5936worker1 # [ 158.303496] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:24Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=05937worker1 # [ 158.303727] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:24Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=15938worker1 # [ 158.311142] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500945939worker1 # [ 158.321934] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:500945940client # [ 158.330216] nix[1511]: copying 1 paths to 'lb:50051'...5941worker1 # [ 158.331811] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:500945942client # [ 158.331728] nix[1511]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...5943worker1 # [ 158.341254] nix-daemon[838]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5944worker1 # [ 158.469717] nix-daemon[838]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)5945worker1 # [ 158.490568] nix-daemon[838]: reaped child process 1673, status = succeeded5946lb # [ 158.489921] niks3-server[1153]: 2026/09/25 15:26:25 INFO Received push request method=POST path=/api/pushes5947worker1 # [ 158.494789] nix-grpc-daemon[1574]: time=2026-09-25T15:26:25.121Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5948worker1 # [ 158.496540] nix-grpc-daemon[1574]: time=2026-09-25T15:26:25.121Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"5949lb # [ 158.507934] niks3-server[1153]: 2026/09/25 15:26:25 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign5950worker1 # [ 158.512042] nix-grpc-daemon[1574]: time=2026-09-25T15:26:25.138Z level=INFO msg="Uploading 1 narinfos"5951lb # [ 158.514869] niks3-server[1153]: 2026/09/25 15:26:25 INFO Signed narinfos id=53 count=15952lb # [ 158.520693] niks3-server[1153]: 2026/09/25 15:26:25 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls5953lb # [ 158.523189] niks3-server[1153]: 2026/09/25 15:26:25 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst5954lb # [ 158.534105] niks3-server[1153]: 2026/09/25 15:26:25 INFO Received complete push request method=POST path=/api/pushes/53/complete5955worker1 # [ 158.538055] nix-grpc-daemon[1574]: time=2026-09-25T15:26:25.165Z level=INFO msg="Upload complete. (186ms)"5956lb # [ 158.538780] niks3-server[1153]: 2026/09/25 15:26:25 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo5957worker1 # [ 158.544554] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50094 duration_s=0 paths=1 nar_bytes_in=4165958worker1 # [ 158.547487] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:500945959worker1 # [ 158.558752] nix-daemon[838]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5960worker1 # [ 158.562581] nix-daemon[838]: reaped child process 1668, status = succeeded5961worker1 # [ 158.570853] nix-daemon[838]: accepted connection from pid 1510, user nix-grpc-daemon (trusted)5962client # [ 158.568655] nix[1511]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...5963worker1: must succeed: systemctl reload nix-grpc-daemon.service5964worker1 # [ 160.519098] systemd[1]: Reloading Nix worker-protocol over gRPC...5965worker1 # [ 160.561227] systemd[1]: Reloaded Nix worker-protocol over gRPC.5966worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)5967worker1: must succeed: pgrep -f 'read -t [9]0 x'5968worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)5969client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&25970worker2 # [ 160.783214] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:437165971client # don't know how to build these paths:5972client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv5973worker2 # [ 160.790243] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437165974worker1 # [ 160.808675] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:27Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=05975worker1 # [ 160.810735] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:27Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=15976worker2 # [ 160.812973] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437165977worker2 # [ 160.822063] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437165978client # copying 1 paths to 'lb:50051'...5979worker2 # [ 160.831353] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437165980client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...5981worker2 # [ 160.844640] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)5982worker1 # [ 160.898048] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:27Z level=info event=draining5983worker2 # [ 160.972218] nix-daemon[857]: accepted connection from pid 1807, user nix-grpc-daemon (trusted)5984worker2 # [ 160.989166] nix-daemon[857]: reaped child process 1810, status = succeeded5985lb # [ 160.998422] niks3-server[1153]: 2026/09/25 15:26:27 INFO Received push request method=POST path=/api/pushes5986worker2 # [ 161.004685] nix-grpc-daemon[1736]: time=2026-09-25T15:26:27.627Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5987worker2 # [ 161.006353] nix-grpc-daemon[1736]: time=2026-09-25T15:26:27.627Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"5988lb # [ 161.023011] niks3-server[1153]: 2026/09/25 15:26:27 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign5989worker2 # [ 161.026677] nix-grpc-daemon[1736]: time=2026-09-25T15:26:27.650Z level=INFO msg="Uploading 1 narinfos"5990lb # [ 161.031654] niks3-server[1153]: 2026/09/25 15:26:27 INFO Signed narinfos id=54 count=15991lb # [ 161.035981] niks3-server[1153]: 2026/09/25 15:26:27 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst5992lb # [ 161.043399] niks3-server[1153]: 2026/09/25 15:26:27 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls5993lb # [ 161.050627] niks3-server[1153]: 2026/09/25 15:26:27 INFO Received complete push request method=POST path=/api/pushes/54/complete5994lb # [ 161.062146] niks3-server[1153]: 2026/09/25 15:26:27 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo5995worker2 # [ 161.061308] nix-grpc-daemon[1736]: time=2026-09-25T15:26:27.683Z level=INFO msg="Upload complete. (205ms)"5996worker2 # [ 161.066797] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=5285997worker2 # [ 161.077436] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437165998worker2 # [ 161.082731] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)5999worker2 # [ 161.086691] nix-daemon[857]: reaped child process 1805, status = succeeded6000client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6001client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6002worker2 # [ 161.096345] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6003client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6004client # farm-a> LOG-farm-a-during-drain6005worker2 # [ 162.552424] nix-daemon[857]: reaped child process 1814, status = succeeded6006worker2 # [ 162.649017] nix-daemon[857]: accepted connection from pid 1830, user nix-grpc-daemon (trusted)6007lb # [ 162.669728] niks3-server[1153]: 2026/09/25 15:26:29 INFO Received push request method=POST path=/api/pushes6008worker2 # [ 162.670229] nix-daemon[857]: reaped child process 1833, status = succeeded6009worker2 # [ 162.680428] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.302Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6010worker2 # [ 162.682006] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.302Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6011lb # [ 162.696709] niks3-server[1153]: 2026/09/25 15:26:29 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6012lb # [ 162.704202] niks3-server[1153]: 2026/09/25 15:26:29 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6013worker2 # [ 162.708171] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.331Z level=INFO msg="Uploading 1 narinfos"6014lb # [ 162.711190] niks3-server[1153]: 2026/09/25 15:26:29 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6015lb # [ 162.717172] niks3-server[1153]: 2026/09/25 15:26:29 INFO Signed narinfos id=55 count=16016lb # [ 162.721014] niks3-server[1153]: 2026/09/25 15:26:29 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6017lb # [ 162.728674] niks3-server[1153]: 2026/09/25 15:26:29 INFO Received complete push request method=POST path=/api/pushes/55/complete6018worker2 # [ 162.733066] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.356Z level=INFO msg="Upload complete. (226ms)"6019lb # [ 162.735871] niks3-server[1153]: 2026/09/25 15:26:29 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6020worker1 # [ 162.745652] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:29Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06021worker2 # [ 162.740674] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06022worker1 # [ 162.747605] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:29Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16023worker2 # [ 162.757025] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166024worker2 # [ 162.761820] nix-daemon[857]: reaped child process 1812, status = succeeded6025worker2 # [ 162.769917] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166026worker2 # [ 162.780458] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.404Z level=INFO msg="All 1 paths already cached"6027client # copying 1 paths to 'lb:50051'...6028worker2 # [ 162.787525] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437166029client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6030worker2 # [ 162.800637] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6031worker2 # [ 162.927920] nix-daemon[857]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6032worker2 # [ 162.951288] nix-daemon[857]: reaped child process 1840, status = succeeded6033lb # [ 162.951823] niks3-server[1153]: 2026/09/25 15:26:29 INFO Received push request method=POST path=/api/pushes6034worker2 # [ 162.957939] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.581Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6035worker2 # [ 162.960813] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.581Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6036lb # [ 162.990401] niks3-server[1153]: 2026/09/25 15:26:29 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6037worker2 # [ 162.994316] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.617Z level=INFO msg="Uploading 1 narinfos"6038lb # [ 162.999387] niks3-server[1153]: 2026/09/25 15:26:29 INFO Signed narinfos id=56 count=16039lb # [ 163.007983] niks3-server[1153]: 2026/09/25 15:26:29 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6040lb # [ 163.014665] niks3-server[1153]: 2026/09/25 15:26:29 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6041lb # [ 163.021995] niks3-server[1153]: 2026/09/25 15:26:29 INFO Received complete push request method=POST path=/api/pushes/56/complete6042worker2 # [ 163.027343] nix-grpc-daemon[1736]: time=2026-09-25T15:26:29.651Z level=INFO msg="Upload complete. (216ms)"6043lb # [ 163.030498] niks3-server[1153]: 2026/09/25 15:26:29 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6044worker2 # [ 163.036331] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=6406045worker2 # [ 163.041846] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166046worker2 # [ 163.055796] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6047worker2 # [ 163.059961] nix-daemon[857]: reaped child process 1835, status = succeeded6048client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6049worker2 # [ 163.070824] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6050client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6051client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6052client # farm-b> LOG-farm-b-during-drain6053worker2 # [ 164.569298] nix-daemon[857]: reaped child process 1844, status = succeeded6054worker2 # [ 164.625881] nix-daemon[857]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)6055worker2 # [ 164.650007] nix-daemon[857]: reaped child process 1863, status = succeeded6056lb # [ 164.657886] niks3-server[1153]: 2026/09/25 15:26:31 INFO Received push request method=POST path=/api/pushes6057worker2 # [ 164.662796] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.285Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6058worker2 # [ 164.664878] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.285Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6059lb # [ 164.683117] niks3-server[1153]: 2026/09/25 15:26:31 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6060worker2 # [ 164.686645] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.310Z level=INFO msg="Uploading 1 narinfos"6061lb # [ 164.693872] niks3-server[1153]: 2026/09/25 15:26:31 INFO Signed narinfos id=57 count=16062lb # [ 164.695630] niks3-server[1153]: 2026/09/25 15:26:31 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6063lb # [ 164.708750] niks3-server[1153]: 2026/09/25 15:26:31 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6064lb # [ 164.721127] niks3-server[1153]: 2026/09/25 15:26:31 INFO Received complete push request method=POST path=/api/pushes/57/complete6065worker2 # [ 164.724831] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.348Z level=INFO msg="Upload complete. (236ms)"6066lb # [ 164.731722] niks3-server[1153]: 2026/09/25 15:26:31 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6067lb # [ 164.736121] niks3-server[1153]: 2026/09/25 15:26:31 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6068worker2 # [ 164.732733] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=4 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06069worker1 # [ 164.742835] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:31Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06070worker1 # [ 164.745276] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:31Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16071worker2 # [ 164.746949] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166072worker2 # [ 164.750142] nix-daemon[857]: reaped child process 1842, status = succeeded6073worker2 # [ 164.758494] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166074worker2 # [ 164.767959] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.391Z level=INFO msg="All 2 paths already cached"6075client # copying 1 paths to 'lb:50051'...6076worker2 # [ 164.774860] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437166077client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6078worker2 # [ 164.788369] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6079worker2 # [ 164.914780] nix-daemon[857]: accepted connection from pid 1867, user nix-grpc-daemon (trusted)6080worker2 # [ 164.934401] nix-daemon[857]: reaped child process 1870, status = succeeded6081lb # [ 164.941543] niks3-server[1153]: 2026/09/25 15:26:31 INFO Received push request method=POST path=/api/pushes6082worker2 # [ 164.947609] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.571Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6083worker2 # [ 164.949322] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.571Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6084lb # [ 164.968055] niks3-server[1153]: 2026/09/25 15:26:31 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6085worker2 # [ 164.972123] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.595Z level=INFO msg="Uploading 1 narinfos"6086lb # [ 164.977029] niks3-server[1153]: 2026/09/25 15:26:31 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6087lb # [ 164.979684] niks3-server[1153]: 2026/09/25 15:26:31 INFO Signed narinfos id=58 count=16088lb # [ 164.992774] niks3-server[1153]: 2026/09/25 15:26:31 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6089lb # [ 164.995514] niks3-server[1153]: 2026/09/25 15:26:31 INFO Received complete push request method=POST path=/api/pushes/58/complete6090worker2 # [ 164.999716] nix-grpc-daemon[1736]: time=2026-09-25T15:26:31.623Z level=INFO msg="Upload complete. (201ms)"6091lb # [ 165.004143] niks3-server[1153]: 2026/09/25 15:26:31 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6092worker2 # [ 165.008557] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=7686093worker2 # [ 165.011938] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166094worker2 # [ 165.022883] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6095worker2 # [ 165.028229] nix-daemon[857]: reaped child process 1865, status = succeeded6096client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6097client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6098worker2 # [ 165.038559] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6099client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6100client # farm-top> LOG-farm-top-during-drain6101worker2 # [ 166.465349] nix-daemon[857]: reaped child process 1874, status = succeeded6102worker2 # [ 166.589510] nix-daemon[857]: accepted connection from pid 1890, user nix-grpc-daemon (trusted)6103worker2 # [ 166.609828] nix-daemon[857]: reaped child process 1893, status = succeeded6104lb # [ 166.619669] niks3-server[1153]: 2026/09/25 15:26:33 INFO Received push request method=POST path=/api/pushes6105worker2 # [ 166.624447] nix-grpc-daemon[1736]: time=2026-09-25T15:26:33.247Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6106worker2 # [ 166.626060] nix-grpc-daemon[1736]: time=2026-09-25T15:26:33.247Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6107lb # [ 166.649958] niks3-server[1153]: 2026/09/25 15:26:33 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6108lb # [ 166.659765] niks3-server[1153]: 2026/09/25 15:26:33 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6109worker2 # [ 166.661867] nix-grpc-daemon[1736]: time=2026-09-25T15:26:33.285Z level=INFO msg="Uploading 1 narinfos"6110lb # [ 166.667371] niks3-server[1153]: 2026/09/25 15:26:33 INFO Signed narinfos id=59 count=16111lb # [ 166.674310] niks3-server[1153]: 2026/09/25 15:26:33 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6112lb # [ 166.680427] niks3-server[1153]: 2026/09/25 15:26:33 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6113lb # [ 166.688734] niks3-server[1153]: 2026/09/25 15:26:33 INFO Received complete push request method=POST path=/api/pushes/59/complete6114worker2 # [ 166.697950] nix-grpc-daemon[1736]: time=2026-09-25T15:26:33.321Z level=INFO msg="Upload complete. (250ms)"6115lb # [ 166.700650] niks3-server[1153]: 2026/09/25 15:26:33 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6116worker2 # [ 166.704713] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=5 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06117worker2 # [ 166.719352] nix-daemon[857]: reaped child process 1872, status = succeeded6118client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.12 seconds)6119client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6120client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.14 seconds)6121worker1: must succeed: pkill -f 'read -t [9]0 x'6122worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6123client: waiting for success: ! systemctl is-active sw6124worker1 # [ 167.047690] nix-daemon[838]: reaped child process 1677, status = succeeded6125worker1 # [ 167.108766] nix-daemon[838]: accepted connection from pid 1724, user nix-grpc-daemon (trusted)6126worker1 # [ 167.124358] nix-daemon[838]: reaped child process 1727, status = succeeded6127lb # [ 167.130035] niks3-server[1153]: 2026/09/25 15:26:33 INFO Received push request method=POST path=/api/pushes6128worker1 # [ 167.136251] nix-grpc-daemon[1574]: time=2026-09-25T15:26:33.762Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6129worker1 # [ 167.137826] nix-grpc-daemon[1574]: time=2026-09-25T15:26:33.762Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6130lb # [ 167.154820] niks3-server[1153]: 2026/09/25 15:26:33 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6131lb # [ 167.161826] niks3-server[1153]: 2026/09/25 15:26:33 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6132worker1 # [ 167.169202] nix-grpc-daemon[1574]: time=2026-09-25T15:26:33.796Z level=INFO msg="Uploading 1 narinfos"6133lb # [ 167.169276] niks3-server[1153]: 2026/09/25 15:26:33 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6134lb # [ 167.171774] niks3-server[1153]: 2026/09/25 15:26:33 INFO Signed narinfos id=60 count=16135lb # [ 167.180563] niks3-server[1153]: 2026/09/25 15:26:33 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6136lb # [ 167.190406] niks3-server[1153]: 2026/09/25 15:26:33 INFO Received complete push request method=POST path=/api/pushes/60/complete6137worker1 # [ 167.195452] nix-grpc-daemon[1574]: time=2026-09-25T15:26:33.822Z level=INFO msg="Upload complete. (208ms)"6138lb # [ 167.195889] niks3-server[1153]: 2026/09/25 15:26:33 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6139worker1 # [ 167.204630] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50094 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=2 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06140worker1 # [ 167.215934] nix-daemon[838]: reaped child process 1675, status = succeeded6141client # [ 167.223040] systemd[1]: sw.service: Deactivated successfully.6142client # [ 167.226174] systemd[1]: sw.service: Consumed 174ms CPU time over 9.124s wall clock time, 41.5M memory peak, 7.1K incoming IP traffic, 8.3K outgoing IP traffic.6143worker1 # [ 167.900410] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:34Z level=info event=signal_exit6144client: (finished: waiting for success: ! systemctl is-active sw, in 1.07 seconds)6145client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6146client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6147worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6148lb # [ 168.114027] envoy[902]: conn peer=[2001:db8:1::1]:41078 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9634 tx=4379 ms=59376149lb # [ 168.116823] envoy[902]: conn peer=[2001:db8:1::1]:41070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2961 tx=1498 ms=89266150lb # [ 168.904653] niks3-server[1153]: 2026/09/25 15:26:35 INFO lead: released remote=192.168.1.3:443966151worker1 # [ 169.900917] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:36Z level=info event=scheduler_restarting6152worker2 # [ 169.901250] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:36Z level=info event=scheduler_restarting addr=lb:500516153lb # [ 169.901671] envoy[902]: [2026-09-25 15:26:36.524][902][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6154worker1 # [ 169.904846] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:36Z level=info event=scheduler_restarting addr=lb:500516155worker1 # [ 169.910326] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:500966156worker1 # [ 169.913773] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:36Z level=info event=worker_gone addr=192.168.1.4:500516157worker1 # [ 169.916098] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:36Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:500966158worker1 # [ 169.918645] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:36Z level=info event=worker_gone addr=192.168.1.3:500516159worker1 # [ 169.933147] nix-daemon[838]: reaped child process 1566, status = succeeded6160worker2 # [ 170.101705] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:36Z level=info event=scheduler_connected addr=lb:500516161worker2 # [ 170.105187] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6162worker1 # [ 170.116791] nix-grpc-daemon[1510]: ts=2026-09-25T15:26:36Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6163worker2 # [ 170.305866] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:36Z level=info event=scheduler_connected addr=lb:500516164worker2 # [ 170.308792] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:36Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6165worker2 # [ 170.509742] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_connected addr=lb:500516166worker2 # [ 170.512833] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6167worker2 # [ 170.713596] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_connected addr=lb:500516168worker2 # [ 170.717004] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6169worker1 # [ 170.913874] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6170worker1 # [ 170.914979] systemd[1]: nix-grpc-daemon.service: Consumed 897ms CPU time over 28.161s wall clock time, 38.1M memory peak, 245.7K incoming IP traffic, 86.3K outgoing IP traffic.6171worker2 # [ 170.918320] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_connected addr=lb:500516172worker2 # [ 170.921250] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6173worker1 # [ 170.928158] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6174worker1 # [ 170.933240] systemd[1]: Starting Nix worker-protocol over gRPC...6175worker1 # [ 171.054123] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6176worker1 # [ 171.056263] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6177worker1 # [ 171.060076] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16178worker1 # [ 171.066616] nix-grpc-daemon[1742]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6179lb # [ 171.076639] niks3-server[1153]: 2026/09/25 15:26:37 INFO lead: acquired remote=192.168.1.3:568386180worker1 # [ 171.088534] nix-grpc-daemon[1742]: E0000 00:00:1790349997.691562 1742 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6181worker1 # [ 171.091010] nix-grpc-daemon[1742]: E0000 00:00:1790349997.693614 1742 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6182worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.08 seconds)6183lb: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux6184lb: must succeed: curl -sf localhost:9901/clusters6185worker1 # [ 171.115305] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6186worker1 # [ 171.117750] nix-grpc-daemon[1742]: E0000 00:00:1790349997.703125 1783 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6187worker1 # [ 171.123398] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=scheduler_take_over6188worker2 # [ 171.122049] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_connected addr=lb:500516189worker1 # [ 171.126772] nix-grpc-daemon[1742]: E0000 00:00:1790349997.725991 1783 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6190worker2 # [ 171.125195] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6191worker1 # [ 171.129317] nix-grpc-daemon[1742]: E0000 00:00:1790349997.726139 1783 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6192worker1 # [ 171.131718] nix-grpc-daemon[1742]: E0000 00:00:1790349997.741574 1783 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6193worker1 # [ 171.134231] systemd[1]: Started Nix worker-protocol over gRPC.6194worker1 # [ 171.135077] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=scheduler_connected addr=lb:500516195worker1 # [ 171.136404] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6196lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6197worker1 # [ 171.327267] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=scheduler_connected addr=lb:500516198worker2 # [ 171.325879] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_connected addr=lb:500516199worker1 # [ 171.330148] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6200worker2 # [ 171.328735] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6201worker1 # [ 171.530600] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:38Z level=info event=scheduler_connected addr=lb:500516202worker2 # [ 171.529467] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:38Z level=info event=scheduler_connected addr=lb:500516203worker1 # [ 171.536099] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:38Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:580106204worker1 # [ 171.538714] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:38Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06205worker1 # [ 171.545437] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:38Z level=info event=oidc_jwks provider=mock keys=16206worker1 # [ 171.547529] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:38Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:580106207worker1 # [ 171.550636] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:38Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06208lb: must succeed: curl -sf localhost:9901/clusters6209lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6210worker1 # [ 172.229739] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:38Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:580206211lb: must succeed: curl -sf localhost:9901/clusters6212lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6213lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of aarch64-linux, in 2.12 seconds)6214lb: must succeed: curl -sf localhost:9901/clusters6215lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6216worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6217worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6218(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 15.24 seconds)6219subtest: queue gauges show a burst while every build blocks6220lb: must succeed: curl -sf localhost:9901/clusters6221lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6222lb: must succeed: curl -sf localhost:9901/clusters6223lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6224worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6225worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6226client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6227client # [ 173.445859] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6228client # Running as unit: burst.service; invocation ID: ea9f9e8e30ef478bbce119b94d9b89f36229client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.05 seconds)6230lb: must succeed: curl -sf localhost:9901/clusters6231lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6232lb: must succeed: curl -sf localhost:9901/clusters6233lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6234worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6235worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)6236worker2 # [ 173.624281] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:437166237client # [ 173.629689] nix[1567]: don't know how to build these paths:6238client # [ 173.630702] nix[1567]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6239client # [ 173.632399] nix[1567]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6240client # [ 173.633451] nix[1567]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6241client # [ 173.634447] nix[1567]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6242worker1 # [ 173.648271] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:40Z level=debug event=want client=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06243worker1 # [ 173.652415] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:40Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16244worker2 # [ 173.651703] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166245worker1 # [ 173.657321] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:40Z level=debug event=want client=1 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06246worker1 # [ 173.661985] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:40Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16247worker2 # [ 173.661934] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166248worker1 # [ 173.670641] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:40Z level=debug event=want client=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06249client # [ 173.671252] nix[1567]: copying 1 paths to 'lb:50051'...6250worker2 # [ 173.670863] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437166251client # [ 173.672725] nix[1567]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6252worker1 # [ 173.679322] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:40Z level=debug event=want client=1 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06253worker2 # [ 173.680617] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6254worker1 # [ 173.684451] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366255worker1 # [ 173.690660] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6256worker1 # [ 173.693933] nix-grpc-daemon[1742]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)6257worker1 # [ 173.742129] nix-grpc-daemon[1742]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 198 ms (attempt 2/5)6258worker2 # [ 173.811609] nix-daemon[857]: accepted connection from pid 1897, user nix-grpc-daemon (trusted)6259worker2 # [ 173.833026] nix-daemon[857]: reaped child process 1900, status = succeeded6260lb # [ 173.833741] niks3-server[1153]: 2026/09/25 15:26:40 INFO Received push request method=POST path=/api/pushes6261worker2 # [ 173.837455] nix-grpc-daemon[1736]: time=2026-09-25T15:26:40.460Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6262worker2 # [ 173.839092] nix-grpc-daemon[1736]: time=2026-09-25T15:26:40.460Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6263lb # [ 173.850377] niks3-server[1153]: 2026/09/25 15:26:40 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6264worker2 # [ 173.851276] nix-grpc-daemon[1736]: time=2026-09-25T15:26:40.474Z level=INFO msg="Uploading 1 narinfos"6265lb # [ 173.858227] niks3-server[1153]: 2026/09/25 15:26:40 INFO Signed narinfos id=61 count=16266lb # [ 173.859941] niks3-server[1153]: 2026/09/25 15:26:40 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6267lb # [ 173.867622] niks3-server[1153]: 2026/09/25 15:26:40 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6268lb # [ 173.876851] niks3-server[1153]: 2026/09/25 15:26:40 INFO Received complete push request method=POST path=/api/pushes/61/complete6269worker2 # [ 173.879457] nix-grpc-daemon[1736]: time=2026-09-25T15:26:40.503Z level=INFO msg="Upload complete. (188ms)"6270lb # [ 173.882966] niks3-server[1153]: 2026/09/25 15:26:40 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6271worker2 # [ 173.887256] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=4166272worker2 # [ 173.896563] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166273worker2 # [ 173.902727] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6274worker2 # [ 173.905159] nix-daemon[857]: reaped child process 1895, status = succeeded6275client # [ 173.913066] nix[1567]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6276worker2 # [ 173.913467] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6277worker1 # [ 173.941375] nix-grpc-daemon[1742]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 396 ms (attempt 3/5)6278worker1 # [ 174.377659] nix-grpc-daemon[1742]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 141 ms (attempt 4/5)6279worker1 # [ 174.519958] nix-grpc-daemon[1742]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6280worker1 # [ 174.523863] nix-grpc-daemon[1742]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6281worker1 # [ 174.534300] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366282client # [ 174.542808] nix[1567]: copying 1 paths to 'lb:50051'...6283worker1 # [ 174.543614] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366284client # [ 174.544316] nix[1567]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6285worker1 # [ 174.552177] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6286lb: must succeed: curl -sf localhost:9901/clusters6287worker1 # [ 174.571736] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:41Z level=info event=niks3_push_started pid=18136288lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6289lb: must succeed: curl -sf localhost:9901/clusters6290lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6291worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6292worker1 # [ 174.713670] nix-grpc-daemon[1813]: time=2026-09-25T15:26:41.338Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6293worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.14 seconds)6294worker1 # [ 174.842934] nix-daemon[838]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6295lb # [ 174.868876] niks3-server[1153]: 2026/09/25 15:26:41 INFO Received push request method=POST path=/api/pushes6296worker1 # [ 174.877627] nix-grpc-daemon[1813]: time=2026-09-25T15:26:41.503Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6297worker1 # [ 174.879201] nix-grpc-daemon[1813]: time=2026-09-25T15:26:41.503Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6298worker1 # [ 174.888147] nix-daemon[838]: reaped child process 1835, status = succeeded6299lb # [ 174.897696] niks3-server[1153]: 2026/09/25 15:26:41 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6300worker1 # [ 174.903339] nix-grpc-daemon[1813]: time=2026-09-25T15:26:41.529Z level=INFO msg="Uploading 1 narinfos"6301lb # [ 174.902603] niks3-server[1153]: 2026/09/25 15:26:41 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6302lb # [ 174.907625] niks3-server[1153]: 2026/09/25 15:26:41 INFO Signed narinfos id=62 count=16303lb # [ 174.913663] niks3-server[1153]: 2026/09/25 15:26:41 INFO Received complete push request method=POST path=/api/pushes/62/complete6304worker1 # [ 174.923207] nix-grpc-daemon[1813]: time=2026-09-25T15:26:41.550Z level=INFO msg="Upload complete. (210ms)"6305lb # [ 174.926829] niks3-server[1153]: 2026/09/25 15:26:41 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6306worker1 # [ 174.928599] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=4166307worker1 # [ 174.937313] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366308worker1 # [ 174.941376] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6309worker1 # [ 174.944648] nix-daemon[838]: reaped child process 1811, status = succeeded6310lb # [ 174.947411] niks3-server[1153]: 2026/09/25 15:26:41 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6311client # [ 174.950288] nix[1567]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6312worker1 # [ 174.953035] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6313worker2 # [ 176.175652] nix-daemon[857]: reaped child process 1904, status = succeeded6314worker2 # [ 176.230588] nix-daemon[857]: accepted connection from pid 1929, user nix-grpc-daemon (trusted)6315worker1 # [ 176.232715] nix-daemon[838]: reaped child process 1840, status = succeeded6316worker2 # [ 176.251352] nix-daemon[857]: reaped child process 1932, status = succeeded6317lb # [ 176.258433] niks3-server[1153]: 2026/09/25 15:26:42 INFO Received push request method=POST path=/api/pushes6318worker2 # [ 176.264378] nix-grpc-daemon[1736]: time=2026-09-25T15:26:42.885Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6319worker2 # [ 176.265914] nix-grpc-daemon[1736]: time=2026-09-25T15:26:42.885Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6320lb # [ 176.288773] niks3-server[1153]: 2026/09/25 15:26:42 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6321worker2 # [ 176.291216] nix-grpc-daemon[1736]: time=2026-09-25T15:26:42.914Z level=INFO msg="Uploading 1 narinfos"6322worker1 # [ 176.294861] nix-daemon[838]: accepted connection from pid 1861, user nix-grpc-daemon (trusted)6323lb # [ 176.297162] niks3-server[1153]: 2026/09/25 15:26:42 INFO Signed narinfos id=63 count=16324lb # [ 176.303914] niks3-server[1153]: 2026/09/25 15:26:42 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6325worker1 # [ 176.315473] nix-daemon[838]: reaped child process 1864, status = succeeded6326lb # [ 176.314956] niks3-server[1153]: 2026/09/25 15:26:42 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6327lb # [ 176.323565] niks3-server[1153]: 2026/09/25 15:26:42 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6328worker1 # [ 176.335835] nix-grpc-daemon[1813]: time=2026-09-25T15:26:42.960Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6329worker2 # [ 176.333718] nix-grpc-daemon[1736]: time=2026-09-25T15:26:42.957Z level=INFO msg="Upload complete. (210ms)"6330lb # [ 176.336778] niks3-server[1153]: 2026/09/25 15:26:42 INFO Received push request method=POST path=/api/pushes6331worker1 # [ 176.338213] nix-grpc-daemon[1813]: time=2026-09-25T15:26:42.960Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6332lb # [ 176.338879] niks3-server[1153]: 2026/09/25 15:26:42 INFO Received complete push request method=POST path=/api/pushes/63/complete6333worker1 # [ 176.346101] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:42Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.4:50051 clients=16334lb # [ 176.348823] niks3-server[1153]: 2026/09/25 15:26:42 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6335worker2 # [ 176.340729] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=2 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06336worker2 # [ 176.355890] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166337worker2 # [ 176.359033] nix-daemon[857]: reaped child process 1902, status = succeeded6338lb # [ 176.361694] niks3-server[1153]: 2026/09/25 15:26:42 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6339lb # [ 176.368510] niks3-server[1153]: 2026/09/25 15:26:42 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6340worker2 # [ 176.370165] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166341worker1 # [ 176.374016] nix-grpc-daemon[1813]: time=2026-09-25T15:26:43.000Z level=INFO msg="Uploading 1 narinfos"6342lb # [ 176.374472] niks3-server[1153]: 2026/09/25 15:26:42 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6343lb # [ 176.380375] niks3-server[1153]: 2026/09/25 15:26:42 INFO Signed narinfos id=64 count=16344lb # [ 176.383937] niks3-server[1153]: 2026/09/25 15:26:43 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6345client # [ 176.381977] nix[1567]: copying 1 paths to 'lb:50051'...6346client # [ 176.385475] nix[1567]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6347worker2 # [ 176.385736] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437166348lb # [ 176.390350] niks3-server[1153]: 2026/09/25 15:26:43 INFO Received complete push request method=POST path=/api/pushes/64/complete6349worker2 # [ 176.393179] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6350lb # [ 176.396860] niks3-server[1153]: 2026/09/25 15:26:43 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6351worker1 # [ 176.396627] nix-grpc-daemon[1813]: time=2026-09-25T15:26:43.023Z level=INFO msg="Upload complete. (236ms)"6352worker1 # [ 176.404841] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=1 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=1 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06353worker1 # [ 176.420659] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:43Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.3:50051 clients=16354worker1 # [ 176.423809] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366355worker1 # [ 176.429330] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366356client # [ 176.430190] nix[1567]: copying 1 paths to 'lb:50051'...6357client # [ 176.431636] nix[1567]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6358worker1 # [ 176.442670] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366359worker1 # [ 176.447363] nix-daemon[838]: reaped child process 1838, status = succeeded6360worker1 # [ 176.449692] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6361worker2 # [ 176.517473] nix-daemon[857]: accepted connection from pid 1936, user nix-grpc-daemon (trusted)6362worker2 # [ 176.537599] nix-daemon[857]: reaped child process 1939, status = succeeded6363lb # [ 176.542735] niks3-server[1153]: 2026/09/25 15:26:43 INFO Received push request method=POST path=/api/pushes6364worker2 # [ 176.545760] nix-grpc-daemon[1736]: time=2026-09-25T15:26:43.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6365worker2 # [ 176.547611] nix-grpc-daemon[1736]: time=2026-09-25T15:26:43.169Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6366worker1 # [ 176.557753] nix-daemon[838]: accepted connection from pid 1868, user nix-grpc-daemon (trusted)6367lb # [ 176.557913] niks3-server[1153]: 2026/09/25 15:26:43 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6368worker2 # [ 176.558812] nix-grpc-daemon[1736]: time=2026-09-25T15:26:43.182Z level=INFO msg="Uploading 1 narinfos"6369lb # [ 176.565887] niks3-server[1153]: 2026/09/25 15:26:43 INFO Signed narinfos id=65 count=16370lb # [ 176.567616] niks3-server[1153]: 2026/09/25 15:26:43 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6371worker1 # [ 176.574476] nix-daemon[838]: reaped child process 1871, status = succeeded6372lb # [ 176.578469] niks3-server[1153]: 2026/09/25 15:26:43 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6373worker1 # [ 176.588242] nix-grpc-daemon[1813]: time=2026-09-25T15:26:43.214Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6374worker1 # [ 176.589867] nix-grpc-daemon[1813]: time=2026-09-25T15:26:43.214Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6375lb # [ 176.589469] niks3-server[1153]: 2026/09/25 15:26:43 INFO Received complete push request method=POST path=/api/pushes/65/complete6376worker2 # [ 176.590318] nix-grpc-daemon[1736]: time=2026-09-25T15:26:43.214Z level=INFO msg="Upload complete. (187ms)"6377lb # [ 176.596381] niks3-server[1153]: 2026/09/25 15:26:43 INFO Received push request method=POST path=/api/pushes6378worker2 # [ 176.596256] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=4166379lb # [ 176.603035] niks3-server[1153]: 2026/09/25 15:26:43 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6380worker2 # [ 176.605425] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166381lb # [ 176.610255] niks3-server[1153]: 2026/09/25 15:26:43 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6382worker2 # [ 176.611245] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6383worker1 # [ 176.616365] nix-grpc-daemon[1813]: time=2026-09-25T15:26:43.243Z level=INFO msg="Uploading 1 narinfos"6384worker2 # [ 176.615329] nix-daemon[857]: reaped child process 1934, status = succeeded6385lb # [ 176.616627] niks3-server[1153]: 2026/09/25 15:26:43 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6386lb # [ 176.622707] niks3-server[1153]: 2026/09/25 15:26:43 INFO Signed narinfos id=66 count=16387client # [ 176.624407] nix[1567]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6388worker2 # [ 176.624593] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6389lb # [ 176.628259] niks3-server[1153]: 2026/09/25 15:26:43 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6390lb # [ 176.638122] niks3-server[1153]: 2026/09/25 15:26:43 INFO Received complete push request method=POST path=/api/pushes/66/complete6391worker1 # [ 176.643848] nix-grpc-daemon[1813]: time=2026-09-25T15:26:43.270Z level=INFO msg="Upload complete. (194ms)"6392lb # [ 176.644230] niks3-server[1153]: 2026/09/25 15:26:43 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6393worker1 # [ 176.650513] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=4166394worker1 # [ 176.662995] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366395worker1 # [ 176.668434] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6396worker1 # [ 176.672791] nix-daemon[838]: reaped child process 1866, status = succeeded6397client # [ 176.679524] nix[1567]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6398worker1 # [ 176.684268] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6399lb # [ 178.114534] envoy[902]: conn peer=[2001:db8:1::3]:37118 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2772 tx=928 ms=271776400worker2 # [ 178.503036] nix-daemon[857]: reaped child process 1943, status = succeeded6401worker1 # [ 178.570969] nix-daemon[838]: reaped child process 1875, status = succeeded6402worker2 # [ 178.568836] nix-daemon[857]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)6403worker2 # [ 178.587903] nix-daemon[857]: reaped child process 1971, status = succeeded6404lb # [ 178.596363] niks3-server[1153]: 2026/09/25 15:26:45 INFO Received push request method=POST path=/api/pushes6405worker2 # [ 178.602132] nix-grpc-daemon[1736]: time=2026-09-25T15:26:45.225Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6406worker2 # [ 178.607056] nix-grpc-daemon[1736]: time=2026-09-25T15:26:45.227Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6407worker1 # [ 178.633745] nix-daemon[838]: accepted connection from pid 1900, user nix-grpc-daemon (trusted)6408worker2 # [ 178.632493] nix-grpc-daemon[1736]: time=2026-09-25T15:26:45.256Z level=INFO msg="Uploading 1 narinfos"6409lb # [ 178.634462] niks3-server[1153]: 2026/09/25 15:26:45 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6410lb # [ 178.642738] niks3-server[1153]: 2026/09/25 15:26:45 INFO Signed narinfos id=67 count=16411lb # [ 178.649404] niks3-server[1153]: 2026/09/25 15:26:45 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6412worker1 # [ 178.652105] nix-daemon[838]: reaped child process 1903, status = succeeded6413lb # [ 178.651985] niks3-server[1153]: 2026/09/25 15:26:45 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6414lb # [ 178.663758] niks3-server[1153]: 2026/09/25 15:26:45 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6415worker1 # [ 178.671396] nix-grpc-daemon[1813]: time=2026-09-25T15:26:45.297Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6416lb # [ 178.673072] niks3-server[1153]: 2026/09/25 15:26:45 INFO Received push request method=POST path=/api/pushes6417worker1 # [ 178.673299] nix-grpc-daemon[1813]: time=2026-09-25T15:26:45.297Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6418lb # [ 178.675033] niks3-server[1153]: 2026/09/25 15:26:45 INFO Received complete push request method=POST path=/api/pushes/67/complete6419lb # [ 178.683588] niks3-server[1153]: 2026/09/25 15:26:45 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6420worker2 # [ 178.677949] nix-grpc-daemon[1736]: time=2026-09-25T15:26:45.301Z level=INFO msg="Upload complete. (241ms)"6421worker2 # [ 178.684784] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=2 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=3 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06422worker2 # [ 178.697144] nix-daemon[857]: reaped child process 1941, status = succeeded6423worker1 # [ 178.699704] nix-grpc-daemon[1813]: time=2026-09-25T15:26:45.326Z level=INFO msg="Uploading 1 narinfos"6424lb # [ 178.699161] niks3-server[1153]: 2026/09/25 15:26:45 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6425lb # [ 178.705544] niks3-server[1153]: 2026/09/25 15:26:45 INFO Signed narinfos id=68 count=16426lb # [ 178.707001] niks3-server[1153]: 2026/09/25 15:26:45 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6427lb # [ 178.717754] niks3-server[1153]: 2026/09/25 15:26:45 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6428lb # [ 178.721232] niks3-server[1153]: 2026/09/25 15:26:45 INFO Received complete push request method=POST path=/api/pushes/68/complete6429worker1 # [ 178.723943] nix-grpc-daemon[1813]: time=2026-09-25T15:26:45.351Z level=INFO msg="Upload complete. (228ms)"6430lb # [ 178.725814] niks3-server[1153]: 2026/09/25 15:26:45 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6431lb # [ 178.729430] niks3-server[1153]: 2026/09/25 15:26:45 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6432worker1 # [ 178.732600] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=4 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06433worker1 # [ 178.744754] nix-daemon[838]: reaped child process 1873, status = succeeded6434client # [ 178.755957] systemd[1]: burst.service: Deactivated successfully.6435client # [ 178.759480] systemd[1]: burst.service: Consumed 189ms CPU time over 5.312s wall clock time, 51.7M memory peak, 13.6K incoming IP traffic, 18.4K outgoing IP traffic.6436client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6437client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6438lb: must succeed: curl -sf localhost:9901/clusters6439lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6440lb: must succeed: curl -sf localhost:9901/clusters6441lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6442worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6443worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6444lb: must succeed: curl -sf localhost:9901/clusters6445lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6446lb: must succeed: curl -sf localhost:9901/clusters6447lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6448worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6449worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6450(finished: subtest: queue gauges show a burst while every build blocks, in 6.45 seconds)6451subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6452client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16453worker1 # [ 179.938454] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580366454worker2 # [ 179.940134] nix-grpc-daemon[1664]: ts=2026-09-25T15:26:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166455worker1 # [ 179.967159] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=want client=2 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06456worker1 # [ 179.971762] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16457worker1 # [ 179.975876] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=want client=2 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06458worker1 # [ 179.980385] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=want client=2 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06459worker1 # [ 179.984585] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366460worker1 # [ 179.988153] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366461worker1 # [ 179.994003] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366462worker1 # [ 180.003260] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6463worker1 # [ 180.128357] nix-daemon[838]: accepted connection from pid 1919, user nix-grpc-daemon (trusted)6464worker1 # [ 180.145842] nix-daemon[838]: reaped child process 1922, status = succeeded6465lb # [ 180.152188] niks3-server[1153]: 2026/09/25 15:26:46 INFO Received push request method=POST path=/api/pushes6466worker1 # [ 180.157830] nix-grpc-daemon[1813]: time=2026-09-25T15:26:46.784Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6467worker1 # [ 180.159384] nix-grpc-daemon[1813]: time=2026-09-25T15:26:46.784Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6468lb # [ 180.169511] niks3-server[1153]: 2026/09/25 15:26:46 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6469worker1 # [ 180.174613] nix-grpc-daemon[1813]: time=2026-09-25T15:26:46.801Z level=INFO msg="Uploading 1 narinfos"6470lb # [ 180.178529] niks3-server[1153]: 2026/09/25 15:26:46 INFO Signed narinfos id=69 count=16471lb # [ 180.181871] niks3-server[1153]: 2026/09/25 15:26:46 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6472lb # [ 180.191466] niks3-server[1153]: 2026/09/25 15:26:46 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6473lb # [ 180.195196] niks3-server[1153]: 2026/09/25 15:26:46 INFO Received complete push request method=POST path=/api/pushes/69/complete6474worker1 # [ 180.199798] nix-grpc-daemon[1813]: time=2026-09-25T15:26:46.826Z level=INFO msg="Upload complete. (186ms)"6475lb # [ 180.200289] niks3-server[1153]: 2026/09/25 15:26:46 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6476worker1 # [ 180.206386] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=5126477worker1 # [ 180.217850] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366478worker1 # [ 180.222518] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6479worker1 # [ 180.226432] nix-daemon[838]: reaped child process 1917, status = succeeded6480worker1 # [ 180.236480] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6481worker1 # [ 181.678299] nix-daemon[838]: reaped child process 1926, status = succeeded6482worker1 # [ 181.786831] nix-daemon[838]: accepted connection from pid 1942, user nix-grpc-daemon (trusted)6483worker1 # [ 181.806434] nix-daemon[838]: reaped child process 1945, status = succeeded6484lb # [ 181.813489] niks3-server[1153]: 2026/09/25 15:26:48 INFO Received push request method=POST path=/api/pushes6485worker1 # [ 181.821462] nix-grpc-daemon[1813]: time=2026-09-25T15:26:48.446Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6486worker1 # [ 181.822966] nix-grpc-daemon[1813]: time=2026-09-25T15:26:48.446Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6487lb # [ 181.839209] niks3-server[1153]: 2026/09/25 15:26:48 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign6488worker1 # [ 181.843703] nix-grpc-daemon[1813]: time=2026-09-25T15:26:48.470Z level=INFO msg="Uploading 1 narinfos"6489lb # [ 181.847744] niks3-server[1153]: 2026/09/25 15:26:48 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6490lb # [ 181.854147] niks3-server[1153]: 2026/09/25 15:26:48 INFO Signed narinfos id=70 count=16491lb # [ 181.855813] niks3-server[1153]: 2026/09/25 15:26:48 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6492lb # [ 181.865875] niks3-server[1153]: 2026/09/25 15:26:48 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6493worker1 # [ 181.869143] nix-grpc-daemon[1813]: time=2026-09-25T15:26:48.496Z level=INFO msg="Upload complete. (222ms)"6494lb # [ 181.870020] niks3-server[1153]: 2026/09/25 15:26:48 INFO Received complete push request method=POST path=/api/pushes/70/complete6495lb # [ 181.872778] niks3-server[1153]: 2026/09/25 15:26:48 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6496worker1 # [ 181.876685] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=5 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06497worker1 # [ 181.894270] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:48Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16498worker1 # [ 181.897561] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366499worker1 # [ 181.901826] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:48Z level=debug event=want client=2 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06500worker1 # [ 181.905854] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366501worker1 # [ 181.909494] nix-daemon[838]: reaped child process 1924, status = succeeded6502worker1 # [ 181.911946] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:48Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366503worker1 # [ 181.919751] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6504worker1 # [ 182.038496] nix-daemon[838]: accepted connection from pid 1951, user nix-grpc-daemon (trusted)6505worker1 # [ 182.058240] nix-daemon[838]: reaped child process 1954, status = succeeded6506lb # [ 182.061552] niks3-server[1153]: 2026/09/25 15:26:48 INFO Received push request method=POST path=/api/pushes6507worker1 # [ 182.066890] nix-grpc-daemon[1813]: time=2026-09-25T15:26:48.693Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6508worker1 # [ 182.068636] nix-grpc-daemon[1813]: time=2026-09-25T15:26:48.693Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6509worker1 # [ 182.080377] nix-grpc-daemon[1813]: time=2026-09-25T15:26:48.707Z level=INFO msg="Uploading 1 narinfos"6510lb # [ 182.080185] niks3-server[1153]: 2026/09/25 15:26:48 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign6511lb # [ 182.083009] niks3-server[1153]: 2026/09/25 15:26:48 INFO Signed narinfos id=71 count=16512lb # [ 182.093064] niks3-server[1153]: 2026/09/25 15:26:48 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6513lb # [ 182.095691] niks3-server[1153]: 2026/09/25 15:26:48 INFO Received complete push request method=POST path=/api/pushes/71/complete6514worker1 # [ 182.101989] nix-grpc-daemon[1813]: time=2026-09-25T15:26:48.729Z level=INFO msg="Upload complete. (173ms)"6515lb # [ 182.103018] niks3-server[1153]: 2026/09/25 15:26:48 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6516lb # [ 182.107319] niks3-server[1153]: 2026/09/25 15:26:48 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6517worker1 # [ 182.108123] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=5126518worker1 # [ 182.117376] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366519worker1 # [ 182.122208] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6520worker1 # [ 182.125932] nix-daemon[838]: reaped child process 1949, status = succeeded6521worker1 # [ 182.135332] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6522worker1 # [ 183.180373] nix-daemon[838]: reaped child process 1958, status = succeeded6523worker1 # [ 183.281354] nix-daemon[838]: accepted connection from pid 1974, user nix-grpc-daemon (trusted)6524worker1 # [ 183.301882] nix-daemon[838]: reaped child process 1977, status = succeeded6525lb # [ 183.303180] niks3-server[1153]: 2026/09/25 15:26:49 INFO Received push request method=POST path=/api/pushes6526worker1 # [ 183.311113] nix-grpc-daemon[1813]: time=2026-09-25T15:26:49.935Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6527worker1 # [ 183.312835] nix-grpc-daemon[1813]: time=2026-09-25T15:26:49.935Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6528lb # [ 183.327787] niks3-server[1153]: 2026/09/25 15:26:49 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6529lb # [ 183.334429] niks3-server[1153]: 2026/09/25 15:26:49 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6530worker1 # [ 183.342554] nix-grpc-daemon[1813]: time=2026-09-25T15:26:49.969Z level=INFO msg="Uploading 1 narinfos"6531lb # [ 183.342529] niks3-server[1153]: 2026/09/25 15:26:49 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign6532lb # [ 183.346886] niks3-server[1153]: 2026/09/25 15:26:49 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6533lb # [ 183.352839] niks3-server[1153]: 2026/09/25 15:26:49 INFO Signed narinfos id=72 count=16534lb # [ 183.360166] niks3-server[1153]: 2026/09/25 15:26:49 INFO Received complete push request method=POST path=/api/pushes/72/complete6535lb # [ 183.366698] niks3-server[1153]: 2026/09/25 15:26:49 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6536worker1 # [ 183.366026] nix-grpc-daemon[1813]: time=2026-09-25T15:26:49.993Z level=INFO msg="Upload complete. (223ms)"6537worker1 # [ 183.376247] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=6 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06538worker1 # [ 183.388230] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:49Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16539worker1 # [ 183.395732] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366540worker1 # [ 183.399628] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:50Z level=debug event=want client=2 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06541worker1 # [ 183.403425] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366542worker1 # [ 183.408216] nix-daemon[838]: reaped child process 1956, status = succeeded6543worker1 # [ 183.409728] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366544worker1 # [ 183.422901] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6545worker1 # [ 183.534470] nix-daemon[838]: accepted connection from pid 1983, user nix-grpc-daemon (trusted)6546lb # [ 183.550736] niks3-server[1153]: 2026/09/25 15:26:50 INFO Received push request method=POST path=/api/pushes6547worker1 # [ 183.554578] nix-daemon[838]: reaped child process 1986, status = succeeded6548worker1 # [ 183.556491] nix-grpc-daemon[1813]: time=2026-09-25T15:26:50.183Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6549worker1 # [ 183.557986] nix-grpc-daemon[1813]: time=2026-09-25T15:26:50.183Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6550lb # [ 183.567975] niks3-server[1153]: 2026/09/25 15:26:50 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign6551worker1 # [ 183.571943] nix-grpc-daemon[1813]: time=2026-09-25T15:26:50.198Z level=INFO msg="Uploading 1 narinfos"6552lb # [ 183.575581] niks3-server[1153]: 2026/09/25 15:26:50 INFO Signed narinfos id=73 count=16553lb # [ 183.580336] niks3-server[1153]: 2026/09/25 15:26:50 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6554lb # [ 183.583795] niks3-server[1153]: 2026/09/25 15:26:50 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6555lb # [ 183.592643] niks3-server[1153]: 2026/09/25 15:26:50 INFO Received complete push request method=POST path=/api/pushes/73/complete6556lb # [ 183.599784] niks3-server[1153]: 2026/09/25 15:26:50 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6557worker1 # [ 183.603083] nix-grpc-daemon[1813]: time=2026-09-25T15:26:50.230Z level=INFO msg="Upload complete. (177ms)"6558worker1 # [ 183.608207] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=5126559worker1 # [ 183.618685] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366560worker1 # [ 183.623461] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6561worker1 # [ 183.627631] nix-daemon[838]: reaped child process 1981, status = succeeded6562worker1 # [ 183.636099] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6563worker1 # [ 184.884212] nix-daemon[838]: reaped child process 1990, status = succeeded6564worker1 # [ 184.981091] nix-daemon[838]: accepted connection from pid 2006, user nix-grpc-daemon (trusted)6565worker1 # [ 185.000181] nix-daemon[838]: reaped child process 2009, status = succeeded6566lb # [ 185.004358] niks3-server[1153]: 2026/09/25 15:26:51 INFO Received push request method=POST path=/api/pushes6567worker1 # [ 185.012403] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.636Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6568worker1 # [ 185.013991] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.636Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6569lb # [ 185.027855] niks3-server[1153]: 2026/09/25 15:26:51 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign6570worker1 # [ 185.031546] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.658Z level=INFO msg="Uploading 1 narinfos"6571lb # [ 185.035914] niks3-server[1153]: 2026/09/25 15:26:51 INFO Signed narinfos id=74 count=16572lb # [ 185.040862] niks3-server[1153]: 2026/09/25 15:26:51 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6573lb # [ 185.046305] niks3-server[1153]: 2026/09/25 15:26:51 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6574lb # [ 185.054818] niks3-server[1153]: 2026/09/25 15:26:51 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6575worker1 # [ 185.058234] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.685Z level=INFO msg="Upload complete. (209ms)"6576lb # [ 185.057884] niks3-server[1153]: 2026/09/25 15:26:51 INFO Received complete push request method=POST path=/api/pushes/74/complete6577lb # [ 185.065154] niks3-server[1153]: 2026/09/25 15:26:51 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6578worker1 # [ 185.069553] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=7 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06579worker1 # [ 185.080331] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:51Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16580worker1 # [ 185.087784] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366581worker1 # [ 185.091320] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:51Z level=debug event=want client=2 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06582worker1 # [ 185.095108] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366583worker1 # [ 185.098869] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.725Z level=INFO msg="All 1 paths already cached"6584worker1 # [ 185.101742] nix-daemon[838]: reaped child process 1988, status = succeeded6585worker1 # [ 185.112243] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366586worker1 # [ 185.114730] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6587worker1 # [ 185.221504] nix-daemon[838]: accepted connection from pid 2015, user nix-grpc-daemon (trusted)6588lb # [ 185.237376] niks3-server[1153]: 2026/09/25 15:26:51 INFO Received push request method=POST path=/api/pushes6589worker1 # [ 185.240440] nix-daemon[838]: reaped child process 2018, status = succeeded6590worker1 # [ 185.242809] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.869Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6591worker1 # [ 185.244350] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.869Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6592lb # [ 185.255604] niks3-server[1153]: 2026/09/25 15:26:51 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6593worker1 # [ 185.260526] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.887Z level=INFO msg="Uploading 1 narinfos"6594lb # [ 185.263381] niks3-server[1153]: 2026/09/25 15:26:51 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign6595lb # [ 185.268881] niks3-server[1153]: 2026/09/25 15:26:51 INFO Signed narinfos id=75 count=16596lb # [ 185.270603] niks3-server[1153]: 2026/09/25 15:26:51 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6597lb # [ 185.280877] niks3-server[1153]: 2026/09/25 15:26:51 INFO Received complete push request method=POST path=/api/pushes/75/complete6598worker1 # [ 185.284944] nix-grpc-daemon[1813]: time=2026-09-25T15:26:51.912Z level=INFO msg="Upload complete. (166ms)"6599lb # [ 185.285548] niks3-server[1153]: 2026/09/25 15:26:51 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6600worker1 # [ 185.291048] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=6246601worker1 # [ 185.298484] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366602worker1 # [ 185.303182] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6603worker1 # [ 185.306941] nix-daemon[838]: reaped child process 2013, status = succeeded6604worker1 # [ 185.318034] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6605worker1 # [ 186.934955] nix-daemon[838]: reaped child process 2022, status = succeeded6606worker1 # [ 187.064895] nix-daemon[838]: accepted connection from pid 2038, user nix-grpc-daemon (trusted)6607worker1 # [ 187.085743] nix-daemon[838]: reaped child process 2041, status = succeeded6608lb # [ 187.087309] niks3-server[1153]: 2026/09/25 15:26:53 INFO Received push request method=POST path=/api/pushes6609worker1 # [ 187.095190] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.719Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6610worker1 # [ 187.096857] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.719Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6611lb # [ 187.115275] niks3-server[1153]: 2026/09/25 15:26:53 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6612worker1 # [ 187.121669] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.748Z level=INFO msg="Uploading 1 narinfos"6613lb # [ 187.122862] niks3-server[1153]: 2026/09/25 15:26:53 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign6614lb # [ 187.127467] niks3-server[1153]: 2026/09/25 15:26:53 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6615lb # [ 187.135398] niks3-server[1153]: 2026/09/25 15:26:53 INFO Signed narinfos id=76 count=16616lb # [ 187.138689] niks3-server[1153]: 2026/09/25 15:26:53 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6617worker1 # [ 187.140872] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.768Z level=INFO msg="Upload complete. (215ms)"6618lb # [ 187.141380] niks3-server[1153]: 2026/09/25 15:26:53 INFO Received complete push request method=POST path=/api/pushes/76/complete6619lb # [ 187.143476] niks3-server[1153]: 2026/09/25 15:26:53 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6620worker1 # [ 187.150081] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=8 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06621worker1 # [ 187.162201] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:53Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16622worker1 # [ 187.166257] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366623worker1 # [ 187.169646] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:53Z level=debug event=want client=2 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06624worker1 # [ 187.173134] nix-daemon[838]: reaped child process 2020, status = succeeded6625worker1 # [ 187.175430] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366626worker1 # [ 187.182331] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.809Z level=INFO msg="All 1 paths already cached"6627worker1 # [ 187.187883] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366628worker1 # [ 187.195798] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6629worker1 # [ 187.304216] nix-daemon[838]: accepted connection from pid 2047, user nix-grpc-daemon (trusted)6630worker1 # [ 187.324679] nix-daemon[838]: reaped child process 2050, status = succeeded6631lb # [ 187.327066] niks3-server[1153]: 2026/09/25 15:26:53 INFO Received push request method=POST path=/api/pushes6632worker1 # [ 187.332211] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.958Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6633worker1 # [ 187.333765] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.958Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6634lb # [ 187.344408] niks3-server[1153]: 2026/09/25 15:26:53 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6635worker1 # [ 187.349404] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.976Z level=INFO msg="Uploading 1 narinfos"6636lb # [ 187.352227] niks3-server[1153]: 2026/09/25 15:26:53 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign6637lb # [ 187.354580] niks3-server[1153]: 2026/09/25 15:26:53 INFO Signed narinfos id=77 count=16638worker1 # [ 187.363739] nix-grpc-daemon[1813]: time=2026-09-25T15:26:53.990Z level=INFO msg="Upload complete. (160ms)"6639lb # [ 187.364084] niks3-server[1153]: 2026/09/25 15:26:53 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6640worker1 # [ 187.368186] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=6246641lb # [ 187.371902] niks3-server[1153]: 2026/09/25 15:26:53 INFO Received complete push request method=POST path=/api/pushes/77/complete6642lb # [ 187.374611] niks3-server[1153]: 2026/09/25 15:26:53 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6643worker1 # [ 187.378235] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366644worker1 # [ 187.381992] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6645worker1 # [ 187.385435] nix-daemon[838]: reaped child process 2045, status = succeeded6646worker1 # [ 187.392880] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6647lb # [ 188.114078] envoy[902]: conn peer=[2001:db8:1::1]:38828 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9641 tx=5163 ms=51186648worker1 # [ 188.431817] nix-daemon[838]: reaped child process 2054, status = succeeded6649worker1 # [ 188.535520] nix-daemon[838]: accepted connection from pid 2070, user nix-grpc-daemon (trusted)6650worker1 # [ 188.553949] nix-daemon[838]: reaped child process 2073, status = succeeded6651lb # [ 188.559196] niks3-server[1153]: 2026/09/25 15:26:55 INFO Received push request method=POST path=/api/pushes6652worker1 # [ 188.567186] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.191Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6653worker1 # [ 188.568876] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.191Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6654lb # [ 188.584099] niks3-server[1153]: 2026/09/25 15:26:55 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign6655worker1 # [ 188.587912] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.214Z level=INFO msg="Uploading 1 narinfos"6656lb # [ 188.592870] niks3-server[1153]: 2026/09/25 15:26:55 INFO Signed narinfos id=78 count=16657lb # [ 188.597023] niks3-server[1153]: 2026/09/25 15:26:55 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6658lb # [ 188.607888] niks3-server[1153]: 2026/09/25 15:26:55 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6659lb # [ 188.610718] niks3-server[1153]: 2026/09/25 15:26:55 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6660lb # [ 188.613715] niks3-server[1153]: 2026/09/25 15:26:55 INFO Received complete push request method=POST path=/api/pushes/78/complete6661worker1 # [ 188.618086] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.245Z level=INFO msg="Upload complete. (216ms)"6662lb # [ 188.619563] niks3-server[1153]: 2026/09/25 15:26:55 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6663worker1 # [ 188.629097] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=9 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06664worker1 # [ 188.642347] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:55Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16665worker1 # [ 188.646143] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366666worker1 # [ 188.649898] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:55Z level=debug event=want client=2 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06667worker1 # [ 188.654198] nix-daemon[838]: reaped child process 2052, status = succeeded6668worker1 # [ 188.656810] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366669worker1 # [ 188.660723] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.286Z level=INFO msg="All 1 paths already cached"6670worker1 # [ 188.663604] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366671worker1 # [ 188.675985] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6672worker1 # [ 188.782284] nix-daemon[838]: accepted connection from pid 2079, user nix-grpc-daemon (trusted)6673worker1 # [ 188.799152] nix-daemon[838]: reaped child process 2082, status = succeeded6674lb # [ 188.806035] niks3-server[1153]: 2026/09/25 15:26:55 INFO Received push request method=POST path=/api/pushes6675worker1 # [ 188.810929] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.437Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6676worker1 # [ 188.812535] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.437Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6677lb # [ 188.821016] niks3-server[1153]: 2026/09/25 15:26:55 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign6678worker1 # [ 188.824105] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.451Z level=INFO msg="Uploading 1 narinfos"6679lb # [ 188.826798] niks3-server[1153]: 2026/09/25 15:26:55 INFO Signed narinfos id=79 count=16680lb # [ 188.832623] niks3-server[1153]: 2026/09/25 15:26:55 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6681lb # [ 188.843277] niks3-server[1153]: 2026/09/25 15:26:55 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6682worker1 # [ 188.848870] nix-grpc-daemon[1813]: time=2026-09-25T15:26:55.476Z level=INFO msg="Upload complete. (169ms)"6683lb # [ 188.849980] niks3-server[1153]: 2026/09/25 15:26:55 INFO Received complete push request method=POST path=/api/pushes/79/complete6684lb # [ 188.853318] niks3-server[1153]: 2026/09/25 15:26:55 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6685worker1 # [ 188.856278] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=6246686worker1 # [ 188.860088] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366687worker1 # [ 188.868178] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6688worker1 # [ 188.872098] nix-daemon[838]: reaped child process 2077, status = succeeded6689worker1 # [ 188.882145] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6690worker1 # [ 190.316380] nix-daemon[838]: reaped child process 2086, status = succeeded6691worker1 # [ 190.409036] nix-daemon[838]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)6692worker1 # [ 190.425733] nix-daemon[838]: reaped child process 2105, status = succeeded6693lb # [ 190.431700] niks3-server[1153]: 2026/09/25 15:26:57 INFO Received push request method=POST path=/api/pushes6694worker1 # [ 190.438606] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.064Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6695worker1 # [ 190.440347] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.064Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6696lb # [ 190.455542] niks3-server[1153]: 2026/09/25 15:26:57 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign6697worker1 # [ 190.460253] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.087Z level=INFO msg="Uploading 1 narinfos"6698lb # [ 190.463333] niks3-server[1153]: 2026/09/25 15:26:57 INFO Signed narinfos id=80 count=16699lb # [ 190.468708] niks3-server[1153]: 2026/09/25 15:26:57 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6700lb # [ 190.471713] niks3-server[1153]: 2026/09/25 15:26:57 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6701lb # [ 190.482525] niks3-server[1153]: 2026/09/25 15:26:57 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6702lb # [ 190.485396] niks3-server[1153]: 2026/09/25 15:26:57 INFO Received complete push request method=POST path=/api/pushes/80/complete6703worker1 # [ 190.490963] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.118Z level=INFO msg="Upload complete. (201ms)"6704lb # [ 190.492369] niks3-server[1153]: 2026/09/25 15:26:57 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6705worker1 # [ 190.501372] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=10 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06706worker1 # [ 190.514553] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:57Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16707worker1 # [ 190.519060] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366708worker1 # [ 190.523299] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:57Z level=debug event=want client=2 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06709worker1 # [ 190.527087] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366710worker1 # [ 190.531041] nix-daemon[838]: reaped child process 2084, status = succeeded6711worker1 # [ 190.532668] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.157Z level=INFO msg="All 2 paths already cached"6712worker1 # [ 190.543891] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366713worker1 # [ 190.548162] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6714worker1 # [ 190.652863] nix-daemon[838]: accepted connection from pid 2111, user nix-grpc-daemon (trusted)6715worker1 # [ 190.668196] nix-daemon[838]: reaped child process 2114, status = succeeded6716lb # [ 190.673479] niks3-server[1153]: 2026/09/25 15:26:57 INFO Received push request method=POST path=/api/pushes6717worker1 # [ 190.678477] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.305Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6718worker1 # [ 190.679980] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.305Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6719lb # [ 190.688977] niks3-server[1153]: 2026/09/25 15:26:57 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6720worker1 # [ 190.694973] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.321Z level=INFO msg="Uploading 1 narinfos"6721lb # [ 190.696906] niks3-server[1153]: 2026/09/25 15:26:57 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign6722lb # [ 190.699307] niks3-server[1153]: 2026/09/25 15:26:57 INFO Signed narinfos id=81 count=16723lb # [ 190.706742] niks3-server[1153]: 2026/09/25 15:26:57 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6724lb # [ 190.715451] niks3-server[1153]: 2026/09/25 15:26:57 INFO Received complete push request method=POST path=/api/pushes/81/complete6725worker1 # [ 190.721119] nix-grpc-daemon[1813]: time=2026-09-25T15:26:57.348Z level=INFO msg="Upload complete. (170ms)"6726lb # [ 190.722466] niks3-server[1153]: 2026/09/25 15:26:57 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6727worker1 # [ 190.728119] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=7526728worker1 # [ 190.731091] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366729worker1 # [ 190.740161] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6730worker1 # [ 190.743877] nix-daemon[838]: reaped child process 2109, status = succeeded6731worker1 # [ 190.753671] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6732worker1 # [ 192.208131] nix-daemon[838]: reaped child process 2118, status = succeeded6733worker1 # [ 192.304132] nix-daemon[838]: accepted connection from pid 2134, user nix-grpc-daemon (trusted)6734worker1 # [ 192.323501] nix-daemon[838]: reaped child process 2137, status = succeeded6735lb # [ 192.328706] niks3-server[1153]: 2026/09/25 15:26:58 INFO Received push request method=POST path=/api/pushes6736worker1 # [ 192.336397] nix-grpc-daemon[1813]: time=2026-09-25T15:26:58.961Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6737worker1 # [ 192.338012] nix-grpc-daemon[1813]: time=2026-09-25T15:26:58.961Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6738lb # [ 192.351003] niks3-server[1153]: 2026/09/25 15:26:58 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign6739worker1 # [ 192.356647] nix-grpc-daemon[1813]: time=2026-09-25T15:26:58.983Z level=INFO msg="Uploading 1 narinfos"6740lb # [ 192.358039] niks3-server[1153]: 2026/09/25 15:26:58 INFO Signed narinfos id=82 count=16741lb # [ 192.363924] niks3-server[1153]: 2026/09/25 15:26:58 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6742lb # [ 192.370515] niks3-server[1153]: 2026/09/25 15:26:58 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6743lb # [ 192.380962] niks3-server[1153]: 2026/09/25 15:26:58 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6744lb # [ 192.383971] niks3-server[1153]: 2026/09/25 15:26:59 INFO Received complete push request method=POST path=/api/pushes/82/complete6745worker1 # [ 192.390262] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.017Z level=INFO msg="Upload complete. (229ms)"6746lb # [ 192.391261] niks3-server[1153]: 2026/09/25 15:26:59 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6747worker1 # [ 192.398845] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=11 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06748worker1 # [ 192.409442] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16749worker1 # [ 192.420180] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366750worker1 # [ 192.426460] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366751worker1 # [ 192.430515] nix-daemon[838]: reaped child process 2116, status = succeeded6752worker1 # [ 192.433436] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.059Z level=INFO msg="All 2 paths already cached"6753worker1 # [ 192.436423] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366754worker1 # [ 192.445117] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6755worker1 # [ 192.557478] nix-daemon[838]: accepted connection from pid 2141, user nix-grpc-daemon (trusted)6756lb # [ 192.574085] niks3-server[1153]: 2026/09/25 15:26:59 INFO Received push request method=POST path=/api/pushes6757worker1 # [ 192.576241] nix-daemon[838]: reaped child process 2144, status = succeeded6758worker1 # [ 192.578656] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.205Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6759worker1 # [ 192.580201] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.205Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6760worker1 # [ 192.596423] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.223Z level=INFO msg="Uploading 1 narinfos"6761lb # [ 192.590071] niks3-server[1153]: 2026/09/25 15:26:59 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6762lb # [ 192.598216] niks3-server[1153]: 2026/09/25 15:26:59 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign6763lb # [ 192.603646] niks3-server[1153]: 2026/09/25 15:26:59 INFO Signed narinfos id=83 count=16764lb # [ 192.611796] niks3-server[1153]: 2026/09/25 15:26:59 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6765worker1 # [ 192.616825] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.243Z level=INFO msg="Upload complete. (164ms)"6766lb # [ 192.617670] niks3-server[1153]: 2026/09/25 15:26:59 INFO Received complete push request method=POST path=/api/pushes/83/complete6767lb # [ 192.623409] niks3-server[1153]: 2026/09/25 15:26:59 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6768worker1 # [ 192.621733] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=7526769worker1 # [ 192.630760] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366770worker1 # [ 192.636765] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6771worker1 # [ 192.638693] nix-daemon[838]: reaped child process 2139, status = succeeded6772worker1 # [ 192.647447] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6773worker1 # [ 193.119020] nix-daemon[838]: reaped child process 2148, status = succeeded6774worker1 # [ 193.187646] nix-daemon[838]: accepted connection from pid 2164, user nix-grpc-daemon (trusted)6775worker1 # [ 193.202935] nix-daemon[838]: reaped child process 2167, status = succeeded6776lb # [ 193.208549] niks3-server[1153]: 2026/09/25 15:26:59 INFO Received push request method=POST path=/api/pushes6777worker1 # [ 193.214579] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.840Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6778worker1 # [ 193.216153] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.840Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6779worker1 # [ 193.240159] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.867Z level=INFO msg="Uploading 1 narinfos"6780lb # [ 193.233139] niks3-server[1153]: 2026/09/25 15:26:59 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6781lb # [ 193.242526] niks3-server[1153]: 2026/09/25 15:26:59 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6782lb # [ 193.248421] niks3-server[1153]: 2026/09/25 15:26:59 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign6783lb # [ 193.256209] niks3-server[1153]: 2026/09/25 15:26:59 INFO Signed narinfos id=84 count=16784lb # [ 193.258028] niks3-server[1153]: 2026/09/25 15:26:59 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6785worker1 # [ 193.263180] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.890Z level=INFO msg="Upload complete. (209ms)"6786lb # [ 193.264993] niks3-server[1153]: 2026/09/25 15:26:59 INFO Received complete push request method=POST path=/api/pushes/84/complete6787lb # [ 193.268862] niks3-server[1153]: 2026/09/25 15:26:59 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6788worker1 # [ 193.272542] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=12 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06789worker1 # [ 193.284245] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16790worker1 # [ 193.287389] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366791worker1 # [ 193.291328] nix-daemon[838]: reaped child process 2146, status = succeeded6792worker1 # [ 193.296583] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:580366793worker1 # [ 193.304764] nix-grpc-daemon[1813]: time=2026-09-25T15:26:59.931Z level=INFO msg="All 2 paths already cached"6794worker1 # [ 193.310705] nix-grpc-daemon[1742]: ts=2026-09-25T15:26:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:580366795worker1 # [ 193.318580] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6796worker1 # [ 193.435230] nix-daemon[838]: accepted connection from pid 2171, user nix-grpc-daemon (trusted)6797lb # [ 193.453749] niks3-server[1153]: 2026/09/25 15:27:00 INFO Received push request method=POST path=/api/pushes6798worker1 # [ 193.456189] nix-daemon[838]: reaped child process 2174, status = succeeded6799worker1 # [ 193.459257] nix-grpc-daemon[1813]: time=2026-09-25T15:27:00.085Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6800worker1 # [ 193.460883] nix-grpc-daemon[1813]: time=2026-09-25T15:27:00.085Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6801worker1 # [ 193.473817] nix-grpc-daemon[1813]: time=2026-09-25T15:27:00.100Z level=INFO msg="Uploading 1 narinfos"6802lb # [ 193.469722] niks3-server[1153]: 2026/09/25 15:27:00 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign6803lb # [ 193.477505] niks3-server[1153]: 2026/09/25 15:27:00 INFO Signed narinfos id=85 count=16804lb # [ 193.481417] niks3-server[1153]: 2026/09/25 15:27:00 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6805lb # [ 193.491391] niks3-server[1153]: 2026/09/25 15:27:00 INFO Received complete push request method=POST path=/api/pushes/85/complete6806lb # [ 193.495796] niks3-server[1153]: 2026/09/25 15:27:00 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6807worker1 # [ 193.499269] nix-grpc-daemon[1813]: time=2026-09-25T15:27:00.126Z level=INFO msg="Upload complete. (171ms)"6808lb # [ 193.500716] niks3-server[1153]: 2026/09/25 15:27:00 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6809worker1 # [ 193.511220] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 paths=1 nar_bytes_in=7526810worker1 # [ 193.515282] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:580366811worker1 # [ 193.518066] nix-daemon[838]: reaped child process 2169, status = succeeded6812worker1 # [ 193.523531] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6813worker1 # [ 193.531873] nix-daemon[838]: accepted connection from pid 1742, user nix-grpc-daemon (trusted)6814worker1 # [ 194.158937] nix-daemon[838]: reaped child process 2178, status = succeeded6815worker1 # [ 194.277129] nix-daemon[838]: accepted connection from pid 2194, user nix-grpc-daemon (trusted)6816worker1 # [ 194.296112] nix-daemon[838]: reaped child process 2197, status = succeeded6817lb # [ 194.302036] niks3-server[1153]: 2026/09/25 15:27:00 INFO Received push request method=POST path=/api/pushes6818worker1 # [ 194.308257] nix-grpc-daemon[1813]: time=2026-09-25T15:27:00.934Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6819worker1 # [ 194.314122] nix-grpc-daemon[1813]: time=2026-09-25T15:27:00.934Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6820lb # [ 194.331919] niks3-server[1153]: 2026/09/25 15:27:00 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6821lb # [ 194.343831] niks3-server[1153]: 2026/09/25 15:27:00 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign6822worker1 # [ 194.348762] nix-grpc-daemon[1813]: time=2026-09-25T15:27:00.975Z level=INFO msg="Uploading 1 narinfos"6823lb # [ 194.350769] niks3-server[1153]: 2026/09/25 15:27:00 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6824lb # [ 194.357602] niks3-server[1153]: 2026/09/25 15:27:00 INFO Signed narinfos id=86 count=16825lb # [ 194.359337] niks3-server[1153]: 2026/09/25 15:27:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6826lb # [ 194.368766] niks3-server[1153]: 2026/09/25 15:27:00 INFO Received complete push request method=POST path=/api/pushes/86/complete6827worker1 # [ 194.372298] nix-grpc-daemon[1813]: time=2026-09-25T15:27:00.999Z level=INFO msg="Upload complete. (231ms)"6828lb # [ 194.373343] niks3-server[1153]: 2026/09/25 15:27:00 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6829worker1 # [ 194.380246] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58036 duration_s=0 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=13 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=06830worker1 # [ 194.392297] nix-daemon[838]: reaped child process 2176, status = succeeded6831client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.66 seconds)6832worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6833worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6834client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6835client # [ 194.489980] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6836client # Running as unit: gpu.service; invocation ID: bdadfbd2dd764cdd9f66b89ce80c6b176837client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.05 seconds)6838lb: must succeed: curl -sf localhost:9901/clusters6839lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6840lb: must succeed: curl -sf localhost:9901/clusters6841lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6842worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6843worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6844worker1 # [ 194.662430] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:580366845client # [ 194.665896] nix[1637]: don't know how to build these paths:6846client # [ 194.666864] nix[1637]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6847worker2 # [ 194.667261] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166848worker1 # [ 194.684917] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:01Z level=debug event=want client=3 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06849client # [ 194.683757] nix[1637]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6850lb: must succeed: curl -sf localhost:9901/clusters6851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6852lb: must succeed: curl -sf localhost:9901/clusters6853lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6854worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6855worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6856client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6857lb # [ 198.113656] envoy[902]: conn peer=[2001:db8:1::1]:38836 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27080 tx=12277 ms=144566858client # [ 224.684681] nix[1637]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6859client # [ 224.692065] nix[1637]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6860client # [ 224.713562] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6861client # [ 224.718050] systemd[1]: gpu.service: Failed with result 'exit-code'.6862client # [ 224.720136] systemd[1]: gpu.service: Consumed 166ms CPU time over 30.226s wall clock time, 33.4M memory peak, 5.7K incoming IP traffic, 5.2K outgoing IP traffic.6863client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.25 seconds)6864client: must succeed: journalctl -u gpu -o cat6865client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)6866lb: must succeed: curl -sf localhost:9901/clusters6867lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6868lb: must succeed: curl -sf localhost:9901/clusters6869lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6870worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6871worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6872(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.34 seconds)6873subtest: low disk drains a worker and builds go to the other6874worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6875worker1 # [ 226.093291] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:32Z level=info event=unhealthy reason=min_free6876worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.17 seconds)6877worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6878worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 0.03 seconds)6879client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26880worker2 # [ 226.471326] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:437166881client # don't know how to build these paths:6882client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6883worker2 # [ 226.477741] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166884worker1 # [ 226.496092] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:33Z level=debug event=want client=4 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06885worker1 # [ 226.498956] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:33Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16886worker2 # [ 226.502299] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166887worker2 # [ 226.511660] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166888client # copying 1 paths to 'lb:50051'...6889client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6890worker2 # [ 226.520855] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437166891worker2 # [ 226.529561] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6892worker2 # [ 226.659688] nix-daemon[857]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)6893worker2 # [ 226.674907] nix-daemon[857]: reaped child process 1982, status = succeeded6894lb # [ 226.683931] niks3-server[1153]: 2026/09/25 15:27:33 INFO Received push request method=POST path=/api/pushes6895worker2 # [ 226.689109] nix-grpc-daemon[1736]: time=2026-09-25T15:27:33.312Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6896worker2 # [ 226.690758] nix-grpc-daemon[1736]: time=2026-09-25T15:27:33.312Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6897lb # [ 226.704619] niks3-server[1153]: 2026/09/25 15:27:33 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign6898worker2 # [ 226.707034] nix-grpc-daemon[1736]: time=2026-09-25T15:27:33.330Z level=INFO msg="Uploading 1 narinfos"6899lb # [ 226.712457] niks3-server[1153]: 2026/09/25 15:27:33 INFO Signed narinfos id=87 count=16900lb # [ 226.722127] niks3-server[1153]: 2026/09/25 15:27:33 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6901lb # [ 226.734203] niks3-server[1153]: 2026/09/25 15:27:33 INFO Received complete push request method=POST path=/api/pushes/87/complete6902lb # [ 226.740257] niks3-server[1153]: 2026/09/25 15:27:33 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6903worker2 # [ 226.741020] nix-grpc-daemon[1736]: time=2026-09-25T15:27:33.364Z level=INFO msg="Upload complete. (201ms)"6904lb # [ 226.747357] niks3-server[1153]: 2026/09/25 15:27:33 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6905worker2 # [ 226.748110] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=5126906worker2 # [ 226.757348] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166907worker2 # [ 226.762490] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6908worker2 # [ 226.766367] nix-daemon[857]: reaped child process 1977, status = succeeded6909client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6910client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6911worker2 # [ 226.776448] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6912client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6913client # farm-a> LOG-farm-a-drain6914worker2 # [ 227.401807] nix-daemon[857]: reaped child process 1986, status = succeeded6915worker2 # [ 227.522508] nix-daemon[857]: accepted connection from pid 2002, user nix-grpc-daemon (trusted)6916lb # [ 227.542842] niks3-server[1153]: 2026/09/25 15:27:34 INFO Received push request method=POST path=/api/pushes6917worker2 # [ 227.542647] nix-daemon[857]: reaped child process 2005, status = succeeded6918worker2 # [ 227.546703] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6919worker2 # [ 227.548489] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.169Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6920worker2 # [ 227.567354] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.191Z level=INFO msg="Uploading 1 narinfos"6921lb # [ 227.566059] niks3-server[1153]: 2026/09/25 15:27:34 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign6922lb # [ 227.573808] niks3-server[1153]: 2026/09/25 15:27:34 INFO Signed narinfos id=88 count=16923lb # [ 227.578964] niks3-server[1153]: 2026/09/25 15:27:34 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6924lb # [ 227.585268] niks3-server[1153]: 2026/09/25 15:27:34 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6925lb # [ 227.593708] niks3-server[1153]: 2026/09/25 15:27:34 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6926lb # [ 227.601084] niks3-server[1153]: 2026/09/25 15:27:34 INFO Received complete push request method=POST path=/api/pushes/88/complete6927worker2 # [ 227.602394] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.226Z level=INFO msg="Upload complete. (216ms)"6928lb # [ 227.606307] niks3-server[1153]: 2026/09/25 15:27:34 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6929worker1 # [ 227.616673] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:34Z level=debug event=want client=4 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06930worker1 # [ 227.619702] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:34Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16931worker2 # [ 227.615869] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=14 outputs=1 inputs=0 inputs_fetched=0 input_bytes=0 input_s=06932worker2 # [ 227.622192] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166933worker2 # [ 227.626647] nix-daemon[857]: reaped child process 1984, status = succeeded6934worker2 # [ 227.631804] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:34Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166935worker2 # [ 227.639911] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.263Z level=INFO msg="All 1 paths already cached"6936client # copying 1 paths to 'lb:50051'...6937client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6938worker2 # [ 227.645952] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:34Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437166939worker2 # [ 227.653491] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6940worker2 # [ 227.772956] nix-daemon[857]: accepted connection from pid 2009, user nix-grpc-daemon (trusted)6941worker2 # [ 227.789166] nix-daemon[857]: reaped child process 2012, status = succeeded6942lb # [ 227.797651] niks3-server[1153]: 2026/09/25 15:27:34 INFO Received push request method=POST path=/api/pushes6943worker2 # [ 227.801250] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.424Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6944worker2 # [ 227.802819] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.424Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6945lb # [ 227.816345] niks3-server[1153]: 2026/09/25 15:27:34 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign6946worker2 # [ 227.817126] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.440Z level=INFO msg="Uploading 1 narinfos"6947lb # [ 227.823399] niks3-server[1153]: 2026/09/25 15:27:34 INFO Signed narinfos id=89 count=16948lb # [ 227.827245] niks3-server[1153]: 2026/09/25 15:27:34 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6949lb # [ 227.836188] niks3-server[1153]: 2026/09/25 15:27:34 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6950lb # [ 227.839106] niks3-server[1153]: 2026/09/25 15:27:34 INFO Received complete push request method=POST path=/api/pushes/89/complete6951worker2 # [ 227.841370] nix-grpc-daemon[1736]: time=2026-09-25T15:27:34.465Z level=INFO msg="Upload complete. (178ms)"6952lb # [ 227.845649] niks3-server[1153]: 2026/09/25 15:27:34 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6953worker2 # [ 227.848682] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=6326954worker2 # [ 227.851890] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166955worker2 # [ 227.863169] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6956worker2 # [ 227.865023] nix-daemon[857]: reaped child process 2007, status = succeeded6957client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6958client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6959worker2 # [ 227.875434] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6960lb # [ 228.114689] envoy[902]: conn peer=[2001:db8:1::1]:59458 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=636 ms=300426961client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6962client # farm-b> LOG-farm-b-drain6963worker2 # [ 229.143661] nix-daemon[857]: reaped child process 2016, status = succeeded6964worker2 # [ 229.199199] nix-daemon[857]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)6965worker2 # [ 229.219168] nix-daemon[857]: reaped child process 2035, status = succeeded6966lb # [ 229.220433] niks3-server[1153]: 2026/09/25 15:27:35 INFO Received push request method=POST path=/api/pushes6967worker2 # [ 229.227717] nix-grpc-daemon[1736]: time=2026-09-25T15:27:35.848Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6968worker2 # [ 229.229574] nix-grpc-daemon[1736]: time=2026-09-25T15:27:35.848Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6969lb # [ 229.246519] niks3-server[1153]: 2026/09/25 15:27:35 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign6970worker2 # [ 229.248195] nix-grpc-daemon[1736]: time=2026-09-25T15:27:35.872Z level=INFO msg="Uploading 1 narinfos"6971lb # [ 229.253343] niks3-server[1153]: 2026/09/25 15:27:35 INFO Signed narinfos id=90 count=16972lb # [ 229.259991] niks3-server[1153]: 2026/09/25 15:27:35 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6973lb # [ 229.268511] niks3-server[1153]: 2026/09/25 15:27:35 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6974lb # [ 229.271522] niks3-server[1153]: 2026/09/25 15:27:35 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6975worker2 # [ 229.276067] nix-grpc-daemon[1736]: time=2026-09-25T15:27:35.899Z level=INFO msg="Upload complete. (192ms)"6976lb # [ 229.279415] niks3-server[1153]: 2026/09/25 15:27:35 INFO Received complete push request method=POST path=/api/pushes/90/complete6977lb # [ 229.283935] niks3-server[1153]: 2026/09/25 15:27:35 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6978worker1 # [ 229.292365] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:35Z level=debug event=want client=4 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06979worker2 # [ 229.284551] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=15 outputs=1 inputs=1 inputs_fetched=0 input_bytes=0 input_s=06980worker1 # [ 229.295386] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:35Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16981worker2 # [ 229.295551] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437166982worker2 # [ 229.299762] nix-daemon[857]: reaped child process 2014, status = succeeded6983worker2 # [ 229.304773] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:437166984worker2 # [ 229.312758] nix-grpc-daemon[1736]: time=2026-09-25T15:27:35.936Z level=INFO msg="All 2 paths already cached"6985client # copying 1 paths to 'lb:50051'...6986client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...6987worker2 # [ 229.318873] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:437166988worker2 # [ 229.326962] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)6989worker2 # [ 229.439904] nix-daemon[857]: accepted connection from pid 2039, user nix-grpc-daemon (trusted)6990lb # [ 229.462091] niks3-server[1153]: 2026/09/25 15:27:36 INFO Received push request method=POST path=/api/pushes6991worker2 # [ 229.461852] nix-daemon[857]: reaped child process 2042, status = succeeded6992worker2 # [ 229.464951] nix-grpc-daemon[1736]: time=2026-09-25T15:27:36.088Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6993worker2 # [ 229.466736] nix-grpc-daemon[1736]: time=2026-09-25T15:27:36.088Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"6994lb # [ 229.480970] niks3-server[1153]: 2026/09/25 15:27:36 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign6995worker2 # [ 229.482634] nix-grpc-daemon[1736]: time=2026-09-25T15:27:36.106Z level=INFO msg="Uploading 1 narinfos"6996lb # [ 229.487856] niks3-server[1153]: 2026/09/25 15:27:36 INFO Signed narinfos id=91 count=16997lb # [ 229.493300] niks3-server[1153]: 2026/09/25 15:27:36 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst6998lb # [ 229.500147] niks3-server[1153]: 2026/09/25 15:27:36 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls6999lb # [ 229.505609] niks3-server[1153]: 2026/09/25 15:27:36 INFO Received complete push request method=POST path=/api/pushes/91/complete7000worker2 # [ 229.509459] nix-grpc-daemon[1736]: time=2026-09-25T15:27:36.133Z level=INFO msg="Upload complete. (174ms)"7001lb # [ 229.512956] niks3-server[1153]: 2026/09/25 15:27:36 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7002worker2 # [ 229.516204] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=0 paths=1 nar_bytes_in=7607003worker2 # [ 229.524286] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:437167004worker2 # [ 229.531051] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7005worker2 # [ 229.532943] nix-daemon[857]: reaped child process 2037, status = succeeded7006client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7007client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7008worker2 # [ 229.543334] nix-daemon[857]: accepted connection from pid 1664, user nix-grpc-daemon (trusted)7009client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7010client # farm-top> LOG-farm-top-drain7011worker2 # [ 230.625460] nix-daemon[857]: reaped child process 2046, status = succeeded7012worker2 # [ 230.693071] nix-daemon[857]: accepted connection from pid 2062, user nix-grpc-daemon (trusted)7013worker2 # [ 230.709963] nix-daemon[857]: reaped child process 2065, status = succeeded7014lb # [ 230.718544] niks3-server[1153]: 2026/09/25 15:27:37 INFO Received push request method=POST path=/api/pushes7015worker2 # [ 230.722834] nix-grpc-daemon[1736]: time=2026-09-25T15:27:37.345Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7016worker2 # [ 230.724473] nix-grpc-daemon[1736]: time=2026-09-25T15:27:37.345Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7017lb # [ 230.743839] niks3-server[1153]: 2026/09/25 15:27:37 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7018worker2 # [ 230.748138] nix-grpc-daemon[1736]: time=2026-09-25T15:27:37.371Z level=INFO msg="Uploading 1 narinfos"7019lb # [ 230.752310] niks3-server[1153]: 2026/09/25 15:27:37 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7020lb # [ 230.755072] niks3-server[1153]: 2026/09/25 15:27:37 INFO Received sign narinfos request method=POST path=/api/pushes/92/sign7021lb # [ 230.766605] niks3-server[1153]: 2026/09/25 15:27:37 INFO Signed narinfos id=92 count=17022worker2 # [ 230.764249] nix-grpc-daemon[1736]: time=2026-09-25T15:27:37.388Z level=INFO msg="Upload complete. (213ms)"7023lb # [ 230.771966] niks3-server[1153]: 2026/09/25 15:27:37 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7024lb # [ 230.776003] niks3-server[1153]: 2026/09/25 15:27:37 INFO Received complete push request method=POST path=/api/pushes/92/complete7025worker2 # [ 230.772622] nix-grpc-daemon[1664]: ts=2026-09-25T15:27:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43716 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=16 outputs=1 inputs=2 inputs_fetched=0 input_bytes=0 input_s=07026lb # [ 230.779086] niks3-server[1153]: 2026/09/25 15:27:37 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7027worker2 # [ 230.783272] nix-daemon[857]: reaped child process 2044, status = succeeded7028client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 4.50 seconds)7029client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7030client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.14 seconds)7031worker1: must succeed: rm /nix/.rw-store/fill7032worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7033worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7034worker1 # [ 232.100498] nix-grpc-daemon[1742]: ts=2026-09-25T15:27:38Z level=info event=healthy reason=min_free7035worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7036(finished: subtest: low disk drains a worker and builds go to the other, in 7.08 seconds)7037(finished: run the VM test script, in 233.04 seconds)7038test script finished in 233.18s7039cleanup7040kill QemuMachine (pid 50)7041client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7042client # [2026-09-25T15:27:38Z INFO virtiofsd] Client disconnected, shutting down7043client # [2026-09-25T15:27:38Z INFO virtiofsd] Client disconnected, shutting down7044client # [2026-09-25T15:27:38Z INFO virtiofsd] Client disconnected, shutting down7045kill QemuMachine (pid 49)7046lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7047lb # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7048lb # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7049lb # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7050kill QemuMachine (pid 52)7051worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7052worker1 # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7053worker1 # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7054worker1 # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7055kill QemuMachine (pid 51)7056worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7057worker2 # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7058worker2 # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7059worker2 # [2026-09-25T15:27:39Z INFO virtiofsd] Client disconnected, shutting down7060(finished: cleanup, in 1.09 seconds)7061additionally exposed symbols:7062 client, lb, worker1, worker2,7063 vlan1,7064 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh