vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #509
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13lb # Disk image does not exist, creating the virtualisation disk image...14client # Disk image does not exist, creating the virtualisation disk image...15client: QEMU running (pid 49)16worker2: QEMU running (pid 52)17worker1 # Disk image does not exist, creating the virtualisation disk image...18worker1: QEMU running (pid 51)19lb: QEMU running (pid 50)20worker2 # Formatting '/build/vm-state-worker2/tmp.UBUbkLo2Ao', fmt=raw size=107374182421client # Formatting '/build/vm-state-client/tmp.q4xRT4xwti', fmt=raw size=107374182422worker1 # Formatting '/build/vm-state-worker1/tmp.pLnZvU534n', fmt=raw size=107374182423lb # Formatting '/build/vm-state-lb/tmp.AA5qMkLJgX', fmt=raw size=107374182424worker2 # mke2fs 1.47.4 (6-Mar-2025)25worker1 # mke2fs 1.47.4 (6-Mar-2025)26lb # mke2fs 1.47.4 (6-Mar-2025)27worker2 # Discarding device blocks: 0/262144 done28worker1 # Discarding device blocks: 0/262144 done29lb # Discarding device blocks: 0/262144 done30worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes31worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes32lb # Creating filesystem with 262144 4k blocks and 65536 inodes33worker2 # Filesystem UUID: 1aed990d-7dc4-4d98-a511-11af0c3377ab34worker1 # Filesystem UUID: 5d94f2b0-e56f-404b-be05-4831c77c48d835lb # Filesystem UUID: 17035f7d-332b-4eff-bed7-2108b75626d236worker2 # Superblock backups stored on blocks:37worker1 # Superblock backups stored on blocks:38lb # Superblock backups stored on blocks:39worker2 # 32768, 98304, 163840, 22937640worker1 # 32768, 98304, 163840, 22937641lb # 32768, 98304, 163840, 22937642worker2 # 43worker1 # 44lb # 45worker2 # Allocating group tables: 0/8 done46worker1 # Allocating group tables: 0/8 done47lb # Allocating group tables: 0/8 done48worker2 # Writing inode tables: 0/8 done49worker1 # Writing inode tables: 0/8 done50lb # Writing inode tables: 0/8 done51worker2 # Creating journal (8192 blocks): done52worker1 # Creating journal (8192 blocks): done53lb # Creating journal (8192 blocks): done54worker2 # Writing superblocks and filesystem accounting information: 0/8 done55worker1 # Writing superblocks and filesystem accounting information: 0/8 done56lb # Writing superblocks and filesystem accounting information: 0/8 done57worker2 # 58worker1 # 59lb # 60worker2 # Virtualisation disk image created.61worker1 # Virtualisation disk image created.62lb # Virtualisation disk image created.63worker2 # Starting virtiofs daemons...64worker1 # Starting virtiofs daemons...65lb # Starting virtiofs daemons...66worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)68lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)69worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether72worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)73worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)74lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)75worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether77lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether81worker2 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82worker1 # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83lb # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...85worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...86lb # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...87worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...88worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...89lb # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...90worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...91worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...92lb # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...93worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests94worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests95lb # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests96worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests97worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests98lb # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests99worker2 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests100worker1 # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests101lb # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests102(finished: start all VMs, in 0.53 seconds)103client # mke2fs 1.47.4 (6-Mar-2025)104client: waiting for unit network-addresses-eth1.service105client # Discarding device blocks: 0/262144 done106client: waiting for the VM to finish booting107client # Creating filesystem with 262144 4k blocks and 65536 inodes108client # Filesystem UUID: 8a2019a4-cdd5-4625-8535-d98bd9f80ddd109client # Superblock backups stored on blocks:110client # 32768, 98304, 163840, 229376111client # 112client # Allocating group tables: 0/8 done113client # Writing inode tables: 0/8 done114client # Creating journal (8192 blocks): done115client # Writing superblocks and filesystem accounting information: 0/8 done116client # 117client # Virtualisation disk image created.118client # Starting virtiofs daemons...119client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)121client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether122client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)124client # [2026-09-22T22:20:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether125client # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...126client # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...127client # [2026-09-22T22:20:11Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-22T22:20:11Z INFO virtiofsd] Client connected, servicing requests131worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]133worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026134worker1 # [ 0.000000] KASLR enabled135worker1 # [ 0.000000] random: crng init done136worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026137worker1 # [ 0.000000] Machine model: linux,dummy-virt138worker2 # [ 0.000000] KASLR enabled139client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]140worker1 # [ 0.000000] efi: UEFI not found.141worker2 # [ 0.000000] random: crng init done142worker2 # [ 0.000000] Machine model: linux,dummy-virt143worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT144worker2 # [ 0.000000] efi: UEFI not found.145client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026146worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]147client # [ 0.000000] KASLR enabled148worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT149client # [ 0.000000] random: crng init done150worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]151worker1 # [ 0.000000] Zone ranges:152worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]153client # [ 0.000000] Machine model: linux,dummy-virt154client # [ 0.000000] efi: UEFI not found.155worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]156worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]157worker2 # [ 0.000000] Zone ranges:158worker1 # [ 0.000000] DMA32 empty159client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT160lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]161worker1 # [ 0.000000] Normal empty162worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]163worker1 # [ 0.000000] Device empty164worker2 # [ 0.000000] DMA32 empty165client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]166worker1 # [ 0.000000] Movable zone start for each node167worker2 # [ 0.000000] Normal empty168worker1 # [ 0.000000] Early memory node ranges169worker2 # [ 0.000000] Device empty170client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]171lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026172client # [ 0.000000] Zone ranges:173lb # [ 0.000000] KASLR enabled174worker2 # [ 0.000000] Movable zone start for each node175worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]176lb # [ 0.000000] random: crng init done177worker2 # [ 0.000000] Early memory node ranges178client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]179lb # [ 0.000000] Machine model: linux,dummy-virt180client # [ 0.000000] DMA32 empty181worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]182worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]183lb # [ 0.000000] efi: UEFI not found.184client # [ 0.000000] Normal empty185worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000186client # [ 0.000000] Device empty187worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]188worker1 # [ 0.000000] psci: probing for conduit method from DT.189lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT190client # [ 0.000000] Movable zone start for each node191client # [ 0.000000] Early memory node ranges192worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000193worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.194lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]195worker2 # [ 0.000000] psci: probing for conduit method from DT.196worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs197client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]198lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]199worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.200worker1 # [ 0.000000] psci: Trusted OS migration not required201lb # [ 0.000000] Zone ranges:202client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]203worker1 # [ 0.000000] psci: SMC Calling Convention v1.1204worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs205lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]206client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000207worker2 # [ 0.000000] psci: Trusted OS migration not required208lb # [ 0.000000] DMA32 empty209client # [ 0.000000] psci: probing for conduit method from DT.210lb # [ 0.000000] Normal empty211worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)212worker2 # [ 0.000000] psci: SMC Calling Convention v1.1213lb # [ 0.000000] Device empty214client # [ 0.000000] psci: PSCIv1.3 detected in firmware.215worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296216lb # [ 0.000000] Movable zone start for each node217worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)218client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs219lb # [ 0.000000] Early memory node ranges220worker1 # [ 0.000000] Detected PIPT I-cache on CPU0221client # [ 0.000000] psci: Trusted OS migration not required222worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296223lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]224client # [ 0.000000] psci: SMC Calling Convention v1.1225worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)226worker2 # [ 0.000000] Detected PIPT I-cache on CPU0227lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]228worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface229client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)230worker1 # [ 0.000000] CPU features: detected: Spectre-v4231lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000232worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)233worker1 # [ 0.000000] CPU features: detected: Spectre-BHB234worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface235client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296236lb # [ 0.000000] psci: probing for conduit method from DT.237worker2 # [ 0.000000] CPU features: detected: Spectre-v4238client # [ 0.000000] Detected PIPT I-cache on CPU0239lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.240worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38241worker2 # [ 0.000000] CPU features: detected: Spectre-BHB242lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs243worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23244client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)245lb # [ 0.000000] psci: Trusted OS migration not required246worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38247worker1 # [ 0.000000] alternatives: applying boot alternatives248client # [ 0.000000] CPU features: detected: GICv3 CPU interface249lb # [ 0.000000] psci: SMC Calling Convention v1.1250worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23251client # [ 0.000000] CPU features: detected: Spectre-v4252worker2 # [ 0.000000] alternatives: applying boot alternatives253lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)254client # [ 0.000000] CPU features: detected: Spectre-BHB255client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38256lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296257lb # [ 0.000000] Detected PIPT I-cache on CPU0258client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23259client # [ 0.000000] alternatives: applying boot alternatives260lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)261worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ry7bqhhhnwa7dlpa75fylilll15fakpb-nixos-system-worker1-test/init regInfo=/nix/store/94nix7rmj7g6brapzr4gg95i44rfy50d-closure-info/registration console=ttyAMA0,115200n8 console=tty0262lb # [ 0.000000] CPU features: detected: GICv3 CPU interface263lb # [ 0.000000] CPU features: detected: Spectre-v4264worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/30asfw61910plj74np6pz2ka43zplm0r-nixos-system-worker2-test/init regInfo=/nix/store/cpmzrhcw14bjzfxvmig0nm1xvcis5jif-closure-info/registration console=ttyAMA0,115200n8 console=tty0265lb # [ 0.000000] CPU features: detected: Spectre-BHB266worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/94nix7rmj7g6brapzr4gg95i44rfy50d-closure-info/registration", will be passed to user space.267lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38268worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/cpmzrhcw14bjzfxvmig0nm1xvcis5jif-closure-info/registration", will be passed to user space.269lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23270worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes271client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/rqv7b5vgrra5y9zkkdndibslzkbjsl4p-nixos-system-client-test/init regInfo=/nix/store/149kn4k15jb67kmb28r78yiz2vw4r88d-closure-info/registration console=ttyAMA0,115200n8 console=tty0272lb # [ 0.000000] alternatives: applying boot alternatives273worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes274worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)275worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)276worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)277worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)278client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/149kn4k15jb67kmb28r78yiz2vw4r88d-closure-info/registration", will be passed to user space.279worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB280worker1 # [ 0.000000] software IO TLB: area num 1.281client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes282worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB283worker2 # [ 0.000000] software IO TLB: area num 1.284worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB285lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test/init regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration console=ttyAMA0,115200n8 console=tty0286client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)287worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB288worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)289client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)290worker1 # [ 0.000000] Fallback order for Node 0: 0291worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)292lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration", will be passed to user space.293worker2 # [ 0.000000] Fallback order for Node 0: 0294client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB295worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216296worker1 # [ 0.000000] Policy zone: DMA297client # [ 0.000000] software IO TLB: area num 1.298lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes299worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216300worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off301worker2 # [ 0.000000] Policy zone: DMA302client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB303lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)304worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1305worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off306client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)307worker1 # [ 0.000000] allocated 3145728 bytes of page_ext308lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)309client # [ 0.000000] Fallback order for Node 0: 0310worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1311worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages312worker2 # [ 0.000000] allocated 3145728 bytes of page_ext313client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216314worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups315worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages316client # [ 0.000000] Policy zone: DMA317lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB318worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.319worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups320lb # [ 0.000000] software IO TLB: area num 1.321client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off322worker1 # [ 0.000000] rcu: RCU event tracing is enabled.323worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.324lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB325client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1326worker2 # [ 0.000000] rcu: RCU event tracing is enabled.327worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.328client # [ 0.000000] allocated 3145728 bytes of page_ext329worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.330lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)331worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.332client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages333lb # [ 0.000000] Fallback order for Node 0: 0334worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.335worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.336client # [ 0.000000] ftrace: allocated 294 pages with 4 groups337worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.338lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216339worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.340client # [ 0.000000] rcu: Hierarchical RCU implementation.341lb # [ 0.000000] Policy zone: DMA342worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.343worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.344client # [ 0.000000] rcu: RCU event tracing is enabled.345lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off346worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1347worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.348client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.349lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1350client # [ 0.000000] Trampoline variant of Tasks RCU enabled.351worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1352lb # [ 0.000000] allocated 3145728 bytes of page_ext353worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.354client # [ 0.000000] Rude variant of Tasks RCU enabled.355lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages356client # [ 0.000000] Tracing variant of Tasks RCU enabled.357worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.358worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.359lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups360client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.361lb # [ 0.000000] rcu: Hierarchical RCU implementation.362worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.363worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.364lb # [ 0.000000] rcu: RCU event tracing is enabled.365client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1366worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0367worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.368lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.369worker1 # [ 0.000000] GICv3: 256 SPIs implemented370client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.371lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.372worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0373worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented374worker2 # [ 0.000000] GICv3: 256 SPIs implemented375lb # [ 0.000000] Rude variant of Tasks RCU enabled.376worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq377client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.378worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented379lb # [ 0.000000] Tracing variant of Tasks RCU enabled.380worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI381worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq382worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0383lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.384worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI385client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.386worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000387worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0388client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0389lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1390worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]391client # [ 0.000000] GICv3: 256 SPIs implemented392worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000393client # [ 0.000000] GICv3: 0 Extended SPIs implemented394lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.395worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]396worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)397client # [ 0.000000] Root IRQ handler: gic_handle_irq398lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.399client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI400worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)401worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)402client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0403lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.404worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000405client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000406worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)407lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0408client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]409worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000410worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000411lb # [ 0.000000] GICv3: 256 SPIs implemented412lb # [ 0.000000] GICv3: 0 Extended SPIs implemented413worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.414client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)415worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000416lb # [ 0.000000] Root IRQ handler: gic_handle_irq417worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.418lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI419worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns420client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)421lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0422worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).423client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000424worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns425lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000426lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]427worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).428client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000429worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns430client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.431lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)432worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns433worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns434worker1 # [ 0.000028] arm-pv: using stolen time PV435client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns436worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns437lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)438worker2 # [ 0.000029] arm-pv: using stolen time PV439client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).440worker1 # [ 0.000385] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)441lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000442worker1 # [ 0.000551] Console: colour dummy device 80x25443lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000444worker1 # [ 0.000558] printk: legacy console [tty0] enabled445worker2 # [ 0.000375] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)446client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns447worker2 # [ 0.000541] Console: colour dummy device 80x25448lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.449client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns450worker2 # [ 0.000548] printk: legacy console [tty0] enabled451worker1 # [ 0.000759] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)452client # [ 0.000030] arm-pv: using stolen time PV453lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns454worker1 # [ 0.000766] pid_max: default: 32768 minimum: 301455worker2 # [ 0.000740] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)456lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).457worker1 # [ 0.000849] LSM: initializing lsm=capability,landlock,yama,bpf,ima458client # [ 0.000403] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)459worker2 # [ 0.000747] pid_max: default: 32768 minimum: 301460worker1 # [ 0.000978] landlock: Up and running.461client # [ 0.000619] Console: colour dummy device 80x25462worker1 # [ 0.000981] Yama: becoming mindful.463worker2 # [ 0.000828] LSM: initializing lsm=capability,landlock,yama,bpf,ima464lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns465client # [ 0.000627] printk: legacy console [tty0] enabled466worker1 # [ 0.001404] LSM support for eBPF active467worker2 # [ 0.000963] landlock: Up and running.468worker2 # [ 0.000966] Yama: becoming mindful.469worker1 # [ 0.001577] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)470lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns471worker2 # [ 0.001379] LSM support for eBPF active472client # [ 0.000845] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)473lb # [ 0.000029] arm-pv: using stolen time PV474client # [ 0.000852] pid_max: default: 32768 minimum: 301475worker1 # [ 0.001619] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)476worker2 # [ 0.001533] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)477worker1 # [ 0.003476] rcu: Hierarchical SRCU implementation.478client # [ 0.000923] LSM: initializing lsm=capability,landlock,yama,bpf,ima479lb # [ 0.000396] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)480client # [ 0.001053] landlock: Up and running.481worker1 # [ 0.003481] rcu: Max phase no-delay instances is 1000.482lb # [ 0.000560] Console: colour dummy device 80x25483worker2 # [ 0.001566] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)484client # [ 0.001056] Yama: becoming mindful.485worker1 # [ 0.004748] fsl-mc MSI: its@8080000 domain created486worker2 # [ 0.003426] rcu: Hierarchical SRCU implementation.487lb # [ 0.000567] printk: legacy console [tty0] enabled488client # [ 0.001510] LSM support for eBPF active489worker1 # [ 0.004839] EFI services will not be available.490worker2 # [ 0.003431] rcu: Max phase no-delay instances is 1000.491worker1 # [ 0.004918] smp: Bringing up secondary CPUs ...492worker2 # [ 0.004650] fsl-mc MSI: its@8080000 domain created493client # [ 0.001667] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)494lb # [ 0.000753] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)495worker1 # [ 0.004926] smp: Brought up 1 node, 1 CPU496worker2 # [ 0.004740] EFI services will not be available.497lb # [ 0.000759] pid_max: default: 32768 minimum: 301498worker1 # [ 0.004929] SMP: Total of 1 processors activated.499client # [ 0.001700] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)500worker2 # [ 0.004806] smp: Bringing up secondary CPUs ...501worker1 # [ 0.004935] CPU: All CPU(s) started at EL1502lb # [ 0.000830] LSM: initializing lsm=capability,landlock,yama,bpf,ima503client # [ 0.003495] rcu: Hierarchical SRCU implementation.504worker2 # [ 0.004815] smp: Brought up 1 node, 1 CPU505lb # [ 0.000969] landlock: Up and running.506client # [ 0.003500] rcu: Max phase no-delay instances is 1000.507worker2 # [ 0.004818] SMP: Total of 1 processors activated.508lb # [ 0.000972] Yama: becoming mindful.509worker1 # [ 0.004945] CPU features: detected: Branch Target Identification510worker2 # [ 0.004823] CPU: All CPU(s) started at EL1511client # [ 0.004793] fsl-mc MSI: its@8080000 domain created512lb # [ 0.001406] LSM support for eBPF active513worker1 # [ 0.004950] CPU features: detected: ARMv8.4 Translation Table Level514client # [ 0.004884] EFI services will not be available.515worker2 # [ 0.004833] CPU features: detected: Branch Target Identification516lb # [ 0.001570] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)517client # [ 0.004957] smp: Bringing up secondary CPUs ...518worker1 # [ 0.004953] CPU features: detected: Instruction cache invalidation not required for I/D coherence519worker2 # [ 0.004838] CPU features: detected: ARMv8.4 Translation Table Level520client # [ 0.004965] smp: Brought up 1 node, 1 CPU521lb # [ 0.001602] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)522client # [ 0.004968] SMP: Total of 1 processors activated.523worker1 # [ 0.004957] CPU features: detected: Data cache clean to the PoU not required for I/D coherence524lb # [ 0.003448] rcu: Hierarchical SRCU implementation.525worker2 # [ 0.004841] CPU features: detected: Instruction cache invalidation not required for I/D coherence526client # [ 0.004974] CPU: All CPU(s) started at EL1527worker1 # [ 0.004961] CPU features: detected: Common not Private translations528lb # [ 0.003452] rcu: Max phase no-delay instances is 1000.529client # [ 0.004984] CPU features: detected: Branch Target Identification530worker2 # [ 0.004845] CPU features: detected: Data cache clean to the PoU not required for I/D coherence531lb # [ 0.004715] fsl-mc MSI: its@8080000 domain created532worker1 # [ 0.004964] CPU features: detected: CRC32 instructions533client # [ 0.004988] CPU features: detected: ARMv8.4 Translation Table Level534lb # [ 0.004813] EFI services will not be available.535worker2 # [ 0.004849] CPU features: detected: Common not Private translations536worker1 # [ 0.004967] CPU features: detected: Data cache clean to Point of Deep Persistence537lb # [ 0.004880] smp: Bringing up secondary CPUs ...538worker2 # [ 0.004852] CPU features: detected: CRC32 instructions539lb # [ 0.004888] smp: Brought up 1 node, 1 CPU540client # [ 0.004991] CPU features: detected: Instruction cache invalidation not required for I/D coherence541worker1 # [ 0.004970] CPU features: detected: Data cache clean to Point of Persistence542lb # [ 0.004892] SMP: Total of 1 processors activated.543worker2 # [ 0.004855] CPU features: detected: Data cache clean to Point of Deep Persistence544worker1 # [ 0.004973] CPU features: detected: Data independent timing control (DIT)545lb # [ 0.004896] CPU: All CPU(s) started at EL1546client # [ 0.004995] CPU features: detected: Data cache clean to the PoU not required for I/D coherence547worker2 # [ 0.004858] CPU features: detected: Data cache clean to Point of Persistence548worker1 # [ 0.004976] CPU features: detected: E0PD549lb # [ 0.004907] CPU features: detected: Branch Target Identification550client # [ 0.004999] CPU features: detected: Common not Private translations551worker1 # [ 0.004979] CPU features: detected: Enhanced Counter Virtualization552worker2 # [ 0.004861] CPU features: detected: Data independent timing control (DIT)553client # [ 0.005002] CPU features: detected: CRC32 instructions554lb # [ 0.004911] CPU features: detected: ARMv8.4 Translation Table Level555worker2 # [ 0.004864] CPU features: detected: E0PD556worker1 # [ 0.004982] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)557client # [ 0.005005] CPU features: detected: Data cache clean to Point of Deep Persistence558worker2 # [ 0.004866] CPU features: detected: Enhanced Counter Virtualization559lb # [ 0.004915] CPU features: detected: Instruction cache invalidation not required for I/D coherence560worker1 # [ 0.004985] CPU features: detected: Enhanced Virtualization Traps561client # [ 0.005008] CPU features: detected: Data cache clean to Point of Persistence562worker2 # [ 0.004869] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)563worker1 # [ 0.004988] CPU features: detected: Fine Grained Traps564lb # [ 0.004918] CPU features: detected: Data cache clean to the PoU not required for I/D coherence565client # [ 0.005011] CPU features: detected: Data independent timing control (DIT)566worker2 # [ 0.004873] CPU features: detected: Enhanced Virtualization Traps567worker1 # [ 0.004991] CPU features: detected: Generic authentication (architected QARMA5 algorithm)568lb # [ 0.004922] CPU features: detected: Common not Private translations569client # [ 0.005015] CPU features: detected: E0PD570worker2 # [ 0.004876] CPU features: detected: Fine Grained Traps571lb # [ 0.004926] CPU features: detected: CRC32 instructions572worker1 # [ 0.004997] CPU features: detected: RCpc load-acquire (LDAPR)573client # [ 0.005017] CPU features: detected: Enhanced Counter Virtualization574worker2 # [ 0.004879] CPU features: detected: Generic authentication (architected QARMA5 algorithm)575worker1 # [ 0.005000] CPU features: detected: LSE atomic instructions576lb # [ 0.004928] CPU features: detected: Data cache clean to Point of Deep Persistence577client # [ 0.005020] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)578worker2 # [ 0.004884] CPU features: detected: RCpc load-acquire (LDAPR)579worker1 # [ 0.005002] CPU features: detected: Privileged Access Never580lb # [ 0.004932] CPU features: detected: Data cache clean to Point of Persistence581worker1 # [ 0.005005] CPU features: detected: PMUv3582client # [ 0.005023] CPU features: detected: Enhanced Virtualization Traps583worker2 # [ 0.004887] CPU features: detected: LSE atomic instructions584lb # [ 0.004935] CPU features: detected: Data independent timing control (DIT)585worker1 # [ 0.005008] CPU features: detected: RAS Extension Support586worker2 # [ 0.004890] CPU features: detected: Privileged Access Never587client # [ 0.005027] CPU features: detected: Fine Grained Traps588lb # [ 0.004938] CPU features: detected: E0PD589worker1 # [ 0.005011] CPU features: detected: RASv1p1 Extension Support590worker2 # [ 0.004892] CPU features: detected: PMUv3591client # [ 0.005031] CPU features: detected: Generic authentication (architected QARMA5 algorithm)592lb # [ 0.004941] CPU features: detected: Enhanced Counter Virtualization593worker2 # [ 0.004895] CPU features: detected: RAS Extension Support594worker1 # [ 0.005013] CPU features: detected: Random Number Generator595client # [ 0.005036] CPU features: detected: RCpc load-acquire (LDAPR)596worker2 # [ 0.004898] CPU features: detected: RASv1p1 Extension Support597worker1 # [ 0.005016] CPU features: detected: Speculation barrier (SB)598lb # [ 0.004944] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)599client # [ 0.005039] CPU features: detected: LSE atomic instructions600worker2 # [ 0.004901] CPU features: detected: Random Number Generator601worker1 # [ 0.005019] CPU features: detected: Stage-2 Force Write-Back602lb # [ 0.004947] CPU features: detected: Enhanced Virtualization Traps603client # [ 0.005041] CPU features: detected: Privileged Access Never604worker2 # [ 0.004903] CPU features: detected: Speculation barrier (SB)605lb # [ 0.004950] CPU features: detected: Fine Grained Traps606client # [ 0.005044] CPU features: detected: PMUv3607worker2 # [ 0.004906] CPU features: detected: Stage-2 Force Write-Back608client # [ 0.005047] CPU features: detected: RAS Extension Support609lb # [ 0.004954] CPU features: detected: Generic authentication (architected QARMA5 algorithm)610worker2 # [ 0.004909] CPU features: detected: TLB range maintenance instructions611client # [ 0.005050] CPU features: detected: RASv1p1 Extension Support612lb # [ 0.004959] CPU features: detected: RCpc load-acquire (LDAPR)613client # [ 0.005053] CPU features: detected: Random Number Generator614worker2 # [ 0.004914] CPU features: detected: Speculative Store Bypassing Safe (SSBS)615lb # [ 0.004962] CPU features: detected: LSE atomic instructions616worker2 # [ 0.004954] alternatives: applying system-wide alternatives617lb # [ 0.004965] CPU features: detected: Privileged Access Never618lb # [ 0.004967] CPU features: detected: PMUv3619worker2 # [ 0.007825] CPU features: detected: BBM Level 2 without TLB conflict abort620lb # [ 0.004970] CPU features: detected: RAS Extension Support621lb # [ 0.004973] CPU features: detected: RASv1p1 Extension Support622worker2 # [ 0.007993] Memory: 1405656K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)623lb # [ 0.004976] CPU features: detected: Random Number Generator624worker2 # [ 0.008372] devtmpfs: initialized625lb # [ 0.004978] CPU features: detected: Speculation barrier (SB)626worker2 # [ 0.010132] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)627lb # [ 0.004981] CPU features: detected: Stage-2 Force Write-Back628lb # [ 0.004984] CPU features: detected: TLB range maintenance instructions629worker2 # [ 0.010154] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).630worker1 # [ 0.005022] CPU features: detected: TLB range maintenance instructions631lb # [ 0.004988] CPU features: detected: Speculative Store Bypassing Safe (SSBS)632worker2 # [ 0.010335] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL633worker1 # [ 0.005027] CPU features: detected: Speculative Store Bypassing Safe (SSBS)634worker2 # [ 0.010339] 0 pages in range for non-PLT usage635worker1 # [ 0.005065] alternatives: applying system-wide alternatives636worker2 # [ 0.010340] 508272 pages in range for PLT usage637worker2 # [ 0.010445] pinctrl core: initialized pinctrl subsystem638worker1 # [ 0.007987] CPU features: detected: BBM Level 2 without TLB conflict abort639worker2 # [ 0.011126] DMI not present or invalid.640client # [ 0.005055] CPU features: detected: Speculation barrier (SB)641worker2 # [ 0.014372] NET: Registered PF_NETLINK/PF_ROUTE protocol family642client # [ 0.005058] CPU features: detected: Stage-2 Force Write-Back643worker1 # [ 0.008126] Memory: 1405880K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)644client # [ 0.005061] CPU features: detected: TLB range maintenance instructions645worker2 # [ 0.016530] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations646worker1 # [ 0.008517] devtmpfs: initialized647client # [ 0.005066] CPU features: detected: Speculative Store Bypassing Safe (SSBS)648worker1 # [ 0.010356] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)649client # [ 0.005103] alternatives: applying system-wide alternatives650worker2 # [ 0.016696] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations651worker1 # [ 0.010378] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).652client # [ 0.008070] CPU features: detected: BBM Level 2 without TLB conflict abort653worker2 # [ 0.016889] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations654worker1 # [ 0.010551] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL655worker2 # [ 0.016909] audit: initializing netlink subsys (disabled)656worker1 # [ 0.010555] 0 pages in range for non-PLT usage657worker2 # [ 0.017436] thermal_sys: Registered thermal governor 'fair_share'658worker1 # [ 0.010556] 508272 pages in range for PLT usage659client # [ 0.008241] Memory: 1405652K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)660client # [ 0.008639] devtmpfs: initialized661worker1 # [ 0.010651] pinctrl core: initialized pinctrl subsystem662worker2 # [ 0.017438] thermal_sys: Registered thermal governor 'bang_bang'663worker1 # [ 0.011365] DMI not present or invalid.664lb # [ 0.005023] alternatives: applying system-wide alternatives665client # [ 0.010401] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)666worker2 # [ 0.017442] thermal_sys: Registered thermal governor 'step_wise'667worker1 # [ 0.014578] NET: Registered PF_NETLINK/PF_ROUTE protocol family668lb # [ 0.007957] CPU features: detected: BBM Level 2 without TLB conflict abort669worker2 # [ 0.017445] thermal_sys: Registered thermal governor 'user_space'670client # [ 0.010422] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).671worker1 # [ 0.016750] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations672worker2 # [ 0.017450] thermal_sys: Registered thermal governor 'power_allocator'673client # [ 0.010618] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL674worker1 # [ 0.016921] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations675lb # [ 0.008191] Memory: 1405676K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)676client # [ 0.010622] 0 pages in range for non-PLT usage677worker2 # [ 0.017472] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1678lb # [ 0.008586] devtmpfs: initialized679client # [ 0.010624] 508272 pages in range for PLT usage680worker2 # [ 0.017482] cpuidle: using governor ladder681worker1 # [ 0.017115] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations682client # [ 0.010719] pinctrl core: initialized pinctrl subsystem683worker2 # [ 0.017488] cpuidle: using governor menu684lb # [ 0.010409] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)685worker1 # [ 0.017135] audit: initializing netlink subsys (disabled)686client # [ 0.011498] DMI not present or invalid.687worker2 # [ 0.017675] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.688worker1 # [ 0.017709] thermal_sys: Registered thermal governor 'fair_share'689lb # [ 0.010430] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).690client # [ 0.014726] NET: Registered PF_NETLINK/PF_ROUTE protocol family691worker2 # [ 0.017690] ASID allocator initialised with 65536 entries692worker1 # [ 0.017711] thermal_sys: Registered thermal governor 'bang_bang'693lb # [ 0.010618] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL694client # [ 0.016894] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations695worker2 # [ 0.018826] Serial: AMBA PL011 UART driver696lb # [ 0.010622] 0 pages in range for non-PLT usage697worker1 # [ 0.017715] thermal_sys: Registered thermal governor 'step_wise'698lb # [ 0.010623] 508272 pages in range for PLT usage699client # [ 0.017068] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations700worker2 # [ 0.023967] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1701worker1 # [ 0.017717] thermal_sys: Registered thermal governor 'user_space'702lb # [ 0.010731] pinctrl core: initialized pinctrl subsystem703worker2 # [ 0.024094] printk: console [ttyAMA0] enabled704client # [ 0.017263] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations705lb # [ 0.011454] DMI not present or invalid.706worker1 # [ 0.017723] thermal_sys: Registered thermal governor 'power_allocator'707client # [ 0.017283] audit: initializing netlink subsys (disabled)708lb # [ 0.014571] NET: Registered PF_NETLINK/PF_ROUTE protocol family709worker1 # [ 0.017747] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1710client # [ 0.017824] thermal_sys: Registered thermal governor 'fair_share'711worker1 # [ 0.017755] cpuidle: using governor ladder712lb # [ 0.016782] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations713worker1 # [ 0.017761] cpuidle: using governor menu714client # [ 0.017826] thermal_sys: Registered thermal governor 'bang_bang'715lb # [ 0.016951] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations716client # [ 0.017830] thermal_sys: Registered thermal governor 'step_wise'717worker1 # [ 0.017945] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.718worker1 # [ 0.017960] ASID allocator initialised with 65536 entries719lb # [ 0.017148] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations720client # [ 0.017832] thermal_sys: Registered thermal governor 'user_space'721worker1 # [ 0.019127] Serial: AMBA PL011 UART driver722lb # [ 0.017168] audit: initializing netlink subsys (disabled)723client # [ 0.017837] thermal_sys: Registered thermal governor 'power_allocator'724lb # [ 0.017685] thermal_sys: Registered thermal governor 'fair_share'725worker1 # [ 0.024326] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1726client # [ 0.017862] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1727worker1 # [ 0.024477] printk: console [ttyAMA0] enabled728lb # [ 0.017687] thermal_sys: Registered thermal governor 'bang_bang'729client # [ 0.017870] cpuidle: using governor ladder730client # [ 0.017876] cpuidle: using governor menu731lb # [ 0.017691] thermal_sys: Registered thermal governor 'step_wise'732worker2 # [ 0.141199] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages733worker1 # [ 0.142917] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages734lb # [ 0.017694] thermal_sys: Registered thermal governor 'user_space'735client # [ 0.018070] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.736worker2 # [ 0.141215] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page737worker1 # [ 0.142935] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page738client # [ 0.018086] ASID allocator initialised with 65536 entries739lb # [ 0.017699] thermal_sys: Registered thermal governor 'power_allocator'740worker2 # [ 0.141220] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages741client # [ 0.019322] Serial: AMBA PL011 UART driver742worker1 # [ 0.142940] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages743lb # [ 0.017724] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1744worker2 # [ 0.141225] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page745client # [ 0.024418] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1746worker1 # [ 0.142944] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page747lb # [ 0.017732] cpuidle: using governor ladder748lb # [ 0.017737] cpuidle: using governor menu749worker2 # [ 0.141229] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages750client # [ 0.024575] printk: console [ttyAMA0] enabled751worker1 # [ 0.142949] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages752lb # [ 0.017913] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.753worker2 # [ 0.141233] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page754worker1 # [ 0.142953] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page755lb # [ 0.017928] ASID allocator initialised with 65536 entries756client # [ 0.143963] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages757worker2 # [ 0.141238] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages758worker1 # [ 0.142958] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages759lb # [ 0.019110] Serial: AMBA PL011 UART driver760client # [ 0.143980] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page761worker2 # [ 0.141242] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page762worker1 # [ 0.142962] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page763lb # [ 0.024388] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1764client # [ 0.143985] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages765lb # [ 0.024507] printk: console [ttyAMA0] enabled766client # [ 0.143990] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page767client # [ 0.143994] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages768worker2 # [ 0.148346] fbcon: Taking over console769lb # [ 0.143051] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages770worker1 # [ 0.150137] fbcon: Taking over console771worker2 # [ 0.148359] ACPI: Interpreter disabled.772client # [ 0.143998] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page773worker1 # [ 0.150150] ACPI: Interpreter disabled.774lb # [ 0.143068] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page775client # [ 0.144003] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages776lb # [ 0.143074] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages777client # [ 0.144007] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page778lb # [ 0.143078] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page779lb # [ 0.143083] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages780lb # [ 0.143087] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page781client # [ 0.151295] fbcon: Taking over console782lb # [ 0.143091] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages783client # [ 0.151307] ACPI: Interpreter disabled.784lb # [ 0.143095] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page785client # [ 0.153168] iommu: Default domain type: Translated786client # [ 0.153178] iommu: DMA domain TLB invalidation policy: strict mode787client # [ 0.154888] SCSI subsystem initialized788lb # [ 0.150476] fbcon: Taking over console789worker1 # [ 0.158915] iommu: Default domain type: Translated790lb # [ 0.150488] ACPI: Interpreter disabled.791worker2 # [ 0.157244] iommu: Default domain type: Translated792worker1 # [ 0.158925] iommu: DMA domain TLB invalidation policy: strict mode793worker2 # [ 0.157254] iommu: DMA domain TLB invalidation policy: strict mode794worker2 # [ 0.157587] SCSI subsystem initialized795worker1 # [ 0.160613] SCSI subsystem initialized796worker2 # [ 0.159557] usbcore: registered new interface driver usbfs797worker1 # [ 0.161292] usbcore: registered new interface driver usbfs798worker2 # [ 0.159593] usbcore: registered new interface driver hub799worker1 # [ 0.161316] usbcore: registered new interface driver hub800worker2 # [ 0.159608] usbcore: registered new device driver usb801worker1 # [ 0.161339] usbcore: registered new device driver usb802worker2 # [ 0.159861] pps_core: LinuxPPS API ver. 1 registered803worker1 # [ 0.161610] pps_core: LinuxPPS API ver. 1 registered804worker2 # [ 0.159867] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>805worker1 # [ 0.161615] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>806lb # [ 0.157512] iommu: Default domain type: Translated807worker2 # [ 0.159876] PTP clock support registered808worker1 # [ 0.161626] PTP clock support registered809worker2 # [ 0.159925] EDAC MC: Ver: 3.0.0810lb # [ 0.157521] iommu: DMA domain TLB invalidation policy: strict mode811worker1 # [ 0.161665] EDAC MC: Ver: 3.0.0812worker2 # [ 0.164523] scmi_core: SCMI protocol bus registered813client # [ 0.162535] usbcore: registered new interface driver usbfs814lb # [ 0.159141] SCSI subsystem initialized815worker1 # [ 0.166231] scmi_core: SCMI protocol bus registered816client # [ 0.162576] usbcore: registered new interface driver hub817worker2 # [ 0.165488] FPGA manager framework818client # [ 0.162592] usbcore: registered new device driver usb819worker1 # [ 0.167178] FPGA manager framework820lb # [ 0.159805] usbcore: registered new interface driver usbfs821worker2 # [ 0.166408] vgaarb: loaded822worker1 # [ 0.168165] vgaarb: loaded823client # [ 0.162872] pps_core: LinuxPPS API ver. 1 registered824lb # [ 0.159836] usbcore: registered new interface driver hub825worker2 # [ 0.167006] clocksource: Switched to clocksource arch_sys_counter826lb # [ 0.159851] usbcore: registered new device driver usb827client # [ 0.162878] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>828lb # [ 0.160138] pps_core: LinuxPPS API ver. 1 registered829client # [ 0.162887] PTP clock support registered830worker1 # [ 0.168749] clocksource: Switched to clocksource arch_sys_counter831client # [ 0.162927] EDAC MC: Ver: 3.0.0832worker1 # [ 0.169317] VFS: Disk quotas dquot_6.6.0833lb # [ 0.160159] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>834client # [ 0.167511] scmi_core: SCMI protocol bus registered835worker1 # [ 0.169347] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)836lb # [ 0.160169] PTP clock support registered837lb # [ 0.160215] EDAC MC: Ver: 3.0.0838worker1 # [ 0.171694] netfs: FS-Cache loaded839client # [ 0.168439] FPGA manager framework840worker1 # [ 0.171812] pnp: PnP ACPI: disabled841client # [ 0.169348] vgaarb: loaded842lb # [ 0.164675] scmi_core: SCMI protocol bus registered843lb # [ 0.165638] FPGA manager framework844lb # [ 0.166568] vgaarb: loaded845client # [ 0.169947] clocksource: Switched to clocksource arch_sys_counter846client # [ 0.170514] VFS: Disk quotas dquot_6.6.0847worker2 # [ 0.172961] VFS: Disk quotas dquot_6.6.0848lb # [ 0.167186] clocksource: Switched to clocksource arch_sys_counter849client # [ 0.170538] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)850worker2 # [ 0.173000] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)851client # [ 0.172870] netfs: FS-Cache loaded852worker2 # [ 0.174595] netfs: FS-Cache loaded853client # [ 0.172981] pnp: PnP ACPI: disabled854worker2 # [ 0.174708] pnp: PnP ACPI: disabled855lb # [ 0.170155] VFS: Disk quotas dquot_6.6.0856lb # [ 0.170190] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)857worker2 # [ 0.178442] NET: Registered PF_INET protocol family858worker1 # [ 0.177697] NET: Registered PF_INET protocol family859lb # [ 0.173808] netfs: FS-Cache loaded860worker2 # [ 0.178695] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)861lb # [ 0.173924] pnp: PnP ACPI: disabled862worker1 # [ 0.177957] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)863client # [ 0.178764] NET: Registered PF_INET protocol family864client # [ 0.179028] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)865lb # [ 0.177639] NET: Registered PF_INET protocol family866lb # [ 0.177898] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)867worker1 # [ 0.206975] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)868worker1 # [ 0.207015] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)869lb # [ 0.206924] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)870worker1 # [ 0.207042] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)871lb # [ 0.206964] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)872worker1 # [ 0.207114] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)873worker1 # [ 0.207255] TCP: Hash tables configured (established 16384 bind 16384)874lb # [ 0.206989] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)875worker2 # [ 0.207360] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)876client # [ 0.207680] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)877worker1 # [ 0.207334] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)878lb # [ 0.207065] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)879worker2 # [ 0.207400] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)880worker1 # [ 0.207368] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)881client # [ 0.207719] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)882lb # [ 0.211039] TCP: Hash tables configured (established 16384 bind 16384)883worker2 # [ 0.207424] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)884worker1 # [ 0.207403] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)885client # [ 0.207743] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)886worker2 # [ 0.207498] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)887lb # [ 0.211120] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)888worker1 # [ 0.207502] NET: Registered PF_UNIX/PF_LOCAL protocol family889client # [ 0.207813] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)890lb # [ 0.211153] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)891worker2 # [ 0.207642] TCP: Hash tables configured (established 16384 bind 16384)892worker1 # [ 0.207523] NET: Registered PF_XDP protocol family893worker1 # [ 0.207539] PCI: CLS 0 bytes, default 64894client # [ 0.207955] TCP: Hash tables configured (established 16384 bind 16384)895lb # [ 0.211188] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)896worker2 # [ 0.207719] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)897worker1 # [ 0.207764] Trying to unpack rootfs image as initramfs...898client # [ 0.208033] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)899worker2 # [ 0.207753] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)900client # [ 0.208065] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)901worker2 # [ 0.207788] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)902worker2 # [ 0.207875] NET: Registered PF_UNIX/PF_LOCAL protocol family903client # [ 0.208100] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)904worker2 # [ 0.207901] NET: Registered PF_XDP protocol family905client # [ 0.208197] NET: Registered PF_UNIX/PF_LOCAL protocol family906worker2 # [ 0.207917] PCI: CLS 0 bytes, default 64907client # [ 0.208236] NET: Registered PF_XDP protocol family908worker2 # [ 0.208144] Trying to unpack rootfs image as initramfs...909client # [ 0.208253] PCI: CLS 0 bytes, default 64910client # [ 0.208491] Trying to unpack rootfs image as initramfs...911worker1 # [ 0.222147] kvm [1]: HYP mode not available912lb # [ 0.214485] NET: Registered PF_UNIX/PF_LOCAL protocol family913lb # [ 0.214505] NET: Registered PF_XDP protocol family914lb # [ 0.214522] PCI: CLS 0 bytes, default 64915lb # [ 0.214757] Trying to unpack rootfs image as initramfs...916worker2 # [ 0.224494] kvm [1]: HYP mode not available917client # [ 0.223745] kvm [1]: HYP mode not available918lb # [ 0.225633] kvm [1]: HYP mode not available919worker1 # [ 0.313336] Initialise system trusted keyrings920worker1 # [ 0.314075] workingset: timestamp_bits=42 max_order=19 bucket_order=0921worker1 # [ 0.315344] squashfs: version 4.0 (2009/01/31) Phillip Lougher922worker1 # [ 0.316106] 9p: Installing v9fs 9p2000 file system support923worker2 # [ 0.315555] Initialise system trusted keyrings924client # [ 0.314469] Initialise system trusted keyrings925worker2 # [ 0.316325] workingset: timestamp_bits=42 max_order=19 bucket_order=0926client # [ 0.315211] workingset: timestamp_bits=42 max_order=19 bucket_order=0927worker2 # [ 0.317621] squashfs: version 4.0 (2009/01/31) Phillip Lougher928client # [ 0.316477] squashfs: version 4.0 (2009/01/31) Phillip Lougher929worker2 # [ 0.318419] 9p: Installing v9fs 9p2000 file system support930client # [ 0.317271] 9p: Installing v9fs 9p2000 file system support931lb # [ 0.319730] Initialise system trusted keyrings932lb # [ 0.320474] workingset: timestamp_bits=42 max_order=19 bucket_order=0933lb # [ 0.321749] squashfs: version 4.0 (2009/01/31) Phillip Lougher934lb # [ 0.322512] 9p: Installing v9fs 9p2000 file system support935client # [ 0.337932] Key type asymmetric registered936worker1 # [ 0.344919] Key type asymmetric registered937worker1 # [ 0.344947] Asymmetric key parser 'x509' registered938worker1 # [ 0.345009] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)939worker1 # [ 0.347087] io scheduler mq-deadline registered940worker1 # [ 0.347097] io scheduler kyber registered941worker2 # [ 0.347128] Key type asymmetric registered942client # [ 0.345988] Asymmetric key parser 'x509' registered943worker2 # [ 0.347152] Asymmetric key parser 'x509' registered944client # [ 0.346061] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)945worker2 # [ 0.347216] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)946lb # [ 0.343145] Key type asymmetric registered947client # [ 0.347697] io scheduler mq-deadline registered948worker2 # [ 0.349367] io scheduler mq-deadline registered949lb # [ 0.343163] Asymmetric key parser 'x509' registered950client # [ 0.347708] io scheduler kyber registered951worker2 # [ 0.349378] io scheduler kyber registered952worker1 # [ 0.356903] pl061_gpio 9030000.pl061: PL061 GPIO chip registered953lb # [ 0.351288] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)954lb # [ 0.352327] io scheduler mq-deadline registered955lb # [ 0.352337] io scheduler kyber registered956worker1 # [ 0.358273] ledtrig-cpu: registered to indicate activity on CPUs957worker2 # [ 0.359150] pl061_gpio 9030000.pl061: PL061 GPIO chip registered958worker1 # [ 0.358681] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:959client # [ 0.358080] pl061_gpio 9030000.pl061: PL061 GPIO chip registered960worker1 # [ 0.358698] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000961worker1 # [ 0.358710] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000962worker2 # [ 0.360496] ledtrig-cpu: registered to indicate activity on CPUs963worker1 # [ 0.358718] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000964client # [ 0.359435] ledtrig-cpu: registered to indicate activity on CPUs965worker2 # [ 0.360861] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:966worker1 # [ 0.358744] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits967client # [ 0.359820] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:968worker2 # [ 0.360878] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000969client # [ 0.359836] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000970worker1 # [ 0.358767] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]971worker2 # [ 0.360890] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000972worker1 # [ 0.358836] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00973client # [ 0.359850] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000974worker1 # [ 0.358845] pci_bus 0000:00: root bus resource [bus 00-ff]975worker2 # [ 0.360899] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000976client # [ 0.359858] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000977worker1 # [ 0.358851] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]978worker2 # [ 0.360920] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits979worker1 # [ 0.358856] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]980client # [ 0.359878] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits981worker2 # [ 0.360944] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]982worker1 # [ 0.358861] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]983client # [ 0.359901] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]984worker2 # [ 0.361020] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00985worker1 # [ 0.358914] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint986client # [ 0.359972] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00987worker2 # [ 0.361030] pci_bus 0000:00: root bus resource [bus 00-ff]988client # [ 0.359981] pci_bus 0000:00: root bus resource [bus 00-ff]989worker1 # [ 0.359346] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint990worker2 # [ 0.361036] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]991client # [ 0.359987] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]992worker1 # [ 0.359558] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]993worker2 # [ 0.361041] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]994worker1 # [ 0.359575] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]995client # [ 0.359992] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]996worker2 # [ 0.361046] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]997worker1 # [ 0.359604] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]998client # [ 0.359997] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]999worker2 # [ 0.361099] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1000worker1 # [ 0.359620] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1001client # [ 0.360085] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1002worker2 # [ 0.361543] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1003worker1 # [ 0.360084] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1004client # [ 0.360533] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1005worker2 # [ 0.361728] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1006worker1 # [ 0.360265] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1007client # [ 0.360719] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1008worker2 # [ 0.361747] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1009worker1 # [ 0.360281] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1010client # [ 0.360735] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1011lb # [ 0.363358] pl061_gpio 9030000.pl061: PL061 GPIO chip registered1012worker1 # [ 0.360310] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1013worker2 # [ 0.361778] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1014client # [ 0.360765] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1015lb # [ 0.363960] ledtrig-cpu: registered to indicate activity on CPUs1016worker2 # [ 0.361795] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1017client # [ 0.360780] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1018lb # [ 0.364349] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:1019worker2 # [ 0.362270] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1020client # [ 0.361234] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1021worker2 # [ 0.362452] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1022worker1 # [ 0.379281] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1023lb # [ 0.364366] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x00000000001024client # [ 0.361419] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1025worker2 # [ 0.362469] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1026worker1 # [ 0.379474] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1027client # [ 0.361435] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1028lb # [ 0.364378] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x00100000001029worker2 # [ 0.362500] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1030worker1 # [ 0.379491] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1031client # [ 0.361464] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1032worker1 # [ 0.379520] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1033lb # [ 0.364386] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x80000000001034worker2 # [ 0.362959] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1035client # [ 0.361918] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1036worker1 # [ 0.379982] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1037lb # [ 0.364406] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits1038worker2 # [ 0.382599] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1039worker1 # [ 0.380163] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1040client # [ 0.381562] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1041worker2 # [ 0.382620] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1042lb # [ 0.364430] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1043worker1 # [ 0.380179] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1044client # [ 0.381582] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1045worker2 # [ 0.382651] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1046lb # [ 0.364502] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001047worker1 # [ 0.380208] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1048client # [ 0.381611] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1049lb # [ 0.364512] pci_bus 0000:00: root bus resource [bus 00-ff]1050worker1 # [ 0.380227] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1051lb # [ 0.364517] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1052worker1 # [ 0.380673] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1053lb # [ 0.364523] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1054worker2 # [ 0.385184] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1055lb # [ 0.364527] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1056client # [ 0.384157] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1057worker2 # [ 0.385371] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1058client # [ 0.384344] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1059lb # [ 0.364604] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1060worker2 # [ 0.385388] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1061client # [ 0.384360] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1062worker2 # [ 0.385419] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1063lb # [ 0.365040] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1064client # [ 0.384389] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1065worker2 # [ 0.385438] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1066lb # [ 0.365224] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1067client # [ 0.384407] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1068lb # [ 0.365241] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1069worker2 # [ 0.385889] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1070lb # [ 0.365270] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1071client # [ 0.384863] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1072worker2 # [ 0.386079] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1073lb # [ 0.365286] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1074client # [ 0.385050] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1075worker2 # [ 0.386110] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1076client # [ 0.385079] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077lb # [ 0.365737] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1078worker2 # [ 0.386559] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1079lb # [ 0.365921] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1080client # [ 0.385535] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1081worker2 # [ 0.386746] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1082lb # [ 0.365936] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1083client # [ 0.385721] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1084worker2 # [ 0.386777] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1085worker1 # [ 0.392901] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1086lb # [ 0.365965] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1087client # [ 0.385751] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1088worker1 # [ 0.392941] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1089lb # [ 0.366410] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1090lb # [ 0.366590] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1091worker1 # [ 0.393421] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1092lb # [ 0.366606] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1093worker1 # [ 0.393614] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1094lb # [ 0.366635] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1095worker1 # [ 0.393644] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1096lb # [ 0.367091] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1097worker1 # [ 0.394045] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1098lb # [ 0.367291] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1099worker1 # [ 0.394227] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1100lb # [ 0.367307] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1101worker1 # [ 0.394500] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1102lb # [ 0.367336] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1103worker1 # [ 0.394687] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1104lb # [ 0.367355] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1105worker1 # [ 0.394716] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1106worker2 # [ 0.399257] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1107lb # [ 0.367817] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1108client # [ 0.398202] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1109worker1 # [ 0.395159] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1110worker2 # [ 0.399449] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1111lb # [ 0.368001] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1112client # [ 0.398397] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1113worker1 # [ 0.395345] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1114lb # [ 0.368031] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1115worker2 # [ 0.399700] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1116worker1 # [ 0.395375] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1117client # [ 0.398691] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1118worker2 # [ 0.399887] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1119lb # [ 0.368494] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1120client # [ 0.398880] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1121worker1 # [ 0.395831] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1122worker2 # [ 0.399919] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1123lb # [ 0.368679] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1124worker1 # [ 0.396017] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1125client # [ 0.398909] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1126lb # [ 0.368708] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1127worker2 # [ 0.400366] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1128worker1 # [ 0.396048] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129client # [ 0.399365] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1130worker2 # [ 0.400553] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1131lb # [ 0.369137] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1132client # [ 0.399552] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1133worker1 # [ 0.396505] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1134worker2 # [ 0.400584] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135lb # [ 0.369320] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1136client # [ 0.399582] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137worker2 # [ 0.401033] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1138lb # [ 0.369569] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1139client # [ 0.400034] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1140worker2 # [ 0.401220] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1141lb # [ 0.369756] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1142client # [ 0.400220] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1143worker2 # [ 0.401251] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1144lb # [ 0.369787] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1145client # [ 0.400249] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1146worker2 # [ 0.401711] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1147lb # [ 0.370239] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1148client # [ 0.400713] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1149worker2 # [ 0.401978] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1150lb # [ 0.370423] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1151client # [ 0.400973] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1152worker2 # [ 0.401996] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1153lb # [ 0.370452] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1154client # [ 0.400991] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1155worker2 # [ 0.402032] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1156lb # [ 0.370903] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1157client # [ 0.401020] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1158worker2 # [ 0.402486] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1159lb # [ 0.371089] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1160client # [ 0.401478] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1161worker2 # [ 0.402668] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1162lb # [ 0.371118] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1163client # [ 0.401662] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1164worker2 # [ 0.402685] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1165client # [ 0.401678] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1166worker2 # [ 0.402716] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1167client # [ 0.401707] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1168worker1 # [ 0.416827] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1169worker1 # [ 0.416854] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1170worker1 # [ 0.416884] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1171worker1 # [ 0.417366] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1172worker1 # [ 0.417552] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1173worker1 # [ 0.417567] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1174worker1 # [ 0.417597] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1175worker1 # [ 0.418177] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1176worker1 # [ 0.418188] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1177worker2 # [ 0.423363] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1178worker1 # [ 0.418194] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1179client # [ 0.422369] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1180worker2 # [ 0.423383] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1181worker1 # [ 0.418238] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1182client # [ 0.422390] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1183worker2 # [ 0.423389] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1184worker1 # [ 0.418285] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1185client # [ 0.422396] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1186lb # [ 0.419697] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1187worker2 # [ 0.423436] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1188lb # [ 0.419983] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1189worker1 # [ 0.418332] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1190client # [ 0.422444] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1191worker2 # [ 0.423485] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1192lb # [ 0.420002] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1193worker1 # [ 0.418380] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1194client # [ 0.422492] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1195lb # [ 0.420032] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1196worker2 # [ 0.423533] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1197worker1 # [ 0.418428] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1198client # [ 0.422539] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1199lb # [ 0.420503] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1200worker2 # [ 0.423581] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1201worker1 # [ 0.418475] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1202lb # [ 0.420686] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1203client # [ 0.422587] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1204worker2 # [ 0.423630] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1205lb # [ 0.420702] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1206worker1 # [ 0.418522] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1207client # [ 0.422634] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1208worker2 # [ 0.423678] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1209lb # [ 0.420732] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1210worker1 # [ 0.418569] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1211client # [ 0.422682] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1212lb # [ 0.421305] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1213worker2 # [ 0.423726] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1214worker1 # [ 0.418616] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1215client # [ 0.422729] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1216lb # [ 0.421316] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1217worker2 # [ 0.423773] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1218worker1 # [ 0.418721] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1219lb # [ 0.421322] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1220client # [ 0.422776] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1221worker2 # [ 0.423820] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1222worker1 # [ 0.418768] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1223client # [ 0.422822] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1224lb # [ 0.421366] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1225worker2 # [ 0.423925] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1226worker1 # [ 0.418789] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1227client # [ 0.422929] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1228lb # [ 0.421411] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1229worker2 # [ 0.423973] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1230worker1 # [ 0.418810] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1231client # [ 0.422983] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1232lb # [ 0.421458] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1233worker2 # [ 0.423995] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1234worker1 # [ 0.418832] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1235client # [ 0.423004] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1236worker2 # [ 0.424018] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1237worker1 # [ 0.418853] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1238lb # [ 0.421505] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1239client # [ 0.423026] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1240worker2 # [ 0.424041] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1241worker1 # [ 0.418875] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1242lb # [ 0.421552] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1243client # [ 0.423046] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1244worker2 # [ 0.424063] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1245worker1 # [ 0.418896] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1246lb # [ 0.421599] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1247client # [ 0.423067] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1248worker2 # [ 0.424086] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1249worker1 # [ 0.418922] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1250lb # [ 0.421646] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1251client # [ 0.423088] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1252worker2 # [ 0.424108] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1253worker1 # [ 0.418944] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1254client # [ 0.423109] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1255lb # [ 0.421726] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1256worker2 # [ 0.424136] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1257worker1 # [ 0.418966] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1258client # [ 0.423134] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1259worker2 # [ 0.424159] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1260lb # [ 0.421772] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1261worker1 # [ 0.418988] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1262client # [ 0.423156] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1263worker2 # [ 0.424183] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1264worker1 # [ 0.419011] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1265lb # [ 0.421844] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1266client # [ 0.423178] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1267worker2 # [ 0.424206] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1268worker1 # [ 0.419032] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1269lb # [ 0.421890] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1270client # [ 0.423200] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1271worker2 # [ 0.424230] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1272worker1 # [ 0.419053] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1273lb # [ 0.421912] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1274worker2 # [ 0.424252] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1275client # [ 0.423221] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1276worker1 # [ 0.419073] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1277lb # [ 0.421933] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1278worker1 # [ 0.419095] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1279client # [ 0.423242] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1280worker2 # [ 0.424275] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1281lb # [ 0.421954] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1282worker1 # [ 0.419115] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1283worker2 # [ 0.424297] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1284client # [ 0.423263] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1285lb # [ 0.421975] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1286worker1 # [ 0.419137] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1287worker2 # [ 0.424319] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1288client # [ 0.423284] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1289worker1 # [ 0.419163] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1290lb # [ 0.421997] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1291worker2 # [ 0.424341] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1292client # [ 0.423304] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1293worker1 # [ 0.419172] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1294lb # [ 0.422018] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1295worker2 # [ 0.424363] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1296client # [ 0.423325] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1297worker1 # [ 0.419177] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1298worker2 # [ 0.424390] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1299client # [ 0.423346] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1300lb # [ 0.422043] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1301worker1 # [ 0.419981] pci 0000:00:07.0: enabling device (0000 -> 0002)1302worker2 # [ 0.424401] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1303client # [ 0.423372] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1304lb # [ 0.422065] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1305client # [ 0.423382] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1306worker2 # [ 0.424405] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1307lb # [ 0.422086] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1308worker2 # [ 0.425204] pci 0000:00:07.0: enabling device (0000 -> 0002)1309client # [ 0.423386] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1310lb # [ 0.422109] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1311client # [ 0.424215] pci 0000:00:07.0: enabling device (0000 -> 0002)1312lb # [ 0.422131] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1313lb # [ 0.422152] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1314lb # [ 0.422173] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1315lb # [ 0.422194] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1316lb # [ 0.422215] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1317lb # [ 0.422235] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1318lb # [ 0.422257] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1319lb # [ 0.422283] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1320lb # [ 0.422292] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1321lb # [ 0.422297] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1322lb # [ 0.423112] pci 0000:00:07.0: enabling device (0000 -> 0002)1323lb # [ 0.458242] pci 0000:00:07.0: quirk_usb_early_handoff+0x0/0xa60 took 34312 usecs1324worker1 # [ 0.473931] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1325worker1 # [ 0.476111] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1326worker2 # [ 0.480932] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1327client # [ 0.480080] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1328worker2 # [ 0.482937] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1329worker2 # [ 0.486480] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1330worker1 # [ 0.487052] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1331client # [ 0.483232] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1332client # [ 0.485190] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1333lb # [ 0.481929] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1334worker2 # [ 0.492847] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1335client # [ 0.492014] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1336worker1 # [ 0.492922] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1337worker1 # [ 0.494886] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1338lb # [ 0.488213] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1339lb # [ 0.490145] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1340worker2 # [ 0.500031] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1341worker2 # [ 0.501997] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1342worker1 # [ 0.504900] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1343client # [ 0.503218] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1344worker1 # [ 0.506859] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1345lb # [ 0.500267] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1346worker1 # [ 0.508624] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1347lb # [ 0.502346] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1348worker2 # [ 0.512066] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1349worker2 # [ 0.513938] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1350worker1 # [ 0.518551] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1351worker1 # [ 0.520543] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1352lb # [ 0.512955] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1353lb # [ 0.514850] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1354client # [ 0.514120] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1355client # [ 0.515917] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1356lb # [ 0.518161] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1357client # [ 0.517688] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1358client # [ 0.519492] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1359client # [ 0.521284] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1360worker2 # [ 0.523654] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1361worker1 # [ 0.530607] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1362worker2 # [ 0.526379] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1363worker2 # [ 0.528801] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1364lb # [ 0.524101] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1365lb # [ 0.526802] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1366client # [ 0.535440] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1367worker1 # [ 0.545437] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1368lb # [ 0.538633] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1369client # [ 0.544452] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1370worker1 # [ 0.547992] msm_serial: driver initialized1371worker1 # [ 0.548132] SuperH (H)SCI(F) driver initialized1372worker2 # [ 0.547460] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1373worker1 # [ 0.548186] STM32 USART driver initialized1374worker2 # [ 0.549935] msm_serial: driver initialized1375worker2 # [ 0.550103] SuperH (H)SCI(F) driver initialized1376worker2 # [ 0.550156] STM32 USART driver initialized1377client # [ 0.551198] msm_serial: driver initialized1378client # [ 0.551336] SuperH (H)SCI(F) driver initialized1379client # [ 0.551387] STM32 USART driver initialized1380lb # [ 0.551084] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1381lb # [ 0.553578] msm_serial: driver initialized1382lb # [ 0.553715] SuperH (H)SCI(F) driver initialized1383lb # [ 0.553771] STM32 USART driver initialized1384worker1 # [ 0.579698] loop: module loaded1385worker1 # [ 0.579877] virtio_blk virtio2: 1/0/0 default/read/poll queues1386worker1 # [ 0.580692] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1387worker2 # [ 0.581432] loop: module loaded1388worker2 # [ 0.581625] virtio_blk virtio2: 1/0/0 default/read/poll queues1389worker2 # [ 0.582546] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1390client # [ 0.582531] loop: module loaded1391client # [ 0.582698] virtio_blk virtio2: 1/0/0 default/read/poll queues1392client # [ 0.583434] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1393worker1 # [ 0.585309] megasas: 07.734.00.00-rc11394worker1 # [ 0.586017] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1395worker2 # [ 0.587553] megasas: 07.734.00.00-rc11396worker2 # [ 0.588219] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1397lb # [ 0.585913] loop: module loaded1398lb # [ 0.586100] virtio_blk virtio2: 1/0/0 default/read/poll queues1399lb # [ 0.586878] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1400worker1 # [ 0.596793] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001401worker1 # [ 0.596818] Intel/Sharp Extended Query Table at 0x00311402worker1 # [ 0.599606] Using buffer write method1403client # [ 0.594534] megasas: 07.734.00.00-rc11404worker1 # [ 0.599686] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1405worker2 # [ 0.599658] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001406client # [ 0.595287] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1407worker2 # [ 0.599724] Intel/Sharp Extended Query Table at 0x00311408worker1 # [ 0.601346] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001409worker2 # [ 0.601264] Using buffer write method1410client # [ 0.597510] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001411worker1 # [ 0.601367] Intel/Sharp Extended Query Table at 0x00311412client # [ 0.597537] Intel/Sharp Extended Query Table at 0x00311413worker2 # [ 0.601326] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1414worker1 # [ 0.602953] Using buffer write method1415client # [ 0.599120] Using buffer write method1416worker1 # [ 0.602976] Concatenating MTD devices:1417client # [ 0.599192] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1418worker1 # [ 0.602980] (0): "0.flash"1419worker1 # [ 0.602984] (1): "0.flash"1420worker1 # [ 0.602994] into device "0.flash"1421worker2 # [ 0.607539] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001422client # [ 0.600862] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001423worker2 # [ 0.607564] Intel/Sharp Extended Query Table at 0x00311424client # [ 0.600884] Intel/Sharp Extended Query Table at 0x00311425worker2 # [ 0.608801] Using buffer write method1426worker2 # [ 0.608826] Concatenating MTD devices:1427worker2 # [ 0.608830] (0): "0.flash"1428worker2 # [ 0.608833] (1): "0.flash"1429worker2 # [ 0.608844] into device "0.flash"1430lb # [ 0.599771] megasas: 07.734.00.00-rc11431lb # [ 0.600555] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1432lb # [ 0.602624] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001433lb # [ 0.602648] Intel/Sharp Extended Query Table at 0x00311434lb # [ 0.604149] Using buffer write method1435lb # [ 0.604213] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1436lb # [ 0.605834] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001437lb # [ 0.605856] Intel/Sharp Extended Query Table at 0x00311438lb # [ 0.615297] Using buffer write method1439client # [ 0.619711] Using buffer write method1440lb # [ 0.615321] Concatenating MTD devices:1441client # [ 0.619744] Concatenating MTD devices:1442lb # [ 0.615325] (0): "0.flash"1443client # [ 0.619748] (0): "0.flash"1444lb # [ 0.615334] (1): "0.flash"1445client # [ 0.619752] (1): "0.flash"1446lb # [ 0.615338] into device "0.flash"1447client # [ 0.619761] into device "0.flash"1448worker2 # [ 0.843133] Freeing initrd memory: 26944K1449worker1 # [ 0.846182] Freeing initrd memory: 26976K1450worker2 # [ 0.849128] tun: Universal TUN/TAP device driver, 1.61451client # [ 0.848686] Freeing initrd memory: 26972K1452worker1 # [ 0.852262] tun: Universal TUN/TAP device driver, 1.61453worker1 # [ 0.855892] thunder_xcv, ver 1.01454worker1 # [ 0.855936] thunder_bgx, ver 1.01455worker1 # [ 0.855958] nicpf, ver 1.01456worker2 # [ 0.852655] thunder_xcv, ver 1.01457worker2 # [ 0.852697] thunder_bgx, ver 1.01458client # [ 0.854611] tun: Universal TUN/TAP device driver, 1.61459worker1 # [ 0.856521] e1000: Intel(R) PRO/1000 Network Driver1460worker2 # [ 0.852718] nicpf, ver 1.01461worker1 # [ 0.856527] e1000: Copyright (c) 1999-2006 Intel Corporation.1462worker2 # [ 0.853255] e1000: Intel(R) PRO/1000 Network Driver1463worker1 # [ 0.856550] e1000e: Intel(R) PRO/1000 Network Driver1464worker2 # [ 0.853263] e1000: Copyright (c) 1999-2006 Intel Corporation.1465worker1 # [ 0.856558] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1466worker2 # [ 0.853287] e1000e: Intel(R) PRO/1000 Network Driver1467worker1 # [ 0.856585] igb: Intel(R) Gigabit Ethernet Network Driver1468worker2 # [ 0.853295] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1469lb # [ 0.851971] Freeing initrd memory: 26948K1470worker1 # [ 0.856590] igb: Copyright (c) 2007-2014 Intel Corporation.1471worker2 # [ 0.853323] igb: Intel(R) Gigabit Ethernet Network Driver1472worker2 # [ 0.853329] igb: Copyright (c) 2007-2014 Intel Corporation.1473worker1 # [ 0.856613] igbvf: Intel(R) Gigabit Virtual Function Network Driver1474worker2 # [ 0.853370] igbvf: Intel(R) Gigabit Virtual Function Network Driver1475worker1 # [ 0.856618] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1476worker2 # [ 0.853376] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1477worker2 # [ 0.853507] sky2: driver version 1.301478worker2 # [ 0.855706] ehci-pci 0000:00:07.0: EHCI Host Controller1479client # [ 0.858279] thunder_xcv, ver 1.01480lb # [ 0.857871] tun: Universal TUN/TAP device driver, 1.61481client # [ 0.858317] thunder_bgx, ver 1.01482worker2 # [ 0.855730] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11483worker1 # [ 0.863491] sky2: driver version 1.301484client # [ 0.858340] nicpf, ver 1.01485worker2 # [ 0.856005] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001486worker1 # [ 0.865367] ehci-pci 0000:00:07.0: EHCI Host Controller1487client # [ 0.858895] e1000: Intel(R) PRO/1000 Network Driver1488client # [ 0.858902] e1000: Copyright (c) 1999-2006 Intel Corporation.1489worker1 # [ 0.865394] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11490client # [ 0.858926] e1000e: Intel(R) PRO/1000 Network Driver1491worker1 # [ 0.865588] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001492client # [ 0.858933] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1493worker2 # [ 0.865340] usbcore: registered new interface driver usb-storage1494worker1 # [ 0.868669] usbcore: registered new interface driver usb-storage1495client # [ 0.858962] igb: Intel(R) Gigabit Ethernet Network Driver1496worker2 # [ 0.865428] usbcore: registered new interface driver usbserial_generic1497client # [ 0.858968] igb: Copyright (c) 2007-2014 Intel Corporation.1498worker2 # [ 0.865442] usbserial: USB Serial support registered for generic1499lb # [ 0.861422] thunder_xcv, ver 1.01500client # [ 0.858992] igbvf: Intel(R) Gigabit Virtual Function Network Driver1501worker2 # [ 0.866100] hv_vmbus: registering driver hyperv_keyboard1502lb # [ 0.861459] thunder_bgx, ver 1.01503worker1 # [ 0.869524] usbcore: registered new interface driver usbserial_generic1504client # [ 0.858997] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1505lb # [ 0.861481] nicpf, ver 1.01506client # [ 0.859144] sky2: driver version 1.301507lb # [ 0.862056] e1000: Intel(R) PRO/1000 Network Driver1508worker1 # [ 0.869541] usbserial: USB Serial support registered for generic1509worker2 # [ 0.869657] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001510client # [ 0.860699] usbcore: registered new interface driver usb-storage1511worker2 # [ 0.869934] hub 1-0:1.0: USB hub found1512worker1 # [ 0.870147] hv_vmbus: registering driver hyperv_keyboard1513lb # [ 0.862063] e1000: Copyright (c) 1999-2006 Intel Corporation.1514worker2 # [ 0.869953] hub 1-0:1.0: 6 ports detected1515lb # [ 0.862087] e1000e: Intel(R) PRO/1000 Network Driver1516client # [ 0.860783] usbcore: registered new interface driver usbserial_generic1517lb # [ 0.862095] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1518client # [ 0.860797] usbserial: USB Serial support registered for generic1519lb # [ 0.862124] igb: Intel(R) Gigabit Ethernet Network Driver1520worker1 # [ 0.873205] rtc-pl031 9010000.pl031: registered as rtc01521client # [ 0.861375] hv_vmbus: registering driver hyperv_keyboard1522worker2 # [ 0.872179] rtc-pl031 9010000.pl031: registered as rtc01523lb # [ 0.862129] igb: Copyright (c) 2007-2014 Intel Corporation.1524worker1 # [ 0.873231] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T22:20:13 UTC (1790115613)1525lb # [ 0.862156] igbvf: Intel(R) Gigabit Virtual Function Network Driver1526worker2 # [ 0.872205] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T22:20:13 UTC (1790115613)1527worker1 # [ 0.873534] i2c_dev: i2c /dev entries driver1528client # [ 0.872635] ehci-pci 0000:00:07.0: EHCI Host Controller1529worker2 # [ 0.872513] i2c_dev: i2c /dev entries driver1530lb # [ 0.862162] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1531lb # [ 0.862293] sky2: driver version 1.301532client # [ 0.872665] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11533client # [ 0.872916] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001534lb # [ 0.870752] usbcore: registered new interface driver usb-storage1535lb # [ 0.870874] usbcore: registered new interface driver usbserial_generic1536client # [ 0.875535] rtc-pl031 9010000.pl031: registered as rtc01537lb # [ 0.870887] usbserial: USB Serial support registered for generic1538worker2 # [ 0.877606] sdhci: Secure Digital Host Controller Interface driver1539client # [ 0.875565] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T22:20:13 UTC (1790115613)1540worker1 # [ 0.878865] sdhci: Secure Digital Host Controller Interface driver1541worker2 # [ 0.877616] sdhci: Copyright(c) Pierre Ossman1542client # [ 0.875890] i2c_dev: i2c /dev entries driver1543worker1 # [ 0.878876] sdhci: Copyright(c) Pierre Ossman1544worker2 # [ 0.877866] Synopsys Designware Multimedia Card Interface Driver1545lb # [ 0.874141] ehci-pci 0000:00:07.0: EHCI Host Controller1546worker1 # [ 0.879147] Synopsys Designware Multimedia Card Interface Driver1547worker2 # [ 0.878235] sdhci-pltfm: SDHCI platform and OF driver helper1548lb # [ 0.874169] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11549worker1 # [ 0.879535] sdhci-pltfm: SDHCI platform and OF driver helper1550lb # [ 0.874455] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001551worker1 # [ 0.883461] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001552worker2 # [ 0.882360] hid: raw HID events driver (C) Jiri Kosina1553worker1 # [ 0.883750] hub 1-0:1.0: USB hub found1554lb # [ 0.877138] hv_vmbus: registering driver hyperv_keyboard1555worker1 # [ 0.883780] hub 1-0:1.0: 6 ports detected1556worker2 # [ 0.882612] usbcore: registered new interface driver usbhid1557client # [ 0.881148] sdhci: Secure Digital Host Controller Interface driver1558worker2 # [ 0.882618] usbhid: USB HID core driver1559client # [ 0.881158] sdhci: Copyright(c) Pierre Ossman1560client # [ 0.881422] Synopsys Designware Multimedia Card Interface Driver1561lb # [ 0.878596] rtc-pl031 9010000.pl031: registered as rtc01562client # [ 0.881781] sdhci-pltfm: SDHCI platform and OF driver helper1563worker1 # [ 0.886099] hid: raw HID events driver (C) Jiri Kosina1564lb # [ 0.878623] rtc-pl031 9010000.pl031: setting system clock to 2026-09-22T22:20:13 UTC (1790115613)1565client # [ 0.884625] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001566worker1 # [ 0.886350] usbcore: registered new interface driver usbhid1567lb # [ 0.878921] i2c_dev: i2c /dev entries driver1568worker1 # [ 0.886356] usbhid: USB HID core driver1569client # [ 0.885642] hub 1-0:1.0: USB hub found1570worker2 # [ 0.885439] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1571client # [ 0.886146] hub 1-0:1.0: 6 ports detected1572worker2 # [ 0.886919] drop_monitor: Initializing network drop monitor service1573worker2 # [ 0.888854] NET: Registered PF_INET6 protocol family1574worker2 # [ 0.890043] Segment Routing with IPv61575worker1 # [ 0.889271] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1576worker2 # [ 0.890064] In-situ OAM (IOAM) with IPv61577worker1 # [ 0.890790] drop_monitor: Initializing network drop monitor service1578worker2 # [ 0.890095] NET: Registered PF_PACKET protocol family1579client # [ 0.888380] hid: raw HID events driver (C) Jiri Kosina1580worker1 # [ 0.890926] NET: Registered PF_INET6 protocol family1581client # [ 0.888641] usbcore: registered new interface driver usbhid1582client # [ 0.888647] usbhid: USB HID core driver1583worker1 # [ 0.894027] Segment Routing with IPv61584worker1 # [ 0.894061] In-situ OAM (IOAM) with IPv61585lb # [ 0.884167] sdhci: Secure Digital Host Controller Interface driver1586worker1 # [ 0.894104] NET: Registered PF_PACKET protocol family1587lb # [ 0.884177] sdhci: Copyright(c) Pierre Ossman1588lb # [ 0.884442] Synopsys Designware Multimedia Card Interface Driver1589worker2 # [ 0.891917] 9pnet: Installing 9P2000 support1590lb # [ 0.884811] sdhci-pltfm: SDHCI platform and OF driver helper1591worker1 # [ 0.895788] 9pnet: Installing 9P2000 support1592client # [ 0.891535] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1593worker2 # [ 0.891962] Key type dns_resolver registered1594worker1 # [ 0.895833] Key type dns_resolver registered1595lb # [ 0.886288] hid: raw HID events driver (C) Jiri Kosina1596client # [ 0.893057] drop_monitor: Initializing network drop monitor service1597lb # [ 0.886524] usbcore: registered new interface driver usbhid1598client # [ 0.893182] NET: Registered PF_INET6 protocol family1599lb # [ 0.886530] usbhid: USB HID core driver1600client # [ 0.896184] Segment Routing with IPv61601lb # [ 0.891675] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001602client # [ 0.896204] In-situ OAM (IOAM) with IPv61603lb # [ 0.891984] hub 1-0:1.0: USB hub found1604client # [ 0.896230] NET: Registered PF_PACKET protocol family1605worker2 # [ 0.898501] registered taskstats version 11606lb # [ 0.892003] hub 1-0:1.0: 6 ports detected1607client # [ 0.897864] 9pnet: Installing 9P2000 support1608worker2 # [ 0.898636] Loading compiled-in X.509 certificates1609client # [ 0.897914] Key type dns_resolver registered1610lb # [ 0.894376] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1611lb # [ 0.896979] drop_monitor: Initializing network drop monitor service1612worker1 # [ 0.902451] registered taskstats version 11613lb # [ 0.897152] NET: Registered PF_INET6 protocol family1614worker1 # [ 0.902598] Loading compiled-in X.509 certificates1615lb # [ 0.899116] Segment Routing with IPv61616lb # [ 0.899135] In-situ OAM (IOAM) with IPv61617lb # [ 0.899174] NET: Registered PF_PACKET protocol family1618client # [ 0.904468] registered taskstats version 11619worker2 # [ 0.907184] Demotion targets for Node 0: null1620worker2 # [ 0.907282] Key type .fscrypt registered1621client # [ 0.904620] Loading compiled-in X.509 certificates1622worker2 # [ 0.907288] Key type fscrypt-provisioning registered1623lb # [ 0.900819] 9pnet: Installing 9P2000 support1624worker2 # [ 0.907372] ima: No TPM chip found, activating TPM-bypass!1625lb # [ 0.900868] Key type dns_resolver registered1626worker2 # [ 0.907391] ima: Allocated hash algorithm: sha11627worker2 # [ 0.907417] ima: No architecture policies found1628worker1 # [ 0.911462] Demotion targets for Node 0: null1629worker1 # [ 0.911557] Key type .fscrypt registered1630worker1 # [ 0.911564] Key type fscrypt-provisioning registered1631worker2 # [ 0.911490] input: gpio-keys as /devices/platform/gpio-keys/input/input01632worker1 # [ 0.911654] ima: No TPM chip found, activating TPM-bypass!1633worker1 # [ 0.911673] ima: Allocated hash algorithm: sha11634worker1 # [ 0.911694] ima: No architecture policies found1635worker1 # [ 0.915692] input: gpio-keys as /devices/platform/gpio-keys/input/input01636client # [ 0.913242] Demotion targets for Node 0: null1637client # [ 0.913355] Key type .fscrypt registered1638client # [ 0.913363] Key type fscrypt-provisioning registered1639client # [ 0.913487] ima: No TPM chip found, activating TPM-bypass!1640lb # [ 0.907512] registered taskstats version 11641client # [ 0.913507] ima: Allocated hash algorithm: sha11642lb # [ 0.907656] Loading compiled-in X.509 certificates1643client # [ 0.913527] ima: No architecture policies found1644client # [ 0.917623] input: gpio-keys as /devices/platform/gpio-keys/input/input01645lb # [ 0.916274] Demotion targets for Node 0: null1646lb # [ 0.916382] Key type .fscrypt registered1647lb # [ 0.916388] Key type fscrypt-provisioning registered1648lb # [ 0.916489] ima: No TPM chip found, activating TPM-bypass!1649lb # [ 0.916508] ima: Allocated hash algorithm: sha11650lb # [ 0.916528] ima: No architecture policies found1651lb # [ 0.920442] input: gpio-keys as /devices/platform/gpio-keys/input/input01652worker2 # [ 0.929058] clk: Disabling unused clocks1653worker2 # [ 0.929080] PM: genpd: Disabling unused power domains1654worker2 # [ 0.933153] Freeing unused kernel memory: 4736K1655worker2 # [ 0.933338] Run /init as init process1656worker1 # [ 0.933560] clk: Disabling unused clocks1657worker1 # [ 0.933582] PM: genpd: Disabling unused power domains1658worker1 # [ 0.937672] Freeing unused kernel memory: 4736K1659worker1 # [ 0.937884] Run /init as init process1660client # [ 0.935500] clk: Disabling unused clocks1661client # [ 0.935524] PM: genpd: Disabling unused power domains1662client # [ 0.939628] Freeing unused kernel memory: 4736K1663client # [ 0.939835] Run /init as init process1664lb # [ 0.938832] clk: Disabling unused clocks1665lb # [ 0.938856] PM: genpd: Disabling unused power domains1666lb # [ 0.942984] Freeing unused kernel memory: 4736K1667worker2 # [ 0.949153] systemd[1]: Successfully made /usr/ read-only.1668lb # [ 0.943181] Run /init as init process1669worker1 # [ 0.954912] systemd[1]: Successfully made /usr/ read-only.1670client # [ 0.956112] systemd[1]: Successfully made /usr/ read-only.1671lb # [ 0.957376] systemd[1]: Successfully made /usr/ read-only.1672worker2 # [ 1.115083] usb 1-1: new high-speed USB device number 2 using ehci-pci1673worker1 # [ 1.128821] usb 1-1: new high-speed USB device number 2 using ehci-pci1674client # [ 1.134022] usb 1-1: new high-speed USB device number 2 using ehci-pci1675lb # [ 1.139267] usb 1-1: new high-speed USB device number 2 using ehci-pci1676worker2 # [ 1.267592] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677worker1 # [ 1.281269] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11678client # [ 1.286426] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11679worker2 # [ 1.283977] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1680worker2 # [ 1.296142] systemd[1]: Detected virtualization qemu.1681worker2 # [ 1.298152] systemd[1]: Detected architecture arm64.1682worker1 # [ 1.289718] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1683worker2 # [ 1.300133] systemd[1]: Running in initrd.1684worker1 # [ 1.302169] systemd[1]: Detected virtualization qemu.1685lb # [ 1.291637] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11686worker1 # [ 1.304286] systemd[1]: Detected architecture arm64.1687worker2 # [ 1.302707] systemd[1]: Initializing machine ID from random generator.1688client # [ 1.292369] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1689worker1 # [ 1.306282] systemd[1]: Running in initrd.1690worker2 # [ 1.305697] systemd[1]: Hostname set to <worker2>.1691client # [ 1.304824] systemd[1]: Detected virtualization qemu.1692worker1 # [ 1.308916] systemd[1]: Initializing machine ID from random generator.1693client # [ 1.306980] systemd[1]: Detected architecture arm64.1694client # [ 1.308887] systemd[1]: Running in initrd.1695worker1 # [ 1.311788] systemd[1]: Hostname set to <worker1>.1696client # [ 1.311481] systemd[1]: Initializing machine ID from random generator.1697lb # [ 1.297521] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1698client # [ 1.314367] systemd[1]: Hostname set to <client>.1699lb # [ 1.297579] systemd[1]: Detected virtualization qemu.1700lb # [ 1.297650] systemd[1]: Detected architecture arm64.1701lb # [ 1.297673] systemd[1]: Running in initrd.1702lb # [ 1.298558] systemd[1]: Initializing machine ID from random generator.1703lb # [ 1.298825] systemd[1]: Hostname set to <lb>.1704worker2 # [ 1.351309] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705worker1 # [ 1.369010] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706client # [ 1.370197] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707lb # [ 1.375436] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01708worker2 # [ 1.471095] usb 1-2: new high-speed USB device number 3 using ehci-pci1709worker1 # [ 1.488794] usb 1-2: new high-speed USB device number 3 using ehci-pci1710client # [ 1.489993] usb 1-2: new high-speed USB device number 3 using ehci-pci1711lb # [ 1.499234] usb 1-2: new high-speed USB device number 3 using ehci-pci1712worker1 # [ 1.607788] systemd[1]: bpf-restrict-fs: LSM BPF program attached1713worker2 # [ 1.607603] systemd[1]: bpf-restrict-fs: LSM BPF program attached1714client # [ 1.618792] systemd[1]: bpf-restrict-fs: LSM BPF program attached1715lb # [ 1.623158] systemd[1]: bpf-restrict-fs: LSM BPF program attached1716worker2 # [ 1.630152] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21717worker2 # [ 1.636513] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01718worker1 # [ 1.641557] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21719worker1 # [ 1.646183] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01720client # [ 1.648674] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21721client # [ 1.654439] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01722lb # [ 1.658766] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21723lb # [ 1.664051] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01724worker1 # [ 1.717417] systemd[1]: Queued start job for default target Initrd Default Target.1725worker2 # [ 1.721726] systemd[1]: Queued start job for default target Initrd Default Target.1726worker1 # [ 1.727122] systemd[1]: Created slice Slice /system/modprobe.1727worker1 # [ 1.728403] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1728worker1 # [ 1.729808] systemd[1]: Expecting device /dev/disk/by-label/nixos...1729worker1 # [ 1.730867] systemd[1]: Reached target Path Units.1730worker1 # [ 1.731686] systemd[1]: Reached target Slice Units.1731worker1 # [ 1.732509] systemd[1]: Reached target Swaps.1732worker1 # [ 1.733312] systemd[1]: Reached target Timer Units.1733worker1 # [ 1.734303] systemd[1]: Listening on D-Bus System Message Bus Socket.1734client # [ 1.731877] systemd[1]: Queued start job for default target Initrd Default Target.1735worker1 # [ 1.735547] systemd[1]: Listening on Journal Socket (/dev/log).1736worker2 # [ 1.734793] systemd[1]: Created slice Slice /system/modprobe.1737worker1 # [ 1.736695] systemd[1]: Listening on Journal Sockets.1738worker2 # [ 1.736100] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1739worker1 # [ 1.737766] systemd[1]: Listening on udev Control Socket.1740worker2 # [ 1.737549] systemd[1]: Expecting device /dev/disk/by-label/nixos...1741worker1 # [ 1.737885] systemd[1]: Listening on udev Kernel Socket.1742worker2 # [ 1.738668] systemd[1]: Reached target Path Units.1743worker1 # [ 1.737909] systemd[1]: Reached target Socket Units.1744worker2 # [ 1.739581] systemd[1]: Reached target Slice Units.1745worker2 # [ 1.740444] systemd[1]: Reached target Swaps.1746worker1 # [ 1.741995] systemd[1]: Starting Create List of Static Device Nodes...1747worker2 # [ 1.741231] systemd[1]: Reached target Timer Units.1748client # [ 1.740612] systemd[1]: Created slice Slice /system/modprobe.1749worker2 # [ 1.742291] systemd[1]: Listening on D-Bus System Message Bus Socket.1750worker1 # [ 1.743148] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1751client # [ 1.742065] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1752worker2 # [ 1.743617] systemd[1]: Listening on Journal Socket (/dev/log).1753client # [ 1.743723] systemd[1]: Expecting device /dev/disk/by-label/nixos...1754worker2 # [ 1.743772] systemd[1]: Listening on Journal Sockets.1755client # [ 1.745004] systemd[1]: Reached target Path Units.1756lb # [ 1.740454] systemd[1]: Queued start job for default target Initrd Default Target.1757worker2 # [ 1.743912] systemd[1]: Listening on udev Control Socket.1758client # [ 1.746019] systemd[1]: Reached target Slice Units.1759worker2 # [ 1.744028] systemd[1]: Listening on udev Kernel Socket.1760client # [ 1.747013] systemd[1]: Reached target Swaps.1761worker2 # [ 1.744052] systemd[1]: Reached target Socket Units.1762client # [ 1.747912] systemd[1]: Reached target Timer Units.1763worker2 # [ 1.750082] systemd[1]: Starting Create List of Static Device Nodes...1764client # [ 1.749111] systemd[1]: Listening on D-Bus System Message Bus Socket.1765worker1 # [ 1.752896] systemd[1]: Mounting Kernel Configuration File System...1766worker2 # [ 1.751300] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1767client # [ 1.750645] systemd[1]: Listening on Journal Socket (/dev/log).1768client # [ 1.750823] systemd[1]: Listening on Journal Sockets.1769lb # [ 1.749615] systemd[1]: Created slice Slice /system/modprobe.1770client # [ 1.750980] systemd[1]: Listening on udev Control Socket.1771lb # [ 1.750955] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1772client # [ 1.751109] systemd[1]: Listening on udev Kernel Socket.1773client # [ 1.751134] systemd[1]: Reached target Socket Units.1774lb # [ 1.752450] systemd[1]: Expecting device /dev/disk/by-label/nixos...1775lb # [ 1.753550] systemd[1]: Reached target Path Units.1776worker1 # [ 1.761021] systemd[1]: Starting Journal Service...1777client # [ 1.758101] systemd[1]: Starting Create List of Static Device Nodes...1778lb # [ 1.754387] systemd[1]: Reached target Slice Units.1779lb # [ 1.755274] systemd[1]: Reached target Swaps.1780lb # [ 1.756044] systemd[1]: Reached target Timer Units.1781client # [ 1.759457] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1782worker2 # [ 1.762393] systemd[1]: Mounting Kernel Configuration File System...1783lb # [ 1.757213] systemd[1]: Listening on D-Bus System Message Bus Socket.1784lb # [ 1.758463] systemd[1]: Listening on Journal Socket (/dev/log).1785lb # [ 1.759673] systemd[1]: Listening on Journal Sockets.1786lb # [ 1.759818] systemd[1]: Listening on udev Control Socket.1787lb # [ 1.759938] systemd[1]: Listening on udev Kernel Socket.1788lb # [ 1.759963] systemd[1]: Reached target Socket Units.1789lb # [ 1.765186] systemd[1]: Starting Create List of Static Device Nodes...1790client # [ 1.770135] systemd[1]: Mounting Kernel Configuration File System...1791lb # [ 1.766377] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1792worker1 # [ 1.777166] systemd[1]: Starting Load Kernel Modules...1793worker1 # [ 1.778704] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1794client # [ 1.778237] systemd[1]: Starting Journal Service...1795lb # [ 1.775361] systemd[1]: Mounting Kernel Configuration File System...1796lb # [ 1.783501] systemd[1]: Starting Journal Service...1797worker2 # [ 1.791938] systemd[1]: Starting Journal Service...1798worker2 # [ 1.794663] systemd[1]: Starting Load Kernel Modules...1799worker2 # [ 1.795872] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1800client # [ 1.802527] systemd[1]: Starting Load Kernel Modules...1801worker2 # [ 1.805391] systemd[1]: Starting Coldplug All udev Devices...1802client # [ 1.803564] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1803worker1 # [ 1.808961] systemd[1]: Starting Coldplug All udev Devices...1804worker1 # [ 1.815023] systemd[1]: Finished Create List of Static Device Nodes.1805lb # [ 1.807459] systemd[1]: Starting Load Kernel Modules...1806worker1 # [ 1.816524] systemd[1]: Mounted Kernel Configuration File System.1807lb # [ 1.807573] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1808worker2 # [ 1.817300] systemd[1]: Finished Create List of Static Device Nodes.1809worker2 # [ 1.819798] systemd[1]: Mounted Kernel Configuration File System.1810worker1 # [ 1.822181] systemd-journald[72]: Collecting audit messages is disabled.1811worker1 # [ 1.823339] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1812client # [ 1.823927] systemd-journald[72]: Collecting audit messages is disabled.1813lb # [ 1.823357] systemd[1]: Starting Coldplug All udev Devices...1814worker2 # [ 1.828887] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1815client # [ 1.830650] systemd[1]: Starting Coldplug All udev Devices...1816lb # [ 1.837080] systemd-journald[72]: Collecting audit messages is disabled.1817lb # [ 1.840464] systemd[1]: Finished Create List of Static Device Nodes.1818lb # [ 1.841946] systemd[1]: Mounted Kernel Configuration File System.1819client # [ 1.850056] systemd[1]: Finished Create List of Static Device Nodes.1820client # [ 1.850799] systemd[1]: Mounted Kernel Configuration File System.1821client # [ 1.859624] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1822worker2 # [ 1.858678] systemd-journald[72]: Collecting audit messages is disabled.1823lb # [ 1.863362] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1824worker1 # [ 1.869786] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1825worker1 # [ 1.870593] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1826worker1 # [ 1.874544] systemd[1]: Starting Create Static Device Nodes in /dev...1827client # [ 1.875841] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1828worker2 # [ 1.879683] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1829lb # [ 1.869636] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1830worker2 # [ 1.882140] systemd[1]: Starting Create Static Device Nodes in /dev...1831worker1 # [ 1.886087] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1832client # [ 1.890000] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1833worker2 # [ 1.895867] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1834lb # [ 1.890305] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1835worker1 # [ 1.897858] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01836worker1 # [ 1.898095] [drm] features: -virgl +edid -resource_blob -host_visible1837lb # [ 1.892767] systemd[1]: Starting Create Static Device Nodes in /dev...1838worker1 # [ 1.898105] [drm] features: -context_init1839worker2 # [ 1.899088] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1840client # [ 1.899434] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01841client # [ 1.899668] [drm] features: -virgl +edid -resource_blob -host_visible1842worker1 # [ 1.898805] [drm] number of scanouts: 11843client # [ 1.899679] [drm] features: -context_init1844worker1 # [ 1.898822] [drm] number of cap sets: 01845client # [ 1.900356] [drm] number of scanouts: 11846client # [ 1.900373] [drm] number of cap sets: 01847worker2 # [ 1.912129] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01848worker2 # [ 1.912368] [drm] features: -virgl +edid -resource_blob -host_visible1849worker2 # [ 1.912378] [drm] features: -context_init1850worker2 # [ 1.913142] [drm] number of scanouts: 11851worker2 # [ 1.913162] [drm] number of cap sets: 01852lb # [ 1.904979] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1853lb # [ 1.911086] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01854worker1 # [ 1.921352] systemd[1]: Finished Create Static Device Nodes in /dev.1855worker1 # [ 1.921665] systemd[1]: Reached target Preparation for Local File Systems.1856worker1 # [ 1.921692] systemd[1]: Reached target Local File Systems.1857client # [ 1.922574] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1858client # [ 1.924935] systemd[1]: Starting Create Static Device Nodes in /dev...1859worker1 # [ 1.925536] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1860worker1 # [ 1.925549] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01861worker2 # [ 1.931624] systemd[1]: Finished Create Static Device Nodes in /dev.1862worker2 # [ 1.931948] systemd[1]: Reached target Preparation for Local File Systems.1863lb # [ 1.927417] [drm] features: -virgl +edid -resource_blob -host_visible1864worker2 # [ 1.931974] systemd[1]: Reached target Local File Systems.1865lb # [ 1.927435] [drm] features: -context_init1866lb # [ 1.928177] [drm] number of scanouts: 11867worker1 # [ 1.933605] systemd[1]: Starting Rule-based Manager for Device Events and Files...1868client # [ 1.929383] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1869lb # [ 1.928196] [drm] number of cap sets: 01870client # [ 1.929402] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01871worker2 # [ 1.939318] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1872worker2 # [ 1.939331] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01873lb # [ 1.938049] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1874lb # [ 1.938069] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01875worker2 # [ 1.947454] systemd[1]: Starting Rule-based Manager for Device Events and Files...1876worker1 # [ 1.949230] Console: switching to colour frame buffer device 160x501877lb # [ 1.947791] systemd[1]: Finished Create Static Device Nodes in /dev.1878lb # [ 1.948112] systemd[1]: Reached target Preparation for Local File Systems.1879lb # [ 1.948152] systemd[1]: Reached target Local File Systems.1880lb # [ 1.951974] systemd[1]: Starting Rule-based Manager for Device Events and Files...1881client # [ 1.954242] Console: switching to colour frame buffer device 160x501882client # [ 1.960682] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1883worker2 # [ 1.963293] Console: switching to colour frame buffer device 160x501884lb # [ 1.964115] Console: switching to colour frame buffer device 160x501885client # [ 1.974599] systemd[1]: Finished Create Static Device Nodes in /dev.1886client # [ 1.974942] systemd[1]: Reached target Preparation for Local File Systems.1887client # [ 1.974976] systemd[1]: Reached target Local File Systems.1888worker1 # [ 1.981338] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1889client # [ 1.982348] systemd[1]: Starting Rule-based Manager for Device Events and Files...1890worker2 # [ 1.987618] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1891lb # [ 1.987791] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1892worker1 # [ 1.998172] systemd[1]: Finished Load Kernel Modules.1893worker1 # [ 2.001206] systemd[1]: Starting Apply Kernel Variables...1894worker2 # [ 2.000340] systemd[1]: Finished Load Kernel Modules.1895client # [ 1.998563] systemd[1]: Finished Load Kernel Modules.1896lb # [ 2.000634] systemd[1]: Finished Load Kernel Modules.1897worker2 # [ 2.007445] systemd[1]: Starting Apply Kernel Variables...1898client # [ 2.006434] systemd[1]: Starting Apply Kernel Variables...1899worker1 # [ 2.000913] systemd-modules-load[73]: Inserted module 'dm_mod'1900worker1 # [ 2.014716] systemd[1]: Started Journal Service.1901worker1 # [ 2.004801] systemd-modules-load[73]: Module 'virtio_balloon' is built in1902worker1 # [ 2.005894] systemd-modules-load[73]: Module 'virtio_console' is built in1903worker1 # [ 2.006957] systemd-modules-load[73]: Inserted module 'virtio_gpu'1904worker1 # [ 2.007921] systemd-modules-load[73]: Module 'virtio_rng' is built in1905lb # [ 2.011805] systemd[1]: Starting Apply Kernel Variables...1906client # [ 2.007096] systemd-modules-load[74]: Inserted module 'dm_mod'1907client # [ 2.008225] systemd-modules-load[74]: Module 'virtio_balloon' is built in1908lb # [ 2.002686] systemd-modules-load[74]: Inserted module 'dm_mod'1909lb # [ 2.003764] systemd-modules-load[74]: Module 'virtio_balloon' is built in1910client # [ 2.009332] systemd-modules-load[74]: Module 'virtio_console' is built in1911client # [ 2.010403] systemd-modules-load[74]: Inserted module 'virtio_gpu'1912client # [ 2.011410] systemd-modules-load[74]: Module 'virtio_rng' is built in1913client # [ 2.026467] systemd[1]: Started Journal Service.1914lb # [ 2.012337] systemd-modules-load[74]: Module 'virtio_console' is built in1915lb # [ 2.013432] systemd-modules-load[74]: Inserted module 'virtio_gpu'1916lb # [ 2.014388] systemd-modules-load[74]: Module 'virtio_rng' is built in1917worker1 # [ 2.024596] systemd-udevd[79]: Using default interface naming scheme 'v261'.1918lb # [ 2.031061] systemd[1]: Started Journal Service.1919worker1 # [ 2.032166] systemd[1]: Starting Create System Files and Directories...1920worker2 # [ 2.029004] systemd-modules-load[73]: Inserted module 'dm_mod'1921worker2 # [ 2.045088] systemd[1]: Started Journal Service.1922worker2 # [ 2.044197] systemd-modules-load[73]: Module 'virtio_balloon' is built in1923client # [ 2.044097] systemd[1]: Starting Create System Files and Directories...1924worker2 # [ 2.045348] systemd-modules-load[73]: Module 'virtio_console' is built in1925worker2 # [ 2.046426] systemd-modules-load[73]: Inserted module 'virtio_gpu'1926worker2 # [ 2.047444] systemd-modules-load[73]: Module 'virtio_rng' is built in1927worker1 # [ 2.056795] systemd[1]: Finished Apply Kernel Variables.1928lb # [ 2.048909] systemd[1]: Starting Create System Files and Directories...1929lb # [ 2.051289] systemd-udevd[78]: Using default interface naming scheme 'v261'.1930client # [ 2.057375] systemd[1]: Finished Apply Kernel Variables.1931worker2 # [ 2.058597] systemd[1]: Starting Create System Files and Directories...1932lb # [ 2.053232] systemd[1]: Finished Apply Kernel Variables.1933worker2 # [ 2.062365] systemd-udevd[79]: Using default interface naming scheme 'v261'.1934worker1 # [ 2.070387] systemd[1]: Finished Create System Files and Directories.1935client # [ 2.072226] systemd[1]: Finished Create System Files and Directories.1936worker2 # [ 2.073012] systemd[1]: Finished Apply Kernel Variables.1937worker1 # [ 2.081166] systemd[1]: Started Rule-based Manager for Device Events and Files.1938client # [ 2.077383] systemd-udevd[78]: Using default interface naming scheme 'v261'.1939worker2 # [ 2.082901] systemd[1]: Finished Create System Files and Directories.1940lb # [ 2.087029] systemd[1]: Finished Create System Files and Directories.1941worker2 # [ 2.098155] systemd[1]: Started Rule-based Manager for Device Events and Files.1942lb # [ 2.092682] systemd[1]: Started Rule-based Manager for Device Events and Files.1943client # [ 2.104528] systemd[1]: Started Rule-based Manager for Device Events and Files.1944worker1 # [ 2.133237] systemd[1]: Starting Virtual Console Setup...1945lb # [ 2.139464] systemd[1]: Starting Virtual Console Setup...1946worker2 # [ 2.148113] systemd[1]: Starting Virtual Console Setup...1947client # [ 2.156879] systemd[1]: Starting Virtual Console Setup...1948worker1 # [ 2.185895] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1949worker1 # [ 2.187571] systemd[1]: Finished Virtual Console Setup.1950worker2 # [ 2.196416] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1951worker2 # [ 2.201769] systemd[1]: Finished Virtual Console Setup.1952lb # [ 2.196470] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1953lb # [ 2.199772] systemd[1]: Finished Virtual Console Setup.1954client # [ 2.217818] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1955client # [ 2.219671] systemd[1]: Finished Virtual Console Setup.1956worker1 # [ 2.785248] systemd[1]: Finished Coldplug All udev Devices.1957worker1 # [ 2.786198] systemd[1]: Reached target System Initialization.1958worker1 # [ 2.788094] systemd[1]: Reached target Basic System.1959worker2 # [ 2.805312] systemd[1]: Finished Coldplug All udev Devices.1960worker2 # [ 2.806288] systemd[1]: Reached target System Initialization.1961worker2 # [ 2.807129] systemd[1]: Reached target Basic System.1962client # [ 2.821264] systemd[1]: Finished Coldplug All udev Devices.1963lb # [ 2.816080] systemd[1]: Finished Coldplug All udev Devices.1964client # [ 2.822229] systemd[1]: Reached target System Initialization.1965lb # [ 2.817424] systemd[1]: Reached target System Initialization.1966lb # [ 2.818291] systemd[1]: Reached target Basic System.1967client # [ 2.824098] systemd[1]: Reached target Basic System.1968worker1 # [ 2.928168] (udev-worker)[91]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1969lb # [ 2.935938] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1970worker1 # [ 2.954563] (udev-worker)[104]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 2.968374] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1972worker2 # [ 2.967301] (udev-worker)[92]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973worker2 # [ 2.975341] (udev-worker)[99]: Network interface NamePolicy= disabled on kernel command line.1974client # [ 2.973981] (udev-worker)[93]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1975client # [ 2.984875] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1976client # [ 2.989584] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1977worker2 # [ 2.993034] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1978lb # [ 2.994376] (udev-worker)[94]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1979lb # [ 3.000264] (udev-worker)[94]: Network interface NamePolicy= disabled on kernel command line.1980worker1 # [ 3.047574] systemd[1]: Found device /dev/disk/by-label/nixos.1981worker1 # [ 3.051176] systemd[1]: Reached target Initrd Root Device.1982worker1 # [ 3.054141] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1983worker2 # [ 3.051936] systemd[1]: Found device /dev/disk/by-label/nixos.1984worker2 # [ 3.054737] systemd[1]: Reached target Initrd Root Device.1985worker2 # [ 3.058399] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1986client # [ 3.064529] systemd[1]: Found device /dev/disk/by-label/nixos.1987lb # [ 3.061921] systemd[1]: Found device /dev/disk/by-label/nixos.1988client # [ 3.070852] systemd[1]: Reached target Initrd Root Device.1989lb # [ 3.068283] systemd[1]: Reached target Initrd Root Device.1990client # [ 3.078646] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1991lb # [ 3.073199] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1992worker1 # [ 3.096967] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1993worker1 # [ 3.103908] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1994worker1 # [ 3.105517] systemd[1]: Mounting /sysroot...1995worker2 # [ 3.112499] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1996worker2 # [ 3.119876] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1997worker2 # [ 3.124142] systemd[1]: Mounting /sysroot...1998client # [ 3.123323] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1999lb # [ 3.125484] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks2000client # [ 3.132208] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.
loading 2,081 hidden lines…
4082worker1 # [ 42.590016] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124083worker1 # [ 42.725050] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124084worker1 # [ 42.795490] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.434Z level=INFO msg="All 1 paths already cached"4085client # copying 1 paths to 'lb:50051'...4086worker1 # [ 42.805382] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124087client # copying path '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' to 'lb:50051'...4088worker1 # [ 42.845609] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4089worker1 # [ 42.999093] nix-daemon[848]: accepted connection from pid 909, user nix-grpc-daemon (trusted)4090worker1 # [ 43.041380] nix-daemon[848]: reaped child process 912, status = succeeded4091lb # [ 43.038621] niks3-server[959]: 2026/09/22 22:20:55 INFO Received uploads request method=POST path=/api/pending_closures4092worker1 # [ 43.096888] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.736Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4093worker1 # [ 43.099251] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.738Z level=INFO msg="Uploading 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv (624B)"4094lb # [ 43.136799] niks3-server[959]: 2026/09/22 22:20:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4095worker1 # [ 43.154218] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.793Z level=INFO msg="Uploading 1 narinfos"4096lb # [ 43.148453] niks3-server[959]: 2026/09/22 22:20:55 INFO Signed narinfos id=3 count=14097lb # [ 43.226190] niks3-server[959]: 2026/09/22 22:20:55 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4098lb # [ 43.241983] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=nar/1xk4kbz8l387hfrqq2xi6as79has336h62f2ra95472yqjclksjg.nar.zst4099lb # [ 43.255475] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.ls4100worker1 # [ 43.319490] nix-grpc-daemon[863]: time=2026-09-22T22:20:55.958Z level=INFO msg="Upload complete. (463ms)"4101lb # [ 43.309914] niks3-server[959]: 2026/09/22 22:20:55 INFO Completed upload id=34102lb # [ 43.318351] niks3-server[959]: 2026/09/22 22:20:55 INFO Registered completed upload object_key=79z38hfi82zcmb9yq1gcmfviimgf6z2l.narinfo4103worker1 # [ 43.332626] nix-grpc-daemon[766]: ts=2026-09-22T22:20:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=6244104worker1 # [ 43.361148] nix-daemon[848]: reaped child process 907, status = succeeded4105worker1 # [ 43.376706] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4106worker1 # [ 43.379868] nix-grpc-daemon[766]: ts=2026-09-22T22:20:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124107worker1 # [ 43.463671] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4108client # building '/nix/store/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv' on lb:50051...4109client # farm-b> node-a: building 79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4110client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4111client # farm-b> LOG-farm-b-t14112worker1 # [ 44.549155] nix-daemon[848]: reaped child process 916, status = succeeded4113worker1 # [ 44.668467] nix-daemon[848]: accepted connection from pid 932, user nix-grpc-daemon (trusted)4114worker1 # [ 44.696836] nix-daemon[848]: reaped child process 935, status = succeeded4115lb # [ 44.685479] niks3-server[959]: 2026/09/22 22:20:57 INFO Received uploads request method=POST path=/api/pending_closures4116worker1 # [ 44.716581] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.355Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4117worker1 # [ 44.718487] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.355Z level=INFO msg="Uploading 8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b (176B)"4118lb # [ 44.798579] niks3-server[959]: 2026/09/22 22:20:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4119lb # [ 44.802159] niks3-server[959]: 2026/09/22 22:20:57 INFO Signed narinfos id=4 count=14120lb # [ 44.807901] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=nar/1mx8h9irf9ykfzxs6q3lwxbzxwv5d2s9mh2x0sky7448mlc1qxrx.nar.zst4121worker1 # [ 44.833583] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.472Z level=INFO msg="Uploading 1 narinfos"4122lb # [ 44.824753] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=log/79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv4123lb # [ 44.834791] niks3-server[959]: 2026/09/22 22:20:57 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4124lb # [ 44.844229] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.ls4125worker1 # [ 44.858004] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.497Z level=INFO msg="Upload complete. (370ms)"4126lb # [ 44.849591] niks3-server[959]: 2026/09/22 22:20:57 INFO Completed upload id=44127lb # [ 44.862384] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=8n9g36vw9957r150a7wg1k6yj9fbqm8r.narinfo4128worker1 # [ 44.867361] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=79z38hfi82zcmb9yq1gcmfviimgf6z2l-farm-b.drv assign_id=2 outputs=14129worker1 # [ 44.885342] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=want client=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=04130worker1 # [ 44.887854] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=assigned drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv worker=192.168.1.3:50051 clients=14131worker1 # [ 44.892218] nix-daemon[848]: reaped child process 914, status = succeeded4132worker1 # [ 44.931018] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124133worker1 # [ 44.941807] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124134worker1 # [ 44.952915] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.592Z level=INFO msg="All 2 paths already cached"4135client # copying 1 paths to 'lb:50051'...4136client # copying path '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' to 'lb:50051'...4137worker1 # [ 44.966277] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124138worker1 # [ 44.975427] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4139worker1 # [ 45.133869] nix-daemon[848]: accepted connection from pid 942, user nix-grpc-daemon (trusted)4140lb # [ 45.160452] niks3-server[959]: 2026/09/22 22:20:57 INFO Received uploads request method=POST path=/api/pending_closures4141worker1 # [ 45.178737] nix-daemon[848]: reaped child process 945, status = succeeded4142worker1 # [ 45.187597] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.826Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4143worker1 # [ 45.190157] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.826Z level=INFO msg="Uploading czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv (752B)"4144lb # [ 45.203198] niks3-server[959]: 2026/09/22 22:20:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4145worker1 # [ 45.219577] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.858Z level=INFO msg="Uploading 1 narinfos"4146lb # [ 45.216513] niks3-server[959]: 2026/09/22 22:20:57 INFO Signed narinfos id=5 count=14147lb # [ 45.224418] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.ls4148lb # [ 45.234273] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=nar/02rh0zwkcxgd7rc74r5h676rr0m8dbayk3glb776b2mdlirxh9lv.nar.zst4149lb # [ 45.250626] niks3-server[959]: 2026/09/22 22:20:57 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4150lb # [ 45.261549] niks3-server[959]: 2026/09/22 22:20:57 INFO Completed upload id=54151worker1 # [ 45.272054] nix-grpc-daemon[863]: time=2026-09-22T22:20:57.911Z level=INFO msg="Upload complete. (285ms)"4152worker1 # [ 45.277009] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=7524153lb # [ 45.266879] niks3-server[959]: 2026/09/22 22:20:57 INFO Registered completed upload object_key=czd6ljynjaw3bl3w8iz1nx68whkdis8f.narinfo4154worker1 # [ 45.282514] nix-grpc-daemon[766]: ts=2026-09-22T22:20:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124155worker1 # [ 45.292687] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4156worker1 # [ 45.296557] nix-daemon[848]: reaped child process 940, status = succeeded4157worker1 # [ 45.305870] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4158client # building '/nix/store/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv' on lb:50051...4159client # farm-top> node-a: building czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4160client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4161client # farm-top> LOG-farm-top-t14162worker1 # [ 46.154743] nix-daemon[848]: reaped child process 950, status = succeeded4163worker1 # [ 46.271591] nix-daemon[848]: accepted connection from pid 966, user nix-grpc-daemon (trusted)4164worker1 # [ 46.297894] nix-daemon[848]: reaped child process 969, status = succeeded4165lb # [ 46.293310] niks3-server[959]: 2026/09/22 22:20:58 INFO Received uploads request method=POST path=/api/pending_closures4166worker1 # [ 46.310294] nix-grpc-daemon[863]: time=2026-09-22T22:20:58.948Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4167worker1 # [ 46.314615] nix-grpc-daemon[863]: time=2026-09-22T22:20:58.948Z level=INFO msg="Uploading nblg066ygaz2avzlmix3rnngr62hld8f-farm-top (232B)"4168lb # [ 46.318836] niks3-server[959]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=nar/1wplyx8g7zja504cjvqmxis1q6laml9bidgiq4ya0i9p2pvqb9qp.nar.zst4169lb # [ 46.330022] niks3-server[959]: 2026/09/22 22:20:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4170worker1 # [ 46.343492] nix-grpc-daemon[863]: time=2026-09-22T22:20:58.982Z level=INFO msg="Uploading 1 narinfos"4171lb # [ 46.336083] niks3-server[959]: 2026/09/22 22:20:58 INFO Signed narinfos id=6 count=14172lb # [ 46.339530] niks3-server[959]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.ls4173lb # [ 46.349578] niks3-server[959]: 2026/09/22 22:20:58 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4174lb # [ 46.354953] niks3-server[959]: 2026/09/22 22:20:58 INFO Registered completed upload object_key=log/czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv4175lb # [ 46.366216] niks3-server[959]: 2026/09/22 22:20:59 INFO Registered completed upload object_key=nblg066ygaz2avzlmix3rnngr62hld8f.narinfo4176worker1 # [ 46.379802] nix-grpc-daemon[863]: time=2026-09-22T22:20:59.019Z level=INFO msg="Upload complete. (243ms)"4177lb # [ 46.370413] niks3-server[959]: 2026/09/22 22:20:59 INFO Completed upload id=64178worker1 # [ 46.386031] nix-grpc-daemon[766]: ts=2026-09-22T22:20:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv assign_id=3 outputs=14179worker1 # [ 46.396293] nix-daemon[848]: reaped child process 948, status = succeeded4180client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 6.71 seconds)4181client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4182client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.17 seconds)4183client: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4184client: (finished: must fail: test -e /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.03 seconds)4185client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4186client # copying 3 paths...4187client # copying path '/nix/store/dg404zlvxfrspnxl6cc1qkc6i83m5a3a-farm-a' from 'http://lb:5751'...4188client # copying path '/nix/store/8n9g36vw9957r150a7wg1k6yj9fbqm8r-farm-b' from 'http://lb:5751'...4189client # copying path '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top' from 'http://lb:5751'...4190client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top && grep farm-top-t1 /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.37 seconds)4191(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 7.39 seconds)4192subtest: repeat is answered Cached by the scheduler without building4193worker1: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4194worker1 # finding garbage collector roots...4195worker1 # deleting specified paths...4196worker1 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4197worker1 # deleting unused links...4198worker1 # note: hard linking is currently saving 0.0 KiB4199worker1: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.25 seconds)4200worker2: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4201worker2 # finding garbage collector roots...4202worker2 # deleting specified paths...4203worker2 # deleting '/nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top'4204worker2 # deleting unused links...4205worker2 # note: hard linking is currently saving 0.0 KiB4206worker2: (finished: must succeed: nix-store --delete /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top, in 0.23 seconds)4207client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&24208worker1 # [ 47.835592] nix-grpc-daemon[766]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400124209client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4210client # /nix/store/nblg066ygaz2avzlmix3rnngr62hld8f-farm-top4211worker2 # [ 47.852703] nix-grpc-daemon[769]: ts=2026-09-22T22:21:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904212worker1 # [ 47.933146] nix-grpc-daemon[766]: ts=2026-09-22T22:21:00Z level=debug event=want client=2 drv=czd6ljynjaw3bl3w8iz1nx68whkdis8f-farm-top.drv cached=14213client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 >&2, in 0.38 seconds)4214client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath4215client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 outPath, in 0.15 seconds)4216lb: must succeed: curl -sf localhost:9901/clusters4217lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)4218lb: must succeed: curl -sf localhost:9901/clusters4219lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4220worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4221worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.11 seconds)4222(finished: subtest: repeat is answered Cached by the scheduler without building, in 1.40 seconds)4223subtest: inputs already in the cache are not sent to the scheduler4224lb: must succeed: curl -sf localhost:9901/clusters4225lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4226lb: must succeed: curl -sf localhost:9901/clusters4227lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4228worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4229worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.07 seconds)4230client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&24231client # don't know how to build these paths:4232client # /nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4233worker2 # [ 48.920172] nix-grpc-daemon[769]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536904234worker2 # [ 48.927364] nix-grpc-daemon[769]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904235worker1 # [ 48.953222] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=want client=3 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv cached=04236worker1 # [ 48.953691] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=assigned drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv worker=192.168.1.3:50051 clients=14237worker1 # [ 49.000747] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124238worker1 # [ 49.019823] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124239worker1 # [ 49.039576] nix-grpc-daemon[863]: time=2026-09-22T22:21:01.677Z level=INFO msg="All 2 paths already cached"4240client # copying 1 paths to 'lb:50051'...4241client # copying path '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' to 'lb:50051'...4242worker1 # [ 49.052668] nix-grpc-daemon[766]: ts=2026-09-22T22:21:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124243worker1 # [ 49.069131] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4244worker1 # [ 49.239528] nix-daemon[848]: accepted connection from pid 998, user nix-grpc-daemon (trusted)4245worker1 # [ 49.268314] nix-daemon[848]: reaped child process 1001, status = succeeded4246lb # [ 49.261107] niks3-server[959]: 2026/09/22 22:21:01 INFO Received uploads request method=POST path=/api/pending_closures4247worker1 # [ 49.286838] nix-grpc-daemon[863]: time=2026-09-22T22:21:01.926Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4248worker1 # [ 49.290588] nix-grpc-daemon[863]: time=2026-09-22T22:21:01.928Z level=INFO msg="Uploading blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv (752B)"4249lb # [ 49.310211] niks3-server[959]: 2026/09/22 22:21:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4250worker1 # [ 49.330790] nix-grpc-daemon[863]: time=2026-09-22T22:21:01.969Z level=INFO msg="Uploading 1 narinfos"4251lb # [ 49.325644] niks3-server[959]: 2026/09/22 22:21:01 INFO Signed narinfos id=7 count=14252lb # [ 49.333119] niks3-server[959]: 2026/09/22 22:21:01 INFO Registered completed upload object_key=nar/1vy3ds565pni9mnkldcs6alimlb6bg39cimm60s4swkk5yjhwlqf.nar.zst4253lb # [ 49.344335] niks3-server[959]: 2026/09/22 22:21:01 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.ls4254lb # [ 49.353653] niks3-server[959]: 2026/09/22 22:21:01 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4255lb # [ 49.365291] niks3-server[959]: 2026/09/22 22:21:01 INFO Completed upload id=74256worker1 # [ 49.372959] nix-grpc-daemon[863]: time=2026-09-22T22:21:02.012Z level=INFO msg="Upload complete. (283ms)"4257lb # [ 49.371887] niks3-server[959]: 2026/09/22 22:21:02 INFO Registered completed upload object_key=blgxcsidls7na1m5da3m0bja1prb87vl.narinfo4258worker1 # [ 49.384711] nix-grpc-daemon[766]: ts=2026-09-22T22:21:02Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=7524259worker1 # [ 49.400656] nix-grpc-daemon[766]: ts=2026-09-22T22:21:02Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124260worker1 # [ 49.417336] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4261worker1 # [ 49.426092] nix-daemon[848]: reaped child process 996, status = succeeded4262client # building '/nix/store/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv' on lb:50051...4263worker1 # [ 49.442812] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4264client # farm-top> node-a: building blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4265lb # [ 49.913716] envoy[895]: conn peer=[2001:db8:1::1]:40296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9549 tx=4390 ms=65194266lb # [ 49.916086] envoy[895]: conn peer=[2001:db8:1::1]:45098 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=480 ms=1284267client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4268worker1 # [ 50.381691] nix-grpc-daemon[766]: E0000 00:00:1790115663.017992 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4269worker1 # [ 50.412110] nix-grpc-daemon[766]: E0000 00:00:1790115663.043100 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4270worker1 # [ 50.414629] nix-grpc-daemon[766]: E0000 00:00:1790115663.044312 808 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.4271worker1 # [ 50.430109] nix-grpc-daemon[766]: E0000 00:00:1790115663.065065 808 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.4272client # farm-top> LOG-farm-top-t1b4273worker1 # [ 50.473709] nix-daemon[848]: reaped child process 1005, status = succeeded4274worker1 # [ 50.657571] nix-daemon[848]: accepted connection from pid 1021, user nix-grpc-daemon (trusted)4275worker1 # [ 50.709653] nix-daemon[848]: reaped child process 1024, status = succeeded4276lb # [ 50.703105] niks3-server[959]: 2026/09/22 22:21:03 INFO Received uploads request method=POST path=/api/pending_closures4277worker1 # [ 50.830596] nix-grpc-daemon[863]: time=2026-09-22T22:21:03.469Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4278worker1 # [ 50.837285] nix-grpc-daemon[863]: time=2026-09-22T22:21:03.471Z level=INFO msg="Uploading 0xnysbbdbmapgf23wb2dbsfhgh18yxvp-farm-top (232B)"4279lb # [ 50.885878] niks3-server[959]: 2026/09/22 22:21:03 INFO Registered completed upload object_key=log/blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv4280lb # [ 50.908327] niks3-server[959]: 2026/09/22 22:21:03 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4281worker1 # [ 50.922349] nix-grpc-daemon[863]: time=2026-09-22T22:21:03.561Z level=INFO msg="Uploading 1 narinfos"4282lb # [ 50.914447] niks3-server[959]: 2026/09/22 22:21:03 INFO Registered completed upload object_key=nar/1f6pjd04v1xmbxqqjhczkkfk4ns9qg6ssnspdalp41q9hkxf9yfr.nar.zst4283lb # [ 50.920863] niks3-server[959]: 2026/09/22 22:21:03 INFO Signed narinfos id=8 count=14284lb # [ 50.922109] niks3-server[959]: 2026/09/22 22:21:03 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.ls4285lb # [ 50.929602] niks3-server[959]: 2026/09/22 22:21:03 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4286lb # [ 50.933880] niks3-server[959]: 2026/09/22 22:21:03 INFO Completed upload id=84287lb # [ 50.935327] niks3-server[959]: 2026/09/22 22:21:03 INFO Registered completed upload object_key=0xnysbbdbmapgf23wb2dbsfhgh18yxvp.narinfo4288worker1 # [ 50.943514] nix-grpc-daemon[863]: time=2026-09-22T22:21:03.582Z level=INFO msg="Upload complete. (487ms)"4289worker1 # [ 50.956411] nix-grpc-daemon[766]: ts=2026-09-22T22:21:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=blgxcsidls7na1m5da3m0bja1prb87vl-farm-top.drv assign_id=4 outputs=14290worker1 # [ 50.974777] nix-daemon[848]: reaped child process 1003, status = succeeded4291client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.30 seconds)4292client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath4293client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.20 seconds)4294lb: must succeed: curl -sf localhost:9901/clusters4295lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)4296lb: must succeed: curl -sf localhost:9901/clusters4297lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4298worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4299worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.06 seconds)4300(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.87 seconds)4301subtest: outputs read back through any worker carry the cache signature4302client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&24303worker2 # [ 51.606672] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536904304client # don't know how to build these paths:4305client # /nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4306worker1 # [ 51.627360] nix-grpc-daemon[766]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124307worker1 # [ 51.653220] nix-grpc-daemon[766]: ts=2026-09-22T22:21:04Z level=debug event=want client=4 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv cached=04308worker1 # [ 51.659620] nix-grpc-daemon[766]: ts=2026-09-22T22:21:04Z level=debug event=assigned drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv worker=192.168.1.4:50051 clients=14309worker2 # [ 51.659866] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904310worker2 # [ 51.677703] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904311client # copying 1 paths to 'lb:50051'...4312client # copying path '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' to 'lb:50051'...4313worker2 # [ 51.696243] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536904314worker2 # [ 51.713230] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4315worker2 # [ 51.747045] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=info event=niks3_push_started pid=8654316worker2 # [ 51.933451] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.574Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""4317worker2 # [ 52.055615] nix-daemon[837]: accepted connection from pid 878, user nix-grpc-daemon (trusted)4318worker2 # [ 52.078211] nix-daemon[837]: reaped child process 881, status = succeeded4319lb # [ 52.081768] niks3-server[959]: 2026/09/22 22:21:04 INFO Received uploads request method=POST path=/api/pending_closures4320worker2 # [ 52.110135] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.751Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4321worker2 # [ 52.114936] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.751Z level=INFO msg="Uploading 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv (512B)"4322lb # [ 52.131661] niks3-server[959]: 2026/09/22 22:21:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4323lb # [ 52.138120] niks3-server[959]: 2026/09/22 22:21:04 INFO Signed narinfos id=9 count=14324worker2 # [ 52.145486] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.787Z level=INFO msg="Uploading 1 narinfos"4325lb # [ 52.148356] niks3-server[959]: 2026/09/22 22:21:04 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4326lb # [ 52.155108] niks3-server[959]: 2026/09/22 22:21:04 INFO Registered completed upload object_key=nar/13mhmmy2qay13imbbdi287vhryz1hwnyg0w6agd9s0ph37wnsqll.nar.zst4327lb # [ 52.161760] niks3-server[959]: 2026/09/22 22:21:04 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.ls4328lb # [ 52.163439] niks3-server[959]: 2026/09/22 22:21:04 INFO Registered completed upload object_key=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y.narinfo4329lb # [ 52.166253] niks3-server[959]: 2026/09/22 22:21:04 INFO Completed upload id=94330worker2 # [ 52.172752] nix-grpc-daemon[865]: time=2026-09-22T22:21:04.815Z level=INFO msg="Upload complete. (236ms)"4331worker2 # [ 52.181008] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=5124332worker2 # [ 52.183984] nix-grpc-daemon[769]: ts=2026-09-22T22:21:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904333worker2 # [ 52.195546] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4334worker2 # [ 52.199634] nix-daemon[837]: reaped child process 863, status = succeeded4335client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4336worker2 # [ 52.211031] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4337client # farm-a> worker2: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4338client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4339client # farm-a> LOG-farm-a-sig4340worker2 # [ 53.244615] nix-daemon[837]: reaped child process 885, status = succeeded4341worker2 # [ 53.364167] nix-daemon[837]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4342worker2 # [ 53.390990] nix-daemon[837]: reaped child process 904, status = succeeded4343lb # [ 53.381832] niks3-server[959]: 2026/09/22 22:21:06 INFO Received uploads request method=POST path=/api/pending_closures4344worker2 # [ 53.404537] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.045Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4345worker2 # [ 53.406130] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.045Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4346lb # [ 53.423829] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4347lb # [ 53.443970] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4348lb # [ 53.454287] niks3-server[959]: 2026/09/22 22:21:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4349worker2 # [ 53.467351] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.109Z level=INFO msg="Uploading 1 narinfos"4350lb # [ 53.463573] niks3-server[959]: 2026/09/22 22:21:06 INFO Signed narinfos id=10 count=14351lb # [ 53.469288] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4352lb # [ 53.481334] niks3-server[959]: 2026/09/22 22:21:06 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4353worker2 # [ 53.492802] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.135Z level=INFO msg="Upload complete. (273ms)"4354lb # [ 53.488197] niks3-server[959]: 2026/09/22 22:21:06 INFO Completed upload id=104355lb # [ 53.492882] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4356worker2 # [ 53.499082] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14357worker1 # [ 53.511336] nix-grpc-daemon[766]: ts=2026-09-22T22:21:06Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04358worker2 # [ 53.511785] nix-daemon[837]: reaped child process 883, status = succeeded4359worker1 # [ 53.518204] nix-grpc-daemon[766]: ts=2026-09-22T22:21:06Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.4:50051 clients=14360worker2 # [ 53.554789] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904361worker2 # [ 53.567214] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904362worker2 # [ 53.582144] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.224Z level=INFO msg="All 1 paths already cached"4363client # copying 1 paths to 'lb:50051'...4364worker2 # [ 53.592816] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536904365client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4366worker2 # [ 53.603722] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4367worker2 # [ 53.751710] nix-daemon[837]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4368lb # [ 53.769709] niks3-server[959]: 2026/09/22 22:21:06 INFO Received uploads request method=POST path=/api/pending_closures4369worker2 # [ 53.779396] nix-daemon[837]: reaped child process 911, status = succeeded4370worker2 # [ 53.788103] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.430Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4371worker2 # [ 53.789803] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.430Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4372lb # [ 53.804888] niks3-server[959]: 2026/09/22 22:21:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4373worker2 # [ 53.818446] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.460Z level=INFO msg="Uploading 1 narinfos"4374lb # [ 53.820549] niks3-server[959]: 2026/09/22 22:21:06 INFO Signed narinfos id=11 count=14375lb # [ 53.829712] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4376lb # [ 53.838271] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4377worker2 # [ 53.858991] nix-grpc-daemon[865]: time=2026-09-22T22:21:06.501Z level=INFO msg="Upload complete. (245ms)"4378lb # [ 53.852474] niks3-server[959]: 2026/09/22 22:21:06 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4379lb # [ 53.859896] niks3-server[959]: 2026/09/22 22:21:06 INFO Completed upload id=114380worker2 # [ 53.868485] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=6244381lb # [ 53.865820] niks3-server[959]: 2026/09/22 22:21:06 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4382worker2 # [ 53.877298] nix-grpc-daemon[769]: ts=2026-09-22T22:21:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904383worker2 # [ 53.890269] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4384worker2 # [ 53.896560] nix-daemon[837]: reaped child process 906, status = succeeded4385client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4386client # farm-b> worker2: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4387worker2 # [ 53.910680] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4388client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4389client # farm-b> LOG-farm-b-sig4390worker2 # [ 54.545666] nix-daemon[837]: reaped child process 915, status = succeeded4391worker2 # [ 54.666765] nix-daemon[837]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4392worker2 # [ 54.692324] nix-daemon[837]: reaped child process 934, status = succeeded4393lb # [ 54.693541] niks3-server[959]: 2026/09/22 22:21:07 INFO Received uploads request method=POST path=/api/pending_closures4394worker2 # [ 54.713349] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.353Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4395worker2 # [ 54.715009] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.353Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4396lb # [ 54.732351] niks3-server[959]: 2026/09/22 22:21:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4397worker2 # [ 54.745618] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.388Z level=INFO msg="Uploading 1 narinfos"4398lb # [ 54.741264] niks3-server[959]: 2026/09/22 22:21:07 INFO Signed narinfos id=12 count=14399lb # [ 54.749222] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4400lb # [ 54.756750] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4401lb # [ 54.758972] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4402lb # [ 54.771364] niks3-server[959]: 2026/09/22 22:21:07 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4403lb # [ 54.778896] niks3-server[959]: 2026/09/22 22:21:07 INFO Completed upload id=124404worker2 # [ 54.784925] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.427Z level=INFO msg="Upload complete. (264ms)"4405lb # [ 54.781861] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4406worker2 # [ 54.797362] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14407worker1 # [ 54.806392] nix-grpc-daemon[766]: ts=2026-09-22T22:21:07Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04408worker1 # [ 54.812954] nix-grpc-daemon[766]: ts=2026-09-22T22:21:07Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.4:50051 clients=14409worker2 # [ 54.815577] nix-daemon[837]: reaped child process 913, status = succeeded4410worker2 # [ 54.847390] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904411worker2 # [ 54.865299] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904412worker2 # [ 54.882631] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.525Z level=INFO msg="All 2 paths already cached"4413client # copying 1 paths to 'lb:50051'...4414client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4415worker2 # [ 54.897595] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536904416worker2 # [ 54.918711] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4417worker2 # [ 55.091908] nix-daemon[837]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4418worker2 # [ 55.114180] nix-daemon[837]: reaped child process 941, status = succeeded4419lb # [ 55.114829] niks3-server[959]: 2026/09/22 22:21:07 INFO Received uploads request method=POST path=/api/pending_closures4420worker2 # [ 55.133073] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.775Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4421worker2 # [ 55.134748] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.775Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4422lb # [ 55.155438] niks3-server[959]: 2026/09/22 22:21:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4423worker2 # [ 55.172590] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.814Z level=INFO msg="Uploading 1 narinfos"4424lb # [ 55.173052] niks3-server[959]: 2026/09/22 22:21:07 INFO Signed narinfos id=13 count=14425lb # [ 55.182211] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4426lb # [ 55.190056] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4427lb # [ 55.205970] niks3-server[959]: 2026/09/22 22:21:07 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4428lb # [ 55.212671] niks3-server[959]: 2026/09/22 22:21:07 INFO Completed upload id=134429lb # [ 55.216139] niks3-server[959]: 2026/09/22 22:21:07 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4430worker2 # [ 55.217594] nix-grpc-daemon[865]: time=2026-09-22T22:21:07.860Z level=INFO msg="Upload complete. (278ms)"4431worker2 # [ 55.228946] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=7524432worker2 # [ 55.244580] nix-grpc-daemon[769]: ts=2026-09-22T22:21:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904433worker2 # [ 55.260156] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4434worker2 # [ 55.266700] nix-daemon[837]: reaped child process 936, status = succeeded4435client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4436worker2 # [ 55.279837] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4437client # farm-top> worker2: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4438client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4439client # farm-top> LOG-farm-top-sig4440worker2 # [ 56.114140] nix-daemon[837]: reaped child process 945, status = succeeded4441worker2 # [ 56.231659] nix-daemon[837]: accepted connection from pid 961, user nix-grpc-daemon (trusted)4442worker2 # [ 56.257106] nix-daemon[837]: reaped child process 964, status = succeeded4443lb # [ 56.260000] niks3-server[959]: 2026/09/22 22:21:08 INFO Received uploads request method=POST path=/api/pending_closures4444worker2 # [ 56.280267] nix-grpc-daemon[865]: time=2026-09-22T22:21:08.920Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4445worker2 # [ 56.281890] nix-grpc-daemon[865]: time=2026-09-22T22:21:08.920Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4446lb # [ 56.310847] niks3-server[959]: 2026/09/22 22:21:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4447worker2 # [ 56.325654] nix-grpc-daemon[865]: time=2026-09-22T22:21:08.968Z level=INFO msg="Uploading 1 narinfos"4448lb # [ 56.326262] niks3-server[959]: 2026/09/22 22:21:08 INFO Signed narinfos id=14 count=14449lb # [ 56.332895] niks3-server[959]: 2026/09/22 22:21:08 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4450lb # [ 56.342788] niks3-server[959]: 2026/09/22 22:21:08 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4451lb # [ 56.352483] niks3-server[959]: 2026/09/22 22:21:08 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4452worker2 # [ 56.368129] nix-grpc-daemon[865]: time=2026-09-22T22:21:09.010Z level=INFO msg="Upload complete. (274ms)"4453lb # [ 56.364640] niks3-server[959]: 2026/09/22 22:21:08 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4454lb # [ 56.368486] niks3-server[959]: 2026/09/22 22:21:08 INFO Completed upload id=144455lb # [ 56.373126] niks3-server[959]: 2026/09/22 22:21:09 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4456worker2 # [ 56.381438] nix-grpc-daemon[769]: ts=2026-09-22T22:21:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14457worker2 # [ 56.405109] nix-daemon[837]: reaped child process 943, status = succeeded4458client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 5.06 seconds)4459client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4460client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.19 seconds)4461client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4462worker1 # [ 56.782005] nix-grpc-daemon[766]: ts=2026-09-22T22:21:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:552984463client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4464client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4465client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4466client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4467worker1 # [ 56.823215] nix-grpc-daemon[766]: ts=2026-09-22T22:21:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:552984468worker1 # [ 56.833322] nix-grpc-daemon[766]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4469worker1 # [ 56.858446] nix-grpc-daemon[766]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4470worker1 # [ 56.877357] nix-grpc-daemon[766]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4471client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.30 seconds)4472client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4473worker2 # [ 57.109197] nix-grpc-daemon[769]: ts=2026-09-22T22:21:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:533904474worker2 # [ 57.152424] nix-grpc-daemon[769]: ts=2026-09-22T22:21:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:533904475client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.30 seconds)4476client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4477client # finding garbage collector roots...4478client # removing stale temporary roots file "/nix/var/nix/temproots/883"4479client # removing stale temporary roots file "/nix/var/nix/temproots/903"4480client # removing stale temporary roots file "/nix/var/nix/temproots/845"4481client # removing stale temporary roots file "/nix/var/nix/temproots/895"4482client # removing stale temporary roots file "/nix/var/nix/temproots/916"4483client # removing stale temporary roots file "/nix/var/nix/temproots/938"4484client # removing stale temporary roots file "/nix/var/nix/temproots/862"4485client # removing stale temporary roots file "/nix/var/nix/temproots/924"4486client # deleting specified paths...4487client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4488client # deleting unused links...4489client # note: hard linking is currently saving 0.0 KiB4490worker2 # [ 57.594237] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536904491worker1 # [ 57.608962] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124492worker1 # [ 57.622784] nix-grpc-daemon[863]: time=2026-09-22T22:21:10.261Z level=INFO msg="All 1 paths already cached"4493worker1 # [ 57.633713] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:400124494worker1 # [ 57.645269] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:400124495worker1 # [ 57.656516] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:400124496client # copying 3 paths...4497worker2 # [ 57.672175] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536904498client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4499worker1 # [ 57.713527] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:400124500worker1 # [ 57.722164] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=2324501worker2 # [ 57.738844] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:536904502worker2 # [ 57.747281] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_out=1764503worker1 # [ 57.761943] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:400124504worker1 # [ 57.769934] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=1284505client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4506client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4507client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.60 seconds)4508(finished: subtest: outputs read back through any worker carry the cache signature, in 6.46 seconds)4509subtest: two clients wanting the same drv build it once4510client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4511client # [ 57.883181] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4512client # Running as unit: dedup1.service; invocation ID: b13591785c854ee78757e801bbfd41f94513client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4514client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4515worker2 # [ 58.091807] nix-grpc-daemon[769]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536904516client # [ 58.099504] nix[991]: don't know how to build these paths:4517client # [ 58.101670] nix[991]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4518worker1 # [ 58.120157] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04519worker1 # [ 58.125982] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.3:50051 clients=14520worker1 # [ 58.163252] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124521worker1 # [ 58.181895] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124522client # [ 58.192268] nix[991]: copying 1 paths to 'lb:50051'...4523client # [ 58.198890] nix[991]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4524worker1 # [ 58.202541] nix-grpc-daemon[766]: ts=2026-09-22T22:21:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124525worker1 # [ 58.220190] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4526worker1 # [ 58.396748] nix-daemon[848]: accepted connection from pid 1044, user nix-grpc-daemon (trusted)4527worker1 # [ 58.424235] nix-daemon[848]: reaped child process 1047, status = succeeded4528lb # [ 58.419788] niks3-server[959]: 2026/09/22 22:21:11 INFO Received uploads request method=POST path=/api/pending_closures4529worker1 # [ 58.440830] nix-grpc-daemon[863]: time=2026-09-22T22:21:11.079Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4530worker1 # [ 58.442717] nix-grpc-daemon[863]: time=2026-09-22T22:21:11.079Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4531lb # [ 58.460899] niks3-server[959]: 2026/09/22 22:21:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4532worker1 # [ 58.484025] nix-grpc-daemon[863]: time=2026-09-22T22:21:11.123Z level=INFO msg="Uploading 1 narinfos"4533lb # [ 58.474119] niks3-server[959]: 2026/09/22 22:21:11 INFO Signed narinfos id=15 count=14534lb # [ 58.484778] niks3-server[959]: 2026/09/22 22:21:11 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4535lb # [ 58.498463] niks3-server[959]: 2026/09/22 22:21:11 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4536lb # [ 58.513633] niks3-server[959]: 2026/09/22 22:21:11 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4537worker1 # [ 58.530843] nix-grpc-daemon[863]: time=2026-09-22T22:21:11.170Z level=INFO msg="Upload complete. (291ms)"4538lb # [ 58.522865] niks3-server[959]: 2026/09/22 22:21:11 INFO Completed upload id=154539lb # [ 58.527150] niks3-server[959]: 2026/09/22 22:21:11 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4540worker1 # [ 58.542848] nix-grpc-daemon[766]: ts=2026-09-22T22:21:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=4324541worker1 # [ 58.559262] nix-grpc-daemon[766]: ts=2026-09-22T22:21:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124542worker1 # [ 58.576510] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4543worker1 # [ 58.589162] nix-daemon[848]: reaped child process 1042, status = succeeded4544client # [ 58.591038] nix[991]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4545worker1 # [ 58.600081] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4546client # [ 58.985817] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4547client # Running as unit: dedup2.service; invocation ID: 185f35890eb74405b3f289db5b4526aa4548client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.12 seconds)4549client # [ 59.244724] nix[1007]: don't know how to build these paths:4550worker2 # [ 59.241067] nix-grpc-daemon[769]: ts=2026-09-22T22:21:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536904551client # [ 59.245688] nix[1007]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4552worker1 # [ 59.262783] nix-grpc-daemon[766]: ts=2026-09-22T22:21:11Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04553worker1 # [ 59.276650] nix-grpc-daemon[766]: ts=2026-09-22T22:21:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124554client # [ 59.280087] nix[1007]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4555lb # [ 59.915204] envoy[895]: conn peer=[2001:db8:1::1]:45100 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4215 tx=1911 ms=20504556lb # [ 59.920507] envoy[895]: conn peer=[2001:db8:1::1]:54570 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9515 tx=4354 ms=47944557lb # [ 59.925420] envoy[895]: conn peer=[2001:db8:1::1]:54580 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1108 tx=2926 ms=2154558lb # [ 59.930289] envoy[895]: conn peer=[2001:db8:1::1]:54602 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=544559lb # [ 59.935005] envoy[895]: conn peer=[2001:db8:1::1]:54588 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=834560lb # [ 59.939726] envoy[895]: conn peer=[2001:db8:1::1]:54584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=1044561client: must succeed: sleep 24562client: (finished: must succeed: sleep 2, in 2.06 seconds)4563client: waiting for success: ! systemctl is-active dedup1 dedup24564worker1 # [ 62.671370] nix-daemon[848]: reaped child process 1051, status = succeeded4565worker1 # [ 62.800588] nix-daemon[848]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)4566worker1 # [ 62.823890] nix-daemon[848]: reaped child process 1086, status = succeeded4567lb # [ 62.822991] niks3-server[959]: 2026/09/22 22:21:15 INFO Received uploads request method=POST path=/api/pending_closures4568worker1 # [ 62.844636] nix-grpc-daemon[863]: time=2026-09-22T22:21:15.482Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4569worker1 # [ 62.846878] nix-grpc-daemon[863]: time=2026-09-22T22:21:15.482Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4570lb # [ 62.879974] niks3-server[959]: 2026/09/22 22:21:15 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4571worker1 # [ 62.905022] nix-grpc-daemon[863]: time=2026-09-22T22:21:15.544Z level=INFO msg="Uploading 1 narinfos"4572lb # [ 62.894688] niks3-server[959]: 2026/09/22 22:21:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4573lb # [ 62.907294] niks3-server[959]: 2026/09/22 22:21:15 INFO Signed narinfos id=16 count=14574lb # [ 62.913739] niks3-server[959]: 2026/09/22 22:21:15 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4575lb # [ 62.923045] niks3-server[959]: 2026/09/22 22:21:15 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4576lb # [ 62.932892] niks3-server[959]: 2026/09/22 22:21:15 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4577lb # [ 62.940120] niks3-server[959]: 2026/09/22 22:21:15 INFO Completed upload id=164578lb # [ 62.944309] niks3-server[959]: 2026/09/22 22:21:15 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4579worker1 # [ 62.949463] nix-grpc-daemon[863]: time=2026-09-22T22:21:15.588Z level=INFO msg="Upload complete. (333ms)"4580worker1 # [ 62.965744] nix-grpc-daemon[766]: ts=2026-09-22T22:21:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14581worker1 # [ 62.987256] nix-grpc-daemon[766]: ts=2026-09-22T22:21:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14582worker1 # [ 62.995196] nix-daemon[848]: reaped child process 1049, status = succeeded4583client # [ 62.997647] systemd[1]: dedup1.service: Deactivated successfully.4584client # [ 63.003444] systemd[1]: dedup1.service: Consumed 193ms CPU time over 5.115s wall clock time, 41.5M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic.4585client # [ 63.051475] systemd[1]: dedup2.service: Deactivated successfully.4586client # [ 63.055856] systemd[1]: dedup2.service: Consumed 230ms CPU time over 4.069s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4587client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.13 seconds)4588client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4589client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.07 seconds)4590worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4591worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4592worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4593worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4594worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4595worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4596worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4597worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4598(finished: subtest: two clients wanting the same drv build it once, in 6.16 seconds)4599subtest: --timeout and --max-silent-time reach the builder4600worker2 # [ 64.221519] nix-grpc-daemon[769]: ts=2026-09-22T22:21:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536904601worker1 # [ 64.238056] nix-grpc-daemon[766]: ts=2026-09-22T22:21:16Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04602worker1 # [ 64.244561] nix-grpc-daemon[766]: ts=2026-09-22T22:21:16Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.4:50051 clients=14603worker2 # [ 64.244760] nix-grpc-daemon[769]: ts=2026-09-22T22:21:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904604worker2 # [ 64.263134] nix-grpc-daemon[769]: ts=2026-09-22T22:21:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904605worker2 # [ 64.281974] nix-grpc-daemon[769]: ts=2026-09-22T22:21:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536904606worker2 # [ 64.296829] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4607worker2 # [ 64.466351] nix-daemon[837]: accepted connection from pid 989, user nix-grpc-daemon (trusted)4608worker2 # [ 64.494743] nix-daemon[837]: reaped child process 992, status = succeeded4609lb # [ 64.487221] niks3-server[959]: 2026/09/22 22:21:17 INFO Received uploads request method=POST path=/api/pending_closures4610worker2 # [ 64.504529] nix-grpc-daemon[865]: time=2026-09-22T22:21:17.146Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4611worker2 # [ 64.506957] nix-grpc-daemon[865]: time=2026-09-22T22:21:17.146Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4612lb # [ 64.525073] niks3-server[959]: 2026/09/22 22:21:17 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4613worker2 # [ 64.543122] nix-grpc-daemon[865]: time=2026-09-22T22:21:17.185Z level=INFO msg="Uploading 1 narinfos"4614lb # [ 64.537809] niks3-server[959]: 2026/09/22 22:21:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4615lb # [ 64.545792] niks3-server[959]: 2026/09/22 22:21:17 INFO Signed narinfos id=17 count=14616lb # [ 64.557581] niks3-server[959]: 2026/09/22 22:21:17 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4617lb # [ 64.566132] niks3-server[959]: 2026/09/22 22:21:17 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4618lb # [ 64.572293] niks3-server[959]: 2026/09/22 22:21:17 INFO Completed upload id=174619worker2 # [ 64.577879] nix-grpc-daemon[865]: time=2026-09-22T22:21:17.220Z level=INFO msg="Upload complete. (262ms)"4620lb # [ 64.578667] niks3-server[959]: 2026/09/22 22:21:17 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4621worker2 # [ 64.588423] nix-grpc-daemon[769]: ts=2026-09-22T22:21:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=4244622worker2 # [ 64.609088] nix-grpc-daemon[769]: ts=2026-09-22T22:21:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904623worker2 # [ 64.617563] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4624worker2 # [ 64.624384] nix-daemon[837]: reaped child process 987, status = succeeded4625worker2 # [ 64.641834] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4626lb # [ 69.914871] envoy[895]: conn peer=[2001:db8:1::1]:54612 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3018 tx=1563 ms=48844627lb # [ 69.915162] envoy[895]: conn peer=[2001:db8:1::1]:54622 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1197 ms=37904628worker2 # [ 71.060869] nix-grpc-daemon[769]: ts=2026-09-22T22:21:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=6 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04629worker2 # [ 71.097751] nix-daemon[837]: reaped child process 994, status = succeeded4630worker2 # [ 71.100673] nix-daemon[837]: reaped child process 996, status = succeeded4631worker1 # [ 71.309022] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400124632worker1 # [ 71.326677] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04633worker1 # [ 71.334284] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.3:50051 clients=14634worker1 # [ 71.343951] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124635worker1 # [ 71.355750] nix-grpc-daemon[766]: ts=2026-09-22T22:21:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124636worker1 # [ 71.371896] nix-grpc-daemon[766]: ts=2026-09-22T22:21:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124637worker1 # [ 71.387571] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4638worker1 # [ 71.548921] nix-daemon[848]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)4639worker1 # [ 71.572678] nix-daemon[848]: reaped child process 1106, status = succeeded4640lb # [ 71.561250] niks3-server[959]: 2026/09/22 22:21:24 INFO Received uploads request method=POST path=/api/pending_closures4641worker1 # [ 71.584438] nix-grpc-daemon[863]: time=2026-09-22T22:21:24.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4642worker1 # [ 71.585974] nix-grpc-daemon[863]: time=2026-09-22T22:21:24.223Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4643lb # [ 71.601349] niks3-server[959]: 2026/09/22 22:21:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4644worker1 # [ 71.622490] nix-grpc-daemon[863]: time=2026-09-22T22:21:24.261Z level=INFO msg="Uploading 1 narinfos"4645lb # [ 71.614308] niks3-server[959]: 2026/09/22 22:21:24 INFO Signed narinfos id=18 count=14646lb # [ 71.624847] niks3-server[959]: 2026/09/22 22:21:24 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4647lb # [ 71.635260] niks3-server[959]: 2026/09/22 22:21:24 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4648lb # [ 71.646912] niks3-server[959]: 2026/09/22 22:21:24 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4649lb # [ 71.653051] niks3-server[959]: 2026/09/22 22:21:24 INFO Completed upload id=184650worker1 # [ 71.662795] nix-grpc-daemon[863]: time=2026-09-22T22:21:24.302Z level=INFO msg="Upload complete. (257ms)"4651lb # [ 71.657907] niks3-server[959]: 2026/09/22 22:21:24 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4652worker1 # [ 71.675206] nix-grpc-daemon[766]: ts=2026-09-22T22:21:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=4244653worker1 # [ 71.692468] nix-grpc-daemon[766]: ts=2026-09-22T22:21:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124654worker1 # [ 71.700245] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4655worker1 # [ 71.706564] nix-daemon[848]: reaped child process 1101, status = succeeded4656worker1 # [ 71.723792] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4657worker1 # [ 77.542414] nix-grpc-daemon[766]: ts=2026-09-22T22:21:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=5 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04658worker1 # [ 77.568537] nix-daemon[848]: reaped child process 1108, status = succeeded4659worker1 # [ 77.570223] nix-daemon[848]: reaped child process 1110, status = succeeded4660(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.77 seconds)4661subtest: the first client leaving does not fail the second4662client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4663client # [ 77.830581] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4664client # Running as unit: share1.service; invocation ID: b16d88b03a2447dc835347e06b7797334665client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.10 seconds)4666client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4667client # [ 78.062348] nix[1079]: don't know how to build these paths:4668client # [ 78.063479] nix[1079]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4669worker2 # [ 78.061290] nix-grpc-daemon[769]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536904670worker1 # [ 78.078843] nix-grpc-daemon[766]: ts=2026-09-22T22:21:30Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04671worker1 # [ 78.085460] nix-grpc-daemon[766]: ts=2026-09-22T22:21:30Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.4:50051 clients=14672worker2 # [ 78.085098] nix-grpc-daemon[769]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904673worker2 # [ 78.102399] nix-grpc-daemon[769]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904674client # [ 78.114934] nix[1079]: copying 1 paths to 'lb:50051'...4675client # [ 78.118985] nix[1079]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4676worker2 # [ 78.119894] nix-grpc-daemon[769]: ts=2026-09-22T22:21:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536904677worker2 # [ 78.135559] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4678worker2 # [ 78.307333] nix-daemon[837]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4679worker2 # [ 78.330303] nix-daemon[837]: reaped child process 1022, status = succeeded4680lb # [ 78.323400] niks3-server[959]: 2026/09/22 22:21:30 INFO Received uploads request method=POST path=/api/pending_closures4681worker2 # [ 78.340712] nix-grpc-daemon[865]: time=2026-09-22T22:21:30.983Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4682worker2 # [ 78.342232] nix-grpc-daemon[865]: time=2026-09-22T22:21:30.983Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4683lb # [ 78.363220] niks3-server[959]: 2026/09/22 22:21:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4684worker2 # [ 78.381180] nix-grpc-daemon[865]: time=2026-09-22T22:21:31.023Z level=INFO msg="Uploading 1 narinfos"4685lb # [ 78.377691] niks3-server[959]: 2026/09/22 22:21:31 INFO Signed narinfos id=19 count=14686lb # [ 78.385473] niks3-server[959]: 2026/09/22 22:21:31 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4687lb # [ 78.396559] niks3-server[959]: 2026/09/22 22:21:31 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4688lb # [ 78.411562] niks3-server[959]: 2026/09/22 22:21:31 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4689worker2 # [ 78.426595] nix-grpc-daemon[865]: time=2026-09-22T22:21:31.069Z level=INFO msg="Upload complete. (273ms)"4690lb # [ 78.422739] niks3-server[959]: 2026/09/22 22:21:31 INFO Completed upload id=194691lb # [ 78.427758] niks3-server[959]: 2026/09/22 22:21:31 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4692worker2 # [ 78.436577] nix-grpc-daemon[769]: ts=2026-09-22T22:21:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=4324693worker2 # [ 78.460287] nix-grpc-daemon[769]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904694worker2 # [ 78.468000] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4695worker2 # [ 78.474871] nix-daemon[837]: reaped child process 1017, status = succeeded4696client # [ 78.486204] nix[1079]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4697worker2 # [ 78.491418] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4698client # [ 78.950485] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4699client # Running as unit: share2.service; invocation ID: 3610e887514745d9b6db841dcc44fc4f4700client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.12 seconds)4701client # [ 79.149866] nix[1095]: don't know how to build these paths:4702client # [ 79.150899] nix[1095]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4703worker1 # [ 79.153129] nix-grpc-daemon[766]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400124704worker1 # [ 79.168230] nix-grpc-daemon[766]: ts=2026-09-22T22:21:31Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04705worker2 # [ 79.218231] nix-grpc-daemon[769]: ts=2026-09-22T22:21:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904706client # [ 79.227713] nix[1095]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4707lb # [ 79.915991] envoy[895]: conn peer=[2001:db8:1::1]:37698 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3001 tx=978 ms=68634708lb # [ 79.916484] envoy[895]: conn peer=[2001:db8:1::1]:35736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3015 tx=977 ms=62924709client: must succeed: sleep 2; systemctl kill -s INT share14710client # [ 82.343047] systemd[1]: share1.service: Sent signal SIGINT to main process 1079 (nix) on client request.4711client # [ 82.350769] nix[1079]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4712client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4713client: waiting for success: ! systemctl is-active share14714client # [ 82.366451] nix[1079]: error: interrupted by the user4715client # [ 82.425552] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4716client # [ 82.428608] systemd[1]: share1.service: Failed with result 'exit-code'.4717client # [ 82.431129] systemd[1]: share1.service: Consumed 190ms CPU time over 4.598s wall clock time, 41.4M memory peak, 6.2K incoming IP traffic, 8.3K outgoing IP traffic.4718client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4719client: waiting for success: ! systemctl is-active share24720worker2 # [ 83.975068] nix-daemon[837]: reaped child process 1026, status = succeeded4721worker2 # [ 84.083420] nix-daemon[837]: accepted connection from pid 1060, user nix-grpc-daemon (trusted)4722worker2 # [ 84.108678] nix-daemon[837]: reaped child process 1063, status = succeeded4723lb # [ 84.100825] niks3-server[959]: 2026/09/22 22:21:36 INFO Received uploads request method=POST path=/api/pending_closures4724worker2 # [ 84.120479] nix-grpc-daemon[865]: time=2026-09-22T22:21:36.762Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4725worker2 # [ 84.122087] nix-grpc-daemon[865]: time=2026-09-22T22:21:36.762Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4726lb # [ 84.153558] niks3-server[959]: 2026/09/22 22:21:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4727worker2 # [ 84.168696] nix-grpc-daemon[865]: time=2026-09-22T22:21:36.811Z level=INFO msg="Uploading 1 narinfos"4728lb # [ 84.164674] niks3-server[959]: 2026/09/22 22:21:36 INFO Signed narinfos id=20 count=14729lb # [ 84.172849] niks3-server[959]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4730lb # [ 84.181917] niks3-server[959]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4731lb # [ 84.190411] niks3-server[959]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4732lb # [ 84.199563] niks3-server[959]: 2026/09/22 22:21:36 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4733lb # [ 84.206748] niks3-server[959]: 2026/09/22 22:21:36 INFO Completed upload id=204734worker2 # [ 84.211643] nix-grpc-daemon[865]: time=2026-09-22T22:21:36.854Z level=INFO msg="Upload complete. (302ms)"4735lb # [ 84.209526] niks3-server[959]: 2026/09/22 22:21:36 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4736worker2 # [ 84.228551] nix-grpc-daemon[769]: ts=2026-09-22T22:21:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14737worker2 # [ 84.244486] nix-grpc-daemon[769]: ts=2026-09-22T22:21:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14738worker2 # [ 84.251222] nix-daemon[837]: reaped child process 1024, status = succeeded4739client # [ 84.303509] systemd[1]: share2.service: Deactivated successfully.4740client # [ 84.308120] systemd[1]: share2.service: Consumed 180ms CPU time over 5.354s wall clock time, 33.3M memory peak, 6.2K incoming IP traffic, 5.6K outgoing IP traffic.4741client: (finished: waiting for success: ! systemctl is-active share2, in 1.10 seconds)4742client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4743client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.04 seconds)4744(finished: subtest: the first client leaving does not fail the second, in 7.12 seconds)4745subtest: build hook: nix-daemon with builders = grpc://lb4746client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4747client # [ 85.008385] systemd[1]: Started Nix Daemon.4748client # [ 85.133064] nix-daemon[1147]: accepted connection from pid 1144, user root (trusted)4749worker1 # [ 85.294350] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:400124750worker1 # [ 85.302311] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124751worker1 # [ 85.316142] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04752worker1 # [ 85.322906] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.3:50051 clients=14753worker1 # [ 85.333601] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124754worker1 # [ 85.345932] nix-grpc-daemon[766]: ts=2026-09-22T22:21:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124755worker1 # [ 85.362277] nix-grpc-daemon[766]: ts=2026-09-22T22:21:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124756worker1 # [ 85.379152] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4757worker1 # [ 85.534183] nix-daemon[848]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)4758worker1 # [ 85.558667] nix-daemon[848]: reaped child process 1158, status = succeeded4759lb # [ 85.547468] niks3-server[959]: 2026/09/22 22:21:38 INFO Received uploads request method=POST path=/api/pending_closures4760worker1 # [ 85.567509] nix-grpc-daemon[863]: time=2026-09-22T22:21:38.206Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4761worker1 # [ 85.569397] nix-grpc-daemon[863]: time=2026-09-22T22:21:38.206Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4762lb # [ 85.584610] niks3-server[959]: 2026/09/22 22:21:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4763worker1 # [ 85.602101] nix-grpc-daemon[863]: time=2026-09-22T22:21:38.241Z level=INFO msg="Uploading 1 narinfos"4764lb # [ 85.594722] niks3-server[959]: 2026/09/22 22:21:38 INFO Signed narinfos id=21 count=14765lb # [ 85.603583] niks3-server[959]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4766lb # [ 85.614711] niks3-server[959]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4767lb # [ 85.624261] niks3-server[959]: 2026/09/22 22:21:38 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4768lb # [ 85.627162] niks3-server[959]: 2026/09/22 22:21:38 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4769lb # [ 85.633075] niks3-server[959]: 2026/09/22 22:21:38 INFO Completed upload id=214770worker1 # [ 85.642284] nix-grpc-daemon[863]: time=2026-09-22T22:21:38.281Z level=INFO msg="Upload complete. (247ms)"4771worker1 # [ 85.653208] nix-grpc-daemon[766]: ts=2026-09-22T22:21:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=5124772worker1 # [ 85.664592] nix-grpc-daemon[766]: ts=2026-09-22T22:21:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124773worker1 # [ 85.677032] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4774worker1 # [ 85.682210] nix-daemon[848]: reaped child process 1153, status = succeeded4775worker1 # [ 85.695336] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4776worker1 # [ 86.596303] nix-daemon[848]: reaped child process 1162, status = succeeded4777worker1 # [ 86.679126] nix-daemon[848]: accepted connection from pid 1178, user nix-grpc-daemon (trusted)4778worker1 # [ 86.703993] nix-daemon[848]: reaped child process 1181, status = succeeded4779lb # [ 86.693527] niks3-server[959]: 2026/09/22 22:21:39 INFO Received uploads request method=POST path=/api/pending_closures4780worker1 # [ 86.715496] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.353Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4781worker1 # [ 86.717172] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.353Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4782lb # [ 86.737024] niks3-server[959]: 2026/09/22 22:21:39 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4783worker1 # [ 86.760470] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.399Z level=INFO msg="Uploading 1 narinfos"4784lb # [ 86.751978] niks3-server[959]: 2026/09/22 22:21:39 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4785lb # [ 86.763422] niks3-server[959]: 2026/09/22 22:21:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4786lb # [ 86.771846] niks3-server[959]: 2026/09/22 22:21:39 INFO Signed narinfos id=22 count=14787lb # [ 86.777347] niks3-server[959]: 2026/09/22 22:21:39 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4788lb # [ 86.779852] niks3-server[959]: 2026/09/22 22:21:39 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4789lb # [ 86.784223] niks3-server[959]: 2026/09/22 22:21:39 INFO Completed upload id=224790lb # [ 86.786361] niks3-server[959]: 2026/09/22 22:21:39 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4791worker1 # [ 86.785993] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.425Z level=INFO msg="Upload complete. (280ms)"4792worker1 # [ 86.801620] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14793worker1 # [ 86.820760] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:400124794worker1 # [ 86.827011] nix-daemon[848]: reaped child process 1160, status = succeeded4795worker1 # [ 86.838550] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:400124796worker1 # [ 86.845176] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=1284797worker1 # [ 87.124136] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:400124798worker1 # [ 87.134550] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124799worker1 # [ 87.144694] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.784Z level=INFO msg="All 1 paths already cached"4800worker1 # [ 87.164618] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04801worker1 # [ 87.171411] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.3:50051 clients=14802worker1 # [ 87.180582] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124803worker1 # [ 87.192741] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124804worker1 # [ 87.206905] nix-grpc-daemon[863]: time=2026-09-22T22:21:39.846Z level=INFO msg="All 1 paths already cached"4805worker1 # [ 87.217215] nix-grpc-daemon[766]: ts=2026-09-22T22:21:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124806worker1 # [ 87.231864] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4807worker1 # [ 87.394881] nix-daemon[848]: accepted connection from pid 1187, user nix-grpc-daemon (trusted)4808worker1 # [ 87.416750] nix-daemon[848]: reaped child process 1190, status = succeeded4809lb # [ 87.412782] niks3-server[959]: 2026/09/22 22:21:40 INFO Received uploads request method=POST path=/api/pending_closures4810worker1 # [ 87.433578] nix-grpc-daemon[863]: time=2026-09-22T22:21:40.072Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4811worker1 # [ 87.435161] nix-grpc-daemon[863]: time=2026-09-22T22:21:40.072Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4812lb # [ 87.456369] niks3-server[959]: 2026/09/22 22:21:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4813worker1 # [ 87.475856] nix-grpc-daemon[863]: time=2026-09-22T22:21:40.115Z level=INFO msg="Uploading 1 narinfos"4814lb # [ 87.471873] niks3-server[959]: 2026/09/22 22:21:40 INFO Signed narinfos id=23 count=14815lb # [ 87.477325] niks3-server[959]: 2026/09/22 22:21:40 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4816lb # [ 87.484868] niks3-server[959]: 2026/09/22 22:21:40 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4817worker1 # [ 87.502000] nix-grpc-daemon[863]: time=2026-09-22T22:21:40.141Z level=INFO msg="Upload complete. (252ms)"4818lb # [ 87.493759] niks3-server[959]: 2026/09/22 22:21:40 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4819lb # [ 87.501485] niks3-server[959]: 2026/09/22 22:21:40 INFO Completed upload id=234820lb # [ 87.504504] niks3-server[959]: 2026/09/22 22:21:40 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4821worker1 # [ 87.512321] nix-grpc-daemon[766]: ts=2026-09-22T22:21:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=6244822worker1 # [ 87.524696] nix-grpc-daemon[766]: ts=2026-09-22T22:21:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124823worker1 # [ 87.539686] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4824worker1 # [ 87.545905] nix-daemon[848]: reaped child process 1185, status = succeeded4825worker1 # [ 87.558798] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4826worker1 # [ 88.644705] nix-daemon[848]: reaped child process 1194, status = succeeded4827worker1 # [ 88.770541] nix-daemon[848]: accepted connection from pid 1210, user nix-grpc-daemon (trusted)4828worker1 # [ 88.797628] nix-daemon[848]: reaped child process 1213, status = succeeded4829lb # [ 88.787600] niks3-server[959]: 2026/09/22 22:21:41 INFO Received uploads request method=POST path=/api/pending_closures4830worker1 # [ 88.815803] nix-grpc-daemon[863]: time=2026-09-22T22:21:41.451Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4831worker1 # [ 88.817553] nix-grpc-daemon[863]: time=2026-09-22T22:21:41.451Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4832lb # [ 88.853668] niks3-server[959]: 2026/09/22 22:21:41 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4833lb # [ 88.866277] niks3-server[959]: 2026/09/22 22:21:41 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4834worker1 # [ 88.892325] nix-grpc-daemon[863]: time=2026-09-22T22:21:41.531Z level=INFO msg="Uploading 1 narinfos"4835lb # [ 88.875879] niks3-server[959]: 2026/09/22 22:21:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4836lb # [ 88.888509] niks3-server[959]: 2026/09/22 22:21:41 INFO Signed narinfos id=24 count=14837lb # [ 88.894426] niks3-server[959]: 2026/09/22 22:21:41 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4838lb # [ 88.912802] niks3-server[959]: 2026/09/22 22:21:41 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4839lb # [ 88.919615] niks3-server[959]: 2026/09/22 22:21:41 INFO Completed upload id=244840worker1 # [ 88.929715] nix-grpc-daemon[863]: time=2026-09-22T22:21:41.568Z level=INFO msg="Upload complete. (355ms)"4841lb # [ 88.922410] niks3-server[959]: 2026/09/22 22:21:41 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4842worker1 # [ 88.945681] nix-grpc-daemon[766]: ts=2026-09-22T22:21:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14843worker1 # [ 88.967100] nix-daemon[848]: reaped child process 1192, status = succeeded4844worker2 # [ 88.993674] nix-grpc-daemon[769]: ts=2026-09-22T22:21:41Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536904845worker2 # [ 89.026581] nix-grpc-daemon[769]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4846worker2 # [ 89.048463] nix-grpc-daemon[769]: copying path '/nix/store/la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b' from 'http://lb:5751'...4847worker1 # [ 89.104813] nix-grpc-daemon[766]: ts=2026-09-22T22:21:41Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:400124848worker1 # [ 89.113512] nix-grpc-daemon[766]: ts=2026-09-22T22:21:41Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=1764849worker2 # [ 89.403802] nix-grpc-daemon[769]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:536904850worker1 # [ 89.419732] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124851worker1 # [ 89.432791] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.072Z level=INFO msg="All 2 paths already cached"4852worker1 # [ 89.459674] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04853worker1 # [ 89.467911] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.3:50051 clients=14854worker1 # [ 89.478296] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124855worker1 # [ 89.492981] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400124856worker1 # [ 89.511335] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.150Z level=INFO msg="All 2 paths already cached"4857worker1 # [ 89.525279] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124858worker1 # [ 89.543141] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4859worker1 # [ 89.727631] nix-daemon[848]: accepted connection from pid 1219, user nix-grpc-daemon (trusted)4860worker1 # [ 89.752659] nix-daemon[848]: reaped child process 1222, status = succeeded4861lb # [ 89.748148] niks3-server[959]: 2026/09/22 22:21:42 INFO Received uploads request method=POST path=/api/pending_closures4862worker1 # [ 89.771512] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.410Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4863worker1 # [ 89.773488] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.410Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4864lb # [ 89.798610] niks3-server[959]: 2026/09/22 22:21:42 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4865worker1 # [ 89.821847] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.459Z level=INFO msg="Uploading 1 narinfos"4866lb # [ 89.814759] niks3-server[959]: 2026/09/22 22:21:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4867lb # [ 89.822479] niks3-server[959]: 2026/09/22 22:21:42 INFO Signed narinfos id=25 count=14868lb # [ 89.827206] niks3-server[959]: 2026/09/22 22:21:42 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4869worker1 # [ 89.846966] nix-grpc-daemon[863]: time=2026-09-22T22:21:42.486Z level=INFO msg="Upload complete. (281ms)"4870lb # [ 89.839572] niks3-server[959]: 2026/09/22 22:21:42 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4871lb # [ 89.846218] niks3-server[959]: 2026/09/22 22:21:42 INFO Completed upload id=254872lb # [ 89.849636] niks3-server[959]: 2026/09/22 22:21:42 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4873worker1 # [ 89.860914] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=7524874worker1 # [ 89.873030] nix-grpc-daemon[766]: ts=2026-09-22T22:21:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400124875worker1 # [ 89.889537] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4876worker1 # [ 89.897709] nix-daemon[848]: reaped child process 1217, status = succeeded4877worker1 # [ 89.914746] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4878lb # [ 89.918510] envoy[895]: conn peer=[2001:db8:1::1]:35752 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3029 tx=790 ms=43134879lb # [ 89.923681] envoy[895]: conn peer=[2001:db8:1::1]:35762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1198 ms=51344880lb # [ 89.929227] envoy[895]: conn peer=[2001:db8:1::1]:53388 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3420 tx=1966 ms=16114881lb # [ 89.934277] envoy[895]: conn peer=[2001:db8:1::1]:53400 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=694882lb # [ 89.939172] envoy[895]: conn peer=[2001:db8:1::1]:53406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3904 tx=2173 ms=20094883lb # [ 89.944396] envoy[895]: conn peer=[2001:db8:1::1]:53418 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=375 ms=404884worker1 # [ 90.838568] nix-daemon[848]: reaped child process 1226, status = succeeded4885worker1 # [ 90.920663] nix-daemon[848]: accepted connection from pid 1242, user nix-grpc-daemon (trusted)4886worker1 # [ 90.945566] nix-daemon[848]: reaped child process 1245, status = succeeded4887lb # [ 90.942415] niks3-server[959]: 2026/09/22 22:21:43 INFO Received uploads request method=POST path=/api/pending_closures4888worker1 # [ 90.968509] nix-grpc-daemon[863]: time=2026-09-22T22:21:43.605Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4889worker1 # [ 90.971001] nix-grpc-daemon[863]: time=2026-09-22T22:21:43.606Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4890lb # [ 91.013088] niks3-server[959]: 2026/09/22 22:21:43 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4891lb # [ 91.035169] niks3-server[959]: 2026/09/22 22:21:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4892lb # [ 91.055989] niks3-server[959]: 2026/09/22 22:21:43 INFO Signed narinfos id=26 count=14893worker1 # [ 91.066159] nix-grpc-daemon[863]: time=2026-09-22T22:21:43.704Z level=INFO msg="Uploading 1 narinfos"4894lb # [ 91.067071] niks3-server[959]: 2026/09/22 22:21:43 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4895lb # [ 91.076815] niks3-server[959]: 2026/09/22 22:21:43 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4896lb # [ 91.092740] niks3-server[959]: 2026/09/22 22:21:43 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4897lb # [ 91.101222] niks3-server[959]: 2026/09/22 22:21:43 INFO Completed upload id=264898worker1 # [ 91.113048] nix-grpc-daemon[863]: time=2026-09-22T22:21:43.751Z level=INFO msg="Upload complete. (382ms)"4899worker1 # [ 91.141828] nix-grpc-daemon[766]: ts=2026-09-22T22:21:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14900worker1 # [ 91.172232] nix-daemon[848]: reaped child process 1224, status = succeeded4901lb # [ 91.733476] niks3-server[959]: 2026/09/22 22:21:43 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4902worker2 # [ 91.955983] nix-grpc-daemon[769]: ts=2026-09-22T22:21:44Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536904903worker2 # [ 91.979955] nix-grpc-daemon[769]: copying path '/nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top' from 'http://lb:5751'...4904worker2 # [ 92.057667] nix-grpc-daemon[769]: ts=2026-09-22T22:21:44Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:536904905worker2 # [ 92.067662] nix-grpc-daemon[769]: ts=2026-09-22T22:21:44Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_out=2324906client # [ 92.208758] nix-daemon[1147]: reaped child process 1149, status = succeeded4907client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.36 seconds)4908client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4909client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.09 seconds)4910client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4911client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4912client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4913client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4914(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.53 seconds)4915subtest: an input only one worker has still reaches the builder4916client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4917client # this derivation will be built:4918client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4919client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4920client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.33 seconds)4921client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4922client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.13 seconds)4923worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4924worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.21 seconds)4925worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4926worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.04 seconds)4927client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24928client # [ 93.303328] nix-daemon[1147]: accepted connection from pid 1230, user root (trusted)4929client # this derivation will be built:4930client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4931client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4932worker1 # [ 93.461661] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:400124933worker2 # [ 93.466113] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904934client # copying 1 paths...4935client # copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' to 'grpc://lb:50051'...4936worker1 # [ 93.482990] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400124937worker1 # [ 93.500560] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4938worker1 # [ 93.675180] nix-daemon[848]: accepted connection from pid 1254, user nix-grpc-daemon (trusted)4939worker1 # [ 93.698261] nix-daemon[848]: reaped child process 1257, status = succeeded4940lb # [ 93.693535] niks3-server[959]: 2026/09/22 22:21:46 INFO Received uploads request method=POST path=/api/pending_closures4941worker1 # [ 93.714544] nix-grpc-daemon[863]: time=2026-09-22T22:21:46.353Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4942worker1 # [ 93.716216] nix-grpc-daemon[863]: time=2026-09-22T22:21:46.353Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4943lb # [ 93.723843] niks3-server[959]: 2026/09/22 22:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4944worker1 # [ 93.745600] nix-grpc-daemon[863]: time=2026-09-22T22:21:46.384Z level=INFO msg="Uploading 1 narinfos"4945lb # [ 93.734815] niks3-server[959]: 2026/09/22 22:21:46 INFO Signed narinfos id=27 count=14946lb # [ 93.747584] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4947lb # [ 93.758080] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4948worker1 # [ 93.780606] nix-grpc-daemon[863]: time=2026-09-22T22:21:46.419Z level=INFO msg="Upload complete. (263ms)"4949lb # [ 93.771748] niks3-server[959]: 2026/09/22 22:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4950lb # [ 93.775767] niks3-server[959]: 2026/09/22 22:21:46 INFO Completed upload id=274951lb # [ 93.779935] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4952worker1 # [ 93.792447] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_in=1204953worker1 # [ 93.814627] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04954worker1 # [ 93.819495] nix-grpc-daemon[766]: ts=2026-09-22T22:21:46Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=14955worker1 # [ 93.824795] nix-daemon[848]: reaped child process 1252, status = succeeded4956worker2 # [ 93.850840] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904957worker2 # [ 93.867996] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536904958client # copying 2 paths to 'lb:50051'...4959client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4960worker2 # [ 93.887851] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536904961client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4962worker2 # [ 93.910289] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4963worker2 # [ 94.086599] nix-daemon[837]: accepted connection from pid 1073, user nix-grpc-daemon (trusted)4964worker2 # [ 94.108127] nix-daemon[837]: reaped child process 1076, status = succeeded4965lb # [ 94.108968] niks3-server[959]: 2026/09/22 22:21:46 INFO Received uploads request method=POST path=/api/pending_closures4966lb # [ 94.118226] niks3-server[959]: 2026/09/22 22:21:46 INFO Received uploads request method=POST path=/api/pending_closures4967worker2 # [ 94.136489] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.777Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4968worker2 # [ 94.138287] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.777Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4969worker2 # [ 94.142325] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.777Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4970lb # [ 94.160115] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4971lb # [ 94.172181] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4972lb # [ 94.186411] niks3-server[959]: 2026/09/22 22:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4973worker2 # [ 94.199114] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.840Z level=INFO msg="Uploading 3 narinfos"4974lb # [ 94.195920] niks3-server[959]: 2026/09/22 22:21:46 INFO Signed narinfos id=28 count=14975lb # [ 94.201636] niks3-server[959]: 2026/09/22 22:21:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4976lb # [ 94.209150] niks3-server[959]: 2026/09/22 22:21:46 INFO Signed narinfos id=29 count=24977lb # [ 94.211035] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4978lb # [ 94.219267] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4979lb # [ 94.231965] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4980lb # [ 94.236110] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4981lb # [ 94.243575] niks3-server[959]: 2026/09/22 22:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4982lb # [ 94.249356] niks3-server[959]: 2026/09/22 22:21:46 INFO Completed upload id=284983lb # [ 94.251115] niks3-server[959]: 2026/09/22 22:21:46 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4984lb # [ 94.254688] niks3-server[959]: 2026/09/22 22:21:46 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4985worker2 # [ 94.260814] nix-grpc-daemon[865]: time=2026-09-22T22:21:46.903Z level=INFO msg="Upload complete. (325ms)"4986lb # [ 94.257826] niks3-server[959]: 2026/09/22 22:21:46 INFO Completed upload id=294987worker2 # [ 94.271543] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=2 nar_bytes_in=9844988worker2 # [ 94.288312] nix-grpc-daemon[769]: ts=2026-09-22T22:21:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536904989worker2 # [ 94.302973] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4990worker2 # [ 94.314232] nix-daemon[837]: reaped child process 1071, status = succeeded4991worker2 # [ 94.325264] nix-grpc-daemon[769]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...4992client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4993client # uses-input-w1onlya> worker2: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4994worker2 # [ 94.346546] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)4995client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4996worker2 # [ 95.608659] nix-daemon[837]: reaped child process 1081, status = succeeded4997worker2 # [ 95.722943] nix-daemon[837]: accepted connection from pid 1097, user nix-grpc-daemon (trusted)4998worker2 # [ 95.745822] nix-daemon[837]: reaped child process 1100, status = succeeded4999lb # [ 95.738402] niks3-server[959]: 2026/09/22 22:21:48 INFO Received uploads request method=POST path=/api/pending_closures5000worker2 # [ 95.755927] nix-grpc-daemon[865]: time=2026-09-22T22:21:48.396Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5001lb # [ 95.776847] niks3-server[959]: 2026/09/22 22:21:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5002worker2 # [ 95.794556] nix-grpc-daemon[865]: time=2026-09-22T22:21:48.437Z level=INFO msg="Uploading 1 narinfos"5003lb # [ 95.791218] niks3-server[959]: 2026/09/22 22:21:48 INFO Signed narinfos id=30 count=15004lb # [ 95.799726] niks3-server[959]: 2026/09/22 22:21:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls5005lb # [ 95.809786] niks3-server[959]: 2026/09/22 22:21:48 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv5006worker2 # [ 95.828842] nix-grpc-daemon[865]: time=2026-09-22T22:21:48.471Z level=INFO msg="Upload complete. (267ms)"5007lb # [ 95.823930] niks3-server[959]: 2026/09/22 22:21:48 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5008lb # [ 95.829234] niks3-server[959]: 2026/09/22 22:21:48 INFO Completed upload id=305009lb # [ 95.833339] niks3-server[959]: 2026/09/22 22:21:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo5010client # copying 1 paths...5011worker2 # [ 95.841916] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=15012client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...5013worker2 # [ 95.864602] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536905014worker2 # [ 95.870543] nix-daemon[837]: reaped child process 1078, status = succeeded5015worker2 # [ 95.888493] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:536905016worker2 # [ 95.895385] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_out=1205017client # [ 96.028232] nix-daemon[1147]: reaped child process 1233, status = succeeded5018client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.93 seconds)5019client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&25020client # [ 96.193540] nix-daemon[1147]: accepted connection from pid 1251, user root (trusted)5021client # this derivation will be built:5022client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5023client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5024worker1 # [ 96.349758] nix-grpc-daemon[766]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:400125025worker2 # [ 96.350305] nix-grpc-daemon[769]: ts=2026-09-22T22:21:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536905026worker2 # [ 96.362919] nix-grpc-daemon[865]: time=2026-09-22T22:21:49.005Z level=INFO msg="All 1 paths already cached"5027worker1 # [ 96.384823] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05028worker1 # [ 96.392625] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15029worker1 # [ 96.403421] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400125030worker1 # [ 96.415621] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:400125031client # copying 1 paths to 'lb:50051'...5032client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5033worker1 # [ 96.438002] nix-grpc-daemon[766]: ts=2026-09-22T22:21:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:400125034worker1 # [ 96.453735] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5035worker1 # [ 96.475494] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 35 ms (attempt 1/5)5036worker1 # [ 96.517967] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 87 ms (attempt 2/5)5037worker1 # [ 96.607127] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 219 ms (attempt 3/5)5038worker1 # [ 96.828209] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 572 ms (attempt 4/5)5039worker1 # [ 97.403728] nix-grpc-daemon[766]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5040worker1 # [ 97.410598] nix-grpc-daemon[766]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5041worker1 # [ 97.418710] nix-grpc-daemon[766]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5042worker1 # [ 97.593798] nix-daemon[848]: accepted connection from pid 1270, user nix-grpc-daemon (trusted)5043worker1 # [ 97.614351] nix-daemon[848]: reaped child process 1273, status = succeeded5044lb # [ 97.610196] niks3-server[959]: 2026/09/22 22:21:50 INFO Received uploads request method=POST path=/api/pending_closures5045worker1 # [ 97.633334] nix-grpc-daemon[863]: time=2026-09-22T22:21:50.271Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5046worker1 # [ 97.634977] nix-grpc-daemon[863]: time=2026-09-22T22:21:50.271Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5047lb # [ 97.648830] niks3-server[959]: 2026/09/22 22:21:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5048worker1 # [ 97.665579] nix-grpc-daemon[863]: time=2026-09-22T22:21:50.304Z level=INFO msg="Uploading 1 narinfos"5049lb # [ 97.660903] niks3-server[959]: 2026/09/22 22:21:50 INFO Signed narinfos id=31 count=15050lb # [ 97.668358] niks3-server[959]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5051lb # [ 97.678619] niks3-server[959]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5052worker1 # [ 97.700333] nix-grpc-daemon[863]: time=2026-09-22T22:21:50.339Z level=INFO msg="Upload complete. (261ms)"5053lb # [ 97.690936] niks3-server[959]: 2026/09/22 22:21:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5054lb # [ 97.697339] niks3-server[959]: 2026/09/22 22:21:50 INFO Completed upload id=315055lb # [ 97.704231] niks3-server[959]: 2026/09/22 22:21:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5056worker1 # [ 97.711957] nix-grpc-daemon[766]: ts=2026-09-22T22:21:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 paths=1 nar_bytes_in=5685057worker1 # [ 97.724480] nix-grpc-daemon[766]: ts=2026-09-22T22:21:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:400125058worker1 # [ 97.737562] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5059worker1 # [ 97.742903] nix-daemon[848]: reaped child process 1263, status = succeeded5060client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5061worker1 # [ 97.757898] nix-daemon[848]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5062client # uses-input-w1onlyb> node-a: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5063client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5064worker1 # [ 99.426939] nix-daemon[848]: reaped child process 1277, status = succeeded5065worker1 # [ 99.548539] nix-daemon[848]: accepted connection from pid 1293, user nix-grpc-daemon (trusted)5066worker1 # [ 99.569477] nix-daemon[848]: reaped child process 1296, status = succeeded5067lb # [ 99.559632] niks3-server[959]: 2026/09/22 22:21:52 INFO Received uploads request method=POST path=/api/pending_closures5068worker1 # [ 99.580723] nix-grpc-daemon[863]: time=2026-09-22T22:21:52.218Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5069lb # [ 99.590685] niks3-server[959]: 2026/09/22 22:21:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5070worker1 # [ 99.609541] nix-grpc-daemon[863]: time=2026-09-22T22:21:52.248Z level=INFO msg="Uploading 1 narinfos"5071lb # [ 99.605059] niks3-server[959]: 2026/09/22 22:21:52 INFO Signed narinfos id=32 count=15072lb # [ 99.615951] niks3-server[959]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5073lb # [ 99.625227] niks3-server[959]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5074lb # [ 99.637202] niks3-server[959]: 2026/09/22 22:21:52 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5075lb # [ 99.643699] niks3-server[959]: 2026/09/22 22:21:52 INFO Completed upload id=325076worker1 # [ 99.652799] nix-grpc-daemon[863]: time=2026-09-22T22:21:52.292Z level=INFO msg="Upload complete. (282ms)"5077lb # [ 99.653186] niks3-server[959]: 2026/09/22 22:21:52 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5078worker1 # [ 99.665280] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15079worker1 # [ 99.688565] nix-daemon[848]: reaped child process 1275, status = succeeded5080client # copying 1 paths...5081worker2 # [ 99.709928] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536905082worker2 # [ 99.723582] nix-grpc-daemon[769]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5083client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5084worker1 # [ 99.771796] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:400125085worker1 # [ 99.779537] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=1205086client # [ 99.859806] nix-daemon[1147]: reaped child process 1254, status = succeeded5087client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.83 seconds)5088client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25089lb # [ 99.919961] envoy[895]: conn peer=[2001:db8:1::1]:53434 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4343 tx=2431 ms=26865090lb # [ 99.925028] envoy[895]: conn peer=[2001:db8:1::1]:40832 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=412 tx=407 ms=485091lb # [ 99.929939] envoy[895]: conn peer=[2001:db8:1::1]:40836 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4262 tx=2073 ms=25025092lb # [ 99.935412] envoy[895]: conn peer=[2001:db8:1::1]:40844 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=745093lb # [ 99.941213] envoy[895]: conn peer=[2001:db8:1::1]:40854 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3757 tx=2122 ms=34485094lb # [ 99.946522] envoy[895]: conn peer=[2001:db8:1::1]:40864 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=275095client # [ 100.022388] nix-daemon[1147]: accepted connection from pid 1273, user root (trusted)5096client # this derivation will be built:5097client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5098worker2 # [ 100.173441] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:536905099client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5100worker2 # [ 100.181219] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536905101worker2 # [ 100.189055] nix-grpc-daemon[865]: time=2026-09-22T22:21:52.829Z level=INFO msg="All 1 paths already cached"5102worker1 # [ 100.204205] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05103worker1 # [ 100.210901] nix-grpc-daemon[766]: ts=2026-09-22T22:21:52Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15104worker2 # [ 100.210470] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536905105worker2 # [ 100.227074] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536905106worker2 # [ 100.239759] nix-grpc-daemon[865]: time=2026-09-22T22:21:52.882Z level=INFO msg="All 1 paths already cached"5107client # copying 1 paths to 'lb:50051'...5108client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5109worker2 # [ 100.250854] nix-grpc-daemon[769]: ts=2026-09-22T22:21:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536905110worker2 # [ 100.266473] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5111worker2 # [ 100.424414] nix-daemon[837]: accepted connection from pid 1105, user nix-grpc-daemon (trusted)5112worker2 # [ 100.448212] nix-daemon[837]: reaped child process 1108, status = succeeded5113lb # [ 100.440306] niks3-server[959]: 2026/09/22 22:21:53 INFO Received uploads request method=POST path=/api/pending_closures5114worker2 # [ 100.457520] nix-grpc-daemon[865]: time=2026-09-22T22:21:53.099Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5115worker2 # [ 100.459075] nix-grpc-daemon[865]: time=2026-09-22T22:21:53.099Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5116lb # [ 100.472212] niks3-server[959]: 2026/09/22 22:21:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5117worker2 # [ 100.484228] nix-grpc-daemon[865]: time=2026-09-22T22:21:53.126Z level=INFO msg="Uploading 1 narinfos"5118lb # [ 100.484269] niks3-server[959]: 2026/09/22 22:21:53 INFO Signed narinfos id=33 count=15119lb # [ 100.487453] niks3-server[959]: 2026/09/22 22:21:53 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5120lb # [ 100.504110] niks3-server[959]: 2026/09/22 22:21:53 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5121worker2 # [ 100.516689] nix-grpc-daemon[865]: time=2026-09-22T22:21:53.159Z level=INFO msg="Upload complete. (233ms)"5122lb # [ 100.507975] niks3-server[959]: 2026/09/22 22:21:53 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5123lb # [ 100.519735] niks3-server[959]: 2026/09/22 22:21:53 INFO Completed upload id=335124worker2 # [ 100.524150] nix-grpc-daemon[769]: ts=2026-09-22T22:21:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=0 paths=1 nar_bytes_in=5685125lb # [ 100.525149] niks3-server[959]: 2026/09/22 22:21:53 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5126worker2 # [ 100.536357] nix-grpc-daemon[769]: ts=2026-09-22T22:21:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536905127worker2 # [ 100.547290] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5128worker2 # [ 100.552314] nix-daemon[837]: reaped child process 1103, status = succeeded5129client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5130client # uses-input-w1onlyc> worker2: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5131worker2 # [ 100.564988] nix-daemon[837]: accepted connection from pid 769, user nix-grpc-daemon (trusted)5132client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5133worker2 # [ 101.636972] nix-daemon[837]: reaped child process 1112, status = succeeded5134worker2 # [ 101.742695] nix-daemon[837]: accepted connection from pid 1128, user nix-grpc-daemon (trusted)5135worker2 # [ 101.762589] nix-daemon[837]: reaped child process 1131, status = succeeded5136lb # [ 101.762259] niks3-server[959]: 2026/09/22 22:21:54 INFO Received uploads request method=POST path=/api/pending_closures5137worker2 # [ 101.782767] nix-grpc-daemon[865]: time=2026-09-22T22:21:54.422Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5138lb # [ 101.797194] niks3-server[959]: 2026/09/22 22:21:54 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5139worker2 # [ 101.815330] nix-grpc-daemon[865]: time=2026-09-22T22:21:54.457Z level=INFO msg="Uploading 1 narinfos"5140lb # [ 101.809254] niks3-server[959]: 2026/09/22 22:21:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5141lb # [ 101.818671] niks3-server[959]: 2026/09/22 22:21:54 INFO Signed narinfos id=34 count=15142lb # [ 101.824562] niks3-server[959]: 2026/09/22 22:21:54 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5143lb # [ 101.832665] niks3-server[959]: 2026/09/22 22:21:54 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5144lb # [ 101.839133] niks3-server[959]: 2026/09/22 22:21:54 INFO Completed upload id=345145worker2 # [ 101.844778] nix-grpc-daemon[865]: time=2026-09-22T22:21:54.487Z level=INFO msg="Upload complete. (270ms)"5146lb # [ 101.842209] niks3-server[959]: 2026/09/22 22:21:54 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5147client # copying 1 paths...5148client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5149worker2 # [ 101.857188] nix-grpc-daemon[769]: ts=2026-09-22T22:21:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53690 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15150worker2 # [ 101.876490] nix-grpc-daemon[769]: ts=2026-09-22T22:21:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:536905151worker2 # [ 101.882112] nix-daemon[837]: reaped child process 1110, status = succeeded5152worker1 # [ 101.897848] nix-grpc-daemon[766]: ts=2026-09-22T22:21:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:400125153worker1 # [ 101.908307] nix-grpc-daemon[766]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5154worker1 # [ 101.919261] nix-grpc-daemon[766]: ts=2026-09-22T22:21:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:40012 duration_s=0 paths=1 nar_bytes_out=1205155client # [ 102.040144] nix-daemon[1147]: reaped child process 1276, status = succeeded5156client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.18 seconds)5157(finished: subtest: an input only one worker has still reaches the builder, in 9.65 seconds)5158subtest: upload whose reference the worker lacks is completed from the cache5159client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5160client # this derivation will be built:5161client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5162client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5163client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.34 seconds)5164client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5165client # this derivation will be built:5166client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5167client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5168client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.29 seconds)5169client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5170client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5171worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5172worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)5173worker2 # [ 103.063221] systemd[1]: Stopping Nix worker-protocol over gRPC...5174worker2 # [ 103.733965] nix-grpc-daemon[769]: ts=2026-09-22T22:21:56Z level=info event=signal_exit5175worker2 # [ 104.866425] nix-grpc-daemon[769]: ts=2026-09-22T22:21:57Z level=info event=scheduler_restarting5176worker1 # [ 104.870179] nix-grpc-daemon[766]: ts=2026-09-22T22:21:57Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400025177worker1 # [ 104.875295] nix-grpc-daemon[766]: ts=2026-09-22T22:21:57Z level=info event=worker_gone addr=192.168.1.4:500515178worker2 # [ 104.893161] nix-grpc-daemon[769]: ts=2026-09-22T22:21:57Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5179worker2 # [ 104.913167] nix-daemon[837]: reaped child process 839, status = succeeded5180worker2 # [ 104.966372] nix-grpc-daemon[769]: E0000 00:00:1790115717.608652 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5181worker2 # [ 104.973043] nix-grpc-daemon[769]: E0000 00:00:1790115717.608976 814 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5182worker2 # [ 106.057340] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5183worker2 # [ 106.062500] systemd[1]: Stopped Nix worker-protocol over gRPC.5184worker2 # [ 106.069437] systemd[1]: nix-grpc-daemon.service: Consumed 3.147s CPU time over 1min 25.923s wall clock time, 60.2M memory peak, 81.4K incoming IP traffic, 186.6K outgoing IP traffic.5185worker2 # [ 106.085039] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5186worker2 # [ 106.090442] systemd[1]: Closed Nix worker-protocol over gRPC.5187worker2 # [ 106.092964] systemd[1]: nix-grpc-daemon.socket: Consumed 661us CPU time over 1min 35.292s wall clock time, 256K memory peak, 93.7K incoming IP traffic, 84.1K outgoing IP traffic.5188lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5189lb: must succeed: curl -sf localhost:9901/clusters5190lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5191lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.09 seconds)5192client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25193worker1 # [ 106.410506] nix-grpc-daemon[766]: ts=2026-09-22T22:21:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:400125194worker1 # [ 106.419536] nix-grpc-daemon[766]: ts=2026-09-22T22:21:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:400125195client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5196client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.27 seconds)5197worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5198worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5199worker2 # [ 106.562163] systemd[1]: Listening on Nix worker-protocol over gRPC.5200worker1 # [ 106.645660] systemd[1]: Stopping Nix worker-protocol over gRPC...5201worker2 # [ 107.056318] systemd[1]: Starting scheduler-token.service...5202worker2 # [ 107.146703] systemd[1]: scheduler-token.service: Deactivated successfully.5203worker2 # [ 107.157018] systemd[1]: Finished scheduler-token.service.5204worker2 # [ 107.159002] systemd[1]: scheduler-token.service: Consumed 61ms CPU time over 95ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5205worker2 # [ 107.166117] systemd[1]: Starting Nix worker-protocol over gRPC...5206worker2 # [ 107.298583] nix-grpc-daemon[1155]: ts=2026-09-22T22:21:59Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5207worker2 # [ 107.300927] nix-grpc-daemon[1155]: ts=2026-09-22T22:21:59Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5208worker2 # [ 107.304787] nix-grpc-daemon[1155]: ts=2026-09-22T22:21:59Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15209worker2 # [ 107.311937] nix-grpc-daemon[1155]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5210worker2 # [ 107.323617] nix-grpc-daemon[1155]: E0000 00:00:1790115719.952161 1155 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5211worker2 # [ 107.356102] nix-grpc-daemon[1155]: E0000 00:00:1790115719.954444 1155 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5212worker2 # [ 107.358476] nix-grpc-daemon[1155]: ts=2026-09-22T22:21:59Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5213worker2 # [ 107.366857] nix-grpc-daemon[1155]: E0000 00:00:1790115719.974366 1195 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5214worker2 # [ 107.369821] nix-grpc-daemon[1155]: E0000 00:00:1790115719.989582 1195 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5215worker2 # [ 107.372332] nix-grpc-daemon[1155]: E0000 00:00:1790115719.989741 1195 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5216worker2 # [ 107.377490] systemd[1]: Started Nix worker-protocol over gRPC.5217worker2 # [ 107.380299] nix-grpc-daemon[1155]: E0000 00:00:1790115720.008681 1195 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5218worker2 # [ 107.382854] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:00Z level=info event=scheduler_connected addr=lb:500515219worker1 # [ 107.387440] nix-grpc-daemon[766]: ts=2026-09-22T22:22:00Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:400025220worker1 # [ 107.393545] nix-grpc-daemon[766]: ts=2026-09-22T22:22:00Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05221worker1 # [ 107.549854] nix-grpc-daemon[766]: ts=2026-09-22T22:22:00Z level=info event=signal_exit5222lb # [ 107.857441] niks3-server[959]: 2026/09/22 22:22:00 INFO lead: released remote=192.168.1.3:410825223worker2 # [ 108.862753] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_restarting addr=lb:500515224worker1 # [ 108.865351] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=scheduler_restarting5225worker1 # [ 108.879600] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400025226worker1 # [ 108.884680] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=worker_gone addr=192.168.1.4:500515227worker1 # [ 108.889801] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=scheduler_restarting addr=lb:500515228worker1 # [ 108.894610] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:400025229worker1 # [ 108.902149] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=worker_gone addr=192.168.1.3:500515230worker1 # [ 108.928672] nix-daemon[848]: reaped child process 850, status = succeeded5231worker1 # [ 108.986921] nix-grpc-daemon[766]: E0000 00:00:1790115721.625898 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5232worker1 # [ 108.993878] nix-grpc-daemon[766]: E0000 00:00:1790115721.626229 811 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5233worker2 # [ 109.063811] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_connected addr=lb:500515234worker2 # [ 109.069367] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5235worker1 # [ 109.104678] nix-grpc-daemon[766]: ts=2026-09-22T22:22:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5236worker2 # [ 109.270420] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_connected addr=lb:500515237worker2 # [ 109.275800] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5238worker2 # [ 109.477008] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:500515239worker2 # [ 109.482253] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5240worker2 # [ 109.683610] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:500515241worker2 # [ 109.689045] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5242worker2 # [ 109.892156] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:500515243worker2 # [ 109.896309] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5244lb # [ 109.919171] envoy[895]: conn peer=[2001:db8:1::1]:40866 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3745 tx=2101 ms=18015245lb # [ 109.924424] envoy[895]: conn peer=[2001:db8:1::1]:33102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=329 ms=835246lb # [ 109.929279] envoy[895]: conn peer=[2001:db8:1::4]:38654 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=54325 tx=3171 ms=843185247lb # [ 109.934447] envoy[895]: conn peer=[2001:db8:1::1]:33106 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=295248lb # [ 109.939166] envoy[895]: conn peer=[2001:db8:1::3]:52148 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10708 tx=3202 ms=876835249worker1 # [ 109.982994] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5250worker1 # [ 109.985818] systemd[1]: Stopped Nix worker-protocol over gRPC.5251worker1 # [ 109.990331] systemd[1]: nix-grpc-daemon.service: Consumed 4.840s CPU time over 1min 30.054s wall clock time, 70.6M memory peak, 116.4K incoming IP traffic, 230.7K outgoing IP traffic.5252worker1 # [ 109.998459] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5253worker1 # [ 110.000478] systemd[1]: Closed Nix worker-protocol over gRPC.5254worker1 # [ 110.002720] systemd[1]: nix-grpc-daemon.socket: Consumed 643us CPU time over 1min 39.241s wall clock time, 256K memory peak, 129.1K incoming IP traffic, 110.5K outgoing IP traffic.5255lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5256lb: must succeed: curl -sf localhost:9901/clusters5257lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5258lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.08 seconds)5259worker2 # [ 110.092833] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:500515260client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25261worker2 # [ 110.095124] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5262worker2 # [ 110.296778] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_connected addr=lb:500515263worker2 # [ 110.302194] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5264worker2 # [ 110.329013] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:02Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579625265worker2 # [ 110.346816] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5266worker2 # [ 110.374404] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 26 ms (attempt 1/5)5267worker2 # [ 110.408466] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 182 ms (attempt 2/5)5268worker2 # [ 110.503512] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:500515269worker2 # [ 110.509242] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5270worker2 # [ 110.593138] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 196 ms (attempt 3/5)5271worker2 # [ 110.710943] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:500515272worker2 # [ 110.716682] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5273worker2 # [ 110.792154] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 680 ms (attempt 4/5)5274worker2 # [ 110.919808] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:500515275worker2 # [ 110.924256] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5276worker2 # [ 111.121180] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:500515277worker2 # [ 111.126776] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5278worker2 # [ 111.327870] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_connected addr=lb:500515279worker2 # [ 111.334122] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5280worker2 # [ 111.477365] nix-grpc-daemon[1155]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5281worker2 # [ 111.483188] nix-grpc-daemon[1155]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5282client # copying 2 paths...5283client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5284worker2 # [ 111.511027] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579625285client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5286worker2 # [ 111.530204] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5287worker2 # [ 111.535802] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_connected addr=lb:500515288worker2 # [ 111.542673] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5289worker2 # [ 111.580459] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=niks3_push_started pid=12155290worker2 # [ 111.726726] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.368Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5291worker2 # [ 111.742887] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_connected addr=lb:500515292worker2 # [ 111.745727] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5293worker2 # [ 111.841336] nix-daemon[837]: accepted connection from pid 1228, user nix-grpc-daemon (trusted)5294worker2 # [ 111.861625] nix-daemon[837]: reaped child process 1231, status = succeeded5295lb # [ 111.861095] niks3-server[959]: 2026/09/22 22:22:04 INFO Received uploads request method=POST path=/api/pending_closures5296lb # [ 111.868764] niks3-server[959]: 2026/09/22 22:22:04 INFO Received uploads request method=POST path=/api/pending_closures5297worker2 # [ 111.884063] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.524Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5298worker2 # [ 111.885635] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.524Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5299worker2 # [ 111.887373] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.524Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5300lb # [ 111.908992] niks3-server[959]: 2026/09/22 22:22:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5301worker2 # [ 111.923436] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.564Z level=INFO msg="Uploading 3 narinfos"5302lb # [ 111.919200] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5303lb # [ 111.927244] niks3-server[959]: 2026/09/22 22:22:04 INFO Signed narinfos id=35 count=15304lb # [ 111.932561] niks3-server[959]: 2026/09/22 22:22:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5305lb # [ 111.934538] niks3-server[959]: 2026/09/22 22:22:04 INFO Signed narinfos id=36 count=25306lb # [ 111.935709] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5307worker2 # [ 111.946532] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_connected addr=lb:500515308worker2 # [ 111.949581] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5309lb # [ 111.945850] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5310lb # [ 111.960901] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5311lb # [ 111.966544] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5312lb # [ 111.973050] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5313lb # [ 111.974762] niks3-server[959]: 2026/09/22 22:22:04 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5314lb # [ 111.978719] niks3-server[959]: 2026/09/22 22:22:04 INFO Completed upload id=355315lb # [ 111.980807] niks3-server[959]: 2026/09/22 22:22:04 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5316lb # [ 111.983779] niks3-server[959]: 2026/09/22 22:22:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5317lb # [ 111.986170] niks3-server[959]: 2026/09/22 22:22:04 INFO Completed upload id=365318worker2 # [ 111.992742] nix-grpc-daemon[1215]: time=2026-09-22T22:22:04.635Z level=INFO msg="Upload complete. (262ms)"5319worker2 # [ 112.004502] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=2 nar_bytes_in=3125320worker2 # [ 112.020480] nix-daemon[837]: reaped child process 1213, status = succeeded5321client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.93 seconds)5322worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5323worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5324worker1 # [ 112.135309] systemd[1]: Listening on Nix worker-protocol over gRPC.5325worker1 # [ 112.145177] systemd[1]: Starting Nix worker-protocol over gRPC...5326worker2 # [ 112.151233] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_connected addr=lb:500515327worker2 # [ 112.156787] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5328worker1 # [ 112.299852] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5329worker1 # [ 112.302425] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5330worker1 # [ 112.306819] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15331worker1 # [ 112.312525] nix-grpc-daemon[1322]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5332worker1 # [ 112.314024] nix-grpc-daemon[1322]: E0000 00:00:1790115724.951602 1322 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5333lb # [ 112.317516] niks3-server[959]: 2026/09/22 22:22:04 INFO lead: acquired remote=192.168.1.3:485085334lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5335lb: must succeed: curl -sf localhost:9901/clusters5336worker1 # [ 112.360220] nix-grpc-daemon[1322]: E0000 00:00:1790115724.957852 1322 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5337worker1 # [ 112.362776] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5338worker2 # [ 112.357844] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515339worker1 # [ 112.366123] nix-grpc-daemon[1322]: E0000 00:00:1790115724.965805 1364 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5340worker1 # [ 112.369243] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:04Z level=info event=scheduler_take_over5341worker2 # [ 112.363437] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5342worker1 # [ 112.370648] nix-grpc-daemon[1322]: E0000 00:00:1790115724.982875 1364 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5343worker1 # [ 112.373007] nix-grpc-daemon[1322]: E0000 00:00:1790115724.983083 1364 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5344worker1 # [ 112.375400] systemd[1]: Started Nix worker-protocol over gRPC.5345worker1 # [ 112.376253] nix-grpc-daemon[1322]: E0000 00:00:1790115725.004442 1364 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5346worker1 # [ 112.385415] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515347worker1 # [ 112.387440] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5348lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5349worker2 # [ 112.564842] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515350worker2 # [ 112.570315] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5351worker1 # [ 112.588852] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515352worker1 # [ 112.594586] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5353worker2 # [ 112.771646] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515354worker2 # [ 112.777203] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5355worker1 # [ 112.795104] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515356worker1 # [ 112.799280] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5357worker2 # [ 112.978649] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515358worker2 # [ 112.981609] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5359worker1 # [ 113.000972] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515360worker1 # [ 113.010670] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:328045361worker1 # [ 113.015766] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05362worker2 # [ 113.183197] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:05Z level=info event=scheduler_connected addr=lb:500515363worker1 # [ 113.194019] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=oidc_jwks provider=mock keys=15364worker1 # [ 113.195478] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:328045365worker1 # [ 113.197581] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05366lb: must succeed: curl -sf localhost:9901/clusters5367lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5368lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)5369(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.44 seconds)5370subtest: a niks3 restart does not cost the leader its role5371lb: must succeed: curl -sf localhost:9901/clusters5372lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5373lb: must succeed: curl -sf localhost:9901/clusters5374lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5375lb # [ 113.610111] niks3-server[959]: 2026/09/22 22:22:06 INFO Shutdown signal received, draining in-flight requests timeout=10s5376lb # [ 113.610655] systemd[1]: Stopping niks3 server...5377worker2 # [ 113.617068] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:06Z level=info event=scheduler_lock_lost status=2005378lb # [ 113.616989] niks3-server[959]: 2026/09/22 22:22:06 INFO lead: released remote=192.168.1.3:485085379worker1 # [ 113.624777] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:06Z level=info event=scheduler_lock_lost status=2005380lb # [ 113.638696] systemd[1]: niks3.service: Deactivated successfully.5381lb # [ 113.640708] systemd[1]: Stopped niks3 server.5382lb # [ 113.642308] systemd[1]: niks3.service: Consumed 1.004s CPU time over 1min 30.303s wall clock time, 28M memory peak, 98.2K incoming IP traffic, 141.6K outgoing IP traffic.5383lb # [ 113.653063] systemd[1]: Starting niks3 server...5384lb # [ 113.723174] niks3-server[1100]: 2026/09/22 22:22:06 goose: no migrations to run. current version: 202609200000005385lb # [ 113.728779] niks3-server[1100]: 2026/09/22 22:22:06 OK 1_commit_pending_closure.sql (7.18ms)5386lb # [ 113.732575] niks3-server[1100]: 2026/09/22 22:22:06 OK 2_object_stats_trigger.sql (3.43ms)5387lb # [ 113.733762] niks3-server[1100]: 2026/09/22 22:22:06 goose: up to current file version: 25388lb # [ 113.735086] niks3-server[1100]: 2026/09/22 22:22:06 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5389lb # [ 113.747316] niks3-server[1100]: 2026/09/22 22:22:06 INFO Read proxy enabled — serving cache objects from S35390lb # [ 113.753245] niks3-server[1100]: 2026/09/22 22:22:06 INFO Using socket-activated listener address=0.0.0.0:57515391lb # [ 113.759441] niks3-server[1100]: 2026/09/22 22:22:06 INFO systemd watchdog enabled interval=15s5392lb: waiting for unit niks3.service5393lb # [ 113.762123] niks3-server[1100]: 2026/09/22 22:22:06 INFO Starting HTTP server address=0.0.0.0:57515394lb # [ 113.767187] systemd[1]: Started niks3 server.5395lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5396lb: must succeed: curl -sf localhost:9901/clusters5397lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5398worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5399worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5400lb: must succeed: curl -sf localhost:9901/clusters5401lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5402lb: must succeed: curl -sf localhost:9901/clusters5403lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5404worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5405worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.08 seconds)5406client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25407worker1 # [ 114.300090] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:328145408worker1 # [ 114.316285] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5409client # don't know how to build these paths:5410client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5411worker2 # [ 114.343841] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579625412worker1 # [ 114.388425] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:07Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05413worker1 # [ 114.395875] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:07Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.3:50051 clients=15414worker1 # [ 114.445161] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328145415worker1 # [ 114.465746] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 13 ms (attempt 1/5)5416worker1 # [ 114.489361] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 2/5)5417worker1 # [ 114.553731] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 242 ms (attempt 3/5)5418lb # [ 114.621227] niks3-server[1100]: 2026/09/22 22:22:07 INFO lead: acquired remote=192.168.1.3:485165419worker1 # [ 114.799279] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 521 ms (attempt 4/5)5420worker1 # [ 115.323023] nix-grpc-daemon[1322]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5421worker1 # [ 115.331353] nix-grpc-daemon[1322]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5422worker1 # [ 115.352150] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:328145423client # copying 1 paths to 'lb:50051'...5424client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5425worker1 # [ 115.369822] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:328145426worker1 # [ 115.386233] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5427worker1 # [ 115.418414] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:08Z level=info event=niks3_push_started pid=13945428worker1 # [ 115.566246] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.205Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5429worker1 # [ 115.716398] nix-daemon[848]: accepted connection from pid 1407, user nix-grpc-daemon (trusted)5430worker1 # [ 115.738667] nix-daemon[848]: reaped child process 1410, status = succeeded5431lb # [ 115.732848] niks3-server[1100]: 2026/09/22 22:22:08 INFO Received uploads request method=POST path=/api/pending_closures5432worker1 # [ 115.760084] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.398Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5433worker1 # [ 115.762032] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.398Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5434lb # [ 115.783223] niks3-server[1100]: 2026/09/22 22:22:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5435lb # [ 115.832100] niks3-server[1100]: 2026/09/22 22:22:08 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5436lb # [ 115.841681] niks3-server[1100]: 2026/09/22 22:22:08 INFO Signed narinfos id=37 count=15437worker1 # [ 115.856211] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.493Z level=INFO msg="Uploading 1 narinfos"5438lb # [ 115.852968] niks3-server[1100]: 2026/09/22 22:22:08 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5439lb # [ 115.861547] niks3-server[1100]: 2026/09/22 22:22:08 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5440lb # [ 115.869151] niks3-server[1100]: 2026/09/22 22:22:08 INFO Completed upload id=375441worker1 # [ 115.879441] nix-grpc-daemon[1394]: time=2026-09-22T22:22:08.518Z level=INFO msg="Upload complete. (309ms)"5442lb # [ 115.871602] niks3-server[1100]: 2026/09/22 22:22:08 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5443worker1 # [ 115.892727] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=0 paths=1 nar_bytes_in=5445444worker1 # [ 115.912625] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328145445worker1 # [ 115.921160] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5446worker1 # [ 115.927214] nix-daemon[848]: reaped child process 1392, status = succeeded5447client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5448worker1 # [ 115.939552] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5449client # farm-a> node-a: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5450client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5451client # farm-a> LOG-farm-a-after-niks3-restart5452worker1 # [ 116.828728] nix-daemon[848]: reaped child process 1414, status = succeeded5453worker1 # [ 116.923460] nix-daemon[848]: accepted connection from pid 1430, user nix-grpc-daemon (trusted)5454worker1 # [ 116.943977] nix-daemon[848]: reaped child process 1433, status = succeeded5455lb # [ 116.938843] niks3-server[1100]: 2026/09/22 22:22:09 INFO Received uploads request method=POST path=/api/pending_closures5456worker1 # [ 116.964511] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.601Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5457worker1 # [ 116.967825] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.601Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5458lb # [ 116.980070] niks3-server[1100]: 2026/09/22 22:22:09 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5459lb # [ 116.994927] niks3-server[1100]: 2026/09/22 22:22:09 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5460lb # [ 117.004121] niks3-server[1100]: 2026/09/22 22:22:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5461worker1 # [ 117.020711] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.659Z level=INFO msg="Uploading 1 narinfos"5462lb # [ 117.014800] niks3-server[1100]: 2026/09/22 22:22:09 INFO Signed narinfos id=38 count=15463lb # [ 117.019615] niks3-server[1100]: 2026/09/22 22:22:09 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5464lb # [ 117.028203] niks3-server[1100]: 2026/09/22 22:22:09 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5465lb # [ 117.033868] niks3-server[1100]: 2026/09/22 22:22:09 INFO Completed upload id=385466worker1 # [ 117.044130] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.683Z level=INFO msg="Upload complete. (290ms)"5467lb # [ 117.036380] niks3-server[1100]: 2026/09/22 22:22:09 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5468worker1 # [ 117.069285] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15469worker1 # [ 117.085182] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05470worker1 # [ 117.088778] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.3:50051 clients=15471worker1 # [ 117.092785] nix-daemon[848]: reaped child process 1412, status = succeeded5472worker1 # [ 117.110672] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328145473worker1 # [ 117.126169] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:328145474worker1 # [ 117.143953] nix-grpc-daemon[1394]: time=2026-09-22T22:22:09.783Z level=INFO msg="All 1 paths already cached"5475client # copying 1 paths to 'lb:50051'...5476client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5477worker1 # [ 117.156633] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:328145478worker1 # [ 117.171285] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5479worker1 # [ 117.327579] nix-daemon[848]: accepted connection from pid 1439, user nix-grpc-daemon (trusted)5480worker1 # [ 117.350040] nix-daemon[848]: reaped child process 1442, status = succeeded5481lb # [ 117.345395] niks3-server[1100]: 2026/09/22 22:22:09 INFO Received uploads request method=POST path=/api/pending_closures5482worker1 # [ 117.367588] nix-grpc-daemon[1394]: time=2026-09-22T22:22:10.006Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5483worker1 # [ 117.369801] nix-grpc-daemon[1394]: time=2026-09-22T22:22:10.006Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5484lb # [ 117.380941] niks3-server[1100]: 2026/09/22 22:22:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5485worker1 # [ 117.397837] nix-grpc-daemon[1394]: time=2026-09-22T22:22:10.036Z level=INFO msg="Uploading 1 narinfos"5486lb # [ 117.396740] niks3-server[1100]: 2026/09/22 22:22:10 INFO Signed narinfos id=39 count=15487lb # [ 117.399565] niks3-server[1100]: 2026/09/22 22:22:10 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5488lb # [ 117.417180] niks3-server[1100]: 2026/09/22 22:22:10 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5489worker1 # [ 117.434992] nix-grpc-daemon[1394]: time=2026-09-22T22:22:10.074Z level=INFO msg="Upload complete. (246ms)"5490lb # [ 117.426934] niks3-server[1100]: 2026/09/22 22:22:10 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5491lb # [ 117.432936] niks3-server[1100]: 2026/09/22 22:22:10 INFO Completed upload id=395492worker1 # [ 117.444390] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=0 paths=1 nar_bytes_in=6565493lb # [ 117.437325] niks3-server[1100]: 2026/09/22 22:22:10 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5494worker1 # [ 117.464964] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328145495worker1 # [ 117.472963] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5496worker1 # [ 117.479172] nix-daemon[848]: reaped child process 1437, status = succeeded5497client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5498worker1 # [ 117.493600] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5499client # farm-b> node-a: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5500client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5501client # farm-b> LOG-farm-b-after-niks3-restart5502worker1 # [ 118.728238] nix-daemon[848]: reaped child process 1446, status = succeeded5503worker1 # [ 118.833421] nix-daemon[848]: accepted connection from pid 1462, user nix-grpc-daemon (trusted)5504worker1 # [ 118.853298] nix-daemon[848]: reaped child process 1465, status = succeeded5505lb # [ 118.851166] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received uploads request method=POST path=/api/pending_closures5506worker1 # [ 118.873861] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.512Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5507worker1 # [ 118.875394] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.512Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5508lb # [ 118.896826] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5509worker1 # [ 118.914662] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.553Z level=INFO msg="Uploading 1 narinfos"5510lb # [ 118.913230] niks3-server[1100]: 2026/09/22 22:22:11 INFO Signed narinfos id=40 count=15511lb # [ 118.929631] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5512lb # [ 118.940154] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5513lb # [ 118.952838] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5514worker1 # [ 118.968296] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.607Z level=INFO msg="Upload complete. (259ms)"5515lb # [ 118.958996] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5516lb # [ 118.963838] niks3-server[1100]: 2026/09/22 22:22:11 INFO Completed upload id=405517lb # [ 118.968746] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5518worker1 # [ 118.981375] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15519worker1 # [ 119.006013] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05520worker1 # [ 119.013852] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.3:50051 clients=15521worker1 # [ 119.018656] nix-daemon[848]: reaped child process 1444, status = succeeded5522worker1 # [ 119.051466] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328145523worker1 # [ 119.067589] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:328145524worker1 # [ 119.083951] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.723Z level=INFO msg="All 2 paths already cached"5525client # copying 1 paths to 'lb:50051'...5526client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5527worker1 # [ 119.097619] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:328145528worker1 # [ 119.112533] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5529worker1 # [ 119.274497] nix-daemon[848]: accepted connection from pid 1471, user nix-grpc-daemon (trusted)5530worker1 # [ 119.294098] nix-daemon[848]: reaped child process 1474, status = succeeded5531lb # [ 119.289921] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received uploads request method=POST path=/api/pending_closures5532worker1 # [ 119.313367] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.952Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5533worker1 # [ 119.314950] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.952Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5534lb # [ 119.317370] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5535worker1 # [ 119.330867] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.969Z level=INFO msg="Uploading 1 narinfos"5536lb # [ 119.322920] niks3-server[1100]: 2026/09/22 22:22:11 INFO Signed narinfos id=41 count=15537lb # [ 119.325965] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5538lb # [ 119.327551] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5539lb # [ 119.334038] niks3-server[1100]: 2026/09/22 22:22:11 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5540lb # [ 119.339332] niks3-server[1100]: 2026/09/22 22:22:11 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5541lb # [ 119.342707] niks3-server[1100]: 2026/09/22 22:22:11 INFO Completed upload id=415542worker1 # [ 119.352617] nix-grpc-daemon[1394]: time=2026-09-22T22:22:11.991Z level=INFO msg="Upload complete. (222ms)"5543worker1 # [ 119.360553] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=0 paths=1 nar_bytes_in=7845544worker1 # [ 119.371004] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:328145545worker1 # [ 119.376922] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5546worker1 # [ 119.381506] nix-daemon[848]: reaped child process 1469, status = succeeded5547client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5548worker1 # [ 119.393824] nix-daemon[848]: accepted connection from pid 1322, user nix-grpc-daemon (trusted)5549client # farm-top> node-a: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5550lb # [ 119.919366] envoy[895]: conn peer=[2001:db8:1::1]:33112 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1010 tx=305 ms=16885551client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5552client # farm-top> LOG-farm-top-after-niks3-restart5553worker1 # [ 121.062904] nix-daemon[848]: reaped child process 1478, status = succeeded5554worker1 # [ 121.180632] nix-daemon[848]: accepted connection from pid 1494, user nix-grpc-daemon (trusted)5555worker1 # [ 121.203181] nix-daemon[848]: reaped child process 1497, status = succeeded5556lb # [ 121.199423] niks3-server[1100]: 2026/09/22 22:22:13 INFO Received uploads request method=POST path=/api/pending_closures5557worker1 # [ 121.220472] nix-grpc-daemon[1394]: time=2026-09-22T22:22:13.858Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5558worker1 # [ 121.222301] nix-grpc-daemon[1394]: time=2026-09-22T22:22:13.859Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5559lb # [ 121.241795] niks3-server[1100]: 2026/09/22 22:22:13 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5560lb # [ 121.256233] niks3-server[1100]: 2026/09/22 22:22:13 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5561worker1 # [ 121.279703] nix-grpc-daemon[1394]: time=2026-09-22T22:22:13.918Z level=INFO msg="Uploading 1 narinfos"5562lb # [ 121.264523] niks3-server[1100]: 2026/09/22 22:22:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5563lb # [ 121.274583] niks3-server[1100]: 2026/09/22 22:22:13 INFO Signed narinfos id=42 count=15564lb # [ 121.280357] niks3-server[1100]: 2026/09/22 22:22:13 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5565lb # [ 121.291826] niks3-server[1100]: 2026/09/22 22:22:13 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5566lb # [ 121.298996] niks3-server[1100]: 2026/09/22 22:22:13 INFO Completed upload id=425567worker1 # [ 121.308927] nix-grpc-daemon[1394]: time=2026-09-22T22:22:13.948Z level=INFO msg="Upload complete. (303ms)"5568lb # [ 121.302596] niks3-server[1100]: 2026/09/22 22:22:13 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5569worker1 # [ 121.321137] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:32814 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15570worker1 # [ 121.338614] nix-daemon[848]: reaped child process 1476, status = succeeded5571client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 7.30 seconds)5572client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5573client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.19 seconds)5574(finished: subtest: a niks3 restart does not cost the leader its role, in 8.06 seconds)5575subtest: scheduler down: the other node takes the lock and keeps it5576lb: must succeed: curl -sf localhost:9901/clusters5577lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5578lb: must succeed: curl -sf localhost:9901/clusters5579lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5580lb: must succeed: curl -sf localhost:9901/clusters5581lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5582lb: must succeed: curl -sf localhost:9901/clusters5583lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5584worker1 # [ 121.817962] systemd[1]: Stopping Nix worker-protocol over gRPC...5585worker1 # [ 122.332926] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:14Z level=info event=signal_exit5586lb # [ 122.629287] niks3-server[1100]: 2026/09/22 22:22:15 INFO lead: released remote=192.168.1.3:485165587worker2 # [ 123.636951] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_restarting addr=lb:500515588worker1 # [ 123.643257] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=scheduler_restarting5589worker1 # [ 123.650420] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:328045590worker1 # [ 123.656326] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=worker_gone addr=192.168.1.4:500515591worker1 # [ 123.660866] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=scheduler_restarting addr=lb:500515592worker1 # [ 123.666264] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:328045593worker1 # [ 123.672448] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=worker_gone addr=192.168.1.3:500515594worker1 # [ 123.693846] nix-daemon[848]: reaped child process 1382, status = succeeded5595worker2 # [ 123.837046] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:500515596worker2 # [ 123.842866] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5597worker1 # [ 123.863634] nix-grpc-daemon[1322]: ts=2026-09-22T22:22:16Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5598worker1 # [ 123.950772] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5599worker1 # [ 123.958696] systemd[1]: Stopped Nix worker-protocol over gRPC.5600worker1 # [ 123.965102] systemd[1]: nix-grpc-daemon.service: Consumed 1.291s CPU time over 11.809s wall clock time, 38M memory peak, 36.9K incoming IP traffic, 59K outgoing IP traffic.5601worker1 # [ 123.979653] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5602worker1 # [ 123.986382] systemd[1]: Closed Nix worker-protocol over gRPC.5603worker1 # [ 123.990847] systemd[1]: nix-grpc-daemon.socket: Consumed 1ms CPU time over 11.858s wall clock time, 256K memory peak, 32.1K incoming IP traffic, 25.4K outgoing IP traffic.5604worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5605worker2 # [ 124.043351] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:500515606worker2 # [ 124.045838] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5607worker2 # [ 124.247138] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_connected addr=lb:500515608worker2 # [ 124.252125] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5609worker2 # [ 124.453426] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:500515610worker2 # [ 124.458802] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5611worker2 # [ 124.628311] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_take_over5612lb # [ 124.621140] niks3-server[1100]: 2026/09/22 22:22:17 INFO lead: acquired remote=192.168.1.4:371825613worker2 # [ 124.661763] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:500515614worker2 # [ 124.665371] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5615worker2 # [ 124.863285] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:500515616worker2 # [ 124.868886] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5617worker2 # [ 125.069408] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:500515618worker2 # [ 125.073257] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5619worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 1.09 seconds)5620worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5621worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5622worker2 # [ 125.273005] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_connected addr=lb:500515623worker2 # [ 125.275176] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5624worker2 # [ 125.476497] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:18Z level=info event=scheduler_connected addr=lb:500515625worker2 # [ 125.504204] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:18Z level=info event=oidc_jwks provider=mock keys=15626worker2 # [ 125.510130] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:18Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:331025627worker2 # [ 125.515609] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:18Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05628worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5629worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.09 seconds)5630client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25631worker2 # [ 126.506366] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579625632client # don't know how to build these paths:5633client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5634worker2 # [ 126.520506] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579625635worker2 # [ 126.560986] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05636worker2 # [ 126.570942] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.4:50051 clients=15637worker2 # [ 126.618003] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579625638worker2 # [ 126.644531] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579625639client # copying 1 paths to 'lb:50051'...5640client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5641worker2 # [ 126.669725] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579625642worker2 # [ 126.694747] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5643worker2 # [ 126.922138] nix-daemon[837]: accepted connection from pid 1260, user nix-grpc-daemon (trusted)5644worker2 # [ 126.944955] nix-daemon[837]: reaped child process 1263, status = succeeded5645lb # [ 126.952823] niks3-server[1100]: 2026/09/22 22:22:19 INFO Received uploads request method=POST path=/api/pending_closures5646worker2 # [ 126.978656] nix-grpc-daemon[1215]: time=2026-09-22T22:22:19.619Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5647worker2 # [ 126.980461] nix-grpc-daemon[1215]: time=2026-09-22T22:22:19.619Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5648lb # [ 127.033151] niks3-server[1100]: 2026/09/22 22:22:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5649lb # [ 127.047794] niks3-server[1100]: 2026/09/22 22:22:19 INFO Signed narinfos id=43 count=15650worker2 # [ 127.056272] nix-grpc-daemon[1215]: time=2026-09-22T22:22:19.696Z level=INFO msg="Uploading 1 narinfos"5651lb # [ 127.073174] niks3-server[1100]: 2026/09/22 22:22:19 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5652lb # [ 127.092938] niks3-server[1100]: 2026/09/22 22:22:19 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5653lb # [ 127.106447] niks3-server[1100]: 2026/09/22 22:22:19 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5654lb # [ 127.118801] niks3-server[1100]: 2026/09/22 22:22:19 INFO Completed upload id=435655worker2 # [ 127.125634] nix-grpc-daemon[1215]: time=2026-09-22T22:22:19.768Z level=INFO msg="Upload complete. (405ms)"5656lb # [ 127.122633] niks3-server[1100]: 2026/09/22 22:22:19 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5657worker2 # [ 127.141101] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=1 nar_bytes_in=5205658worker2 # [ 127.156801] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579625659worker2 # [ 127.174738] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5660worker2 # [ 127.184321] nix-daemon[837]: reaped child process 1258, status = succeeded5661client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5662client # farm-a> worker2: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5663worker2 # [ 127.203533] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5664client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5665client # farm-a> LOG-farm-a-on-standby5666worker2 # [ 128.860654] nix-daemon[837]: reaped child process 1267, status = succeeded5667worker2 # [ 128.982649] nix-daemon[837]: accepted connection from pid 1283, user nix-grpc-daemon (trusted)5668worker2 # [ 129.003400] nix-daemon[837]: reaped child process 1286, status = succeeded5669lb # [ 129.005697] niks3-server[1100]: 2026/09/22 22:22:21 INFO Received uploads request method=POST path=/api/pending_closures5670worker2 # [ 129.024523] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.666Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5671worker2 # [ 129.026329] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.666Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5672lb # [ 129.059076] niks3-server[1100]: 2026/09/22 22:22:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5673worker2 # [ 129.075190] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.717Z level=INFO msg="Uploading 1 narinfos"5674lb # [ 129.077379] niks3-server[1100]: 2026/09/22 22:22:21 INFO Signed narinfos id=44 count=15675lb # [ 129.087229] niks3-server[1100]: 2026/09/22 22:22:21 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5676lb # [ 129.097818] niks3-server[1100]: 2026/09/22 22:22:21 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5677lb # [ 129.108331] niks3-server[1100]: 2026/09/22 22:22:21 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5678lb # [ 129.111751] niks3-server[1100]: 2026/09/22 22:22:21 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5679lb # [ 129.117522] niks3-server[1100]: 2026/09/22 22:22:21 INFO Completed upload id=445680worker2 # [ 129.123262] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.765Z level=INFO msg="Upload complete. (303ms)"5681lb # [ 129.119737] niks3-server[1100]: 2026/09/22 22:22:21 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5682worker2 # [ 129.154961] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15683worker2 # [ 129.171743] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05684worker2 # [ 129.175270] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.4:50051 clients=15685worker2 # [ 129.179242] nix-daemon[837]: reaped child process 1265, status = succeeded5686worker2 # [ 129.200913] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579625687worker2 # [ 129.222038] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579625688worker2 # [ 129.241321] nix-grpc-daemon[1215]: time=2026-09-22T22:22:21.883Z level=INFO msg="All 1 paths already cached"5689client # copying 1 paths to 'lb:50051'...5690client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5691worker2 # [ 129.255817] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579625692worker2 # [ 129.275001] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5693worker2 # [ 129.443302] nix-daemon[837]: accepted connection from pid 1292, user nix-grpc-daemon (trusted)5694worker2 # [ 129.466254] nix-daemon[837]: reaped child process 1295, status = succeeded5695lb # [ 129.465020] niks3-server[1100]: 2026/09/22 22:22:22 INFO Received uploads request method=POST path=/api/pending_closures5696worker2 # [ 129.481241] nix-grpc-daemon[1215]: time=2026-09-22T22:22:22.123Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5697worker2 # [ 129.482854] nix-grpc-daemon[1215]: time=2026-09-22T22:22:22.123Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5698lb # [ 129.496963] niks3-server[1100]: 2026/09/22 22:22:22 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5699worker2 # [ 129.514968] nix-grpc-daemon[1215]: time=2026-09-22T22:22:22.157Z level=INFO msg="Uploading 1 narinfos"5700lb # [ 129.513971] niks3-server[1100]: 2026/09/22 22:22:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5701lb # [ 129.523146] niks3-server[1100]: 2026/09/22 22:22:22 INFO Signed narinfos id=45 count=15702lb # [ 129.528365] niks3-server[1100]: 2026/09/22 22:22:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5703lb # [ 129.538588] niks3-server[1100]: 2026/09/22 22:22:22 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5704lb # [ 129.544513] niks3-server[1100]: 2026/09/22 22:22:22 INFO Completed upload id=455705lb # [ 129.548419] niks3-server[1100]: 2026/09/22 22:22:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5706worker2 # [ 129.551084] nix-grpc-daemon[1215]: time=2026-09-22T22:22:22.193Z level=INFO msg="Upload complete. (253ms)"5707worker2 # [ 129.560574] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=1 nar_bytes_in=6405708worker2 # [ 129.576860] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579625709worker2 # [ 129.584634] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5710worker2 # [ 129.590211] nix-daemon[837]: reaped child process 1290, status = succeeded5711client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5712client # farm-b> worker2: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5713worker2 # [ 129.604786] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5714lb # [ 129.918710] envoy[895]: conn peer=[2001:db8:1::1]:49296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9727 tx=4459 ms=70415715lb # [ 129.924519] envoy[895]: conn peer=[2001:db8:1::3]:46084 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1487 tx=609 ms=114765716client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5717client # farm-b> LOG-farm-b-on-standby5718worker2 # [ 131.073682] nix-daemon[837]: reaped child process 1299, status = succeeded5719worker2 # [ 131.204123] nix-daemon[837]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5720worker2 # [ 131.223666] nix-daemon[837]: reaped child process 1318, status = succeeded5721lb # [ 131.230634] niks3-server[1100]: 2026/09/22 22:22:23 INFO Received uploads request method=POST path=/api/pending_closures5722worker2 # [ 131.248709] nix-grpc-daemon[1215]: time=2026-09-22T22:22:23.889Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5723worker2 # [ 131.251053] nix-grpc-daemon[1215]: time=2026-09-22T22:22:23.889Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5724lb # [ 131.274626] niks3-server[1100]: 2026/09/22 22:22:23 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5725lb # [ 131.296458] niks3-server[1100]: 2026/09/22 22:22:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5726worker2 # [ 131.308170] nix-grpc-daemon[1215]: time=2026-09-22T22:22:23.950Z level=INFO msg="Uploading 1 narinfos"5727lb # [ 131.305403] niks3-server[1100]: 2026/09/22 22:22:23 INFO Signed narinfos id=46 count=15728lb # [ 131.314224] niks3-server[1100]: 2026/09/22 22:22:23 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5729lb # [ 131.322703] niks3-server[1100]: 2026/09/22 22:22:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5730lb # [ 131.332222] niks3-server[1100]: 2026/09/22 22:22:23 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5731lb # [ 131.343683] niks3-server[1100]: 2026/09/22 22:22:23 INFO Completed upload id=465732worker2 # [ 131.348413] nix-grpc-daemon[1215]: time=2026-09-22T22:22:23.990Z level=INFO msg="Upload complete. (331ms)"5733lb # [ 131.348561] niks3-server[1100]: 2026/09/22 22:22:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5734worker2 # [ 131.367902] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15735worker2 # [ 131.397711] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05736worker2 # [ 131.403968] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.4:50051 clients=15737worker2 # [ 131.408860] nix-daemon[837]: reaped child process 1297, status = succeeded5738worker2 # [ 131.430895] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579625739worker2 # [ 131.444852] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579625740worker2 # [ 131.459962] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.102Z level=INFO msg="All 2 paths already cached"5741client # copying 1 paths to 'lb:50051'...5742client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5743worker2 # [ 131.469840] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579625744worker2 # [ 131.482495] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5745worker2 # [ 131.622985] nix-daemon[837]: accepted connection from pid 1324, user nix-grpc-daemon (trusted)5746worker2 # [ 131.646031] nix-daemon[837]: reaped child process 1327, status = succeeded5747lb # [ 131.637180] niks3-server[1100]: 2026/09/22 22:22:24 INFO Received uploads request method=POST path=/api/pending_closures5748worker2 # [ 131.657320] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.299Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5749worker2 # [ 131.659105] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.299Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5750lb # [ 131.674955] niks3-server[1100]: 2026/09/22 22:22:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5751worker2 # [ 131.689746] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.332Z level=INFO msg="Uploading 1 narinfos"5752lb # [ 131.690932] niks3-server[1100]: 2026/09/22 22:22:24 INFO Signed narinfos id=47 count=15753lb # [ 131.698162] niks3-server[1100]: 2026/09/22 22:22:24 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5754worker2 # [ 131.718643] nix-grpc-daemon[1215]: time=2026-09-22T22:22:24.361Z level=INFO msg="Upload complete. (223ms)"5755lb # [ 131.713667] niks3-server[1100]: 2026/09/22 22:22:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5756worker2 # [ 131.726143] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=1 nar_bytes_in=7685757lb # [ 131.719562] niks3-server[1100]: 2026/09/22 22:22:24 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5758lb # [ 131.727144] niks3-server[1100]: 2026/09/22 22:22:24 INFO Completed upload id=475759lb # [ 131.730937] niks3-server[1100]: 2026/09/22 22:22:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5760worker2 # [ 131.737101] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579625761worker2 # [ 131.749346] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5762worker2 # [ 131.754711] nix-daemon[837]: reaped child process 1322, status = succeeded5763worker2 # [ 131.766130] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5764client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5765client # farm-top> worker2: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5766client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5767client # farm-top> LOG-farm-top-on-standby5768worker2 # [ 133.229456] nix-daemon[837]: reaped child process 1331, status = succeeded5769worker2 # [ 133.364063] nix-daemon[837]: accepted connection from pid 1347, user nix-grpc-daemon (trusted)5770worker2 # [ 133.385334] nix-daemon[837]: reaped child process 1350, status = succeeded5771lb # [ 133.386345] niks3-server[1100]: 2026/09/22 22:22:26 INFO Received uploads request method=POST path=/api/pending_closures5772worker2 # [ 133.408632] nix-grpc-daemon[1215]: time=2026-09-22T22:22:26.049Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5773worker2 # [ 133.411006] nix-grpc-daemon[1215]: time=2026-09-22T22:22:26.049Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5774lb # [ 133.436596] niks3-server[1100]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5775worker2 # [ 133.452317] nix-grpc-daemon[1215]: time=2026-09-22T22:22:26.094Z level=INFO msg="Uploading 1 narinfos"5776lb # [ 133.448647] niks3-server[1100]: 2026/09/22 22:22:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5777lb # [ 133.451695] niks3-server[1100]: 2026/09/22 22:22:26 INFO Signed narinfos id=48 count=15778lb # [ 133.461081] niks3-server[1100]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5779lb # [ 133.470544] niks3-server[1100]: 2026/09/22 22:22:26 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5780lb # [ 133.481316] niks3-server[1100]: 2026/09/22 22:22:26 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5781worker2 # [ 133.491115] nix-grpc-daemon[1215]: time=2026-09-22T22:22:26.133Z level=INFO msg="Upload complete. (310ms)"5782lb # [ 133.489721] niks3-server[1100]: 2026/09/22 22:22:26 INFO Completed upload id=485783worker2 # [ 133.508646] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15784worker2 # [ 133.526571] nix-daemon[837]: reaped child process 1329, status = succeeded5785client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 7.37 seconds)5786client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5787client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.20 seconds)5788worker1 # [ 133.842949] systemd[1]: Listening on Nix worker-protocol over gRPC.5789worker1 # [ 133.853413] systemd[1]: Starting Nix worker-protocol over gRPC...5790worker1 # [ 133.999257] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5791worker1 # [ 134.001993] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5792worker1 # [ 134.006443] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15793worker1 # [ 134.012045] nix-grpc-daemon[1515]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5794worker1 # [ 134.028195] nix-grpc-daemon[1515]: E0000 00:00:1790115746.651217 1515 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5795lb: must succeed: curl -sf localhost:9901/clusters5796worker1 # [ 134.060325] nix-grpc-daemon[1515]: E0000 00:00:1790115746.655463 1515 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5797worker1 # [ 134.062821] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5798worker1 # [ 134.074469] nix-grpc-daemon[1515]: E0000 00:00:1790115746.665519 1557 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5799worker1 # [ 134.077547] nix-grpc-daemon[1515]: E0000 00:00:1790115746.689486 1557 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5800worker1 # [ 134.080046] nix-grpc-daemon[1515]: E0000 00:00:1790115746.689640 1557 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5801worker1 # [ 134.082431] systemd[1]: Started Nix worker-protocol over gRPC.5802worker1 # [ 134.083250] nix-grpc-daemon[1515]: E0000 00:00:1790115746.713048 1557 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5803worker1 # [ 134.091161] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:26Z level=info event=scheduler_connected addr=lb:500515804worker2 # [ 134.089423] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:26Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:331025805worker2 # [ 134.094839] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:26Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05806lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.09 seconds)5807worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5808worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5809lb: must succeed: curl -sf localhost:9901/clusters5810lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5811lb: must succeed: curl -sf localhost:9901/clusters5812lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5813worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5814worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5815(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.79 seconds)5816subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5817client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5818client # [ 134.414878] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5819client # Running as unit: rs.service; invocation ID: a68250cdb6b54c778086b27cb8b40d585820client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5821worker2 # [ 134.609312] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:579625822client # [ 134.616591] nix[1400]: don't know how to build these paths:5823client # [ 134.617616] nix[1400]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5824worker2 # [ 134.629788] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05825worker2 # [ 134.636939] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.4:50051 clients=15826worker2 # [ 134.675408] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579625827worker2 # [ 134.689569] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:579625828client # [ 134.703895] nix[1400]: copying 1 paths to 'lb:50051'...5829worker2 # [ 134.703520] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:579625830client # [ 134.706451] nix[1400]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5831worker2 # [ 134.718854] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5832worker2 # [ 134.885961] nix-daemon[837]: accepted connection from pid 1363, user nix-grpc-daemon (trusted)5833worker2 # [ 134.907968] nix-daemon[837]: reaped child process 1366, status = succeeded5834lb # [ 134.903908] niks3-server[1100]: 2026/09/22 22:22:27 INFO Received uploads request method=POST path=/api/pending_closures5835worker2 # [ 134.924975] nix-grpc-daemon[1215]: time=2026-09-22T22:22:27.566Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5836worker2 # [ 134.926521] nix-grpc-daemon[1215]: time=2026-09-22T22:22:27.566Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5837lb # [ 134.939047] niks3-server[1100]: 2026/09/22 22:22:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5838worker2 # [ 134.958258] nix-grpc-daemon[1215]: time=2026-09-22T22:22:27.600Z level=INFO msg="Uploading 1 narinfos"5839lb # [ 134.958732] niks3-server[1100]: 2026/09/22 22:22:27 INFO Signed narinfos id=49 count=15840lb # [ 134.972615] niks3-server[1100]: 2026/09/22 22:22:27 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5841lb # [ 134.988289] niks3-server[1100]: 2026/09/22 22:22:27 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5842lb # [ 134.993871] niks3-server[1100]: 2026/09/22 22:22:27 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5843lb # [ 135.000946] niks3-server[1100]: 2026/09/22 22:22:27 INFO Completed upload id=495844worker2 # [ 135.007470] nix-grpc-daemon[1215]: time=2026-09-22T22:22:27.650Z level=INFO msg="Upload complete. (270ms)"5845lb # [ 135.005438] niks3-server[1100]: 2026/09/22 22:22:27 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5846worker2 # [ 135.020244] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=0 paths=1 nar_bytes_in=4165847worker2 # [ 135.032691] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:579625848worker2 # [ 135.046867] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5849worker2 # [ 135.053229] nix-daemon[837]: reaped child process 1361, status = succeeded5850client # [ 135.063900] nix[1400]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5851worker2 # [ 135.068210] nix-daemon[837]: accepted connection from pid 1155, user nix-grpc-daemon (trusted)5852lb: must succeed: curl -sf localhost:9901/clusters5853lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5854lb: must succeed: curl -sf localhost:9901/clusters5855lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5856lb: must succeed: curl -sf localhost:9901/clusters5857lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5858lb: must succeed: curl -sf localhost:9901/clusters5859lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5860worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5861worker1 # [ 137.273634] systemd-journald[270]: Received client request to rotate journal, rotating.5862worker1 # [ 137.302736] systemd-journald[270]: Vacuuming done, freed 0B of archived journals from /var/log/journal/283d1111edd04c2c9541ffd544a9bf18.5863worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.10 seconds)5864worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service5865worker2 # [ 137.355778] systemd[1]: Stopping Nix worker-protocol over gRPC...5866worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.06 seconds)5867client: must succeed: sleep 5.05868worker2 # [ 138.334185] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:30Z level=info event=draining5869lb # [ 139.918924] envoy[895]: conn peer=[2001:db8:1::1]:51130 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9675 tx=4436 ms=70185870client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5871worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5872worker2 # [ 142.978199] nix-daemon[837]: reaped child process 1370, status = succeeded5873worker2 # [ 143.058772] nix-daemon[837]: accepted connection from pid 1411, user nix-grpc-daemon (trusted)5874worker2 # [ 143.080900] nix-daemon[837]: reaped child process 1414, status = succeeded5875lb # [ 143.073498] niks3-server[1100]: 2026/09/22 22:22:35 INFO Received uploads request method=POST path=/api/pending_closures5876worker2 # [ 143.092474] nix-grpc-daemon[1215]: time=2026-09-22T22:22:35.734Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5877worker2 # [ 143.094183] nix-grpc-daemon[1215]: time=2026-09-22T22:22:35.734Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5878lb # [ 143.111468] niks3-server[1100]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5879worker2 # [ 143.124682] nix-grpc-daemon[1215]: time=2026-09-22T22:22:35.767Z level=INFO msg="Uploading 1 narinfos"5880lb # [ 143.116451] niks3-server[1100]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5881lb # [ 143.125746] niks3-server[1100]: 2026/09/22 22:22:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5882lb # [ 143.131060] niks3-server[1100]: 2026/09/22 22:22:35 INFO Signed narinfos id=50 count=15883lb # [ 143.135370] niks3-server[1100]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=log/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5884worker2 # [ 143.149447] nix-grpc-daemon[1215]: time=2026-09-22T22:22:35.792Z level=INFO msg="Upload complete. (263ms)"5885lb # [ 143.144243] niks3-server[1100]: 2026/09/22 22:22:35 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5886lb # [ 143.147796] niks3-server[1100]: 2026/09/22 22:22:35 INFO Completed upload id=505887lb # [ 143.151546] niks3-server[1100]: 2026/09/22 22:22:35 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5888worker2 # [ 143.161001] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57962 duration_s=8 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15889worker2 # [ 143.175691] nix-daemon[837]: reaped child process 1368, status = succeeded5890client # [ 143.204431] systemd[1]: rs.service: Deactivated successfully.5891client # [ 143.209724] systemd[1]: rs.service: Consumed 193ms CPU time over 8.793s wall clock time, 41.2M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.5892worker2 # [ 143.334545] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:35Z level=info event=signal_exit5893lb # [ 143.628273] niks3-server[1100]: 2026/09/22 22:22:36 INFO lead: released remote=192.168.1.4:371825894lb # [ 144.014776] niks3-server[1100]: 2026/09/22 22:22:36 INFO lead: acquired remote=192.168.1.3:432765895worker1 # [ 144.026407] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:36Z level=info event=scheduler_take_over5896worker1 # [ 144.637456] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_restarting addr=lb:500515897worker2 # [ 144.638667] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=scheduler_restarting5898worker2 # [ 144.644306] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=scheduler_restarting addr=lb:500515899worker2 # [ 144.649463] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:331025900worker2 # [ 144.654796] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=worker_gone addr=192.168.1.4:500515901worker2 # [ 144.659914] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:331025902worker2 # [ 144.664821] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=worker_gone addr=192.168.1.3:500515903worker2 # [ 144.682479] nix-daemon[837]: reaped child process 1207, status = succeeded5904worker1 # [ 144.838534] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_connected addr=lb:500515905worker1 # [ 144.843766] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5906worker2 # [ 144.857179] nix-grpc-daemon[1155]: ts=2026-09-22T22:22:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5907worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.19 seconds)5908client: waiting for success: ! systemctl is-active rs5909client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5910client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5911client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5912client: must fail: journalctl -u rs | grep -q 'reconnecting'5913client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5914lb: must succeed: curl -sf localhost:9901/clusters5915worker1 # [ 145.043141] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_connected addr=lb:500515916worker1 # [ 145.046650] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5917lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)5918worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5919worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5920worker1 # [ 145.247746] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=scheduler_connected addr=lb:500515921worker1 # [ 145.257736] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:483405922worker1 # [ 145.263222] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:37Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05923worker2 # [ 145.396652] nix-grpc-daemon[1155]: E0000 00:00:1790115758.039029 1200 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5924worker2 # [ 145.421820] nix-grpc-daemon[1155]: E0000 00:00:1790115758.064341 1200 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5925lb: must succeed: curl -sf localhost:9901/clusters5926lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5927worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5928worker2 # [ 146.240297] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5929worker2 # [ 146.246919] systemd[1]: Stopped Nix worker-protocol over gRPC.5930worker2 # [ 146.253214] systemd[1]: nix-grpc-daemon.service: Consumed 1.983s CPU time over 39.076s wall clock time, 39.7M memory peak, 62.8K incoming IP traffic, 142.3K outgoing IP traffic.5931worker2 # [ 146.271702] systemd[1]: Starting scheduler-token.service...5932worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5933worker2 # [ 146.350578] systemd[1]: scheduler-token.service: Deactivated successfully.5934worker2 # [ 146.355706] systemd[1]: Finished scheduler-token.service.5935worker2 # [ 146.360724] systemd[1]: scheduler-token.service: Consumed 48ms CPU time over 81ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5936worker2 # [ 146.368908] systemd[1]: Starting Nix worker-protocol over gRPC...5937worker2 # [ 146.477489] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5938worker2 # [ 146.479878] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5939worker2 # [ 146.483953] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15940worker2 # [ 146.491284] nix-grpc-daemon[1424]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5941worker2 # [ 146.504151] nix-grpc-daemon[1424]: E0000 00:00:1790115759.131527 1424 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5942worker2 # [ 146.506575] nix-grpc-daemon[1424]: E0000 00:00:1790115759.133786 1424 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5943worker2 # [ 146.536928] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5944worker2 # [ 146.539209] nix-grpc-daemon[1424]: E0000 00:00:1790115759.143174 1463 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5945worker2 # [ 146.542025] nix-grpc-daemon[1424]: E0000 00:00:1790115759.161047 1463 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5946worker2 # [ 146.547436] nix-grpc-daemon[1424]: E0000 00:00:1790115759.161232 1463 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5947worker2 # [ 146.552345] nix-grpc-daemon[1424]: E0000 00:00:1790115759.178351 1463 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5948worker2 # [ 146.555049] systemd[1]: Started Nix worker-protocol over gRPC.5949worker2 # [ 146.555992] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:39Z level=info event=scheduler_connected addr=lb:500515950worker1 # [ 146.568978] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:39Z level=info event=oidc_jwks provider=mock keys=15951worker1 # [ 146.573147] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:39Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:483405952worker1 # [ 146.578557] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:39Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05953lb: must succeed: curl -sf localhost:9901/clusters5954lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5955worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5956worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5957(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.06 seconds)5958subtest: scheduler restart mid-build: build finishes, no double build5959client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5960client # [ 147.474821] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5961client # Running as unit: mid.service; invocation ID: e73fbeb486b14434ab5c58dad2b751145962client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.10 seconds)5963worker1 # [ 147.691881] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:483425964worker1 # [ 147.706656] nix-daemon[848]: accepted connection from pid 1515, user nix-grpc-daemon (trusted)5965client # [ 147.724999] nix[1436]: don't know how to build these paths:5966client # [ 147.726895] nix[1436]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5967worker1 # [ 147.745111] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:40Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05968worker1 # [ 147.752527] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:40Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=15969worker1 # [ 147.790342] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483425970worker1 # [ 147.799816] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 1/5)5971worker1 # [ 147.809090] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 193 ms (attempt 2/5)5972worker1 # [ 148.010850] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 297 ms (attempt 3/5)5973worker1 # [ 148.310003] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 314 ms (attempt 4/5)5974worker1 # [ 148.626746] nix-grpc-daemon[1515]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5975worker1 # [ 148.634695] nix-grpc-daemon[1515]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5976worker1 # [ 148.654944] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483425977client # [ 148.667723] nix[1436]: copying 1 paths to 'lb:50051'...5978client # [ 148.673356] nix[1436]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5979worker1 # [ 148.678997] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483425980worker1 # [ 148.700266] nix-daemon[848]: accepted connection from pid 1515, user nix-grpc-daemon (trusted)5981worker1 # [ 148.759308] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=info event=niks3_push_started pid=16445982worker1 # [ 148.894136] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.532Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5983worker1 # [ 149.007477] nix-daemon[848]: accepted connection from pid 1657, user nix-grpc-daemon (trusted)5984worker1 # [ 149.029014] nix-daemon[848]: reaped child process 1660, status = succeeded5985lb # [ 149.023773] niks3-server[1100]: 2026/09/22 22:22:41 INFO Received uploads request method=POST path=/api/pending_closures5986worker1 # [ 149.044061] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.683Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5987worker1 # [ 149.045550] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.683Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5988lb # [ 149.067713] niks3-server[1100]: 2026/09/22 22:22:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5989worker1 # [ 149.088815] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.727Z level=INFO msg="Uploading 1 narinfos"5990lb # [ 149.080592] niks3-server[1100]: 2026/09/22 22:22:41 INFO Signed narinfos id=51 count=15991lb # [ 149.090071] niks3-server[1100]: 2026/09/22 22:22:41 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5992lb # [ 149.099867] niks3-server[1100]: 2026/09/22 22:22:41 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5993lb # [ 149.116802] niks3-server[1100]: 2026/09/22 22:22:41 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5994worker1 # [ 149.133726] nix-grpc-daemon[1644]: time=2026-09-22T22:22:41.773Z level=INFO msg="Upload complete. (236ms)"5995lb # [ 149.125112] niks3-server[1100]: 2026/09/22 22:22:41 INFO Completed upload id=515996lb # [ 149.130299] niks3-server[1100]: 2026/09/22 22:22:41 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5997worker1 # [ 149.145260] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_in=4245998worker1 # [ 149.152348] nix-grpc-daemon[1515]: ts=2026-09-22T22:22:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483425999worker1 # [ 149.164568] nix-daemon[848]: accepted connection from pid 1515, user nix-grpc-daemon (trusted)6000worker1 # [ 149.169582] nix-daemon[848]: reaped child process 1641, status = succeeded6001worker1 # [ 149.181900] nix-daemon[848]: accepted connection from pid 1515, user nix-grpc-daemon (trusted)6002client # [ 149.176097] nix[1436]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6003lb # [ 149.920851] envoy[895]: conn peer=[2001:db8:1::1]:59574 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1541 ms=85756004lb: must succeed: curl -sf localhost:9901/clusters6005lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.10 seconds)6006lb: must succeed: curl -sf localhost:9901/clusters6007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6008worker1: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6009worker1 # [ 150.346636] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1515 (nix-grpc-daemon) on client request.6010worker1 # [ 150.348697] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6011lb # [ 149.925932] envoy[895]: conn peer=[2001:db8:1::4]:53934 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=24909 tx=1639 ms=374666012lb # [ 150.349402] niks3-server[1100]: 2026/09/22 22:22:42 INFO lead: released remote=192.168.1.3:432766013worker1 # [ 150.367607] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6014client # [ 150.369806] nix[1436]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6015worker2 # [ 150.370329] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:43Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6016worker1 # [ 150.382918] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6017client # [ 150.374997] nix[1436]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6018worker1 # [ 150.394239] systemd[1]: nix-grpc-daemon.service: Consumed 513ms CPU time over 16.515s wall clock time, 34.6M memory peak, 15.9K incoming IP traffic, 26K outgoing IP traffic.6019worker1 # [ 150.404224] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6020worker1 # [ 150.405751] systemd[1]: Starting Nix worker-protocol over gRPC...6021worker1 # [ 150.456328] nix-daemon[848]: reaped child process 1662, status = succeeded6022worker1 # [ 150.460327] nix-daemon[848]: reaped child process 1630, status = succeeded6023worker1 # [ 150.466016] nix-daemon[1664]: unexpected Nix daemon error: error: interrupted by the user6024worker1 # [ 150.479706] nix-daemon[848]: reaped child process 1664, status = failed with exit code 16025worker1 # [ 150.523872] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6026worker1 # [ 150.526267] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6027worker1 # [ 150.529984] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16028worker1 # [ 150.535247] nix-grpc-daemon[1696]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6029lb # [ 150.537143] niks3-server[1100]: 2026/09/22 22:22:43 INFO lead: acquired remote=192.168.1.3:557686030worker1: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.24 seconds)6031client: must succeed: sleep 5.06032worker1 # [ 150.552193] nix-grpc-daemon[1696]: E0000 00:00:1790115763.173886 1696 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6033worker2 # [ 150.571091] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:43Z level=info event=scheduler_connected addr=lb:500516034worker1 # [ 150.580147] nix-grpc-daemon[1696]: E0000 00:00:1790115763.177441 1696 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6035worker1 # [ 150.582415] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6036worker1 # [ 150.586894] nix-grpc-daemon[1696]: E0000 00:00:1790115763.186023 1738 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6037worker1 # [ 150.589651] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=scheduler_take_over6038worker1 # [ 150.590738] nix-grpc-daemon[1696]: E0000 00:00:1790115763.204366 1738 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6039worker1 # [ 150.594133] nix-grpc-daemon[1696]: E0000 00:00:1790115763.204503 1738 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6040worker1 # [ 150.599941] nix-grpc-daemon[1696]: E0000 00:00:1790115763.219374 1738 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6041worker1 # [ 150.605339] systemd[1]: Started Nix worker-protocol over gRPC.6042worker1 # [ 150.607970] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=scheduler_connected addr=lb:500516043worker1 # [ 150.609439] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:483586044worker1 # [ 150.610947] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06045worker1 # [ 150.612723] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=oidc_jwks provider=mock keys=16046worker1 # [ 150.613939] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:483586047worker1 # [ 150.615581] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06048worker1 # [ 150.883819] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06049worker1 # [ 150.891006] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.3:50051 clients=16050worker1 # [ 150.946299] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395886051worker1 # [ 150.959761] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6052worker1 # [ 150.980229] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6053worker1 # [ 150.997098] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6054client # [ 150.991768] nix[1436]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6055client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6056worker1 # [ 155.817430] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:48Z level=info event=niks3_push_started pid=17806057worker1 # [ 155.845870] nix-daemon[848]: reaped child process 1755, status = succeeded6058client: waiting for success: ! systemctl is-active mid6059worker1 # [ 155.944332] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.582Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6060worker1 # [ 156.050706] nix-daemon[848]: accepted connection from pid 1793, user nix-grpc-daemon (trusted)6061worker1 # [ 156.069202] nix-daemon[848]: reaped child process 1796, status = succeeded6062lb # [ 156.072069] niks3-server[1100]: 2026/09/22 22:22:48 INFO Received uploads request method=POST path=/api/pending_closures6063worker1 # [ 156.096080] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.733Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6064worker1 # [ 156.097774] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.733Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6065lb # [ 156.143468] niks3-server[1100]: 2026/09/22 22:22:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6066lb # [ 156.154112] niks3-server[1100]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6067lb # [ 156.165693] niks3-server[1100]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6068lb # [ 156.174511] niks3-server[1100]: 2026/09/22 22:22:48 INFO Signed narinfos id=52 count=16069worker1 # [ 156.182180] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.819Z level=INFO msg="Uploading 1 narinfos"6070lb # [ 156.185831] niks3-server[1100]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6071lb # [ 156.195507] niks3-server[1100]: 2026/09/22 22:22:48 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6072lb # [ 156.201481] niks3-server[1100]: 2026/09/22 22:22:48 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6073lb # [ 156.206453] niks3-server[1100]: 2026/09/22 22:22:48 INFO Completed upload id=526074worker1 # [ 156.216827] nix-grpc-daemon[1780]: time=2026-09-22T22:22:48.856Z level=INFO msg="Upload complete. (269ms)"6075worker1 # [ 156.232542] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39588 duration_s=5 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16076worker1 # [ 156.253709] nix-daemon[848]: reaped child process 1753, status = succeeded6077client # [ 156.260196] systemd[1]: mid.service: Deactivated successfully.6078client # [ 156.265225] systemd[1]: mid.service: Consumed 202ms CPU time over 8.786s wall clock time, 41.5M memory peak, 8.7K incoming IP traffic, 10.3K outgoing IP traffic.6079client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6080client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6081client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)6082(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.62 seconds)6083subtest: interrupting the client stops the build on the worker6084client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6085client # [ 157.070790] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6086client # Running as unit: intr.service; invocation ID: d8d6f2d69ff14e83ac676d0946853a136087client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6088worker1 # [ 157.251449] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:49Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395886089client # [ 157.256479] nix[1472]: don't know how to build these paths:6090client # [ 157.257415] nix[1472]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6091worker1 # [ 157.271565] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:49Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06092worker1 # [ 157.277794] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:49Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.4:50051 clients=16093worker2 # [ 157.331245] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866094worker2 # [ 157.345876] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6095worker2 # [ 157.374685] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 24 ms (attempt 1/5)6096worker2 # [ 157.407408] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 4 ms (attempt 2/5)6097worker2 # [ 157.420877] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 3/5)6098worker2 # [ 157.545343] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 4/5)6099worker2 # [ 157.684105] nix-grpc-daemon[1424]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6100worker2 # [ 157.689838] nix-grpc-daemon[1424]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6101worker2 # [ 157.709266] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866102client # [ 157.724436] nix[1472]: copying 1 paths to 'lb:50051'...6103worker2 # [ 157.725899] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368866104client # [ 157.726863] nix[1472]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6105worker2 # [ 157.742563] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6106worker2 # [ 157.771579] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=info event=niks3_push_started pid=15126107worker2 # [ 157.928845] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.570Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6108worker2 # [ 158.043426] nix-daemon[837]: accepted connection from pid 1525, user nix-grpc-daemon (trusted)6109worker2 # [ 158.063745] nix-daemon[837]: reaped child process 1528, status = succeeded6110lb # [ 158.063797] niks3-server[1100]: 2026/09/22 22:22:50 INFO Received uploads request method=POST path=/api/pending_closures6111worker2 # [ 158.083119] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.724Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6112worker2 # [ 158.087340] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.724Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6113lb # [ 158.099439] niks3-server[1100]: 2026/09/22 22:22:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6114lb # [ 158.106032] niks3-server[1100]: 2026/09/22 22:22:50 INFO Signed narinfos id=53 count=16115worker2 # [ 158.115437] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.757Z level=INFO msg="Uploading 1 narinfos"6116lb # [ 158.111697] niks3-server[1100]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6117lb # [ 158.117762] niks3-server[1100]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6118lb # [ 158.123664] niks3-server[1100]: 2026/09/22 22:22:50 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6119worker2 # [ 158.134942] nix-grpc-daemon[1512]: time=2026-09-22T22:22:50.777Z level=INFO msg="Upload complete. (206ms)"6120lb # [ 158.130077] niks3-server[1100]: 2026/09/22 22:22:50 INFO Completed upload id=536121lb # [ 158.131994] niks3-server[1100]: 2026/09/22 22:22:50 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6122worker2 # [ 158.144588] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=4246123worker2 # [ 158.147794] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866124worker2 # [ 158.158324] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6125worker2 # [ 158.162179] nix-daemon[837]: reaped child process 1510, status = succeeded6126worker2 # [ 158.172140] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6127client # [ 158.170757] nix[1472]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6128client: must succeed: systemctl kill -s INT intr6129client # [ 159.548940] systemd[1]: intr.service: Sent signal SIGINT to main process 1472 (nix) on client request.6130worker2 # [ 159.555663] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:52Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6131client # [ 159.559461] nix[1472]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6132client # [ 159.574073] nix[1472]: error: interrupted by the user6133client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6134worker2 # [ 159.599341] nix-daemon[837]: reaped child process 1530, status = succeeded6135client # [ 159.599058] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6136worker2 # [ 159.605398] nix-daemon[1532]: unexpected Nix daemon error: error: interrupted by the user6137client # [ 159.603779] systemd[1]: intr.service: Failed with result 'exit-code'.6138client # [ 159.606861] systemd[1]: intr.service: Consumed 173ms CPU time over 2.531s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6139worker2 # [ 159.609933] nix-daemon[837]: reaped child process 1532, status = failed with exit code 16140(finished: subtest: interrupting the client stops the build on the worker, in 2.65 seconds)6141subtest: a deploy mid-build drains: the build finishes, the new generation takes over6142client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6143client # [ 159.742945] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6144client # Running as unit: sw.service; invocation ID: 2232792ef1ea4f93928a67ab9bef37d86145client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)6146lb # [ 159.919421] envoy[895]: conn peer=[2001:db8:1::3]:54920 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=953 tx=431 ms=162926147lb # [ 159.924659] envoy[895]: conn peer=[2001:db8:1::1]:59092 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3884 tx=1922 ms=85696148lb # [ 159.935125] envoy[895]: conn peer=[2001:db8:1::1]:50056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2987 tx=829 ms=23356149client # [ 159.954938] nix[1493]: don't know how to build these paths:6150client # [ 159.956342] nix[1493]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6151worker2 # [ 159.953308] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:368866152worker1 # [ 159.970091] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:52Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06153worker1 # [ 159.970378] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:52Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.3:50051 clients=16154worker1 # [ 160.014401] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395886155worker1 # [ 160.029568] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 1/5)6156worker1 # [ 160.059606] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 2/5)6157worker1 # [ 160.069149] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 205 ms (attempt 3/5)6158worker1 # [ 160.276333] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 68 ms (attempt 4/5)6159worker1 # [ 160.346459] nix-grpc-daemon[1696]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6160worker1 # [ 160.353561] nix-grpc-daemon[1696]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6161worker1 # [ 160.370577] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395886162client # [ 160.379552] nix[1493]: copying 1 paths to 'lb:50051'...6163client # [ 160.384513] nix[1493]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6164worker1 # [ 160.389077] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395886165worker1 # [ 160.405245] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6166worker1 # [ 160.571780] nix-daemon[848]: accepted connection from pid 1828, user nix-grpc-daemon (trusted)6167worker1 # [ 160.590248] nix-daemon[848]: reaped child process 1831, status = succeeded6168lb # [ 160.589464] niks3-server[1100]: 2026/09/22 22:22:53 INFO Received uploads request method=POST path=/api/pending_closures6169worker1 # [ 160.609392] nix-grpc-daemon[1780]: time=2026-09-22T22:22:53.248Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6170worker1 # [ 160.611882] nix-grpc-daemon[1780]: time=2026-09-22T22:22:53.250Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6171lb # [ 160.623202] niks3-server[1100]: 2026/09/22 22:22:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6172worker1 # [ 160.642062] nix-grpc-daemon[1780]: time=2026-09-22T22:22:53.280Z level=INFO msg="Uploading 1 narinfos"6173lb # [ 160.638366] niks3-server[1100]: 2026/09/22 22:22:53 INFO Signed narinfos id=54 count=16174lb # [ 160.645772] niks3-server[1100]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6175lb # [ 160.657540] niks3-server[1100]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6176lb # [ 160.665366] niks3-server[1100]: 2026/09/22 22:22:53 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6177lb # [ 160.673952] niks3-server[1100]: 2026/09/22 22:22:53 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6178lb # [ 160.680324] niks3-server[1100]: 2026/09/22 22:22:53 INFO Completed upload id=546179worker1 # [ 160.689986] nix-grpc-daemon[1780]: time=2026-09-22T22:22:53.329Z level=INFO msg="Upload complete. (267ms)"6180worker1 # [ 160.700649] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39588 duration_s=0 paths=1 nar_bytes_in=4166181worker1 # [ 160.719140] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395886182worker1 # [ 160.731595] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6183worker1 # [ 160.737869] nix-daemon[848]: reaped child process 1826, status = succeeded6184client # [ 160.742563] nix[1493]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6185worker1 # [ 160.748370] nix-daemon[848]: accepted connection from pid 1696, user nix-grpc-daemon (trusted)6186worker1: must succeed: systemctl reload nix-grpc-daemon.service6187worker1 # [ 162.359478] systemd[1]: Reloading Nix worker-protocol over gRPC...6188worker1 # [ 162.402385] systemd[1]: Reloaded Nix worker-protocol over gRPC.6189worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.08 seconds)6190worker1: must succeed: pgrep -f 'read -t [9]0 x'6191worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6192client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26193worker1 # [ 162.547255] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:55Z level=info event=draining6194worker2 # [ 162.664618] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:368866195client # don't know how to build these paths:6196client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6197worker2 # [ 162.674628] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866198worker1 # [ 162.702178] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:55Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06199worker1 # [ 162.704155] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:55Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.4:50051 clients=16200worker2 # [ 162.751844] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866201worker2 # [ 162.767434] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866202client # copying 1 paths to 'lb:50051'...6203worker2 # [ 162.783964] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368866204client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6205worker2 # [ 162.808328] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6206worker2 # [ 162.977539] nix-daemon[837]: accepted connection from pid 1579, user nix-grpc-daemon (trusted)6207worker2 # [ 162.999323] nix-daemon[837]: reaped child process 1582, status = succeeded6208lb # [ 162.999685] niks3-server[1100]: 2026/09/22 22:22:55 INFO Received uploads request method=POST path=/api/pending_closures6209worker2 # [ 163.018091] nix-grpc-daemon[1512]: time=2026-09-22T22:22:55.659Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6210worker2 # [ 163.019770] nix-grpc-daemon[1512]: time=2026-09-22T22:22:55.659Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6211lb # [ 163.027646] niks3-server[1100]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6212worker2 # [ 163.046875] nix-grpc-daemon[1512]: time=2026-09-22T22:22:55.688Z level=INFO msg="Uploading 1 narinfos"6213lb # [ 163.040173] niks3-server[1100]: 2026/09/22 22:22:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6214lb # [ 163.051611] niks3-server[1100]: 2026/09/22 22:22:55 INFO Signed narinfos id=55 count=16215lb # [ 163.061206] niks3-server[1100]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6216lb # [ 163.075786] niks3-server[1100]: 2026/09/22 22:22:55 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6217lb # [ 163.082471] niks3-server[1100]: 2026/09/22 22:22:55 INFO Completed upload id=556218worker2 # [ 163.086995] nix-grpc-daemon[1512]: time=2026-09-22T22:22:55.729Z level=INFO msg="Upload complete. (261ms)"6219lb # [ 163.084902] niks3-server[1100]: 2026/09/22 22:22:55 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6220worker2 # [ 163.098465] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=5286221worker2 # [ 163.116275] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866222worker2 # [ 163.122864] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6223worker2 # [ 163.128214] nix-daemon[837]: reaped child process 1577, status = succeeded6224client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6225worker2 # [ 163.141871] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6226client # farm-a> worker2: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6227client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6228client # farm-a> LOG-farm-a-during-drain6229worker2 # [ 164.808944] nix-daemon[837]: reaped child process 1586, status = succeeded6230worker2 # [ 164.925917] nix-daemon[837]: accepted connection from pid 1602, user nix-grpc-daemon (trusted)6231worker2 # [ 164.944840] nix-daemon[837]: reaped child process 1605, status = succeeded6232lb # [ 164.949369] niks3-server[1100]: 2026/09/22 22:22:57 INFO Received uploads request method=POST path=/api/pending_closures6233worker2 # [ 164.968399] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.608Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6234worker2 # [ 164.970104] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.608Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6235lb # [ 164.995613] niks3-server[1100]: 2026/09/22 22:22:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6236worker2 # [ 165.013957] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.656Z level=INFO msg="Uploading 1 narinfos"6237lb # [ 165.010451] niks3-server[1100]: 2026/09/22 22:22:57 INFO Signed narinfos id=56 count=16238lb # [ 165.027202] niks3-server[1100]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6239lb # [ 165.042180] niks3-server[1100]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6240worker2 # [ 165.055488] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.697Z level=INFO msg="Upload complete. (301ms)"6241lb # [ 165.052518] niks3-server[1100]: 2026/09/22 22:22:57 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6242lb # [ 165.058462] niks3-server[1100]: 2026/09/22 22:22:57 INFO Completed upload id=566243lb # [ 165.061500] niks3-server[1100]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6244worker2 # [ 165.069502] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16245worker1 # [ 165.079818] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:57Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06246worker1 # [ 165.081815] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:57Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.4:50051 clients=16247lb # [ 165.070469] niks3-server[1100]: 2026/09/22 22:22:57 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6248worker2 # [ 165.091736] nix-daemon[837]: reaped child process 1584, status = succeeded6249worker2 # [ 165.128532] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866250worker2 # [ 165.145988] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866251worker2 # [ 165.161584] nix-grpc-daemon[1512]: time=2026-09-22T22:22:57.804Z level=INFO msg="All 1 paths already cached"6252client # copying 1 paths to 'lb:50051'...6253worker2 # [ 165.173084] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368866254client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6255worker2 # [ 165.202233] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6256worker2 # [ 165.363767] nix-daemon[837]: accepted connection from pid 1609, user nix-grpc-daemon (trusted)6257worker2 # [ 165.386445] nix-daemon[837]: reaped child process 1612, status = succeeded6258lb # [ 165.378801] niks3-server[1100]: 2026/09/22 22:22:58 INFO Received uploads request method=POST path=/api/pending_closures6259worker2 # [ 165.396107] nix-grpc-daemon[1512]: time=2026-09-22T22:22:58.038Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6260worker2 # [ 165.397725] nix-grpc-daemon[1512]: time=2026-09-22T22:22:58.038Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6261lb # [ 165.414810] niks3-server[1100]: 2026/09/22 22:22:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6262worker2 # [ 165.430414] nix-grpc-daemon[1512]: time=2026-09-22T22:22:58.072Z level=INFO msg="Uploading 1 narinfos"6263lb # [ 165.430798] niks3-server[1100]: 2026/09/22 22:22:58 INFO Signed narinfos id=57 count=16264lb # [ 165.439826] niks3-server[1100]: 2026/09/22 22:22:58 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6265worker2 # [ 165.459911] nix-grpc-daemon[1512]: time=2026-09-22T22:22:58.102Z level=INFO msg="Upload complete. (240ms)"6266lb # [ 165.453877] niks3-server[1100]: 2026/09/22 22:22:58 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6267lb # [ 165.460778] niks3-server[1100]: 2026/09/22 22:22:58 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6268worker2 # [ 165.468361] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=6406269lb # [ 165.467481] niks3-server[1100]: 2026/09/22 22:22:58 INFO Completed upload id=576270lb # [ 165.471831] niks3-server[1100]: 2026/09/22 22:22:58 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6271worker2 # [ 165.480363] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866272worker2 # [ 165.493207] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6273worker2 # [ 165.498502] nix-daemon[837]: reaped child process 1607, status = succeeded6274client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6275worker2 # [ 165.512300] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6276client # farm-b> worker2: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6277client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6278client # farm-b> LOG-farm-b-during-drain6279worker2 # [ 166.779317] nix-daemon[837]: reaped child process 1616, status = succeeded6280worker2 # [ 166.913102] nix-daemon[837]: accepted connection from pid 1632, user nix-grpc-daemon (trusted)6281worker2 # [ 166.939783] nix-daemon[837]: reaped child process 1635, status = succeeded6282lb # [ 166.933027] niks3-server[1100]: 2026/09/22 22:22:59 INFO Received uploads request method=POST path=/api/pending_closures6283worker2 # [ 166.952341] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.593Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6284worker2 # [ 166.954566] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.593Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6285lb # [ 166.975003] niks3-server[1100]: 2026/09/22 22:22:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6286worker2 # [ 166.989241] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.631Z level=INFO msg="Uploading 1 narinfos"6287lb # [ 166.988955] niks3-server[1100]: 2026/09/22 22:22:59 INFO Signed narinfos id=58 count=16288lb # [ 166.991704] niks3-server[1100]: 2026/09/22 22:22:59 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6289lb # [ 167.006027] niks3-server[1100]: 2026/09/22 22:22:59 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6290lb # [ 167.019679] niks3-server[1100]: 2026/09/22 22:22:59 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6291worker2 # [ 167.025675] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.668Z level=INFO msg="Upload complete. (301ms)"6292lb # [ 167.028531] niks3-server[1100]: 2026/09/22 22:22:59 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6293lb # [ 167.033129] niks3-server[1100]: 2026/09/22 22:22:59 INFO Completed upload id=586294worker1 # [ 167.047758] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:59Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06295lb # [ 167.037227] niks3-server[1100]: 2026/09/22 22:22:59 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6296worker1 # [ 167.049730] nix-grpc-daemon[1696]: ts=2026-09-22T22:22:59Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.4:50051 clients=16297worker2 # [ 167.041447] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16298worker2 # [ 167.062730] nix-daemon[837]: reaped child process 1614, status = succeeded6299worker2 # [ 167.096501] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866300worker2 # [ 167.119751] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866301worker2 # [ 167.135458] nix-grpc-daemon[1512]: time=2026-09-22T22:22:59.776Z level=INFO msg="All 2 paths already cached"6302client # copying 1 paths to 'lb:50051'...6303worker2 # [ 167.147422] nix-grpc-daemon[1424]: ts=2026-09-22T22:22:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368866304client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6305worker2 # [ 167.171973] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6306worker2 # [ 167.336448] nix-daemon[837]: accepted connection from pid 1639, user nix-grpc-daemon (trusted)6307worker2 # [ 167.356617] nix-daemon[837]: reaped child process 1642, status = succeeded6308lb # [ 167.355883] niks3-server[1100]: 2026/09/22 22:22:59 INFO Received uploads request method=POST path=/api/pending_closures6309worker2 # [ 167.373166] nix-grpc-daemon[1512]: time=2026-09-22T22:23:00.015Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6310worker2 # [ 167.375025] nix-grpc-daemon[1512]: time=2026-09-22T22:23:00.015Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6311lb # [ 167.390820] niks3-server[1100]: 2026/09/22 22:23:00 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6312worker2 # [ 167.409066] nix-grpc-daemon[1512]: time=2026-09-22T22:23:00.051Z level=INFO msg="Uploading 1 narinfos"6313lb # [ 167.402966] niks3-server[1100]: 2026/09/22 22:23:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6314lb # [ 167.413454] niks3-server[1100]: 2026/09/22 22:23:00 INFO Signed narinfos id=59 count=16315lb # [ 167.419138] niks3-server[1100]: 2026/09/22 22:23:00 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6316lb # [ 167.431379] niks3-server[1100]: 2026/09/22 22:23:00 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6317worker2 # [ 167.442118] nix-grpc-daemon[1512]: time=2026-09-22T22:23:00.084Z level=INFO msg="Upload complete. (251ms)"6318lb # [ 167.438914] niks3-server[1100]: 2026/09/22 22:23:00 INFO Completed upload id=596319lb # [ 167.441377] niks3-server[1100]: 2026/09/22 22:23:00 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6320worker2 # [ 167.452385] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=7686321worker2 # [ 167.472266] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866322worker2 # [ 167.479465] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6323worker2 # [ 167.485399] nix-daemon[837]: reaped child process 1637, status = succeeded6324client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6325worker2 # [ 167.497637] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6326client # farm-top> worker2: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6327client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6328client # farm-top> LOG-farm-top-during-drain6329worker2 # [ 168.388645] nix-daemon[837]: reaped child process 1646, status = succeeded6330worker2 # [ 168.483761] nix-daemon[837]: accepted connection from pid 1662, user nix-grpc-daemon (trusted)6331worker2 # [ 168.508056] nix-daemon[837]: reaped child process 1665, status = succeeded6332lb # [ 168.501150] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received uploads request method=POST path=/api/pending_closures6333worker2 # [ 168.519387] nix-grpc-daemon[1512]: time=2026-09-22T22:23:01.160Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6334worker2 # [ 168.521133] nix-grpc-daemon[1512]: time=2026-09-22T22:23:01.160Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6335lb # [ 168.544638] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6336lb # [ 168.558435] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6337worker2 # [ 168.572492] nix-grpc-daemon[1512]: time=2026-09-22T22:23:01.214Z level=INFO msg="Uploading 1 narinfos"6338lb # [ 168.570300] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6339lb # [ 168.576998] niks3-server[1100]: 2026/09/22 22:23:01 INFO Signed narinfos id=60 count=16340lb # [ 168.578973] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6341lb # [ 168.589808] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6342lb # [ 168.592582] niks3-server[1100]: 2026/09/22 22:23:01 INFO Completed upload id=606343worker2 # [ 168.595709] nix-grpc-daemon[1512]: time=2026-09-22T22:23:01.238Z level=INFO msg="Upload complete. (283ms)"6344lb # [ 168.594371] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6345worker2 # [ 168.609263] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16346worker2 # [ 168.626096] nix-daemon[837]: reaped child process 1644, status = succeeded6347client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 6.21 seconds)6348client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6349client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.20 seconds)6350worker1: must succeed: pkill -f 'read -t [9]0 x'6351worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6352client: waiting for success: ! systemctl is-active sw6353worker1 # [ 168.984200] nix-daemon[848]: reaped child process 1835, status = succeeded6354worker1 # [ 169.087172] nix-daemon[848]: accepted connection from pid 1882, user nix-grpc-daemon (trusted)6355worker1 # [ 169.107661] nix-daemon[848]: reaped child process 1885, status = succeeded6356lb # [ 169.104180] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received uploads request method=POST path=/api/pending_closures6357worker1 # [ 169.124523] nix-grpc-daemon[1780]: time=2026-09-22T22:23:01.763Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6358worker1 # [ 169.126938] nix-grpc-daemon[1780]: time=2026-09-22T22:23:01.765Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6359lb # [ 169.148403] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6360lb # [ 169.162305] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6361worker1 # [ 169.185231] nix-grpc-daemon[1780]: time=2026-09-22T22:23:01.824Z level=INFO msg="Uploading 1 narinfos"6362lb # [ 169.174996] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6363lb # [ 169.185376] niks3-server[1100]: 2026/09/22 22:23:01 INFO Signed narinfos id=61 count=16364lb # [ 169.191145] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6365lb # [ 169.201387] niks3-server[1100]: 2026/09/22 22:23:01 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6366lb # [ 169.209642] niks3-server[1100]: 2026/09/22 22:23:01 INFO Completed upload id=616367worker1 # [ 169.216987] nix-grpc-daemon[1780]: time=2026-09-22T22:23:01.856Z level=INFO msg="Upload complete. (247ms)"6368lb # [ 169.212404] niks3-server[1100]: 2026/09/22 22:23:01 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6369worker1 # [ 169.233274] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39588 duration_s=8 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16370worker1 # [ 169.252485] nix-daemon[848]: reaped child process 1833, status = succeeded6371client # [ 169.269489] systemd[1]: sw.service: Deactivated successfully.6372client # [ 169.274243] systemd[1]: sw.service: Consumed 196ms CPU time over 9.530s wall clock time, 41.4M memory peak, 7.6K incoming IP traffic, 8.5K outgoing IP traffic.6373worker1 # [ 169.548832] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:02Z level=info event=signal_exit6374lb # [ 169.919439] envoy[895]: conn peer=[2001:db8:1::1]:55582 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4381 ms=59646375lb # [ 169.924738] envoy[895]: conn peer=[2001:db8:1::1]:50070 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2995 tx=1558 ms=92936376client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6377client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6378client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.06 seconds)6379worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6380lb # [ 170.541469] niks3-server[1100]: 2026/09/22 22:23:03 INFO lead: released remote=192.168.1.3:557686381lb # [ 171.494177] niks3-server[1100]: 2026/09/22 22:23:04 INFO lead: acquired remote=192.168.1.4:524766382worker2 # [ 171.506738] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_take_over6383worker1 # [ 171.550756] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=scheduler_restarting6384worker2 # [ 171.547925] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_restarting addr=lb:500516385worker1 # [ 171.562598] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:483586386worker1 # [ 171.568262] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=worker_gone addr=192.168.1.4:500516387worker1 # [ 171.572479] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=scheduler_restarting addr=lb:500516388worker1 # [ 171.577087] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:483586389worker1 # [ 171.582192] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=worker_gone addr=192.168.1.3:500516390worker1 # [ 171.606020] nix-daemon[848]: reaped child process 1751, status = succeeded6391worker2 # [ 171.749231] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_connected addr=lb:500516392worker2 # [ 171.754662] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6393worker1 # [ 171.779703] nix-grpc-daemon[1696]: ts=2026-09-22T22:23:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6394worker2 # [ 171.955738] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_connected addr=lb:500516395worker2 # [ 171.961436] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6396worker1 # [ 172.156473] nix-grpc-daemon[1696]: E0000 00:00:1790115784.795663 1742 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 20, started true, shutdown true6397worker1 # [ 172.164176] nix-grpc-daemon[1696]: E0000 00:00:1790115784.803180 1742 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6398worker2 # [ 172.162654] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_connected addr=lb:500516399worker2 # [ 172.167876] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6400worker2 # [ 172.368886] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=scheduler_connected addr=lb:500516401worker2 # [ 172.398123] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=oidc_jwks provider=mock keys=16402worker2 # [ 172.402699] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:506486403worker2 # [ 172.408107] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06404worker1 # [ 172.582784] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6405worker1 # [ 172.589277] systemd[1]: nix-grpc-daemon.service: Consumed 765ms CPU time over 22.209s wall clock time, 37.4M memory peak, 25.1K incoming IP traffic, 32.6K outgoing IP traffic.6406worker1 # [ 172.616458] systemd[1]: Starting Nix worker-protocol over gRPC...6407worker1 # [ 172.751732] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6408worker1 # [ 172.754144] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6409worker1 # [ 172.757910] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16410worker1 # [ 172.763247] nix-grpc-daemon[1899]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6411worker1 # [ 172.781333] nix-grpc-daemon[1899]: E0000 00:00:1790115785.401887 1899 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6412worker1 # [ 172.783713] nix-grpc-daemon[1899]: E0000 00:00:1790115785.405446 1899 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6413worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.74 seconds)6414lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6415lb: must succeed: curl -sf localhost:9901/clusters6416worker1 # [ 172.812511] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6417worker1 # [ 172.814724] nix-grpc-daemon[1899]: E0000 00:00:1790115785.413313 1940 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6418worker1 # [ 172.817526] systemd[1]: Started Nix worker-protocol over gRPC.6419worker1 # [ 172.818327] nix-grpc-daemon[1899]: E0000 00:00:1790115785.435336 1940 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6420worker1 # [ 172.820591] nix-grpc-daemon[1899]: E0000 00:00:1790115785.435483 1940 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6421worker1 # [ 172.822872] nix-grpc-daemon[1899]: E0000 00:00:1790115785.450940 1940 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6422worker1 # [ 172.830865] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:05Z level=info event=scheduler_connected addr=lb:500516423worker2 # [ 172.831915] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:506486424worker2 # [ 172.836956] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06425lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6426lb: must succeed: curl -sf localhost:9901/clusters6427lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6428lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.15 seconds)6429lb: must succeed: curl -sf localhost:9901/clusters6430lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6431worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6432worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6433(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 14.38 seconds)6434subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6435client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16436worker2 # [ 174.280684] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:368866437worker2 # [ 174.288463] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866438worker2 # [ 174.333857] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=want client=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06439worker2 # [ 174.339901] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16440worker2 # [ 174.346016] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=want client=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06441worker2 # [ 174.351209] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:06Z level=debug event=want client=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06442worker1 # [ 174.388960] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426443worker1 # [ 174.403997] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6444worker1 # [ 174.428373] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 0 ms (attempt 1/5)6445worker1 # [ 174.436874] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 116 ms (attempt 2/5)6446worker1 # [ 174.554727] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 309 ms (attempt 3/5)6447worker1 # [ 174.866121] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 134 ms (attempt 4/5)6448worker1 # [ 175.002760] nix-grpc-daemon[1899]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6449worker1 # [ 175.009820] nix-grpc-daemon[1899]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6450worker1 # [ 175.027695] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426451worker1 # [ 175.044109] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426452worker1 # [ 175.058558] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6453worker1 # [ 175.099195] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:07Z level=info event=niks3_push_started pid=19596454worker1 # [ 175.252788] nix-grpc-daemon[1959]: time=2026-09-22T22:23:07.891Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6455worker1 # [ 175.361178] nix-daemon[848]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6456worker1 # [ 175.382004] nix-daemon[848]: reaped child process 1975, status = succeeded6457lb # [ 175.373824] niks3-server[1100]: 2026/09/22 22:23:08 INFO Received uploads request method=POST path=/api/pending_closures6458worker1 # [ 175.392168] nix-grpc-daemon[1959]: time=2026-09-22T22:23:08.031Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6459worker1 # [ 175.396922] nix-grpc-daemon[1959]: time=2026-09-22T22:23:08.033Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6460lb # [ 175.413411] niks3-server[1100]: 2026/09/22 22:23:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6461worker1 # [ 175.439440] nix-grpc-daemon[1959]: time=2026-09-22T22:23:08.077Z level=INFO msg="Uploading 1 narinfos"6462lb # [ 175.430248] niks3-server[1100]: 2026/09/22 22:23:08 INFO Signed narinfos id=62 count=16463lb # [ 175.439990] niks3-server[1100]: 2026/09/22 22:23:08 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6464lb # [ 175.454024] niks3-server[1100]: 2026/09/22 22:23:08 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6465worker1 # [ 175.471292] nix-grpc-daemon[1959]: time=2026-09-22T22:23:08.110Z level=INFO msg="Upload complete. (215ms)"6466lb # [ 175.465441] niks3-server[1100]: 2026/09/22 22:23:08 INFO Completed upload id=626467lb # [ 175.471059] niks3-server[1100]: 2026/09/22 22:23:08 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6468worker1 # [ 175.480399] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=5126469lb # [ 175.477370] niks3-server[1100]: 2026/09/22 22:23:08 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6470worker1 # [ 175.488513] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426471worker1 # [ 175.499328] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6472worker1 # [ 175.505120] nix-daemon[848]: reaped child process 1957, status = succeeded6473worker1 # [ 175.516915] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6474worker1 # [ 176.370308] nix-daemon[848]: reaped child process 1979, status = succeeded6475worker1 # [ 176.508892] nix-daemon[848]: accepted connection from pid 1995, user nix-grpc-daemon (trusted)6476worker1 # [ 176.529725] nix-daemon[848]: reaped child process 1998, status = succeeded6477worker1 # [ 176.550453] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6478worker1 # [ 176.553512] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.187Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6479lb # [ 176.523908] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received uploads request method=POST path=/api/pending_closures6480worker1 # [ 176.588988] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.227Z level=INFO msg="Uploading 1 narinfos"6481lb # [ 176.570100] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6482lb # [ 176.585811] niks3-server[1100]: 2026/09/22 22:23:09 INFO Signed narinfos id=63 count=16483lb # [ 176.596318] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6484lb # [ 176.606002] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6485lb # [ 176.617117] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6486worker1 # [ 176.626771] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.266Z level=INFO msg="Upload complete. (300ms)"6487lb # [ 176.625809] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6488lb # [ 176.630482] niks3-server[1100]: 2026/09/22 22:23:09 INFO Completed upload id=636489worker2 # [ 176.635688] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:09Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16490lb # [ 176.634184] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6491worker1 # [ 176.641948] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=1 outputs=16492worker2 # [ 176.648735] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:09Z level=debug event=want client=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06493worker1 # [ 176.661587] nix-daemon[848]: reaped child process 1977, status = succeeded6494worker1 # [ 176.686604] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426495worker1 # [ 176.703068] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426496worker1 # [ 176.717890] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426497worker1 # [ 176.732767] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6498worker1 # [ 176.881165] nix-daemon[848]: accepted connection from pid 2002, user nix-grpc-daemon (trusted)6499worker1 # [ 176.901992] nix-daemon[848]: reaped child process 2005, status = succeeded6500lb # [ 176.889920] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received uploads request method=POST path=/api/pending_closures6501worker1 # [ 176.909174] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.548Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6502worker1 # [ 176.910685] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.548Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6503lb # [ 176.922282] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6504lb # [ 176.936456] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6505worker1 # [ 176.951616] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.590Z level=INFO msg="Uploading 1 narinfos"6506lb # [ 176.947379] niks3-server[1100]: 2026/09/22 22:23:09 INFO Signed narinfos id=64 count=16507lb # [ 176.953007] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6508lb # [ 176.965221] niks3-server[1100]: 2026/09/22 22:23:09 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6509lb # [ 176.972151] niks3-server[1100]: 2026/09/22 22:23:09 INFO Completed upload id=646510worker1 # [ 176.981349] nix-grpc-daemon[1959]: time=2026-09-22T22:23:09.620Z level=INFO msg="Upload complete. (234ms)"6511lb # [ 176.974902] niks3-server[1100]: 2026/09/22 22:23:09 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6512worker1 # [ 176.992348] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=5126513worker1 # [ 177.000654] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426514worker1 # [ 177.013772] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6515worker1 # [ 177.018984] nix-daemon[848]: reaped child process 2000, status = succeeded6516worker1 # [ 177.029521] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6517worker1 # [ 178.116553] nix-daemon[848]: reaped child process 2009, status = succeeded6518worker1 # [ 178.210780] nix-daemon[848]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)6519worker1 # [ 178.232115] nix-daemon[848]: reaped child process 2028, status = succeeded6520lb # [ 178.221642] niks3-server[1100]: 2026/09/22 22:23:10 INFO Received uploads request method=POST path=/api/pending_closures6521worker1 # [ 178.241953] nix-grpc-daemon[1959]: time=2026-09-22T22:23:10.880Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6522worker1 # [ 178.243382] nix-grpc-daemon[1959]: time=2026-09-22T22:23:10.880Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6523lb # [ 178.250990] niks3-server[1100]: 2026/09/22 22:23:10 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6524lb # [ 178.275530] niks3-server[1100]: 2026/09/22 22:23:10 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6525worker1 # [ 178.296260] nix-grpc-daemon[1959]: time=2026-09-22T22:23:10.935Z level=INFO msg="Uploading 1 narinfos"6526lb # [ 178.291879] niks3-server[1100]: 2026/09/22 22:23:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6527lb # [ 178.302129] niks3-server[1100]: 2026/09/22 22:23:10 INFO Signed narinfos id=65 count=16528lb # [ 178.309090] niks3-server[1100]: 2026/09/22 22:23:10 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6529worker1 # [ 178.329284] nix-grpc-daemon[1959]: time=2026-09-22T22:23:10.968Z level=INFO msg="Upload complete. (287ms)"6530lb # [ 178.319883] niks3-server[1100]: 2026/09/22 22:23:10 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6531lb # [ 178.329168] niks3-server[1100]: 2026/09/22 22:23:10 INFO Completed upload id=656532worker2 # [ 178.334732] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:10Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16533lb # [ 178.332772] niks3-server[1100]: 2026/09/22 22:23:10 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6534worker1 # [ 178.341181] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=2 outputs=16535worker2 # [ 178.348560] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:10Z level=debug event=want client=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06536worker1 # [ 178.364462] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426537worker1 # [ 178.370715] nix-daemon[848]: reaped child process 2007, status = succeeded6538worker1 # [ 178.379157] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426539worker1 # [ 178.393048] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426540worker1 # [ 178.405844] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6541worker1 # [ 178.552557] nix-daemon[848]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)6542worker1 # [ 178.573469] nix-daemon[848]: reaped child process 2035, status = succeeded6543lb # [ 178.564700] niks3-server[1100]: 2026/09/22 22:23:11 INFO Received uploads request method=POST path=/api/pending_closures6544worker1 # [ 178.584178] nix-grpc-daemon[1959]: time=2026-09-22T22:23:11.223Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6545worker1 # [ 178.586238] nix-grpc-daemon[1959]: time=2026-09-22T22:23:11.223Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6546lb # [ 178.601046] niks3-server[1100]: 2026/09/22 22:23:11 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6547worker1 # [ 178.620195] nix-grpc-daemon[1959]: time=2026-09-22T22:23:11.259Z level=INFO msg="Uploading 1 narinfos"6548lb # [ 178.613467] niks3-server[1100]: 2026/09/22 22:23:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6549lb # [ 178.624698] niks3-server[1100]: 2026/09/22 22:23:11 INFO Signed narinfos id=66 count=16550lb # [ 178.627254] niks3-server[1100]: 2026/09/22 22:23:11 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6551lb # [ 178.639894] niks3-server[1100]: 2026/09/22 22:23:11 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6552lb # [ 178.646902] niks3-server[1100]: 2026/09/22 22:23:11 INFO Completed upload id=666553worker1 # [ 178.657000] nix-grpc-daemon[1959]: time=2026-09-22T22:23:11.296Z level=INFO msg="Upload complete. (237ms)"6554lb # [ 178.649968] niks3-server[1100]: 2026/09/22 22:23:11 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6555worker1 # [ 178.667763] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=5126556worker1 # [ 178.683718] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426557worker1 # [ 178.690573] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6558worker1 # [ 178.695650] nix-daemon[848]: reaped child process 2030, status = succeeded6559worker1 # [ 178.706497] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6560worker1 # [ 179.605191] nix-daemon[848]: reaped child process 2039, status = succeeded6561worker1 # [ 179.674762] nix-daemon[848]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)6562worker1 # [ 179.697661] nix-daemon[848]: reaped child process 2058, status = succeeded6563lb # [ 179.687113] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received uploads request method=POST path=/api/pending_closures6564worker1 # [ 179.706718] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.345Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6565worker1 # [ 179.710297] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.348Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6566lb # [ 179.730500] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6567worker1 # [ 179.748887] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.388Z level=INFO msg="Uploading 1 narinfos"6568lb # [ 179.745623] niks3-server[1100]: 2026/09/22 22:23:12 INFO Signed narinfos id=67 count=16569lb # [ 179.753213] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6570lb # [ 179.761058] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6571lb # [ 179.771448] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6572worker1 # [ 179.785380] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.424Z level=INFO msg="Upload complete. (267ms)"6573lb # [ 179.776706] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6574lb # [ 179.781557] niks3-server[1100]: 2026/09/22 22:23:12 INFO Completed upload id=676575lb # [ 179.784790] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6576worker2 # [ 179.793289] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:12Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16577worker1 # [ 179.797167] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=3 outputs=16578worker2 # [ 179.806804] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:12Z level=debug event=want client=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06579worker1 # [ 179.819419] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426580worker1 # [ 179.825700] nix-daemon[848]: reaped child process 2037, status = succeeded6581worker1 # [ 179.833653] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426582worker1 # [ 179.846477] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.485Z level=INFO msg="All 1 paths already cached"6583worker1 # [ 179.855725] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426584worker1 # [ 179.868878] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6585lb # [ 179.918837] envoy[895]: conn peer=[2001:db8:1::3]:55530 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=911 tx=361 ms=211746586worker1 # [ 180.018239] nix-daemon[848]: accepted connection from pid 2062, user nix-grpc-daemon (trusted)6587worker1 # [ 180.036782] nix-daemon[848]: reaped child process 2065, status = succeeded6588lb # [ 180.031748] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received uploads request method=POST path=/api/pending_closures6589worker1 # [ 180.050905] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.689Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6590worker1 # [ 180.052919] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.689Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6591lb # [ 180.062699] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6592worker1 # [ 180.078640] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.717Z level=INFO msg="Uploading 1 narinfos"6593lb # [ 180.077886] niks3-server[1100]: 2026/09/22 22:23:12 INFO Signed narinfos id=68 count=16594lb # [ 180.085428] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6595lb # [ 180.091875] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6596lb # [ 180.100339] niks3-server[1100]: 2026/09/22 22:23:12 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6597lb # [ 180.109432] niks3-server[1100]: 2026/09/22 22:23:12 INFO Completed upload id=686598lb # [ 180.111847] niks3-server[1100]: 2026/09/22 22:23:12 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6599worker1 # [ 180.116663] nix-grpc-daemon[1959]: time=2026-09-22T22:23:12.755Z level=INFO msg="Upload complete. (234ms)"6600worker1 # [ 180.128511] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=6246601worker1 # [ 180.144920] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426602worker1 # [ 180.151571] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6603worker1 # [ 180.156980] nix-daemon[848]: reaped child process 2060, status = succeeded6604worker1 # [ 180.169538] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6605worker1 # [ 181.608856] nix-daemon[848]: reaped child process 2069, status = succeeded6606worker1 # [ 181.716432] nix-daemon[848]: accepted connection from pid 2085, user nix-grpc-daemon (trusted)6607worker1 # [ 181.738317] nix-daemon[848]: reaped child process 2088, status = succeeded6608lb # [ 181.727128] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received uploads request method=POST path=/api/pending_closures6609worker1 # [ 181.748431] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.386Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6610worker1 # [ 181.749952] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.386Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6611lb # [ 181.767305] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6612worker1 # [ 181.792145] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.431Z level=INFO msg="Uploading 1 narinfos"6613lb # [ 181.781870] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6614lb # [ 181.794446] niks3-server[1100]: 2026/09/22 22:23:14 INFO Signed narinfos id=69 count=16615lb # [ 181.801777] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6616lb # [ 181.810525] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6617lb # [ 181.822784] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6618lb # [ 181.827894] niks3-server[1100]: 2026/09/22 22:23:14 INFO Completed upload id=696619worker1 # [ 181.838070] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.477Z level=INFO msg="Upload complete. (258ms)"6620lb # [ 181.833206] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6621worker2 # [ 181.847379] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:14Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16622worker1 # [ 181.849141] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=4 outputs=16623worker2 # [ 181.861913] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:14Z level=debug event=want client=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06624worker1 # [ 181.871306] nix-daemon[848]: reaped child process 2067, status = succeeded6625worker1 # [ 181.906486] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426626worker1 # [ 181.922242] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426627worker1 # [ 181.938024] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.577Z level=INFO msg="All 1 paths already cached"6628worker1 # [ 181.947950] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426629worker1 # [ 181.962735] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6630worker1 # [ 182.118654] nix-daemon[848]: accepted connection from pid 2092, user nix-grpc-daemon (trusted)6631worker1 # [ 182.140057] nix-daemon[848]: reaped child process 2095, status = succeeded6632lb # [ 182.131163] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received uploads request method=POST path=/api/pending_closures6633worker1 # [ 182.150397] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.789Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6634worker1 # [ 182.151918] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.789Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6635lb # [ 182.161080] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6636worker1 # [ 182.176618] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.815Z level=INFO msg="Uploading 1 narinfos"6637lb # [ 182.174834] niks3-server[1100]: 2026/09/22 22:23:14 INFO Signed narinfos id=70 count=16638lb # [ 182.183155] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6639lb # [ 182.196340] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6640worker1 # [ 182.213573] nix-grpc-daemon[1959]: time=2026-09-22T22:23:14.852Z level=INFO msg="Upload complete. (233ms)"6641lb # [ 182.204960] niks3-server[1100]: 2026/09/22 22:23:14 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6642lb # [ 182.214300] niks3-server[1100]: 2026/09/22 22:23:14 INFO Completed upload id=706643worker1 # [ 182.223464] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=6246644lb # [ 182.218421] niks3-server[1100]: 2026/09/22 22:23:14 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6645worker1 # [ 182.240515] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426646worker1 # [ 182.246392] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6647worker1 # [ 182.251165] nix-daemon[848]: reaped child process 2090, status = succeeded6648worker1 # [ 182.261910] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6649worker1 # [ 183.126664] nix-daemon[848]: reaped child process 2099, status = succeeded6650worker1 # [ 183.240367] nix-daemon[848]: accepted connection from pid 2115, user nix-grpc-daemon (trusted)6651worker1 # [ 183.262703] nix-daemon[848]: reaped child process 2118, status = succeeded6652lb # [ 183.255656] niks3-server[1100]: 2026/09/22 22:23:15 INFO Received uploads request method=POST path=/api/pending_closures6653worker1 # [ 183.276498] nix-grpc-daemon[1959]: time=2026-09-22T22:23:15.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6654worker1 # [ 183.278275] nix-grpc-daemon[1959]: time=2026-09-22T22:23:15.914Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6655lb # [ 183.293353] niks3-server[1100]: 2026/09/22 22:23:15 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6656lb # [ 183.306708] niks3-server[1100]: 2026/09/22 22:23:15 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6657worker1 # [ 183.326521] nix-grpc-daemon[1959]: time=2026-09-22T22:23:15.965Z level=INFO msg="Uploading 1 narinfos"6658lb # [ 183.318599] niks3-server[1100]: 2026/09/22 22:23:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6659lb # [ 183.330134] niks3-server[1100]: 2026/09/22 22:23:15 INFO Signed narinfos id=71 count=16660lb # [ 183.335802] niks3-server[1100]: 2026/09/22 22:23:15 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6661lb # [ 183.345642] niks3-server[1100]: 2026/09/22 22:23:15 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6662worker1 # [ 183.361567] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.000Z level=INFO msg="Upload complete. (288ms)"6663lb # [ 183.353513] niks3-server[1100]: 2026/09/22 22:23:15 INFO Completed upload id=716664lb # [ 183.356477] niks3-server[1100]: 2026/09/22 22:23:15 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6665worker2 # [ 183.369495] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:16Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16666worker1 # [ 183.377200] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=5 outputs=16667worker2 # [ 183.383039] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:16Z level=debug event=want client=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06668worker1 # [ 183.392696] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426669worker1 # [ 183.406212] nix-daemon[848]: reaped child process 2097, status = succeeded6670worker1 # [ 183.411364] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426671worker1 # [ 183.422073] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.061Z level=INFO msg="All 1 paths already cached"6672worker1 # [ 183.431340] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426673worker1 # [ 183.446171] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6674worker1 # [ 183.585404] nix-daemon[848]: accepted connection from pid 2122, user nix-grpc-daemon (trusted)6675worker1 # [ 183.607252] nix-daemon[848]: reaped child process 2125, status = succeeded6676lb # [ 183.602507] niks3-server[1100]: 2026/09/22 22:23:16 INFO Received uploads request method=POST path=/api/pending_closures6677worker1 # [ 183.621115] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.260Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6678worker1 # [ 183.622629] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.260Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6679worker1 # [ 183.641263] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.280Z level=INFO msg="Uploading 1 narinfos"6680lb # [ 183.631436] niks3-server[1100]: 2026/09/22 22:23:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6681lb # [ 183.644157] niks3-server[1100]: 2026/09/22 22:23:16 INFO Signed narinfos id=72 count=16682lb # [ 183.652590] niks3-server[1100]: 2026/09/22 22:23:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6683worker1 # [ 183.679357] nix-grpc-daemon[1959]: time=2026-09-22T22:23:16.318Z level=INFO msg="Upload complete. (221ms)"6684lb # [ 183.671206] niks3-server[1100]: 2026/09/22 22:23:16 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6685worker1 # [ 183.687887] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=6246686lb # [ 183.682615] niks3-server[1100]: 2026/09/22 22:23:16 INFO Completed upload id=726687lb # [ 183.687366] niks3-server[1100]: 2026/09/22 22:23:16 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6688worker1 # [ 183.703198] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426689lb # [ 183.693167] niks3-server[1100]: 2026/09/22 22:23:16 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6690worker1 # [ 183.709496] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6691worker1 # [ 183.714717] nix-daemon[848]: reaped child process 2120, status = succeeded6692worker1 # [ 183.725081] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6693worker1 # [ 184.596263] nix-daemon[848]: reaped child process 2129, status = succeeded6694worker1 # [ 184.685475] nix-daemon[848]: accepted connection from pid 2145, user nix-grpc-daemon (trusted)6695worker1 # [ 184.706503] nix-daemon[848]: reaped child process 2148, status = succeeded6696lb # [ 184.702198] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received uploads request method=POST path=/api/pending_closures6697worker1 # [ 184.723162] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.361Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6698worker1 # [ 184.725301] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.361Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6699lb # [ 184.742112] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6700worker1 # [ 184.759047] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.398Z level=INFO msg="Uploading 1 narinfos"6701lb # [ 184.756215] niks3-server[1100]: 2026/09/22 22:23:17 INFO Signed narinfos id=73 count=16702lb # [ 184.758896] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6703lb # [ 184.775120] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6704lb # [ 184.784127] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6705lb # [ 184.787923] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6706lb # [ 184.792308] niks3-server[1100]: 2026/09/22 22:23:17 INFO Completed upload id=736707worker1 # [ 184.792290] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.431Z level=INFO msg="Upload complete. (256ms)"6708lb # [ 184.795609] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6709worker2 # [ 184.800981] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:17Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16710worker1 # [ 184.806150] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=6 outputs=16711worker2 # [ 184.814214] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:17Z level=debug event=want client=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06712worker1 # [ 184.824450] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426713worker1 # [ 184.831196] nix-daemon[848]: reaped child process 2127, status = succeeded6714worker1 # [ 184.839567] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426715worker1 # [ 184.852256] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.491Z level=INFO msg="All 2 paths already cached"6716worker1 # [ 184.861805] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426717worker1 # [ 184.873600] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6718worker1 # [ 185.011655] nix-daemon[848]: accepted connection from pid 2152, user nix-grpc-daemon (trusted)6719worker1 # [ 185.031912] nix-daemon[848]: reaped child process 2155, status = succeeded6720lb # [ 185.025755] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received uploads request method=POST path=/api/pending_closures6721worker1 # [ 185.044174] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.683Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6722worker1 # [ 185.046190] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.685Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6723lb # [ 185.058552] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6724worker1 # [ 185.080389] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.719Z level=INFO msg="Uploading 1 narinfos"6725lb # [ 185.074040] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6726lb # [ 185.084620] niks3-server[1100]: 2026/09/22 22:23:17 INFO Signed narinfos id=74 count=16727lb # [ 185.087137] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6728lb # [ 185.101730] niks3-server[1100]: 2026/09/22 22:23:17 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6729worker1 # [ 185.116171] nix-grpc-daemon[1959]: time=2026-09-22T22:23:17.755Z level=INFO msg="Upload complete. (229ms)"6730lb # [ 185.106198] niks3-server[1100]: 2026/09/22 22:23:17 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6731lb # [ 185.110229] niks3-server[1100]: 2026/09/22 22:23:17 INFO Completed upload id=746732worker1 # [ 185.126490] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=7526733worker1 # [ 185.142854] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426734worker1 # [ 185.150023] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6735worker1 # [ 185.155307] nix-daemon[848]: reaped child process 2150, status = succeeded6736worker1 # [ 185.168889] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6737worker1 # [ 185.833572] nix-daemon[848]: reaped child process 2159, status = succeeded6738worker1 # [ 185.958903] nix-daemon[848]: accepted connection from pid 2175, user nix-grpc-daemon (trusted)6739worker1 # [ 185.977293] nix-daemon[848]: reaped child process 2178, status = succeeded6740lb # [ 185.973457] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received uploads request method=POST path=/api/pending_closures6741worker1 # [ 185.996487] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.632Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6742worker1 # [ 185.998324] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.632Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6743lb # [ 186.016477] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6744worker1 # [ 186.035290] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.674Z level=INFO msg="Uploading 1 narinfos"6745lb # [ 186.031983] niks3-server[1100]: 2026/09/22 22:23:18 INFO Signed narinfos id=75 count=16746lb # [ 186.039262] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6747lb # [ 186.049113] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6748worker1 # [ 186.069372] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.708Z level=INFO msg="Upload complete. (286ms)"6749lb # [ 186.060153] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6750lb # [ 186.063756] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6751lb # [ 186.068736] niks3-server[1100]: 2026/09/22 22:23:18 INFO Completed upload id=756752lb # [ 186.071802] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6753worker2 # [ 186.076815] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:18Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16754worker1 # [ 186.081194] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=7 outputs=16755worker1 # [ 186.103531] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426756worker1 # [ 186.110084] nix-daemon[848]: reaped child process 2157, status = succeeded6757worker1 # [ 186.117793] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426758worker1 # [ 186.129605] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.768Z level=INFO msg="All 2 paths already cached"6759worker1 # [ 186.139389] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426760worker1 # [ 186.153011] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6761worker1 # [ 186.299900] nix-daemon[848]: accepted connection from pid 2182, user nix-grpc-daemon (trusted)6762worker1 # [ 186.318368] nix-daemon[848]: reaped child process 2185, status = succeeded6763lb # [ 186.313005] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received uploads request method=POST path=/api/pending_closures6764worker1 # [ 186.332103] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.971Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6765worker1 # [ 186.333567] nix-grpc-daemon[1959]: time=2026-09-22T22:23:18.971Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"6766lb # [ 186.345864] niks3-server[1100]: 2026/09/22 22:23:18 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst6767lb # [ 186.362447] niks3-server[1100]: 2026/09/22 22:23:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6768worker1 # [ 186.378070] nix-grpc-daemon[1959]: time=2026-09-22T22:23:19.017Z level=INFO msg="Uploading 1 narinfos"6769lb # [ 186.374560] niks3-server[1100]: 2026/09/22 22:23:19 INFO Signed narinfos id=76 count=16770lb # [ 186.381548] niks3-server[1100]: 2026/09/22 22:23:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls6771lb # [ 186.392945] niks3-server[1100]: 2026/09/22 22:23:19 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6772lb # [ 186.399071] niks3-server[1100]: 2026/09/22 22:23:19 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo6773lb # [ 186.404831] niks3-server[1100]: 2026/09/22 22:23:19 INFO Completed upload id=766774worker1 # [ 186.414819] nix-grpc-daemon[1959]: time=2026-09-22T22:23:19.054Z level=INFO msg="Upload complete. (248ms)"6775worker1 # [ 186.425707] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=7526776worker1 # [ 186.442952] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426777worker1 # [ 186.453785] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6778worker1 # [ 186.460408] nix-daemon[848]: reaped child process 2180, status = succeeded6779worker1 # [ 186.469565] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6780worker1 # [ 187.378031] nix-daemon[848]: reaped child process 2189, status = succeeded6781worker1 # [ 187.447009] nix-daemon[848]: accepted connection from pid 2205, user nix-grpc-daemon (trusted)6782worker1 # [ 187.469780] nix-daemon[848]: reaped child process 2208, status = succeeded6783lb # [ 187.459248] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received uploads request method=POST path=/api/pending_closures6784worker1 # [ 187.479739] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.118Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6785worker1 # [ 187.482684] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.120Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"6786lb # [ 187.500947] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6787worker1 # [ 187.519936] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.159Z level=INFO msg="Uploading 1 narinfos"6788lb # [ 187.514281] niks3-server[1100]: 2026/09/22 22:23:20 INFO Signed narinfos id=77 count=16789lb # [ 187.525144] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls6790lb # [ 187.536979] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv6791lb # [ 187.543351] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst6792worker1 # [ 187.558658] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.198Z level=INFO msg="Upload complete. (280ms)"6793lb # [ 187.559830] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6794worker2 # [ 187.565441] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:20Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=16795lb # [ 187.566018] niks3-server[1100]: 2026/09/22 22:23:20 INFO Completed upload id=776796worker1 # [ 187.573221] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=8 outputs=16797lb # [ 187.569468] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo6798worker1 # [ 187.592325] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426799worker1 # [ 187.599976] nix-daemon[848]: reaped child process 2187, status = succeeded6800worker1 # [ 187.629846] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426801worker1 # [ 187.644893] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.284Z level=INFO msg="All 2 paths already cached"6802worker1 # [ 187.654983] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:536426803worker1 # [ 187.672344] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6804worker1 # [ 187.823465] nix-daemon[848]: accepted connection from pid 2212, user nix-grpc-daemon (trusted)6805worker1 # [ 187.843771] nix-daemon[848]: reaped child process 2215, status = succeeded6806lb # [ 187.834941] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received uploads request method=POST path=/api/pending_closures6807worker1 # [ 187.854528] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.493Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6808worker1 # [ 187.856225] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.493Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"6809lb # [ 187.865366] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6810worker1 # [ 187.883549] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.522Z level=INFO msg="Uploading 1 narinfos"6811lb # [ 187.879993] niks3-server[1100]: 2026/09/22 22:23:20 INFO Signed narinfos id=78 count=16812lb # [ 187.882480] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls6813lb # [ 187.895886] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst6814worker1 # [ 187.911001] nix-grpc-daemon[1959]: time=2026-09-22T22:23:20.550Z level=INFO msg="Upload complete. (226ms)"6815lb # [ 187.901645] niks3-server[1100]: 2026/09/22 22:23:20 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6816lb # [ 187.906794] niks3-server[1100]: 2026/09/22 22:23:20 INFO Completed upload id=786817lb # [ 187.911826] niks3-server[1100]: 2026/09/22 22:23:20 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo6818worker1 # [ 187.920646] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=0 paths=1 nar_bytes_in=7526819worker1 # [ 187.928866] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:536426820worker1 # [ 187.940098] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6821worker1 # [ 187.945445] nix-daemon[848]: reaped child process 2210, status = succeeded6822worker1 # [ 187.958224] nix-daemon[848]: accepted connection from pid 1899, user nix-grpc-daemon (trusted)6823worker1 # [ 189.261805] nix-daemon[848]: reaped child process 2219, status = succeeded6824worker1 # [ 189.331184] nix-daemon[848]: accepted connection from pid 2235, user nix-grpc-daemon (trusted)6825worker1 # [ 189.351860] nix-daemon[848]: reaped child process 2238, status = succeeded6826lb # [ 189.340716] niks3-server[1100]: 2026/09/22 22:23:21 INFO Received uploads request method=POST path=/api/pending_closures6827worker1 # [ 189.364407] nix-grpc-daemon[1959]: time=2026-09-22T22:23:22.000Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6828worker1 # [ 189.365975] nix-grpc-daemon[1959]: time=2026-09-22T22:23:22.000Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"6829worker1 # [ 189.408483] nix-grpc-daemon[1959]: time=2026-09-22T22:23:22.047Z level=INFO msg="Uploading 1 narinfos"6830lb # [ 189.390605] niks3-server[1100]: 2026/09/22 22:23:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6831lb # [ 189.405085] niks3-server[1100]: 2026/09/22 22:23:22 INFO Signed narinfos id=79 count=16832lb # [ 189.407836] niks3-server[1100]: 2026/09/22 22:23:22 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst6833worker1 # [ 189.440438] nix-grpc-daemon[1959]: time=2026-09-22T22:23:22.079Z level=INFO msg="Upload complete. (271ms)"6834lb # [ 189.425720] niks3-server[1100]: 2026/09/22 22:23:22 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv6835lb # [ 189.438075] niks3-server[1100]: 2026/09/22 22:23:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls6836lb # [ 189.443151] niks3-server[1100]: 2026/09/22 22:23:22 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6837lb # [ 189.448107] niks3-server[1100]: 2026/09/22 22:23:22 INFO Completed upload id=796838worker1 # [ 189.453166] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:53642 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=9 outputs=16839lb # [ 189.451788] niks3-server[1100]: 2026/09/22 22:23:22 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo6840worker1 # [ 189.472292] nix-daemon[848]: reaped child process 2217, status = succeeded6841client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 15.46 seconds)6842worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6843worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6844client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'6845client # [ 189.653711] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".6846client # Running as unit: gpu.service; invocation ID: 912e5a58db124e7e8f771605eb9140c06847client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)6848lb: must succeed: curl -sf localhost:9901/clusters6849lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6850lb: must succeed: curl -sf localhost:9901/clusters6851lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6852worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6853worker1 # [ 189.851790] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:22Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:536426854client # [ 189.855549] nix[1572]: don't know how to build these paths:6855client # [ 189.856723] nix[1572]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv6856worker1 # [ 189.864754] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426857worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6858client # [ 189.887475] nix[1572]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker6859worker2 # [ 189.889108] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:22Z level=debug event=want client=2 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=06860lb # [ 189.919761] envoy[895]: conn peer=[2001:db8:1::1]:50512 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27170 tx=12400 ms=151966861lb: must succeed: curl -sf localhost:9901/clusters6862lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6863lb: must succeed: curl -sf localhost:9901/clusters6864lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6865worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6866worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6867client: waiting for success: systemctl show -p Result gpu | grep -q exit-code6868client # [ 219.893771] nix[1572]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}6869client # [ 219.906816] nix[1572]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed6870lb # [ 219.919643] envoy[895]: conn peer=[2001:db8:1::1]:58762 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300706871client # [ 219.945500] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE6872client # [ 219.953363] systemd[1]: gpu.service: Failed with result 'exit-code'.6873client # [ 219.956863] systemd[1]: gpu.service: Consumed 191ms CPU time over 30.297s wall clock time, 35.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.6874client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.10 seconds)6875client: must succeed: journalctl -u gpu -o cat6876client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)6877lb: must succeed: curl -sf localhost:9901/clusters6878lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6879lb: must succeed: curl -sf localhost:9901/clusters6880lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6881worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6882worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6883(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 46.30 seconds)6884subtest: low disk drains a worker and builds go to the other6885worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill6886worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.21 seconds)6887worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'6888worker1 # [ 221.798881] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:54Z level=info event=unhealthy reason=min_free6889worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.10 seconds)6890client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&26891client # don't know how to build these paths:6892client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv6893worker2 # [ 222.906145] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:368866894worker1 # [ 222.914881] nix-grpc-daemon[1899]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:536426895worker2 # [ 222.937576] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=want client=3 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=06896worker2 # [ 222.944779] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=16897worker2 # [ 222.983700] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866898worker2 # [ 223.001086] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866899client # copying 1 paths to 'lb:50051'...6900client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...6901worker2 # [ 223.018067] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368866902worker2 # [ 223.033252] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6903worker2 # [ 223.197111] nix-daemon[837]: accepted connection from pid 1704, user nix-grpc-daemon (trusted)6904worker2 # [ 223.213387] nix-daemon[837]: reaped child process 1707, status = succeeded6905lb # [ 223.216588] niks3-server[1100]: 2026/09/22 22:23:55 INFO Received uploads request method=POST path=/api/pending_closures6906worker2 # [ 223.236377] nix-grpc-daemon[1512]: time=2026-09-22T22:23:55.878Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6907worker2 # [ 223.238164] nix-grpc-daemon[1512]: time=2026-09-22T22:23:55.878Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"6908lb # [ 223.251433] niks3-server[1100]: 2026/09/22 22:23:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6909worker2 # [ 223.267086] nix-grpc-daemon[1512]: time=2026-09-22T22:23:55.909Z level=INFO msg="Uploading 1 narinfos"6910lb # [ 223.267335] niks3-server[1100]: 2026/09/22 22:23:55 INFO Signed narinfos id=80 count=16911lb # [ 223.276912] niks3-server[1100]: 2026/09/22 22:23:55 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls6912lb # [ 223.288643] niks3-server[1100]: 2026/09/22 22:23:55 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst6913worker2 # [ 223.304813] nix-grpc-daemon[1512]: time=2026-09-22T22:23:55.947Z level=INFO msg="Upload complete. (254ms)"6914lb # [ 223.299650] niks3-server[1100]: 2026/09/22 22:23:55 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6915lb # [ 223.309155] niks3-server[1100]: 2026/09/22 22:23:55 INFO Completed upload id=806916lb # [ 223.311356] niks3-server[1100]: 2026/09/22 22:23:55 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo6917worker2 # [ 223.315865] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=5126918worker2 # [ 223.328540] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866919worker2 # [ 223.340391] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6920worker2 # [ 223.345814] nix-daemon[837]: reaped child process 1702, status = succeeded6921client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...6922worker2 # [ 223.358872] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6923client # farm-a> worker2: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6924client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6925client # farm-a> LOG-farm-a-drain6926worker2 # [ 224.421373] nix-daemon[837]: reaped child process 1711, status = succeeded6927worker2 # [ 224.547992] nix-daemon[837]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6928worker2 # [ 224.569196] nix-daemon[837]: reaped child process 1730, status = succeeded6929lb # [ 224.566861] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received uploads request method=POST path=/api/pending_closures6930worker2 # [ 224.587830] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.227Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6931worker2 # [ 224.589855] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.227Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"6932lb # [ 224.612073] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6933worker2 # [ 224.626570] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.269Z level=INFO msg="Uploading 1 narinfos"6934lb # [ 224.625981] niks3-server[1100]: 2026/09/22 22:23:57 INFO Signed narinfos id=81 count=16935lb # [ 224.633382] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls6936lb # [ 224.641038] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv6937lb # [ 224.650366] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst6938worker2 # [ 224.661478] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.304Z level=INFO msg="Upload complete. (293ms)"6939lb # [ 224.656283] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete6940lb # [ 224.660573] niks3-server[1100]: 2026/09/22 22:23:57 INFO Completed upload id=816941lb # [ 224.663649] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo6942worker2 # [ 224.677639] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=10 outputs=16943worker2 # [ 224.700631] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=want client=3 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=06944worker2 # [ 224.706980] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=16945worker2 # [ 224.712403] nix-daemon[837]: reaped child process 1709, status = succeeded6946worker2 # [ 224.734617] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866947worker2 # [ 224.750288] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866948worker2 # [ 224.765108] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.407Z level=INFO msg="All 1 paths already cached"6949client # copying 1 paths to 'lb:50051'...6950client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...6951worker2 # [ 224.776416] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368866952worker2 # [ 224.790449] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6953worker2 # [ 224.945171] nix-daemon[837]: accepted connection from pid 1736, user nix-grpc-daemon (trusted)6954worker2 # [ 224.964309] nix-daemon[837]: reaped child process 1739, status = succeeded6955lb # [ 224.962655] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received uploads request method=POST path=/api/pending_closures6956worker2 # [ 224.978207] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.620Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6957worker2 # [ 224.980236] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.620Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"6958lb # [ 224.993861] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign6959worker2 # [ 225.009062] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.651Z level=INFO msg="Uploading 1 narinfos"6960lb # [ 225.009294] niks3-server[1100]: 2026/09/22 22:23:57 INFO Signed narinfos id=82 count=16961lb # [ 225.015934] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst6962worker2 # [ 225.032988] nix-grpc-daemon[1512]: time=2026-09-22T22:23:57.675Z level=INFO msg="Upload complete. (226ms)"6963lb # [ 225.029109] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls6964worker2 # [ 225.040136] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=6326965lb # [ 225.034032] niks3-server[1100]: 2026/09/22 22:23:57 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete6966lb # [ 225.039522] niks3-server[1100]: 2026/09/22 22:23:57 INFO Completed upload id=826967lb # [ 225.043981] niks3-server[1100]: 2026/09/22 22:23:57 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo6968worker2 # [ 225.053340] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866969worker2 # [ 225.059687] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6970worker2 # [ 225.064665] nix-daemon[837]: reaped child process 1734, status = succeeded6971client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...6972worker2 # [ 225.077363] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6973client # farm-b> worker2: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6974client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6975client # farm-b> LOG-farm-b-drain6976worker2 # [ 225.923831] nix-daemon[837]: reaped child process 1743, status = succeeded6977worker2 # [ 226.054992] nix-daemon[837]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6978worker2 # [ 226.073269] nix-daemon[837]: reaped child process 1762, status = succeeded6979lb # [ 226.073261] niks3-server[1100]: 2026/09/22 22:23:58 INFO Received uploads request method=POST path=/api/pending_closures6980worker2 # [ 226.092410] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.731Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6981worker2 # [ 226.093961] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.731Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"6982lb # [ 226.118470] niks3-server[1100]: 2026/09/22 22:23:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign6983worker2 # [ 226.137597] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.780Z level=INFO msg="Uploading 1 narinfos"6984lb # [ 226.133089] niks3-server[1100]: 2026/09/22 22:23:58 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv6985lb # [ 226.142080] niks3-server[1100]: 2026/09/22 22:23:58 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls6986lb # [ 226.154116] niks3-server[1100]: 2026/09/22 22:23:58 INFO Signed narinfos id=83 count=16987worker2 # [ 226.165264] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.807Z level=INFO msg="Upload complete. (278ms)"6988lb # [ 226.159598] niks3-server[1100]: 2026/09/22 22:23:58 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst6989lb # [ 226.164743] niks3-server[1100]: 2026/09/22 22:23:58 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete6990lb # [ 226.172362] niks3-server[1100]: 2026/09/22 22:23:58 INFO Completed upload id=836991lb # [ 226.175330] niks3-server[1100]: 2026/09/22 22:23:58 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo6992worker2 # [ 226.180561] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=11 outputs=16993worker2 # [ 226.201092] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=want client=3 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=06994worker2 # [ 226.205479] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=16995worker2 # [ 226.210005] nix-daemon[837]: reaped child process 1741, status = succeeded6996worker2 # [ 226.242572] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368866997worker2 # [ 226.257662] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:368866998worker2 # [ 226.271921] nix-grpc-daemon[1512]: time=2026-09-22T22:23:58.914Z level=INFO msg="All 2 paths already cached"6999client # copying 1 paths to 'lb:50051'...7000client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7001worker2 # [ 226.283494] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:368867002worker2 # [ 226.297122] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)7003worker2 # [ 226.454254] nix-daemon[837]: accepted connection from pid 1768, user nix-grpc-daemon (trusted)7004worker2 # [ 226.474519] nix-daemon[837]: reaped child process 1771, status = succeeded7005lb # [ 226.466409] niks3-server[1100]: 2026/09/22 22:23:59 INFO Received uploads request method=POST path=/api/pending_closures7006worker2 # [ 226.481778] nix-grpc-daemon[1512]: time=2026-09-22T22:23:59.124Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7007worker2 # [ 226.483597] nix-grpc-daemon[1512]: time=2026-09-22T22:23:59.124Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7008lb # [ 226.493877] niks3-server[1100]: 2026/09/22 22:23:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7009worker2 # [ 226.508640] nix-grpc-daemon[1512]: time=2026-09-22T22:23:59.151Z level=INFO msg="Uploading 1 narinfos"7010lb # [ 226.508736] niks3-server[1100]: 2026/09/22 22:23:59 INFO Signed narinfos id=84 count=17011lb # [ 226.517816] niks3-server[1100]: 2026/09/22 22:23:59 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7012lb # [ 226.528343] niks3-server[1100]: 2026/09/22 22:23:59 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7013worker2 # [ 226.543482] nix-grpc-daemon[1512]: time=2026-09-22T22:23:59.186Z level=INFO msg="Upload complete. (229ms)"7014lb # [ 226.540709] niks3-server[1100]: 2026/09/22 22:23:59 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7015lb # [ 226.547608] niks3-server[1100]: 2026/09/22 22:23:59 INFO Completed upload id=847016lb # [ 226.550276] niks3-server[1100]: 2026/09/22 22:23:59 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7017worker2 # [ 226.552298] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=0 paths=1 nar_bytes_in=7607018worker2 # [ 226.567514] nix-grpc-daemon[1424]: ts=2026-09-22T22:23:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:368867019worker2 # [ 226.574861] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)7020worker2 # [ 226.579904] nix-daemon[837]: reaped child process 1766, status = succeeded7021client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7022client # farm-top> worker2: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7023worker2 # [ 226.594873] nix-daemon[837]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)7024client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7025client # farm-top> LOG-farm-top-drain7026worker2 # [ 228.103215] nix-daemon[837]: reaped child process 1775, status = succeeded7027worker2 # [ 228.173745] nix-daemon[837]: accepted connection from pid 1791, user nix-grpc-daemon (trusted)7028worker2 # [ 228.191555] nix-daemon[837]: reaped child process 1794, status = succeeded7029lb # [ 228.193568] niks3-server[1100]: 2026/09/22 22:24:00 INFO Received uploads request method=POST path=/api/pending_closures7030worker2 # [ 228.210656] nix-grpc-daemon[1512]: time=2026-09-22T22:24:00.851Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7031worker2 # [ 228.212474] nix-grpc-daemon[1512]: time=2026-09-22T22:24:00.852Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7032lb # [ 228.227625] niks3-server[1100]: 2026/09/22 22:24:00 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7033lb # [ 228.247851] niks3-server[1100]: 2026/09/22 22:24:00 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7034worker2 # [ 228.261338] nix-grpc-daemon[1512]: time=2026-09-22T22:24:00.903Z level=INFO msg="Uploading 1 narinfos"7035lb # [ 228.257513] niks3-server[1100]: 2026/09/22 22:24:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7036lb # [ 228.265263] niks3-server[1100]: 2026/09/22 22:24:00 INFO Signed narinfos id=85 count=17037lb # [ 228.267788] niks3-server[1100]: 2026/09/22 22:24:00 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7038lb # [ 228.279515] niks3-server[1100]: 2026/09/22 22:24:00 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7039lb # [ 228.285173] niks3-server[1100]: 2026/09/22 22:24:00 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7040worker2 # [ 228.290823] nix-grpc-daemon[1512]: time=2026-09-22T22:24:00.933Z level=INFO msg="Upload complete. (284ms)"7041lb # [ 228.289961] niks3-server[1100]: 2026/09/22 22:24:00 INFO Completed upload id=857042worker2 # [ 228.305199] nix-grpc-daemon[1424]: ts=2026-09-22T22:24:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:36886 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=12 outputs=17043worker2 # [ 228.322587] nix-daemon[837]: reaped child process 1773, status = succeeded7044client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.67 seconds)7045client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7046client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.18 seconds)7047worker1: must succeed: rm /nix/.rw-store/fill7048worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7049worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7050worker1 # [ 228.806103] nix-grpc-daemon[1899]: ts=2026-09-22T22:24:01Z level=info event=healthy reason=min_free7051worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7052(finished: subtest: low disk drains a worker and builds go to the other, in 9.51 seconds)7053(finished: run the VM test script, in 230.84 seconds)7054lb # [ 229.920375] envoy[895]: conn peer=[2001:db8:1::1]:37740 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9545 tx=4358 ms=54137055test script finished in 231.11s7056cleanup7057kill QemuMachine (pid 49)7058client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7059client # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down7060client # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down7061client # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down7062kill QemuMachine (pid 50)7063lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7064lb # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down7065lb # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down7066lb # [2026-09-22T22:24:02Z INFO virtiofsd] Client disconnected, shutting down7067kill QemuMachine (pid 51)7068worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7069worker1 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down7070worker1 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down7071worker1 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down7072kill QemuMachine (pid 52)7073worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7074worker2 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down7075worker2 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down7076worker2 # [2026-09-22T22:24:03Z INFO virtiofsd] Client disconnected, shutting down7077(finished: cleanup, in 1.25 seconds)7078additionally exposed symbols:7079 client, lb, worker1, worker2,7080 vlan1,7081 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh