vm-test-run-nix-grpc-farm
checks.aarch64-linux.farm
· build #511
· raw
1tribuchet: building on eliza2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13client # Disk image does not exist, creating the virtualisation disk image...14worker2: QEMU running (pid 52)15worker2 # Formatting '/build/vm-state-worker2/tmp.A6jrz2Hj3M', fmt=raw size=107374182416worker1 # Disk image does not exist, creating the virtualisation disk image...17client: QEMU running (pid 49)18lb: QEMU running (pid 50)19worker2 # mke2fs 1.47.4 (6-Mar-2025)20worker1 # Formatting '/build/vm-state-worker1/tmp.iHodXA3zOw', fmt=raw size=107374182421client # Formatting '/build/vm-state-client/tmp.N1k3HOkeLj', fmt=raw size=107374182422worker2 # Discarding device blocks: 0/262144 done23worker1 # mke2fs 1.47.4 (6-Mar-2025)24client # mke2fs 1.47.4 (6-Mar-2025)25worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes26worker1 # Discarding device blocks: 0/262144 done27client # Discarding device blocks: 0/262144 done28worker2 # Filesystem UUID: 849c7a3f-1ab2-4693-b52d-af9fe158a12b29worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes30client # Creating filesystem with 262144 4k blocks and 65536 inodes31worker2 # Superblock backups stored on blocks:32worker1 # Filesystem UUID: 534bdfd8-5116-405d-9f63-cc51d12f4ec333client # Filesystem UUID: 1fc68248-1ae2-42bc-8827-5f4f41b076d334worker2 # 32768, 98304, 163840, 22937635worker1 # Superblock backups stored on blocks:36client # Superblock backups stored on blocks:37worker2 # 38worker1 # 32768, 98304, 163840, 22937639client # 32768, 98304, 163840, 22937640worker2 # Allocating group tables: 0/8 done41worker1 # 42client # 43worker2 # Writing inode tables: 0/8 done44worker1 # Allocating group tables: 0/8 done45client # Allocating group tables: 0/8 done46worker2 # Creating journal (8192 blocks): done47worker1 # Writing inode tables: 0/8 done48client # Writing inode tables: 0/8 done49worker2 # Writing superblocks and filesystem accounting information: 0/8 done50worker1 # Creating journal (8192 blocks): done51client # Creating journal (8192 blocks): done52worker2 # 53worker1 # Writing superblocks and filesystem accounting information: 0/8 done54client # Writing superblocks and filesystem accounting information: 0/8 done55worker2 # Virtualisation disk image created.56worker1 # 57client # 58worker2 # Starting virtiofs daemons...59worker1 # Virtualisation disk image created.60client # Virtualisation disk image created.61worker2 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)62worker1 # Starting virtiofs daemons...63client # Starting virtiofs daemons...64worker2 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether65worker1 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)66client # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)67worker2 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)68worker1 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether69client # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether70worker2 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether71worker1 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72client # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)73worker2 # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...74worker1 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether75client # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether76worker2 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)77worker1 # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...78client # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)79worker2 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether80worker1 # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...81client # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether82worker2 # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...83worker1 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)84client # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...85worker2 # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...86worker1 # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87client # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...88worker2 # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests89worker1 # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...90client # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...91worker2 # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests92worker1 # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests93client # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests94worker2 # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests95worker1 # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests96client # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests97lb # Disk image does not exist, creating the virtualisation disk image...98worker1: QEMU running (pid 51)99worker1 # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests100client # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests101lb # Formatting '/build/vm-state-lb/tmp.KtLnoNLIN1', fmt=raw size=1073741824102lb # mke2fs 1.47.4 (6-Mar-2025)103lb # Discarding device blocks: 0/262144 done104lb # Creating filesystem with 262144 4k blocks and 65536 inodes105(finished: start all VMs, in 0.49 seconds)106lb # Filesystem UUID: 8ad90e9a-c614-4d36-9ae9-6371c810d3cf107client: waiting for unit network-addresses-eth1.service108lb # Superblock backups stored on blocks:109client: waiting for the VM to finish booting110lb # 32768, 98304, 163840, 229376111lb # 112lb # Allocating group tables: 0/8 done113lb # Writing inode tables: 0/8 done114lb # Creating journal (8192 blocks): done115lb # Writing superblocks and filesystem accounting information: 0/8 done116lb # 117lb # Virtualisation disk image created.118lb # Starting virtiofs daemons...119lb # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)122lb # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether123lb # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)124lb # [2026-09-23T04:11:14Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether125lb # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...126lb # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...127lb # [2026-09-23T04:11:14Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-23T04:11:14Z INFO virtiofsd] Client connected, servicing requests131client # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]132client # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026133client # [ 0.000000] KASLR enabled134client # [ 0.000000] random: crng init done135lb # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]136worker2 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]137client # [ 0.000000] Machine model: linux,dummy-virt138client # [ 0.000000] efi: UEFI not found.139worker1 # [ 0.000000] Booting Linux on physical CPU 0x0000000000 [0xc00fac40]140lb # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026141worker2 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026142client # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT143lb # [ 0.000000] KASLR enabled144worker2 # [ 0.000000] KASLR enabled145lb # [ 0.000000] random: crng init done146worker2 # [ 0.000000] random: crng init done147client # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]148worker1 # [ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT Mon Sep 14 11:36:19 UTC 2026149lb # [ 0.000000] Machine model: linux,dummy-virt150worker2 # [ 0.000000] Machine model: linux,dummy-virt151worker1 # [ 0.000000] KASLR enabled152lb # [ 0.000000] efi: UEFI not found.153client # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]154worker2 # [ 0.000000] efi: UEFI not found.155worker1 # [ 0.000000] random: crng init done156client # [ 0.000000] Zone ranges:157worker1 # [ 0.000000] Machine model: linux,dummy-virt158lb # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT159worker2 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT160worker1 # [ 0.000000] efi: UEFI not found.161client # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]162client # [ 0.000000] DMA32 empty163lb # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]164worker2 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]165client # [ 0.000000] Normal empty166worker1 # [ 0.000000] OF: reserved mem: Reserved memory: No reserved-memory node in the DT167client # [ 0.000000] Device empty168lb # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]169worker2 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]170client # [ 0.000000] Movable zone start for each node171lb # [ 0.000000] Zone ranges:172worker2 # [ 0.000000] Zone ranges:173worker1 # [ 0.000000] NUMA: Faking a node at [mem 0x0000000040000000-0x000000009fffffff]174client # [ 0.000000] Early memory node ranges175lb # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]176worker2 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]177worker1 # [ 0.000000] NODE_DATA(0) allocated [mem 0x9fdec700-0x9fdefe7f]178lb # [ 0.000000] DMA32 empty179worker1 # [ 0.000000] Zone ranges:180client # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]181worker2 # [ 0.000000] DMA32 empty182lb # [ 0.000000] Normal empty183worker2 # [ 0.000000] Normal empty184worker1 # [ 0.000000] DMA [mem 0x0000000040000000-0x000000009fffffff]185lb # [ 0.000000] Device empty186worker2 # [ 0.000000] Device empty187client # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]188worker1 # [ 0.000000] DMA32 empty189lb # [ 0.000000] Movable zone start for each node190worker2 # [ 0.000000] Movable zone start for each node191worker1 # [ 0.000000] Normal empty192client # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000193lb # [ 0.000000] Early memory node ranges194worker1 # [ 0.000000] Device empty195worker2 # [ 0.000000] Early memory node ranges196client # [ 0.000000] psci: probing for conduit method from DT.197worker1 # [ 0.000000] Movable zone start for each node198lb # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]199worker2 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]200client # [ 0.000000] psci: PSCIv1.3 detected in firmware.201worker1 # [ 0.000000] Early memory node ranges202lb # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]203client # [ 0.000000] psci: Using standard PSCI v0.2 function IDs204worker2 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]205worker1 # [ 0.000000] node 0: [mem 0x0000000040000000-0x000000009fffffff]206client # [ 0.000000] psci: Trusted OS migration not required207lb # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000208worker2 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000209worker1 # [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000009fffffff]210client # [ 0.000000] psci: SMC Calling Convention v1.1211lb # [ 0.000000] psci: probing for conduit method from DT.212worker2 # [ 0.000000] psci: probing for conduit method from DT.213worker1 # [ 0.000000] cma: Reserved 32 MiB at 0x000000009c400000214lb # [ 0.000000] psci: PSCIv1.3 detected in firmware.215worker2 # [ 0.000000] psci: PSCIv1.3 detected in firmware.216client # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)217worker1 # [ 0.000000] psci: probing for conduit method from DT.218lb # [ 0.000000] psci: Using standard PSCI v0.2 function IDs219worker2 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs220worker1 # [ 0.000000] psci: PSCIv1.3 detected in firmware.221client # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296222lb # [ 0.000000] psci: Trusted OS migration not required223worker2 # [ 0.000000] psci: Trusted OS migration not required224worker1 # [ 0.000000] psci: Using standard PSCI v0.2 function IDs225client # [ 0.000000] Detected PIPT I-cache on CPU0226lb # [ 0.000000] psci: SMC Calling Convention v1.1227worker2 # [ 0.000000] psci: SMC Calling Convention v1.1228worker1 # [ 0.000000] psci: Trusted OS migration not required229client # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)230worker1 # [ 0.000000] psci: SMC Calling Convention v1.1231lb # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)232worker2 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)233lb # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296234worker2 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296235worker1 # [ 0.000000] smccc: KVM: hypervisor services detected (0x00000000 0x00000000 0x00000000 0x00000003)236client # [ 0.000000] CPU features: detected: GICv3 CPU interface237lb # [ 0.000000] Detected PIPT I-cache on CPU0238worker2 # [ 0.000000] Detected PIPT I-cache on CPU0239client # [ 0.000000] CPU features: detected: Spectre-v4240worker1 # [ 0.000000] percpu: Embedded 76 pages/cpu s186712 r8192 d116392 u311296241client # [ 0.000000] CPU features: detected: Spectre-BHB242worker1 # [ 0.000000] Detected PIPT I-cache on CPU0243lb # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)244client # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38245worker2 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)246worker1 # [ 0.000000] CPU features: detected: Address authentication (architected QARMA5 algorithm)247worker2 # [ 0.000000] CPU features: detected: GICv3 CPU interface248lb # [ 0.000000] CPU features: detected: GICv3 CPU interface249client # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23250worker1 # [ 0.000000] CPU features: detected: GICv3 CPU interface251worker2 # [ 0.000000] CPU features: detected: Spectre-v4252client # [ 0.000000] alternatives: applying boot alternatives253lb # [ 0.000000] CPU features: detected: Spectre-v4254worker1 # [ 0.000000] CPU features: detected: Spectre-v4255worker2 # [ 0.000000] CPU features: detected: Spectre-BHB256lb # [ 0.000000] CPU features: detected: Spectre-BHB257worker1 # [ 0.000000] CPU features: detected: Spectre-BHB258worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38259lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38260worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC03_CPU_38261worker2 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23262lb # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23263worker1 # [ 0.000000] CPU features: detected: AmpereOne erratum AC04_CPU_23264worker2 # [ 0.000000] alternatives: applying boot alternatives265lb # [ 0.000000] alternatives: applying boot alternatives266worker1 # [ 0.000000] alternatives: applying boot alternatives267client # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/pka8rsnbb5rl489pq588cv7y1qycbh45-nixos-system-client-test/init regInfo=/nix/store/gmykddx7xl27s6hilr51wva292wc8vv2-closure-info/registration console=ttyAMA0,115200n8 console=tty0268client # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/gmykddx7xl27s6hilr51wva292wc8vv2-closure-info/registration", will be passed to user space.269worker2 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gnimypw801f4lzy6a5zb7cjxr1n5hqxf-nixos-system-worker2-test/init regInfo=/nix/store/gpp1yfkwxlp6j7wbj4qdvyv3hsy2wnhd-closure-info/registration console=ttyAMA0,115200n8 console=tty0270lb # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/kgjzg78npi7z07am2szqxkr41f06jp5w-nixos-system-lb-test/init regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration console=ttyAMA0,115200n8 console=tty0271client # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes272worker1 # [ 0.000000] Kernel command line: console=ttyAMA0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/i9c4dcq9i981i967gfs22mhmyydrkc8s-nixos-system-worker1-test/init regInfo=/nix/store/q0yhvq411jnv0dm976pqpl3xd1a1rg1q-closure-info/registration console=ttyAMA0,115200n8 console=tty0273client # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)274worker2 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/gpp1yfkwxlp6j7wbj4qdvyv3hsy2wnhd-closure-info/registration", will be passed to user space.275lb # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/lcw0xq38k7svc2ifmp8wza1s56mbyq61-closure-info/registration", will be passed to user space.276client # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)277worker1 # [ 0.000000] Unknown kernel command line parameters "regInfo=/nix/store/q0yhvq411jnv0dm976pqpl3xd1a1rg1q-closure-info/registration", will be passed to user space.278worker2 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes279lb # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes280worker1 # [ 0.000000] printk: log buffer data + meta data: 131072 + 458752 = 589824 bytes281client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB282client # [ 0.000000] software IO TLB: area num 1.283worker2 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)284lb # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)285worker1 # [ 0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)286client # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB287worker2 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)288lb # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)289worker1 # [ 0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)290client # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)291worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB292lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB293client # [ 0.000000] Fallback order for Node 0: 0294worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size adjusted to 1MB295worker2 # [ 0.000000] software IO TLB: area num 1.296lb # [ 0.000000] software IO TLB: area num 1.297worker1 # [ 0.000000] software IO TLB: area num 1.298client # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216299worker2 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB300client # [ 0.000000] Policy zone: DMA301lb # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB302worker1 # [ 0.000000] software IO TLB: SWIOTLB bounce buffer size roundup to 2MB303client # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off304worker2 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)305lb # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)306worker1 # [ 0.000000] software IO TLB: mapped [mem 0x000000009c000000-0x000000009c200000] (2MB)307worker2 # [ 0.000000] Fallback order for Node 0: 0308lb # [ 0.000000] Fallback order for Node 0: 0309worker1 # [ 0.000000] Fallback order for Node 0: 0310client # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1311worker2 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216312client # [ 0.000000] allocated 3145728 bytes of page_ext313lb # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216314worker1 # [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 393216315worker2 # [ 0.000000] Policy zone: DMA316lb # [ 0.000000] Policy zone: DMA317worker1 # [ 0.000000] Policy zone: DMA318client # [ 0.000000] ftrace: allocating 74950 entries in 294 pages319worker2 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off320client # [ 0.000000] ftrace: allocated 294 pages with 4 groups321lb # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off322worker1 # [ 0.000000] mem auto-init: stack:all(zero), heap alloc:on, heap free:off323client # [ 0.000000] rcu: Hierarchical RCU implementation.324worker2 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1325lb # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1326worker1 # [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1327client # [ 0.000000] rcu: RCU event tracing is enabled.328worker2 # [ 0.000000] allocated 3145728 bytes of page_ext329worker1 # [ 0.000000] allocated 3145728 bytes of page_ext330lb # [ 0.000000] allocated 3145728 bytes of page_ext331worker2 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages332client # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.333worker1 # [ 0.000000] ftrace: allocating 74950 entries in 294 pages334lb # [ 0.000000] ftrace: allocating 74950 entries in 294 pages335worker2 # [ 0.000000] ftrace: allocated 294 pages with 4 groups336client # [ 0.000000] Trampoline variant of Tasks RCU enabled.337worker1 # [ 0.000000] ftrace: allocated 294 pages with 4 groups338lb # [ 0.000000] ftrace: allocated 294 pages with 4 groups339worker2 # [ 0.000000] rcu: Hierarchical RCU implementation.340client # [ 0.000000] Rude variant of Tasks RCU enabled.341worker1 # [ 0.000000] rcu: Hierarchical RCU implementation.342lb # [ 0.000000] rcu: Hierarchical RCU implementation.343worker2 # [ 0.000000] rcu: RCU event tracing is enabled.344client # [ 0.000000] Tracing variant of Tasks RCU enabled.345worker1 # [ 0.000000] rcu: RCU event tracing is enabled.346lb # [ 0.000000] rcu: RCU event tracing is enabled.347worker2 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.348client # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.349worker1 # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.350lb # [ 0.000000] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.351worker2 # [ 0.000000] Trampoline variant of Tasks RCU enabled.352worker1 # [ 0.000000] Trampoline variant of Tasks RCU enabled.353lb # [ 0.000000] Trampoline variant of Tasks RCU enabled.354client # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1355worker2 # [ 0.000000] Rude variant of Tasks RCU enabled.356worker1 # [ 0.000000] Rude variant of Tasks RCU enabled.357lb # [ 0.000000] Rude variant of Tasks RCU enabled.358worker2 # [ 0.000000] Tracing variant of Tasks RCU enabled.359worker1 # [ 0.000000] Tracing variant of Tasks RCU enabled.360client # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.361lb # [ 0.000000] Tracing variant of Tasks RCU enabled.362worker2 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.363worker1 # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.364lb # [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.365client # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.366worker2 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1367worker1 # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1368lb # [ 0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1369client # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.370worker2 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.371worker1 # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.372lb # [ 0.000000] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.373client # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0374client # [ 0.000000] GICv3: 256 SPIs implemented375worker2 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.376worker1 # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.377client # [ 0.000000] GICv3: 0 Extended SPIs implemented378lb # [ 0.000000] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.379client # [ 0.000000] Root IRQ handler: gic_handle_irq380worker2 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.381worker1 # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.382client # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI383lb # [ 0.000000] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.384worker2 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0385worker1 # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0386client # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0387lb # [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0388worker2 # [ 0.000000] GICv3: 256 SPIs implemented389worker1 # [ 0.000000] GICv3: 256 SPIs implemented390lb # [ 0.000000] GICv3: 256 SPIs implemented391worker2 # [ 0.000000] GICv3: 0 Extended SPIs implemented392client # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000393worker1 # [ 0.000000] GICv3: 0 Extended SPIs implemented394lb # [ 0.000000] GICv3: 0 Extended SPIs implemented395worker2 # [ 0.000000] Root IRQ handler: gic_handle_irq396client # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]397worker1 # [ 0.000000] Root IRQ handler: gic_handle_irq398lb # [ 0.000000] Root IRQ handler: gic_handle_irq399worker2 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI400worker1 # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI401lb # [ 0.000000] GICv3: GICv3 features: 16 PPIs, DirectLPI402client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)403worker2 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0404worker1 # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0405lb # [ 0.000000] GICv3: GICD_CTLR.DS=1, SCR_EL3.FIQ=0406worker2 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000407worker1 # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000408client # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)409lb # [ 0.000000] GICv3: CPU0: found redistributor 0 region 0:0x00000000080a0000410worker2 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]411worker1 # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]412lb # [ 0.000000] ITS [mem 0x08080000-0x0809ffff]413client # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000414worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)415worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449e0000 (indirect, esz 8, psz 64K, shr 1)416client # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000417lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Devices @449f0000 (indirect, esz 8, psz 64K, shr 1)418client # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.419worker2 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)420worker1 # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @449f0000 (flat, esz 8, psz 64K, shr 1)421lb # [ 0.000000] ITS@0x0000000008080000: allocated 8192 Interrupt Collections @44a00000 (flat, esz 8, psz 64K, shr 1)422worker2 # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000423worker1 # [ 0.000000] GICv3: using LPI property table @0x0000000044a00000424client # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns425lb # [ 0.000000] GICv3: using LPI property table @0x0000000044a10000426worker2 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000427worker1 # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000428client # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).429lb # [ 0.000000] GICv3: CPU0: using allocated LPI pending table @0x0000000044a20000430worker2 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.431worker1 # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.432lb # [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.433client # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns434worker2 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns435worker1 # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns436lb # [ 0.000000] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns437client # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns438worker2 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).439worker1 # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).440client # [ 0.000028] arm-pv: using stolen time PV441lb # [ 0.000000] arch_timer: cp15 timer running at 1000.00MHz (virt).442worker2 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns443client # [ 0.000397] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)444worker1 # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns445lb # [ 0.000000] clocksource: arch_sys_counter: mask: 0x1fffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns446client # [ 0.000576] Console: colour dummy device 80x25447worker2 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns448worker1 # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns449client # [ 0.000583] printk: legacy console [tty0] enabled450lb # [ 0.000000] sched_clock: 61 bits at 1000MHz, resolution 1ns, wraps every 4398046511103ns451worker1 # [ 0.000028] arm-pv: using stolen time PV452worker2 # [ 0.000029] arm-pv: using stolen time PV453lb # [ 0.000029] arm-pv: using stolen time PV454client # [ 0.000776] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)455worker2 # [ 0.000409] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)456worker1 # [ 0.000402] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)457lb # [ 0.000375] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)458client # [ 0.000783] pid_max: default: 32768 minimum: 301459worker2 # [ 0.000588] Console: colour dummy device 80x25460worker1 # [ 0.000594] Console: colour dummy device 80x25461lb # [ 0.000576] Console: colour dummy device 80x25462worker2 # [ 0.000596] printk: legacy console [tty0] enabled463worker1 # [ 0.000601] printk: legacy console [tty0] enabled464client # [ 0.000863] LSM: initializing lsm=capability,landlock,yama,bpf,ima465lb # [ 0.000582] printk: legacy console [tty0] enabled466client # [ 0.001001] landlock: Up and running.467client # [ 0.001004] Yama: becoming mindful.468worker2 # [ 0.000791] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)469worker1 # [ 0.000789] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)470client # [ 0.001453] LSM support for eBPF active471lb # [ 0.000775] Calibrating delay loop (skipped), value calculated using timer frequency.. 2000.00 BogoMIPS (lpj=4000000)472worker2 # [ 0.000798] pid_max: default: 32768 minimum: 301473worker1 # [ 0.000796] pid_max: default: 32768 minimum: 301474lb # [ 0.000781] pid_max: default: 32768 minimum: 301475client # [ 0.001583] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)476worker2 # [ 0.000869] LSM: initializing lsm=capability,landlock,yama,bpf,ima477worker1 # [ 0.000867] LSM: initializing lsm=capability,landlock,yama,bpf,ima478lb # [ 0.000855] LSM: initializing lsm=capability,landlock,yama,bpf,ima479worker1 # [ 0.000985] landlock: Up and running.480worker2 # [ 0.000999] landlock: Up and running.481client # [ 0.001620] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)482worker2 # [ 0.001001] Yama: becoming mindful.483worker1 # [ 0.000988] Yama: becoming mindful.484lb # [ 0.001011] landlock: Up and running.485client # [ 0.003425] rcu: Hierarchical SRCU implementation.486lb # [ 0.001014] Yama: becoming mindful.487worker1 # [ 0.001431] LSM support for eBPF active488worker2 # [ 0.001460] LSM support for eBPF active489lb # [ 0.001458] LSM support for eBPF active490client # [ 0.003430] rcu: Max phase no-delay instances is 1000.491worker2 # [ 0.001615] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)492worker1 # [ 0.001570] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)493client # [ 0.004661] fsl-mc MSI: its@8080000 domain created494lb # [ 0.001593] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)495client # [ 0.004750] EFI services will not be available.496worker1 # [ 0.001602] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)497worker2 # [ 0.001648] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)498client # [ 0.004857] smp: Bringing up secondary CPUs ...499lb # [ 0.001625] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)500worker1 # [ 0.003359] rcu: Hierarchical SRCU implementation.501worker2 # [ 0.003511] rcu: Hierarchical SRCU implementation.502client # [ 0.004865] smp: Brought up 1 node, 1 CPU503lb # [ 0.003382] rcu: Hierarchical SRCU implementation.504worker2 # [ 0.003515] rcu: Max phase no-delay instances is 1000.505worker1 # [ 0.003364] rcu: Max phase no-delay instances is 1000.506client # [ 0.004868] SMP: Total of 1 processors activated.507lb # [ 0.003386] rcu: Max phase no-delay instances is 1000.508worker1 # [ 0.004566] fsl-mc MSI: its@8080000 domain created509worker2 # [ 0.004770] fsl-mc MSI: its@8080000 domain created510client # [ 0.004874] CPU: All CPU(s) started at EL1511lb # [ 0.004599] fsl-mc MSI: its@8080000 domain created512worker1 # [ 0.004657] EFI services will not be available.513worker2 # [ 0.004861] EFI services will not be available.514lb # [ 0.004694] EFI services will not be available.515client # [ 0.004884] CPU features: detected: Branch Target Identification516worker1 # [ 0.004728] smp: Bringing up secondary CPUs ...517worker2 # [ 0.004931] smp: Bringing up secondary CPUs ...518worker1 # [ 0.004736] smp: Brought up 1 node, 1 CPU519lb # [ 0.004759] smp: Bringing up secondary CPUs ...520worker2 # [ 0.004939] smp: Brought up 1 node, 1 CPU521client # [ 0.004888] CPU features: detected: ARMv8.4 Translation Table Level522lb # [ 0.004767] smp: Brought up 1 node, 1 CPU523worker1 # [ 0.004739] SMP: Total of 1 processors activated.524worker2 # [ 0.004942] SMP: Total of 1 processors activated.525worker1 # [ 0.004744] CPU: All CPU(s) started at EL1526lb # [ 0.004770] SMP: Total of 1 processors activated.527worker2 # [ 0.004947] CPU: All CPU(s) started at EL1528client # [ 0.004891] CPU features: detected: Instruction cache invalidation not required for I/D coherence529lb # [ 0.004776] CPU: All CPU(s) started at EL1530worker1 # [ 0.004753] CPU features: detected: Branch Target Identification531worker2 # [ 0.004957] CPU features: detected: Branch Target Identification532client # [ 0.004895] CPU features: detected: Data cache clean to the PoU not required for I/D coherence533lb # [ 0.004786] CPU features: detected: Branch Target Identification534worker1 # [ 0.004758] CPU features: detected: ARMv8.4 Translation Table Level535worker2 # [ 0.004962] CPU features: detected: ARMv8.4 Translation Table Level536client # [ 0.004899] CPU features: detected: Common not Private translations537lb # [ 0.004790] CPU features: detected: ARMv8.4 Translation Table Level538worker1 # [ 0.004761] CPU features: detected: Instruction cache invalidation not required for I/D coherence539client # [ 0.004903] CPU features: detected: CRC32 instructions540worker2 # [ 0.004965] CPU features: detected: Instruction cache invalidation not required for I/D coherence541lb # [ 0.004794] CPU features: detected: Instruction cache invalidation not required for I/D coherence542client # [ 0.004905] CPU features: detected: Data cache clean to Point of Deep Persistence543worker1 # [ 0.004765] CPU features: detected: Data cache clean to the PoU not required for I/D coherence544worker2 # [ 0.004969] CPU features: detected: Data cache clean to the PoU not required for I/D coherence545lb # [ 0.004797] CPU features: detected: Data cache clean to the PoU not required for I/D coherence546worker1 # [ 0.004769] CPU features: detected: Common not Private translations547client # [ 0.004909] CPU features: detected: Data cache clean to Point of Persistence548worker2 # [ 0.004972] CPU features: detected: Common not Private translations549worker1 # [ 0.004772] CPU features: detected: CRC32 instructions550lb # [ 0.004801] CPU features: detected: Common not Private translations551worker2 # [ 0.004976] CPU features: detected: CRC32 instructions552client # [ 0.004912] CPU features: detected: Data independent timing control (DIT)553lb # [ 0.004804] CPU features: detected: CRC32 instructions554client # [ 0.004915] CPU features: detected: E0PD555worker1 # [ 0.004775] CPU features: detected: Data cache clean to Point of Deep Persistence556worker2 # [ 0.004978] CPU features: detected: Data cache clean to Point of Deep Persistence557client # [ 0.004917] CPU features: detected: Enhanced Counter Virtualization558lb # [ 0.004807] CPU features: detected: Data cache clean to Point of Deep Persistence559worker1 # [ 0.004778] CPU features: detected: Data cache clean to Point of Persistence560worker2 # [ 0.004982] CPU features: detected: Data cache clean to Point of Persistence561lb # [ 0.004810] CPU features: detected: Data cache clean to Point of Persistence562client # [ 0.004921] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)563worker1 # [ 0.004781] CPU features: detected: Data independent timing control (DIT)564worker2 # [ 0.004985] CPU features: detected: Data independent timing control (DIT)565worker1 # [ 0.004784] CPU features: detected: E0PD566lb # [ 0.004813] CPU features: detected: Data independent timing control (DIT)567client # [ 0.004924] CPU features: detected: Enhanced Virtualization Traps568worker2 # [ 0.004988] CPU features: detected: E0PD569lb # [ 0.004816] CPU features: detected: E0PD570worker1 # [ 0.004787] CPU features: detected: Enhanced Counter Virtualization571client # [ 0.004927] CPU features: detected: Fine Grained Traps572worker2 # [ 0.004991] CPU features: detected: Enhanced Counter Virtualization573lb # [ 0.004818] CPU features: detected: Enhanced Counter Virtualization574worker1 # [ 0.004790] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)575client # [ 0.004931] CPU features: detected: Generic authentication (architected QARMA5 algorithm)576worker2 # [ 0.004994] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)577worker1 # [ 0.004793] CPU features: detected: Enhanced Virtualization Traps578client # [ 0.004936] CPU features: detected: RCpc load-acquire (LDAPR)579worker2 # [ 0.004997] CPU features: detected: Enhanced Virtualization Traps580worker1 # [ 0.004796] CPU features: detected: Fine Grained Traps581client # [ 0.004939] CPU features: detected: LSE atomic instructions582worker2 # [ 0.005000] CPU features: detected: Fine Grained Traps583client # [ 0.004942] CPU features: detected: Privileged Access Never584worker1 # [ 0.004800] CPU features: detected: Generic authentication (architected QARMA5 algorithm)585client # [ 0.004945] CPU features: detected: PMUv3586worker2 # [ 0.005004] CPU features: detected: Generic authentication (architected QARMA5 algorithm)587worker1 # [ 0.004805] CPU features: detected: RCpc load-acquire (LDAPR)588client # [ 0.004947] CPU features: detected: RAS Extension Support589worker2 # [ 0.005008] CPU features: detected: RCpc load-acquire (LDAPR)590worker1 # [ 0.004808] CPU features: detected: LSE atomic instructions591client # [ 0.004950] CPU features: detected: RASv1p1 Extension Support592worker2 # [ 0.005011] CPU features: detected: LSE atomic instructions593worker1 # [ 0.004811] CPU features: detected: Privileged Access Never594client # [ 0.004953] CPU features: detected: Random Number Generator595worker2 # [ 0.005014] CPU features: detected: Privileged Access Never596worker1 # [ 0.004813] CPU features: detected: PMUv3597worker2 # [ 0.005016] CPU features: detected: PMUv3598client # [ 0.004955] CPU features: detected: Speculation barrier (SB)599worker1 # [ 0.004816] CPU features: detected: RAS Extension Support600worker2 # [ 0.005019] CPU features: detected: RAS Extension Support601client # [ 0.004958] CPU features: detected: Stage-2 Force Write-Back602worker1 # [ 0.004819] CPU features: detected: RASv1p1 Extension Support603worker2 # [ 0.005022] CPU features: detected: RASv1p1 Extension Support604worker1 # [ 0.004821] CPU features: detected: Random Number Generator605worker2 # [ 0.005025] CPU features: detected: Random Number Generator606worker1 # [ 0.004824] CPU features: detected: Speculation barrier (SB)607worker1 # [ 0.004827] CPU features: detected: Stage-2 Force Write-Back608lb # [ 0.004822] CPU features: detected: Enhanced Counter Virtualization (CNTPOFF)609worker1 # [ 0.004830] CPU features: detected: TLB range maintenance instructions610lb # [ 0.004825] CPU features: detected: Enhanced Virtualization Traps611worker1 # [ 0.004834] CPU features: detected: Speculative Store Bypassing Safe (SSBS)612lb # [ 0.004828] CPU features: detected: Fine Grained Traps613worker1 # [ 0.004871] alternatives: applying system-wide alternatives614lb # [ 0.004831] CPU features: detected: Generic authentication (architected QARMA5 algorithm)615worker1 # [ 0.007771] CPU features: detected: BBM Level 2 without TLB conflict abort616lb # [ 0.004836] CPU features: detected: RCpc load-acquire (LDAPR)617lb # [ 0.004838] CPU features: detected: LSE atomic instructions618worker1 # [ 0.008053] Memory: 1405620K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124664K reserved, 32768K cma-reserved)619lb # [ 0.004841] CPU features: detected: Privileged Access Never620worker1 # [ 0.008409] devtmpfs: initialized621lb # [ 0.004844] CPU features: detected: PMUv3622worker1 # [ 0.010257] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)623lb # [ 0.004847] CPU features: detected: RAS Extension Support624client # [ 0.004961] CPU features: detected: TLB range maintenance instructions625lb # [ 0.004849] CPU features: detected: RASv1p1 Extension Support626worker1 # [ 0.010280] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).627client # [ 0.004966] CPU features: detected: Speculative Store Bypassing Safe (SSBS)628lb # [ 0.004852] CPU features: detected: Random Number Generator629worker2 # [ 0.005027] CPU features: detected: Speculation barrier (SB)630worker1 # [ 0.010445] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL631client # [ 0.005004] alternatives: applying system-wide alternatives632lb # [ 0.004854] CPU features: detected: Speculation barrier (SB)633worker1 # [ 0.010449] 0 pages in range for non-PLT usage634worker2 # [ 0.005030] CPU features: detected: Stage-2 Force Write-Back635client # [ 0.007883] CPU features: detected: BBM Level 2 without TLB conflict abort636worker1 # [ 0.010450] 508272 pages in range for PLT usage637lb # [ 0.004857] CPU features: detected: Stage-2 Force Write-Back638worker2 # [ 0.005033] CPU features: detected: TLB range maintenance instructions639worker1 # [ 0.010544] pinctrl core: initialized pinctrl subsystem640lb # [ 0.004860] CPU features: detected: TLB range maintenance instructions641worker1 # [ 0.011297] DMI not present or invalid.642worker2 # [ 0.005038] CPU features: detected: Speculative Store Bypassing Safe (SSBS)643lb # [ 0.004864] CPU features: detected: Speculative Store Bypassing Safe (SSBS)644client # [ 0.008075] Memory: 1405648K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124660K reserved, 32768K cma-reserved)645worker2 # [ 0.005075] alternatives: applying system-wide alternatives646worker1 # [ 0.014406] NET: Registered PF_NETLINK/PF_ROUTE protocol family647client # [ 0.008461] devtmpfs: initialized648lb # [ 0.004902] alternatives: applying system-wide alternatives649worker2 # [ 0.007949] CPU features: detected: BBM Level 2 without TLB conflict abort650worker1 # [ 0.016584] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations651client # [ 0.010294] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)652lb # [ 0.007753] CPU features: detected: BBM Level 2 without TLB conflict abort653worker1 # [ 0.016772] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations654client # [ 0.010316] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).655worker2 # [ 0.008177] Memory: 1405656K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124632K reserved, 32768K cma-reserved)656worker1 # [ 0.016968] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations657worker2 # [ 0.008551] devtmpfs: initialized658client # [ 0.010492] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL659lb # [ 0.007945] Memory: 1405628K/1572864K available (24448K kernel code, 7094K rwdata, 26372K rodata, 4736K init, 1107K bss, 124636K reserved, 32768K cma-reserved)660worker1 # [ 0.016988] audit: initializing netlink subsys (disabled)661client # [ 0.010497] 0 pages in range for non-PLT usage662lb # [ 0.008334] devtmpfs: initialized663worker2 # [ 0.010359] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)664client # [ 0.010498] 508272 pages in range for PLT usage665worker1 # [ 0.017522] thermal_sys: Registered thermal governor 'fair_share'666lb # [ 0.010143] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)667client # [ 0.010585] pinctrl core: initialized pinctrl subsystem668worker1 # [ 0.017524] thermal_sys: Registered thermal governor 'bang_bang'669worker2 # [ 0.010386] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).670client # [ 0.011309] DMI not present or invalid.671lb # [ 0.010164] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).672worker1 # [ 0.017528] thermal_sys: Registered thermal governor 'step_wise'673worker2 # [ 0.010568] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL674client # [ 0.014487] NET: Registered PF_NETLINK/PF_ROUTE protocol family675worker2 # [ 0.010572] 0 pages in range for non-PLT usage676lb # [ 0.010327] 2G module region forced by RANDOMIZE_MODULE_REGION_FULL677worker1 # [ 0.017531] thermal_sys: Registered thermal governor 'user_space'678lb # [ 0.010332] 0 pages in range for non-PLT usage679worker2 # [ 0.010573] 508272 pages in range for PLT usage680client # [ 0.016690] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations681worker1 # [ 0.017536] thermal_sys: Registered thermal governor 'power_allocator'682lb # [ 0.010333] 508272 pages in range for PLT usage683worker2 # [ 0.010680] pinctrl core: initialized pinctrl subsystem684client # [ 0.016870] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations685worker2 # [ 0.011405] DMI not present or invalid.686worker1 # [ 0.017556] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1687lb # [ 0.010422] pinctrl core: initialized pinctrl subsystem688lb # [ 0.011164] DMI not present or invalid.689worker2 # [ 0.014685] NET: Registered PF_NETLINK/PF_ROUTE protocol family690worker1 # [ 0.017566] cpuidle: using governor ladder691client # [ 0.017069] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations692worker1 # [ 0.017571] cpuidle: using governor menu693lb # [ 0.014398] NET: Registered PF_NETLINK/PF_ROUTE protocol family694client # [ 0.017089] audit: initializing netlink subsys (disabled)695worker2 # [ 0.016881] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations696worker1 # [ 0.017747] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.697lb # [ 0.016565] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations698client # [ 0.017689] thermal_sys: Registered thermal governor 'fair_share'699worker2 # [ 0.017068] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations700worker1 # [ 0.017760] ASID allocator initialised with 65536 entries701client # [ 0.017691] thermal_sys: Registered thermal governor 'bang_bang'702worker1 # [ 0.018932] Serial: AMBA PL011 UART driver703lb # [ 0.016748] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations704worker2 # [ 0.017267] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations705client # [ 0.017695] thermal_sys: Registered thermal governor 'step_wise'706lb # [ 0.016946] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations707worker2 # [ 0.017287] audit: initializing netlink subsys (disabled)708client # [ 0.017698] thermal_sys: Registered thermal governor 'user_space'709lb # [ 0.016966] audit: initializing netlink subsys (disabled)710worker1 # [ 0.024006] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1711worker2 # [ 0.017891] thermal_sys: Registered thermal governor 'fair_share'712client # [ 0.017704] thermal_sys: Registered thermal governor 'power_allocator'713worker1 # [ 0.024175] printk: console [ttyAMA0] enabled714lb # [ 0.017501] thermal_sys: Registered thermal governor 'fair_share'715worker2 # [ 0.017893] thermal_sys: Registered thermal governor 'bang_bang'716client # [ 0.017728] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1717lb # [ 0.017504] thermal_sys: Registered thermal governor 'bang_bang'718worker2 # [ 0.017896] thermal_sys: Registered thermal governor 'step_wise'719client # [ 0.017738] cpuidle: using governor ladder720lb # [ 0.017507] thermal_sys: Registered thermal governor 'step_wise'721worker2 # [ 0.017899] thermal_sys: Registered thermal governor 'user_space'722client # [ 0.017743] cpuidle: using governor menu723lb # [ 0.017510] thermal_sys: Registered thermal governor 'user_space'724worker2 # [ 0.017904] thermal_sys: Registered thermal governor 'power_allocator'725client # [ 0.017940] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.726lb # [ 0.017515] thermal_sys: Registered thermal governor 'power_allocator'727client # [ 0.017956] ASID allocator initialised with 65536 entries728worker2 # [ 0.017929] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1729client # [ 0.019144] Serial: AMBA PL011 UART driver730worker2 # [ 0.017937] cpuidle: using governor ladder731lb # [ 0.017537] audit: type=2000 audit(0.016:1): state=initialized audit_enabled=0 res=1732worker2 # [ 0.017943] cpuidle: using governor menu733lb # [ 0.017547] cpuidle: using governor ladder734client # [ 0.024299] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1735lb # [ 0.017552] cpuidle: using governor menu736worker2 # [ 0.018122] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.737client # [ 0.024433] printk: console [ttyAMA0] enabled738worker2 # [ 0.018135] ASID allocator initialised with 65536 entries739lb # [ 0.017733] hw-breakpoint: found 16 breakpoint and 4 watchpoint registers.740worker2 # [ 0.019363] Serial: AMBA PL011 UART driver741lb # [ 0.017746] ASID allocator initialised with 65536 entries742client # [ 0.143512] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages743lb # [ 0.018934] Serial: AMBA PL011 UART driver744worker2 # [ 0.024568] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1745worker1 # [ 0.140420] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages746client # [ 0.143528] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page747worker2 # [ 0.024723] printk: console [ttyAMA0] enabled748lb # [ 0.024172] 9000000.pl011: ttyAMA0 at MMIO 0x9000000 (irq = 13, base_baud = 0) is a PL011 rev1749worker1 # [ 0.140435] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page750client # [ 0.143532] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages751lb # [ 0.024288] printk: console [ttyAMA0] enabled752worker1 # [ 0.140440] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages753client # [ 0.143536] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page754worker2 # [ 0.143811] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages755worker1 # [ 0.140444] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page756client # [ 0.143541] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages757worker2 # [ 0.143828] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page758lb # [ 0.144257] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages759worker1 # [ 0.140449] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages760client # [ 0.143545] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page761lb # [ 0.144272] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page762worker2 # [ 0.143833] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages763worker1 # [ 0.140453] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page764client # [ 0.143549] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages765worker2 # [ 0.143837] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page766lb # [ 0.144277] HugeTLB: registered 32.0 MiB page size, pre-allocated 0 pages767worker1 # [ 0.140458] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages768client # [ 0.143553] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page769lb # [ 0.144282] HugeTLB: 0 KiB vmemmap can be freed for a 32.0 MiB page770worker2 # [ 0.143841] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages771worker1 # [ 0.140462] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page772worker2 # [ 0.143846] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page773lb # [ 0.144286] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages774worker2 # [ 0.143850] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages775lb # [ 0.144290] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page776client # [ 0.150707] fbcon: Taking over console777worker2 # [ 0.143855] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page778lb # [ 0.144294] HugeTLB: registered 64.0 KiB page size, pre-allocated 0 pages779client # [ 0.150721] ACPI: Interpreter disabled.780worker1 # [ 0.147557] fbcon: Taking over console781lb # [ 0.144298] HugeTLB: 0 KiB vmemmap can be freed for a 64.0 KiB page782worker1 # [ 0.147571] ACPI: Interpreter disabled.783worker2 # [ 0.151102] fbcon: Taking over console784worker2 # [ 0.151114] ACPI: Interpreter disabled.785lb # [ 0.151555] fbcon: Taking over console786lb # [ 0.151568] ACPI: Interpreter disabled.787client # [ 0.159302] iommu: Default domain type: Translated788client # [ 0.159312] iommu: DMA domain TLB invalidation policy: strict mode789worker2 # [ 0.158140] iommu: Default domain type: Translated790worker1 # [ 0.156468] iommu: Default domain type: Translated791client # [ 0.161057] SCSI subsystem initialized792worker2 # [ 0.158149] iommu: DMA domain TLB invalidation policy: strict mode793worker1 # [ 0.156477] iommu: DMA domain TLB invalidation policy: strict mode794worker1 # [ 0.156805] SCSI subsystem initialized795worker2 # [ 0.159808] SCSI subsystem initialized796client # [ 0.161722] usbcore: registered new interface driver usbfs797lb # [ 0.160315] iommu: Default domain type: Translated798client # [ 0.161747] usbcore: registered new interface driver hub799worker1 # [ 0.158763] usbcore: registered new interface driver usbfs800worker2 # [ 0.160480] usbcore: registered new interface driver usbfs801client # [ 0.161786] usbcore: registered new device driver usb802lb # [ 0.160325] iommu: DMA domain TLB invalidation policy: strict mode803worker1 # [ 0.158799] usbcore: registered new interface driver hub804worker2 # [ 0.160512] usbcore: registered new interface driver hub805client # [ 0.162053] pps_core: LinuxPPS API ver. 1 registered806worker1 # [ 0.158815] usbcore: registered new device driver usb807lb # [ 0.162028] SCSI subsystem initialized808worker2 # [ 0.160528] usbcore: registered new device driver usb809worker1 # [ 0.159095] pps_core: LinuxPPS API ver. 1 registered810worker2 # [ 0.160806] pps_core: LinuxPPS API ver. 1 registered811client # [ 0.162060] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>812client # [ 0.162070] PTP clock support registered813lb # [ 0.162668] usbcore: registered new interface driver usbfs814worker1 # [ 0.159101] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>815client # [ 0.162122] EDAC MC: Ver: 3.0.0816worker2 # [ 0.160813] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>817worker1 # [ 0.159111] PTP clock support registered818lb # [ 0.162691] usbcore: registered new interface driver hub819worker2 # [ 0.160822] PTP clock support registered820worker1 # [ 0.159157] EDAC MC: Ver: 3.0.0821client # [ 0.166588] scmi_core: SCMI protocol bus registered822worker2 # [ 0.160876] EDAC MC: Ver: 3.0.0823lb # [ 0.162720] usbcore: registered new device driver usb824lb # [ 0.162984] pps_core: LinuxPPS API ver. 1 registered825worker1 # [ 0.163541] scmi_core: SCMI protocol bus registered826worker2 # [ 0.165321] scmi_core: SCMI protocol bus registered827client # [ 0.167503] FPGA manager framework828client # [ 0.168421] vgaarb: loaded829lb # [ 0.162990] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>830worker1 # [ 0.164493] FPGA manager framework831worker2 # [ 0.166301] FPGA manager framework832worker1 # [ 0.165445] vgaarb: loaded833lb # [ 0.163000] PTP clock support registered834worker2 # [ 0.167233] vgaarb: loaded835lb # [ 0.163042] EDAC MC: Ver: 3.0.0836client # [ 0.169045] clocksource: Switched to clocksource arch_sys_counter837worker1 # [ 0.166064] clocksource: Switched to clocksource arch_sys_counter838worker2 # [ 0.167854] clocksource: Switched to clocksource arch_sys_counter839lb # [ 0.167576] scmi_core: SCMI protocol bus registered840client # [ 0.169622] VFS: Disk quotas dquot_6.6.0841client # [ 0.169655] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)842lb # [ 0.168520] FPGA manager framework843lb # [ 0.169478] vgaarb: loaded844client # [ 0.171965] netfs: FS-Cache loaded845client # [ 0.172082] pnp: PnP ACPI: disabled846worker1 # [ 0.169525] VFS: Disk quotas dquot_6.6.0847lb # [ 0.170103] clocksource: Switched to clocksource arch_sys_counter848lb # [ 0.170688] VFS: Disk quotas dquot_6.6.0849worker1 # [ 0.169566] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)850lb # [ 0.170720] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)851worker2 # [ 0.170831] VFS: Disk quotas dquot_6.6.0852lb # [ 0.173132] netfs: FS-Cache loaded853worker2 # [ 0.170865] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)854lb # [ 0.173239] pnp: PnP ACPI: disabled855worker2 # [ 0.174513] netfs: FS-Cache loaded856worker1 # [ 0.173115] netfs: FS-Cache loaded857worker2 # [ 0.174621] pnp: PnP ACPI: disabled858worker1 # [ 0.173217] pnp: PnP ACPI: disabled859client # [ 0.177978] NET: Registered PF_INET protocol family860worker1 # [ 0.177088] NET: Registered PF_INET protocol family861client # [ 0.178249] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)862worker2 # [ 0.178534] NET: Registered PF_INET protocol family863worker1 # [ 0.177344] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)864worker2 # [ 0.178789] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)865lb # [ 0.179164] NET: Registered PF_INET protocol family866lb # [ 0.179462] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)867client # [ 0.207094] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)868client # [ 0.207130] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)869client # [ 0.207156] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)870client # [ 0.207227] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)871lb # [ 0.208231] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)872client # [ 0.207370] TCP: Hash tables configured (established 16384 bind 16384)873client # [ 0.207446] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)874lb # [ 0.208270] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)875worker1 # [ 0.206173] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)876worker2 # [ 0.207998] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)877client # [ 0.207479] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)878lb # [ 0.208295] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)879worker1 # [ 0.206217] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)880worker2 # [ 0.208040] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)881client # [ 0.207514] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)882lb # [ 0.208367] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)883worker1 # [ 0.206241] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)884client # [ 0.207648] NET: Registered PF_UNIX/PF_LOCAL protocol family885worker2 # [ 0.208065] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)886lb # [ 0.208510] TCP: Hash tables configured (established 16384 bind 16384)887client # [ 0.207673] NET: Registered PF_XDP protocol family888worker1 # [ 0.206314] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)889worker2 # [ 0.208137] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)890client # [ 0.207689] PCI: CLS 0 bytes, default 64891lb # [ 0.208584] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)892worker1 # [ 0.206456] TCP: Hash tables configured (established 16384 bind 16384)893worker2 # [ 0.208279] TCP: Hash tables configured (established 16384 bind 16384)894client # [ 0.207898] Trying to unpack rootfs image as initramfs...895lb # [ 0.208616] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)896worker1 # [ 0.206532] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)897worker2 # [ 0.208356] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)898lb # [ 0.208652] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)899worker1 # [ 0.206565] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)900worker2 # [ 0.208389] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)901lb # [ 0.208743] NET: Registered PF_UNIX/PF_LOCAL protocol family902worker1 # [ 0.206600] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)903lb # [ 0.208763] NET: Registered PF_XDP protocol family904worker2 # [ 0.208424] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)905lb # [ 0.208778] PCI: CLS 0 bytes, default 64906worker1 # [ 0.206719] NET: Registered PF_UNIX/PF_LOCAL protocol family907worker2 # [ 0.208521] NET: Registered PF_UNIX/PF_LOCAL protocol family908client # [ 0.222575] kvm [1]: HYP mode not available909worker1 # [ 0.206757] NET: Registered PF_XDP protocol family910lb # [ 0.209010] Trying to unpack rootfs image as initramfs...911worker2 # [ 0.208547] NET: Registered PF_XDP protocol family912worker1 # [ 0.206773] PCI: CLS 0 bytes, default 64913worker2 # [ 0.208563] PCI: CLS 0 bytes, default 64914worker1 # [ 0.207003] Trying to unpack rootfs image as initramfs...915worker2 # [ 0.208824] Trying to unpack rootfs image as initramfs...916lb # [ 0.223478] kvm [1]: HYP mode not available917worker1 # [ 0.223310] kvm [1]: HYP mode not available918worker2 # [ 0.225143] kvm [1]: HYP mode not available919client # [ 0.315012] Initialise system trusted keyrings920client # [ 0.315740] workingset: timestamp_bits=42 max_order=19 bucket_order=0921client # [ 0.316956] squashfs: version 4.0 (2009/01/31) Phillip Lougher922lb # [ 0.314631] Initialise system trusted keyrings923lb # [ 0.315384] workingset: timestamp_bits=42 max_order=19 bucket_order=0924worker1 # [ 0.314621] Initialise system trusted keyrings925worker2 # [ 0.316399] Initialise system trusted keyrings926lb # [ 0.316722] squashfs: version 4.0 (2009/01/31) Phillip Lougher927worker1 # [ 0.315376] workingset: timestamp_bits=42 max_order=19 bucket_order=0928worker2 # [ 0.317124] workingset: timestamp_bits=42 max_order=19 bucket_order=0929worker1 # [ 0.316643] squashfs: version 4.0 (2009/01/31) Phillip Lougher930lb # [ 0.317475] 9p: Installing v9fs 9p2000 file system support931worker2 # [ 0.318358] squashfs: version 4.0 (2009/01/31) Phillip Lougher932client # [ 0.317812] 9p: Installing v9fs 9p2000 file system support933worker1 # [ 0.317379] 9p: Installing v9fs 9p2000 file system support934worker2 # [ 0.319096] 9p: Installing v9fs 9p2000 file system support935lb # [ 0.338015] Key type asymmetric registered936lb # [ 0.338031] Asymmetric key parser 'x509' registered937worker1 # [ 0.337975] Key type asymmetric registered938worker2 # [ 0.339779] Key type asymmetric registered939lb # [ 0.338088] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)940worker1 # [ 0.337992] Asymmetric key parser 'x509' registered941worker2 # [ 0.339796] Asymmetric key parser 'x509' registered942worker1 # [ 0.338059] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)943worker2 # [ 0.339851] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)944client # [ 0.345845] Key type asymmetric registered945client # [ 0.345865] Asymmetric key parser 'x509' registered946client # [ 0.345939] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 242)947client # [ 0.347986] io scheduler mq-deadline registered948client # [ 0.347996] io scheduler kyber registered949lb # [ 0.346157] io scheduler mq-deadline registered950lb # [ 0.346171] io scheduler kyber registered951worker1 # [ 0.346242] io scheduler mq-deadline registered952worker2 # [ 0.348002] io scheduler mq-deadline registered953worker1 # [ 0.346257] io scheduler kyber registered954worker2 # [ 0.348010] io scheduler kyber registered955lb # [ 0.354245] pl061_gpio 9030000.pl061: PL061 GPIO chip registered956client # [ 0.357186] pl061_gpio 9030000.pl061: PL061 GPIO chip registered957worker1 # [ 0.354214] pl061_gpio 9030000.pl061: PL061 GPIO chip registered958worker2 # [ 0.356002] pl061_gpio 9030000.pl061: PL061 GPIO chip registered959lb # [ 0.355621] ledtrig-cpu: registered to indicate activity on CPUs960lb # [ 0.356003] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:961worker1 # [ 0.355560] ledtrig-cpu: registered to indicate activity on CPUs962worker2 # [ 0.357350] ledtrig-cpu: registered to indicate activity on CPUs963lb # [ 0.356021] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000964worker1 # [ 0.355970] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:965worker2 # [ 0.357743] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:966lb # [ 0.356033] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000967worker1 # [ 0.355987] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000968worker2 # [ 0.357760] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000969lb # [ 0.356041] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000970worker1 # [ 0.355999] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000971worker2 # [ 0.357772] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000972client # [ 0.358547] ledtrig-cpu: registered to indicate activity on CPUs973lb # [ 0.356060] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits974worker1 # [ 0.356007] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000975client # [ 0.358942] pci-host-generic 4010000000.pcie: host bridge /pcie@10000000 ranges:976worker2 # [ 0.357780] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000977worker1 # [ 0.356028] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits978lb # [ 0.356083] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]979client # [ 0.358959] pci-host-generic 4010000000.pcie: IO 0x003eff0000..0x003effffff -> 0x0000000000980worker2 # [ 0.357800] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits981lb # [ 0.356171] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00982worker1 # [ 0.356050] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]983client # [ 0.358980] pci-host-generic 4010000000.pcie: MEM 0x0010000000..0x003efeffff -> 0x0010000000984worker2 # [ 0.357824] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]985lb # [ 0.356180] pci_bus 0000:00: root bus resource [bus 00-ff]986worker1 # [ 0.356118] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00987worker2 # [ 0.357907] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:00988lb # [ 0.356186] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]989client # [ 0.358990] pci-host-generic 4010000000.pcie: MEM 0x8000000000..0xffffffffff -> 0x8000000000990worker1 # [ 0.356127] pci_bus 0000:00: root bus resource [bus 00-ff]991worker2 # [ 0.357916] pci_bus 0000:00: root bus resource [bus 00-ff]992lb # [ 0.356191] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]993worker1 # [ 0.356133] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]994client # [ 0.359010] pci-host-generic 4010000000.pcie: Memory resource size exceeds max for 32 bits995worker2 # [ 0.357922] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]996lb # [ 0.356196] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]997worker1 # [ 0.356138] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]998worker2 # [ 0.357928] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]999client # [ 0.359033] pci-host-generic 4010000000.pcie: ECAM at [mem 0x4010000000-0x401fffffff] for [bus 00-ff]1000lb # [ 0.356278] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1001worker1 # [ 0.356143] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1002client # [ 0.359108] pci-host-generic 4010000000.pcie: PCI host bridge to bus 0000:001003worker2 # [ 0.357933] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1004client # [ 0.359118] pci_bus 0000:00: root bus resource [bus 00-ff]1005lb # [ 0.356725] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1006worker1 # [ 0.356195] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1007worker2 # [ 0.358024] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1008lb # [ 0.356967] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1009client # [ 0.359124] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]1010worker1 # [ 0.356631] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1011worker2 # [ 0.358463] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1012lb # [ 0.356984] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1013client # [ 0.359128] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff]1014worker1 # [ 0.356825] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1015worker2 # [ 0.358662] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1016lb # [ 0.357014] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1017worker1 # [ 0.356843] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1018client # [ 0.359134] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff]1019worker2 # [ 0.358679] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1020lb # [ 0.357030] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1021worker1 # [ 0.356872] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1022client # [ 0.359191] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000 conventional PCI endpoint1023worker2 # [ 0.358709] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1024worker1 # [ 0.356888] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1025lb # [ 0.357473] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1026worker2 # [ 0.358726] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1027client # [ 0.359636] pci 0000:00:01.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1028lb # [ 0.357655] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1029worker1 # [ 0.357334] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1030client # [ 0.359832] pci 0000:00:01.0: BAR 0 [io 0x0000-0x001f]1031worker2 # [ 0.359175] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1032lb # [ 0.357671] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1033worker1 # [ 0.357514] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1034client # [ 0.359848] pci 0000:00:01.0: BAR 1 [mem 0x00000000-0x00000fff]1035worker2 # [ 0.359359] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1036lb # [ 0.357701] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1037worker1 # [ 0.357530] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1038worker2 # [ 0.359375] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1039client # [ 0.359878] pci 0000:00:01.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1040worker1 # [ 0.357559] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1041client # [ 0.359894] pci 0000:00:01.0: ROM [mem 0x00000000-0x0003ffff pref]1042worker2 # [ 0.359405] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1043worker1 # [ 0.358027] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1044client # [ 0.360352] pci 0000:00:02.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1045client # [ 0.360536] pci 0000:00:02.0: BAR 0 [io 0x0000-0x001f]1046client # [ 0.360551] pci 0000:00:02.0: BAR 1 [mem 0x00000000-0x00000fff]1047client # [ 0.360580] pci 0000:00:02.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1048client # [ 0.361033] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1049client # [ 0.361235] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1050client # [ 0.361251] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1051client # [ 0.361281] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1052client # [ 0.361733] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1053client # [ 0.362214] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1054client # [ 0.362231] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1055client # [ 0.362261] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1056client # [ 0.362280] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1057client # [ 0.362743] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1058client # [ 0.362943] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1059client # [ 0.362972] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1060lb # [ 0.382218] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1061client # [ 0.363430] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1062lb # [ 0.382414] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1063worker1 # [ 0.382258] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1064client # [ 0.363620] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1065lb # [ 0.382430] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1066worker2 # [ 0.383923] pci 0000:00:03.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1067worker1 # [ 0.382287] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1068worker2 # [ 0.384122] pci 0000:00:03.0: BAR 0 [io 0x0000-0x007f]1069client # [ 0.363649] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1070lb # [ 0.382461] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1071worker1 # [ 0.382322] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1072worker2 # [ 0.384139] pci 0000:00:03.0: BAR 1 [mem 0x00000000-0x00000fff]1073lb # [ 0.382938] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1074client # [ 0.364056] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1075worker1 # [ 0.382805] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1076worker2 # [ 0.384169] pci 0000:00:03.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1077lb # [ 0.383120] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1078client # [ 0.364239] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1079worker1 # [ 0.382992] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1080lb # [ 0.383136] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1081worker2 # [ 0.384662] pci 0000:00:04.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1082worker1 # [ 0.383008] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1083client # [ 0.364484] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1084worker2 # [ 0.384848] pci 0000:00:04.0: BAR 0 [io 0x0000-0x001f]1085lb # [ 0.383166] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1086worker1 # [ 0.383037] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1087client # [ 0.364672] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1088worker2 # [ 0.384863] pci 0000:00:04.0: BAR 1 [mem 0x00000000-0x00000fff]1089lb # [ 0.383185] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1090worker1 # [ 0.383055] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1091client # [ 0.364701] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1092worker2 # [ 0.384893] pci 0000:00:04.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1093lb # [ 0.383660] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1094worker1 # [ 0.383513] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1095worker2 # [ 0.384911] pci 0000:00:04.0: ROM [mem 0x00000000-0x0003ffff pref]1096lb # [ 0.383848] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1097worker1 # [ 0.383698] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1098worker2 # [ 0.385368] pci 0000:00:05.0: [1af4:1050] type 00 class 0x038000 conventional PCI endpoint1099lb # [ 0.383878] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1100worker1 # [ 0.383727] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1101worker2 # [ 0.385557] pci 0000:00:05.0: BAR 1 [mem 0x00000000-0x00000fff]1102lb # [ 0.384330] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1103worker1 # [ 0.384210] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1104worker2 # [ 0.385587] pci 0000:00:05.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1105lb # [ 0.384517] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1106worker1 # [ 0.384396] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1107worker2 # [ 0.386040] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1108lb # [ 0.384546] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1109worker1 # [ 0.384426] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1110worker2 # [ 0.386228] pci 0000:00:06.0: BAR 1 [mem 0x00000000-0x00000fff]1111lb # [ 0.384929] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1112worker1 # [ 0.384810] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1113worker2 # [ 0.386257] pci 0000:00:06.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1114lb # [ 0.385108] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1115worker1 # [ 0.384989] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1116worker2 # [ 0.386646] pci 0000:00:07.0: [8086:24cd] type 00 class 0x0c0320 conventional PCI endpoint1117worker1 # [ 0.385236] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1118lb # [ 0.385395] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1119worker2 # [ 0.386828] pci 0000:00:07.0: BAR 0 [mem 0x00000000-0x00000fff]1120lb # [ 0.385581] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1121worker1 # [ 0.385423] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1122worker2 # [ 0.387114] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1123worker1 # [ 0.385453] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1124lb # [ 0.385611] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1125worker2 # [ 0.387306] pci 0000:00:08.0: BAR 1 [mem 0x00000000-0x00000fff]1126worker1 # [ 0.385901] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1127lb # [ 0.386057] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1128worker2 # [ 0.387336] pci 0000:00:08.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1129client # [ 0.405230] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1130worker1 # [ 0.386108] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1131lb # [ 0.386255] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1132client # [ 0.405431] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1133worker2 # [ 0.387797] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1134lb # [ 0.386285] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1135worker1 # [ 0.386137] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1136client # [ 0.405463] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1137worker2 # [ 0.388003] pci 0000:00:09.0: BAR 1 [mem 0x00000000-0x00000fff]1138worker1 # [ 0.386587] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1139lb # [ 0.386731] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1140worker2 # [ 0.388033] pci 0000:00:09.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1141client # [ 0.405945] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1142worker1 # [ 0.386772] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1143lb # [ 0.386916] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1144client # [ 0.406134] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1145worker2 # [ 0.388492] pci 0000:00:0a.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1146worker1 # [ 0.386802] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1147lb # [ 0.386946] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1148client # [ 0.406164] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1149worker2 # [ 0.388680] pci 0000:00:0a.0: BAR 1 [mem 0x00000000-0x00000fff]1150worker1 # [ 0.387259] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1151lb # [ 0.387415] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1152worker2 # [ 0.388710] pci 0000:00:0a.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1153client # [ 0.406630] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1154worker1 # [ 0.387541] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1155lb # [ 0.387675] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1156client # [ 0.406878] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1157worker1 # [ 0.387559] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1158lb # [ 0.387692] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1159worker2 # [ 0.389178] pci 0000:00:0b.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1160client # [ 0.406895] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1161worker2 # [ 0.389464] pci 0000:00:0b.0: BAR 0 [io 0x0000-0x003f]1162lb # [ 0.387722] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1163worker1 # [ 0.387588] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1164client # [ 0.406925] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1165worker2 # [ 0.389481] pci 0000:00:0b.0: BAR 1 [mem 0x00000000-0x00000fff]1166worker1 # [ 0.388041] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1167lb # [ 0.388173] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1168worker2 # [ 0.389511] pci 0000:00:0b.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1169client # [ 0.407384] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1170worker1 # [ 0.388222] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1171lb # [ 0.388354] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1172client # [ 0.407568] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1173worker1 # [ 0.388238] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1174lb # [ 0.388370] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1175worker2 # [ 0.389967] pci 0000:00:0c.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1176client # [ 0.407584] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1177worker1 # [ 0.388267] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1178worker2 # [ 0.390151] pci 0000:00:0c.0: BAR 0 [io 0x0000-0x001f]1179lb # [ 0.388400] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1180client # [ 0.407614] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1181worker2 # [ 0.390166] pci 0000:00:0c.0: BAR 1 [mem 0x00000000-0x00000fff]1182worker1 # [ 0.388832] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1183lb # [ 0.388966] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1184client # [ 0.408186] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1185worker2 # [ 0.390196] pci 0000:00:0c.0: BAR 4 [mem 0x00000000-0x00003fff 64bit pref]1186worker1 # [ 0.388843] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1187lb # [ 0.388977] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1188client # [ 0.408197] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1189worker2 # [ 0.390769] pci 0000:00:01.0: ROM [mem 0x10000000-0x1003ffff pref]: assigned1190worker1 # [ 0.388848] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1191lb # [ 0.388982] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1192worker2 # [ 0.390781] pci 0000:00:04.0: ROM [mem 0x10040000-0x1007ffff pref]: assigned1193client # [ 0.408202] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1194worker1 # [ 0.388891] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1195lb # [ 0.389025] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1196client # [ 0.408246] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1197worker2 # [ 0.390786] pci 0000:00:01.0: BAR 4 [mem 0x8000000000-0x8000003fff 64bit pref]: assigned1198worker1 # [ 0.388937] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1199lb # [ 0.389070] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1200client # [ 0.408293] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1201worker2 # [ 0.390830] pci 0000:00:02.0: BAR 4 [mem 0x8000004000-0x8000007fff 64bit pref]: assigned1202worker1 # [ 0.388983] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1203lb # [ 0.389117] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1204client # [ 0.408339] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1205worker2 # [ 0.390876] pci 0000:00:03.0: BAR 4 [mem 0x8000008000-0x800000bfff 64bit pref]: assigned1206worker1 # [ 0.389028] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1207lb # [ 0.389162] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1208client # [ 0.408387] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1209worker2 # [ 0.390922] pci 0000:00:04.0: BAR 4 [mem 0x800000c000-0x800000ffff 64bit pref]: assigned1210worker1 # [ 0.389074] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1211lb # [ 0.389209] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1212client # [ 0.408434] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1213worker2 # [ 0.390968] pci 0000:00:05.0: BAR 4 [mem 0x8000010000-0x8000013fff 64bit pref]: assigned1214worker1 # [ 0.389119] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1215lb # [ 0.389256] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1216client # [ 0.408480] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1217worker2 # [ 0.391015] pci 0000:00:06.0: BAR 4 [mem 0x8000014000-0x8000017fff 64bit pref]: assigned1218worker1 # [ 0.389165] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1219lb # [ 0.389303] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1220client # [ 0.408527] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1221worker2 # [ 0.391062] pci 0000:00:08.0: BAR 4 [mem 0x8000018000-0x800001bfff 64bit pref]: assigned1222worker1 # [ 0.389209] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1223lb # [ 0.389349] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1224client # [ 0.408573] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1225worker2 # [ 0.391108] pci 0000:00:09.0: BAR 4 [mem 0x800001c000-0x800001ffff 64bit pref]: assigned1226worker1 # [ 0.389254] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1227lb # [ 0.389395] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1228client # [ 0.408620] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1229worker2 # [ 0.391154] pci 0000:00:0a.0: BAR 4 [mem 0x8000020000-0x8000023fff 64bit pref]: assigned1230worker1 # [ 0.389319] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1231lb # [ 0.389468] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1232client # [ 0.408688] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1233worker1 # [ 0.389364] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1234worker2 # [ 0.391200] pci 0000:00:0b.0: BAR 4 [mem 0x8000024000-0x8000027fff 64bit pref]: assigned1235lb # [ 0.389514] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1236client # [ 0.408734] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1237worker1 # [ 0.389385] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1238worker2 # [ 0.391269] pci 0000:00:0c.0: BAR 4 [mem 0x8000028000-0x800002bfff 64bit pref]: assigned1239lb # [ 0.389536] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1240client # [ 0.408755] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1241worker1 # [ 0.389406] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1242worker2 # [ 0.391316] pci 0000:00:01.0: BAR 1 [mem 0x10080000-0x10080fff]: assigned1243lb # [ 0.389557] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1244client # [ 0.408776] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1245worker1 # [ 0.389428] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1246worker2 # [ 0.391337] pci 0000:00:02.0: BAR 1 [mem 0x10081000-0x10081fff]: assigned1247lb # [ 0.389578] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1248client # [ 0.408797] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1249worker1 # [ 0.389449] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1250worker2 # [ 0.391359] pci 0000:00:03.0: BAR 1 [mem 0x10082000-0x10082fff]: assigned1251lb # [ 0.389600] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1252client # [ 0.408818] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1253worker1 # [ 0.389470] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1254worker2 # [ 0.391381] pci 0000:00:04.0: BAR 1 [mem 0x10083000-0x10083fff]: assigned1255lb # [ 0.389621] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1256client # [ 0.408840] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1257worker1 # [ 0.389492] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1258worker2 # [ 0.391402] pci 0000:00:05.0: BAR 1 [mem 0x10084000-0x10084fff]: assigned1259lb # [ 0.389643] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1260client # [ 0.408861] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1261worker1 # [ 0.389515] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1262worker2 # [ 0.391424] pci 0000:00:06.0: BAR 1 [mem 0x10085000-0x10085fff]: assigned1263lb # [ 0.389667] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1264client # [ 0.408885] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1265worker1 # [ 0.389537] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1266worker2 # [ 0.391446] pci 0000:00:07.0: BAR 0 [mem 0x10086000-0x10086fff]: assigned1267lb # [ 0.389689] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1268client # [ 0.408907] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1269worker1 # [ 0.389559] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1270worker2 # [ 0.391471] pci 0000:00:08.0: BAR 1 [mem 0x10087000-0x10087fff]: assigned1271lb # [ 0.389711] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1272client # [ 0.408929] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1273worker1 # [ 0.389581] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1274worker2 # [ 0.391493] pci 0000:00:09.0: BAR 1 [mem 0x10088000-0x10088fff]: assigned1275lb # [ 0.389732] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1276worker1 # [ 0.389603] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1277client # [ 0.408951] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1278worker2 # [ 0.391515] pci 0000:00:0a.0: BAR 1 [mem 0x10089000-0x10089fff]: assigned1279lb # [ 0.389754] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1280worker1 # [ 0.389624] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1281client # [ 0.408973] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1282worker2 # [ 0.391538] pci 0000:00:0b.0: BAR 1 [mem 0x1008a000-0x1008afff]: assigned1283lb # [ 0.389775] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1284worker1 # [ 0.389644] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1285client # [ 0.408993] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1286worker2 # [ 0.391560] pci 0000:00:0c.0: BAR 1 [mem 0x1008b000-0x1008bfff]: assigned1287lb # [ 0.389797] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1288worker1 # [ 0.389665] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1289client # [ 0.409014] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1290lb # [ 0.389818] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1291worker2 # [ 0.391586] pci 0000:00:03.0: BAR 0 [io 0x1000-0x107f]: assigned1292worker1 # [ 0.389686] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1293client # [ 0.409035] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1294lb # [ 0.389839] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1295worker2 # [ 0.391608] pci 0000:00:0b.0: BAR 0 [io 0x1080-0x10bf]: assigned1296worker1 # [ 0.389707] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1297worker2 # [ 0.391629] pci 0000:00:01.0: BAR 0 [io 0x10c0-0x10df]: assigned1298lb # [ 0.389859] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1299worker1 # [ 0.389727] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1300worker2 # [ 0.391650] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1301lb # [ 0.389881] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1302worker1 # [ 0.389752] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1303lb # [ 0.389906] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1304worker2 # [ 0.391672] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1305worker1 # [ 0.389761] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1306worker2 # [ 0.391693] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1307lb # [ 0.389915] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1308worker1 # [ 0.389766] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1309worker2 # [ 0.391720] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1310lb # [ 0.389920] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1311worker2 # [ 0.391729] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1312worker2 # [ 0.391735] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1313client # [ 0.449097] pci 0000:00:02.0: BAR 0 [io 0x10e0-0x10ff]: assigned1314client # [ 0.449129] pci 0000:00:04.0: BAR 0 [io 0x1100-0x111f]: assigned1315client # [ 0.449152] pci 0000:00:0c.0: BAR 0 [io 0x1120-0x113f]: assigned1316client # [ 0.449180] pci_bus 0000:00: resource 4 [io 0x0000-0xffff]1317client # [ 0.449190] pci_bus 0000:00: resource 5 [mem 0x10000000-0x3efeffff]1318lb # [ 0.450803] pci 0000:00:07.0: enabling device (0000 -> 0002)1319client # [ 0.449195] pci_bus 0000:00: resource 6 [mem 0x8000000000-0xffffffffff]1320worker1 # [ 0.450658] pci 0000:00:07.0: enabling device (0000 -> 0002)1321client # [ 0.450001] pci 0000:00:07.0: enabling device (0000 -> 0002)1322worker2 # [ 0.452634] pci 0000:00:07.0: enabling device (0000 -> 0002)1323lb # [ 0.476416] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1324worker1 # [ 0.476035] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1325worker2 # [ 0.477949] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1326client # [ 0.483271] virtio-pci 0000:00:01.0: enabling device (0000 -> 0003)1327worker1 # [ 0.478992] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1328worker1 # [ 0.480919] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1329client # [ 0.486078] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1330client # [ 0.488030] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1331worker2 # [ 0.484077] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1332worker2 # [ 0.485998] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1333client # [ 0.491759] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1334lb # [ 0.486289] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)1335lb # [ 0.489774] virtio-pci 0000:00:03.0: enabling device (0000 -> 0003)1336worker1 # [ 0.491073] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1337lb # [ 0.491897] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1338worker1 # [ 0.493091] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1339lb # [ 0.493836] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1340worker2 # [ 0.496021] virtio-pci 0000:00:04.0: enabling device (0000 -> 0003)1341worker2 # [ 0.497971] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1342client # [ 0.502008] virtio-pci 0000:00:05.0: enabling device (0000 -> 0002)1343client # [ 0.504147] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1344lb # [ 0.503969] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1345client # [ 0.507489] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1346lb # [ 0.505927] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1347worker1 # [ 0.503451] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1348worker1 # [ 0.505339] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1349worker1 # [ 0.508744] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1350client # [ 0.513366] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1351client # [ 0.516133] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1352worker2 # [ 0.508010] virtio-pci 0000:00:06.0: enabling device (0000 -> 0002)1353worker2 # [ 0.510159] virtio-pci 0000:00:08.0: enabling device (0000 -> 0002)1354lb # [ 0.516408] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1355worker2 # [ 0.511949] virtio-pci 0000:00:09.0: enabling device (0000 -> 0002)1356worker2 # [ 0.513706] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1357worker1 # [ 0.514962] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1358worker2 # [ 0.515501] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1359worker1 # [ 0.518005] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1360lb # [ 0.519549] virtio-pci 0000:00:0a.0: enabling device (0000 -> 0002)1361lb # [ 0.521343] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1362client # [ 0.526131] virtio-pci 0000:00:0b.0: enabling device (0000 -> 0003)1363client # [ 0.528556] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1364worker1 # [ 0.528279] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1365worker2 # [ 0.529337] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1366lb # [ 0.531603] virtio-pci 0000:00:0c.0: enabling device (0000 -> 0003)1367client # [ 0.542554] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1368lb # [ 0.540499] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1369worker2 # [ 0.544528] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1370worker1 # [ 0.543321] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1371worker1 # [ 0.545762] msm_serial: driver initialized1372worker2 # [ 0.547035] msm_serial: driver initialized1373worker1 # [ 0.545944] SuperH (H)SCI(F) driver initialized1374worker2 # [ 0.547171] SuperH (H)SCI(F) driver initialized1375worker1 # [ 0.545997] STM32 USART driver initialized1376worker2 # [ 0.547224] STM32 USART driver initialized1377lb # [ 0.547207] msm_serial: driver initialized1378client # [ 0.550250] msm_serial: driver initialized1379lb # [ 0.547356] SuperH (H)SCI(F) driver initialized1380client # [ 0.550411] SuperH (H)SCI(F) driver initialized1381lb # [ 0.547412] STM32 USART driver initialized1382client # [ 0.550464] STM32 USART driver initialized1383lb # [ 0.576850] loop: module loaded1384client # [ 0.579767] loop: module loaded1385worker2 # [ 0.577664] loop: module loaded1386lb # [ 0.577012] virtio_blk virtio2: 1/0/0 default/read/poll queues1387client # [ 0.579928] virtio_blk virtio2: 1/0/0 default/read/poll queues1388worker2 # [ 0.577826] virtio_blk virtio2: 1/0/0 default/read/poll queues1389lb # [ 0.577844] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1390client # [ 0.580705] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1391worker2 # [ 0.578572] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1392worker1 # [ 0.578965] loop: module loaded1393worker1 # [ 0.579147] virtio_blk virtio2: 1/0/0 default/read/poll queues1394worker1 # [ 0.579930] virtio_blk virtio2: [vda] 2097152 512-byte logical blocks (1.07 GB/1.00 GiB)1395client # [ 0.585580] megasas: 07.734.00.00-rc11396client # [ 0.586382] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1397worker2 # [ 0.584379] megasas: 07.734.00.00-rc11398client # [ 0.588476] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001399worker2 # [ 0.585061] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1400client # [ 0.588495] Intel/Sharp Extended Query Table at 0x00311401lb # [ 0.590683] megasas: 07.734.00.00-rc11402client # [ 0.598038] Using buffer write method1403lb # [ 0.591463] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1404worker1 # [ 0.590666] megasas: 07.734.00.00-rc11405client # [ 0.598106] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1406lb # [ 0.593602] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001407worker1 # [ 0.591329] physmap-flash 0.flash: physmap platform flash device: [mem 0x00000000-0x03ffffff]1408lb # [ 0.593626] Intel/Sharp Extended Query Table at 0x00311409client # [ 0.600878] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001410lb # [ 0.595116] Using buffer write method1411worker1 # [ 0.593446] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001412worker2 # [ 0.596970] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001413client # [ 0.600900] Intel/Sharp Extended Query Table at 0x00311414worker1 # [ 0.593471] Intel/Sharp Extended Query Table at 0x00311415worker2 # [ 0.597032] Intel/Sharp Extended Query Table at 0x00311416lb # [ 0.595178] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1417worker1 # [ 0.595019] Using buffer write method1418worker2 # [ 0.598486] Using buffer write method1419worker1 # [ 0.595098] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1420worker2 # [ 0.598558] physmap-flash 0.flash: physmap platform flash device: [mem 0x04000000-0x07ffffff]1421lb # [ 0.596833] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001422lb # [ 0.596854] Intel/Sharp Extended Query Table at 0x00311423worker1 # [ 0.596726] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001424lb # [ 0.605954] Using buffer write method1425worker1 # [ 0.596746] Intel/Sharp Extended Query Table at 0x00311426lb # [ 0.605983] Concatenating MTD devices:1427worker2 # [ 0.605008] 0.flash: Found 2 x16 devices at 0x0 in 32-bit bank. Manufacturer ID 0x000000 Chip ID 0x0000001428lb # [ 0.605987] (0): "0.flash"1429lb # [ 0.605991] (1): "0.flash"1430worker2 # [ 0.605033] Intel/Sharp Extended Query Table at 0x00311431lb # [ 0.606002] into device "0.flash"1432worker2 # [ 0.606333] Using buffer write method1433worker2 # [ 0.606359] Concatenating MTD devices:1434worker2 # [ 0.606363] (0): "0.flash"1435worker2 # [ 0.606367] (1): "0.flash"1436worker2 # [ 0.606378] into device "0.flash"1437client # [ 0.614172] Using buffer write method1438worker1 # [ 0.610551] Using buffer write method1439client # [ 0.614207] Concatenating MTD devices:1440worker1 # [ 0.610579] Concatenating MTD devices:1441client # [ 0.614212] (0): "0.flash"1442worker1 # [ 0.610584] (0): "0.flash"1443client # [ 0.614216] (1): "0.flash"1444worker1 # [ 0.610588] (1): "0.flash"1445client # [ 0.614226] into device "0.flash"1446worker1 # [ 0.610595] into device "0.flash"1447lb # [ 0.869994] Freeing initrd memory: 26948K1448client # [ 0.876799] Freeing initrd memory: 26972K1449lb # [ 0.876563] tun: Universal TUN/TAP device driver, 1.61450client # [ 0.883649] tun: Universal TUN/TAP device driver, 1.61451worker1 # [ 0.880189] Freeing initrd memory: 26976K1452lb # [ 0.880385] thunder_xcv, ver 1.01453lb # [ 0.880429] thunder_bgx, ver 1.01454lb # [ 0.880452] nicpf, ver 1.01455lb # [ 0.881107] e1000: Intel(R) PRO/1000 Network Driver1456lb # [ 0.881118] e1000: Copyright (c) 1999-2006 Intel Corporation.1457lb # [ 0.881142] e1000e: Intel(R) PRO/1000 Network Driver1458lb # [ 0.881150] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1459lb # [ 0.881178] igb: Intel(R) Gigabit Ethernet Network Driver1460client # [ 0.887579] thunder_xcv, ver 1.01461lb # [ 0.881185] igb: Copyright (c) 2007-2014 Intel Corporation.1462client # [ 0.887635] thunder_bgx, ver 1.01463client # [ 0.887674] nicpf, ver 1.01464lb # [ 0.881207] igbvf: Intel(R) Gigabit Virtual Function Network Driver1465client # [ 0.888306] e1000: Intel(R) PRO/1000 Network Driver1466lb # [ 0.881214] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1467worker1 # [ 0.887062] tun: Universal TUN/TAP device driver, 1.61468client # [ 0.888316] e1000: Copyright (c) 1999-2006 Intel Corporation.1469lb # [ 0.881367] sky2: driver version 1.301470client # [ 0.888343] e1000e: Intel(R) PRO/1000 Network Driver1471client # [ 0.888351] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1472client # [ 0.888385] igb: Intel(R) Gigabit Ethernet Network Driver1473client # [ 0.888391] igb: Copyright (c) 2007-2014 Intel Corporation.1474client # [ 0.888441] igbvf: Intel(R) Gigabit Virtual Function Network Driver1475worker2 # [ 0.890958] Freeing initrd memory: 26944K1476client # [ 0.888448] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1477lb # [ 0.890380] usbcore: registered new interface driver usb-storage1478client # [ 0.888598] sky2: driver version 1.301479lb # [ 0.890484] usbcore: registered new interface driver usbserial_generic1480lb # [ 0.890498] usbserial: USB Serial support registered for generic1481lb # [ 0.891177] hv_vmbus: registering driver hyperv_keyboard1482lb # [ 0.892523] ehci-pci 0000:00:07.0: EHCI Host Controller1483worker1 # [ 0.890882] thunder_xcv, ver 1.01484worker1 # [ 0.890928] thunder_bgx, ver 1.01485lb # [ 0.892548] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11486client # [ 0.897832] usbcore: registered new interface driver usb-storage1487worker1 # [ 0.890952] nicpf, ver 1.01488lb # [ 0.892832] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001489worker1 # [ 0.891507] e1000: Intel(R) PRO/1000 Network Driver1490client # [ 0.897958] usbcore: registered new interface driver usbserial_generic1491worker1 # [ 0.891514] e1000: Copyright (c) 1999-2006 Intel Corporation.1492client # [ 0.897977] usbserial: USB Serial support registered for generic1493worker1 # [ 0.891538] e1000e: Intel(R) PRO/1000 Network Driver1494client # [ 0.898641] hv_vmbus: registering driver hyperv_keyboard1495worker1 # [ 0.891545] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1496lb # [ 0.898500] rtc-pl031 9010000.pl031: registered as rtc01497worker1 # [ 0.891575] igb: Intel(R) Gigabit Ethernet Network Driver1498client # [ 0.902946] ehci-pci 0000:00:07.0: EHCI Host Controller1499lb # [ 0.898534] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T04:11:15 UTC (1790136675)1500worker1 # [ 0.891581] igb: Copyright (c) 2007-2014 Intel Corporation.1501lb # [ 0.898905] i2c_dev: i2c /dev entries driver1502client # [ 0.902974] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11503worker1 # [ 0.891626] igbvf: Intel(R) Gigabit Virtual Function Network Driver1504client # [ 0.903252] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001505worker1 # [ 0.891632] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1506worker1 # [ 0.891790] sky2: driver version 1.301507worker1 # [ 0.893437] usbcore: registered new interface driver usb-storage1508client # [ 0.905882] rtc-pl031 9010000.pl031: registered as rtc01509worker1 # [ 0.893558] usbcore: registered new interface driver usbserial_generic1510worker1 # [ 0.893581] usbserial: USB Serial support registered for generic1511client # [ 0.905908] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T04:11:15 UTC (1790136675)1512worker1 # [ 0.894807] ehci-pci 0000:00:07.0: EHCI Host Controller1513client # [ 0.906252] i2c_dev: i2c /dev entries driver1514lb # [ 0.904635] sdhci: Secure Digital Host Controller Interface driver1515lb # [ 0.904658] sdhci: Copyright(c) Pierre Ossman1516worker1 # [ 0.894831] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11517lb # [ 0.904961] Synopsys Designware Multimedia Card Interface Driver1518lb # [ 0.905352] sdhci-pltfm: SDHCI platform and OF driver helper1519worker1 # [ 0.895140] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001520worker1 # [ 0.906390] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001521lb # [ 0.908862] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001522worker1 # [ 0.906716] hub 1-0:1.0: USB hub found1523lb # [ 0.909258] hub 1-0:1.0: USB hub found1524worker1 # [ 0.906734] hub 1-0:1.0: 6 ports detected1525worker2 # [ 0.910533] tun: Universal TUN/TAP device driver, 1.61526lb # [ 0.910513] hub 1-0:1.0: 6 ports detected1527client # [ 0.912014] sdhci: Secure Digital Host Controller Interface driver1528worker1 # [ 0.909681] hv_vmbus: registering driver hyperv_keyboard1529client # [ 0.912029] sdhci: Copyright(c) Pierre Ossman1530client # [ 0.912312] Synopsys Designware Multimedia Card Interface Driver1531client # [ 0.912711] sdhci-pltfm: SDHCI platform and OF driver helper1532client # [ 0.913106] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001533lb # [ 0.912411] hid: raw HID events driver (C) Jiri Kosina1534worker1 # [ 0.911399] rtc-pl031 9010000.pl031: registered as rtc01535client # [ 0.916924] hub 1-0:1.0: USB hub found1536lb # [ 0.912707] usbcore: registered new interface driver usbhid1537worker1 # [ 0.911429] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T04:11:15 UTC (1790136675)1538lb # [ 0.912717] usbhid: USB HID core driver1539client # [ 0.917464] hub 1-0:1.0: 6 ports detected1540worker2 # [ 0.914376] thunder_xcv, ver 1.01541worker1 # [ 0.911778] i2c_dev: i2c /dev entries driver1542worker2 # [ 0.914422] thunder_bgx, ver 1.01543worker2 # [ 0.914444] nicpf, ver 1.01544worker2 # [ 0.915081] e1000: Intel(R) PRO/1000 Network Driver1545worker2 # [ 0.915092] e1000: Copyright (c) 1999-2006 Intel Corporation.1546worker2 # [ 0.915121] e1000e: Intel(R) PRO/1000 Network Driver1547lb # [ 0.915740] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1548client # [ 0.919703] hid: raw HID events driver (C) Jiri Kosina1549worker2 # [ 0.915128] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.1550lb # [ 0.917445] drop_monitor: Initializing network drop monitor service1551client # [ 0.919989] usbcore: registered new interface driver usbhid1552worker2 # [ 0.915172] igb: Intel(R) Gigabit Ethernet Network Driver1553worker1 # [ 0.917274] sdhci: Secure Digital Host Controller Interface driver1554client # [ 0.920000] usbhid: USB HID core driver1555lb # [ 0.917632] NET: Registered PF_INET6 protocol family1556worker1 # [ 0.917289] sdhci: Copyright(c) Pierre Ossman1557worker2 # [ 0.915181] igb: Copyright (c) 2007-2014 Intel Corporation.1558worker1 # [ 0.917560] Synopsys Designware Multimedia Card Interface Driver1559lb # [ 0.920763] Segment Routing with IPv61560worker2 # [ 0.915222] igbvf: Intel(R) Gigabit Virtual Function Network Driver1561lb # [ 0.920797] In-situ OAM (IOAM) with IPv61562worker1 # [ 0.917953] sdhci-pltfm: SDHCI platform and OF driver helper1563worker2 # [ 0.915228] igbvf: Copyright (c) 2009 - 2012 Intel Corporation.1564lb # [ 0.920828] NET: Registered PF_PACKET protocol family1565worker2 # [ 0.915407] sky2: driver version 1.301566client # [ 0.922991] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1567client # [ 0.924574] drop_monitor: Initializing network drop monitor service1568client # [ 0.924698] NET: Registered PF_INET6 protocol family1569worker1 # [ 0.922382] hid: raw HID events driver (C) Jiri Kosina1570client # [ 0.927855] Segment Routing with IPv61571worker1 # [ 0.922664] usbcore: registered new interface driver usbhid1572client # [ 0.927889] In-situ OAM (IOAM) with IPv61573lb # [ 0.922489] 9pnet: Installing 9P2000 support1574worker1 # [ 0.922674] usbhid: USB HID core driver1575client # [ 0.927921] NET: Registered PF_PACKET protocol family1576lb # [ 0.922538] Key type dns_resolver registered1577worker2 # [ 0.924448] usbcore: registered new interface driver usb-storage1578worker2 # [ 0.924559] usbcore: registered new interface driver usbserial_generic1579worker1 # [ 0.925729] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1580worker2 # [ 0.924576] usbserial: USB Serial support registered for generic1581worker2 # [ 0.925201] hv_vmbus: registering driver hyperv_keyboard1582worker2 # [ 0.926536] ehci-pci 0000:00:07.0: EHCI Host Controller1583lb # [ 0.929558] registered taskstats version 11584worker1 # [ 0.928621] drop_monitor: Initializing network drop monitor service1585client # [ 0.929627] 9pnet: Installing 9P2000 support1586worker2 # [ 0.926571] ehci-pci 0000:00:07.0: new USB bus registered, assigned bus number 11587lb # [ 0.929704] Loading compiled-in X.509 certificates1588client # [ 0.929672] Key type dns_resolver registered1589worker1 # [ 0.928793] NET: Registered PF_INET6 protocol family1590worker2 # [ 0.926805] ehci-pci 0000:00:07.0: irq 16, io mem 0x100860001591worker1 # [ 0.930990] Segment Routing with IPv61592worker1 # [ 0.931021] In-situ OAM (IOAM) with IPv61593worker1 # [ 0.931051] NET: Registered PF_PACKET protocol family1594worker2 # [ 0.932440] rtc-pl031 9010000.pl031: registered as rtc01595client # [ 0.936882] registered taskstats version 11596worker2 # [ 0.932472] rtc-pl031 9010000.pl031: setting system clock to 2026-09-23T04:11:15 UTC (1790136675)1597worker2 # [ 0.932841] i2c_dev: i2c /dev entries driver1598worker1 # [ 0.932723] 9pnet: Installing 9P2000 support1599worker1 # [ 0.932811] Key type dns_resolver registered1600worker2 # [ 0.936043] ehci-pci 0000:00:07.0: USB 2.0 started, EHCI 1.001601worker2 # [ 0.936340] hub 1-0:1.0: USB hub found1602worker2 # [ 0.936380] hub 1-0:1.0: 6 ports detected1603client # [ 0.937593] Loading compiled-in X.509 certificates1604lb # [ 0.938871] Demotion targets for Node 0: null1605lb # [ 0.938987] Key type .fscrypt registered1606lb # [ 0.938993] Key type fscrypt-provisioning registered1607lb # [ 0.939091] ima: No TPM chip found, activating TPM-bypass!1608lb # [ 0.939111] ima: Allocated hash algorithm: sha11609lb # [ 0.939141] ima: No architecture policies found1610worker1 # [ 0.939948] registered taskstats version 11611lb # [ 0.943372] input: gpio-keys as /devices/platform/gpio-keys/input/input01612worker2 # [ 0.941041] sdhci: Secure Digital Host Controller Interface driver1613worker1 # [ 0.940104] Loading compiled-in X.509 certificates1614worker2 # [ 0.941062] sdhci: Copyright(c) Pierre Ossman1615client # [ 0.946415] Demotion targets for Node 0: null1616worker2 # [ 0.941323] Synopsys Designware Multimedia Card Interface Driver1617client # [ 0.946542] Key type .fscrypt registered1618client # [ 0.946548] Key type fscrypt-provisioning registered1619worker2 # [ 0.941847] sdhci-pltfm: SDHCI platform and OF driver helper1620client # [ 0.946651] ima: No TPM chip found, activating TPM-bypass!1621worker2 # [ 0.943616] hid: raw HID events driver (C) Jiri Kosina1622client # [ 0.946674] ima: Allocated hash algorithm: sha11623client # [ 0.946695] ima: No architecture policies found1624worker2 # [ 0.947256] usbcore: registered new interface driver usbhid1625worker2 # [ 0.947270] usbhid: USB HID core driver1626client # [ 0.950745] input: gpio-keys as /devices/platform/gpio-keys/input/input01627worker1 # [ 0.949341] Demotion targets for Node 0: null1628worker1 # [ 0.949470] Key type .fscrypt registered1629worker2 # [ 0.949716] hw perfevents: enabled with armv8_pmuv3 PMU driver, 11 (0,800003ff) counters available1630worker1 # [ 0.949476] Key type fscrypt-provisioning registered1631worker2 # [ 0.951432] drop_monitor: Initializing network drop monitor service1632worker1 # [ 0.949574] ima: No TPM chip found, activating TPM-bypass!1633worker2 # [ 0.951642] NET: Registered PF_INET6 protocol family1634worker1 # [ 0.949593] ima: Allocated hash algorithm: sha11635worker1 # [ 0.949617] ima: No architecture policies found1636worker2 # [ 0.954870] Segment Routing with IPv61637worker2 # [ 0.954902] In-situ OAM (IOAM) with IPv61638worker1 # [ 0.953851] input: gpio-keys as /devices/platform/gpio-keys/input/input01639worker2 # [ 0.954953] NET: Registered PF_PACKET protocol family1640worker2 # [ 0.956698] 9pnet: Installing 9P2000 support1641worker2 # [ 0.956757] Key type dns_resolver registered1642lb # [ 0.961759] clk: Disabling unused clocks1643lb # [ 0.961782] PM: genpd: Disabling unused power domains1644lb # [ 0.966680] Freeing unused kernel memory: 4736K1645lb # [ 0.966895] Run /init as init process1646worker2 # [ 0.964665] registered taskstats version 11647worker2 # [ 0.964845] Loading compiled-in X.509 certificates1648client # [ 0.970023] clk: Disabling unused clocks1649client # [ 0.970049] PM: genpd: Disabling unused power domains1650client # [ 0.974824] Freeing unused kernel memory: 4736K1651client # [ 0.975037] Run /init as init process1652worker2 # [ 0.974577] Demotion targets for Node 0: null1653worker1 # [ 0.972819] clk: Disabling unused clocks1654worker2 # [ 0.974696] Key type .fscrypt registered1655worker1 # [ 0.972844] PM: genpd: Disabling unused power domains1656worker2 # [ 0.974702] Key type fscrypt-provisioning registered1657worker2 # [ 0.974820] ima: No TPM chip found, activating TPM-bypass!1658worker2 # [ 0.974846] ima: Allocated hash algorithm: sha11659worker2 # [ 0.974875] ima: No architecture policies found1660worker2 # [ 0.979139] input: gpio-keys as /devices/platform/gpio-keys/input/input01661worker1 # [ 0.977583] Freeing unused kernel memory: 4736K1662worker1 # [ 0.977808] Run /init as init process1663lb # [ 0.984623] systemd[1]: Successfully made /usr/ read-only.1664client # [ 0.991424] systemd[1]: Successfully made /usr/ read-only.1665worker1 # [ 0.996435] systemd[1]: Successfully made /usr/ read-only.1666worker2 # [ 0.998260] clk: Disabling unused clocks1667worker2 # [ 0.998285] PM: genpd: Disabling unused power domains1668worker2 # [ 1.002839] Freeing unused kernel memory: 4736K1669worker2 # [ 1.003075] Run /init as init process1670worker2 # [ 1.018347] systemd[1]: Successfully made /usr/ read-only.1671worker1 # [ 1.154186] usb 1-1: new high-speed USB device number 2 using ehci-pci1672lb # [ 1.162251] usb 1-1: new high-speed USB device number 2 using ehci-pci1673client # [ 1.165246] usb 1-1: new high-speed USB device number 2 using ehci-pci1674worker2 # [ 1.183941] usb 1-1: new high-speed USB device number 2 using ehci-pci1675worker1 # [ 1.309204] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11676lb # [ 1.315156] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11677client # [ 1.318337] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11678lb # [ 1.322257] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1679client # [ 1.326714] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1680lb # [ 1.335111] systemd[1]: Detected virtualization qemu.1681client # [ 1.339514] systemd[1]: Detected virtualization qemu.1682lb # [ 1.337344] systemd[1]: Detected architecture arm64.1683lb # [ 1.339481] systemd[1]: Running in initrd.1684client # [ 1.341748] systemd[1]: Detected architecture arm64.1685client # [ 1.343731] systemd[1]: Running in initrd.1686worker2 # [ 1.338758] input: QEMU QEMU USB Keyboard as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-1/1-1:1.0/0003:0627:0001.0001/input/input11687worker1 # [ 1.331551] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1688lb # [ 1.343145] systemd[1]: Initializing machine ID from random generator.1689worker1 # [ 1.343957] systemd[1]: Detected virtualization qemu.1690client # [ 1.347126] systemd[1]: Initializing machine ID from random generator.1691lb # [ 1.346545] systemd[1]: Hostname set to <lb>.1692worker1 # [ 1.346176] systemd[1]: Detected architecture arm64.1693client # [ 1.350389] systemd[1]: Hostname set to <client>.1694worker1 # [ 1.348139] systemd[1]: Running in initrd.1695worker1 # [ 1.351321] systemd[1]: Initializing machine ID from random generator.1696worker1 # [ 1.354659] systemd[1]: Hostname set to <worker1>.1697worker2 # [ 1.353562] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1698worker2 # [ 1.365986] systemd[1]: Detected virtualization qemu.1699worker2 # [ 1.368343] systemd[1]: Detected architecture arm64.1700worker2 # [ 1.370363] systemd[1]: Running in initrd.1701worker2 # [ 1.373554] systemd[1]: Initializing machine ID from random generator.1702worker2 # [ 1.376618] systemd[1]: Hostname set to <worker2>.1703client # [ 1.401641] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01704worker1 # [ 1.398477] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01705lb # [ 1.402549] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01706worker2 # [ 1.428440] hid-generic 0003:0627:0001.0001: input,hidraw0: USB HID v1.11 Keyboard [QEMU QEMU USB Keyboard] on usb-0000:00:07.0-1/input01707client # [ 1.525118] usb 1-2: new high-speed USB device number 3 using ehci-pci1708worker1 # [ 1.522175] usb 1-2: new high-speed USB device number 3 using ehci-pci1709lb # [ 1.526146] usb 1-2: new high-speed USB device number 3 using ehci-pci1710worker2 # [ 1.552060] usb 1-2: new high-speed USB device number 3 using ehci-pci1711lb # [ 1.676765] systemd[1]: bpf-restrict-fs: LSM BPF program attached1712client # [ 1.683784] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21713worker1 # [ 1.680849] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21714client # [ 1.693385] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01715worker1 # [ 1.690433] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01716lb # [ 1.692074] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21717lb # [ 1.698573] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01718client # [ 1.707722] systemd[1]: bpf-restrict-fs: LSM BPF program attached1719worker2 # [ 1.709785] input: QEMU QEMU USB Tablet as /devices/platform/4010000000.pcie/pci0000:00/0000:00:07.0/usb1/1-2/1-2:1.0/0003:0627:0001.0002/input/input21720worker2 # [ 1.715843] hid-generic 0003:0627:0001.0002: input,hidraw1: USB HID v0.01 Mouse [QEMU QEMU USB Tablet] on usb-0000:00:07.0-2/input01721worker1 # [ 1.734969] systemd[1]: bpf-restrict-fs: LSM BPF program attached1722worker2 # [ 1.754113] systemd[1]: bpf-restrict-fs: LSM BPF program attached1723lb # [ 1.796412] systemd[1]: Queued start job for default target Initrd Default Target.1724client # [ 1.804203] systemd[1]: Queued start job for default target Initrd Default Target.1725lb # [ 1.804710] systemd[1]: Created slice Slice /system/modprobe.1726lb # [ 1.805925] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1727lb # [ 1.807237] systemd[1]: Expecting device /dev/disk/by-label/nixos...1728lb # [ 1.808244] systemd[1]: Reached target Path Units.1729lb # [ 1.808991] systemd[1]: Reached target Slice Units.1730lb # [ 1.809744] systemd[1]: Reached target Swaps.1731lb # [ 1.810479] systemd[1]: Reached target Timer Units.1732lb # [ 1.811446] systemd[1]: Listening on D-Bus System Message Bus Socket.1733lb # [ 1.812589] systemd[1]: Listening on Journal Socket (/dev/log).1734client # [ 1.815731] systemd[1]: Created slice Slice /system/modprobe.1735lb # [ 1.813636] systemd[1]: Listening on Journal Sockets.1736client # [ 1.816980] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1737client # [ 1.818369] systemd[1]: Expecting device /dev/disk/by-label/nixos...1738lb # [ 1.814636] systemd[1]: Listening on udev Control Socket.1739client # [ 1.819400] systemd[1]: Reached target Path Units.1740lb # [ 1.814799] systemd[1]: Listening on udev Kernel Socket.1741lb # [ 1.814832] systemd[1]: Reached target Socket Units.1742client # [ 1.820303] systemd[1]: Reached target Slice Units.1743client # [ 1.821139] systemd[1]: Reached target Swaps.1744lb # [ 1.819134] systemd[1]: Starting Create List of Static Device Nodes...1745client # [ 1.821874] systemd[1]: Reached target Timer Units.1746client # [ 1.822914] systemd[1]: Listening on D-Bus System Message Bus Socket.1747lb # [ 1.820221] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1748client # [ 1.824141] systemd[1]: Listening on Journal Socket (/dev/log).1749client # [ 1.825325] systemd[1]: Listening on Journal Sockets.1750client # [ 1.825496] systemd[1]: Listening on udev Control Socket.1751client # [ 1.825656] systemd[1]: Listening on udev Kernel Socket.1752client # [ 1.825691] systemd[1]: Reached target Socket Units.1753lb # [ 1.827518] systemd[1]: Mounting Kernel Configuration File System...1754client # [ 1.830732] systemd[1]: Starting Create List of Static Device Nodes...1755client # [ 1.831852] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1756worker1 # [ 1.837019] systemd[1]: Queued start job for default target Initrd Default Target.1757lb # [ 1.839467] systemd[1]: Starting Journal Service...1758client # [ 1.841326] systemd[1]: Mounting Kernel Configuration File System...1759client # [ 1.849408] systemd[1]: Starting Journal Service...1760worker1 # [ 1.849858] systemd[1]: Created slice Slice /system/modprobe.1761worker2 # [ 1.852706] systemd[1]: Queued start job for default target Initrd Default Target.1762worker1 # [ 1.851200] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1763worker1 # [ 1.852493] systemd[1]: Expecting device /dev/disk/by-label/nixos...1764worker1 # [ 1.853516] systemd[1]: Reached target Path Units.1765worker1 # [ 1.854427] systemd[1]: Reached target Slice Units.1766worker1 # [ 1.855234] systemd[1]: Reached target Swaps.1767worker1 # [ 1.855964] systemd[1]: Reached target Timer Units.1768worker1 # [ 1.856969] systemd[1]: Listening on D-Bus System Message Bus Socket.1769worker1 # [ 1.858197] systemd[1]: Listening on Journal Socket (/dev/log).1770worker1 # [ 1.859329] systemd[1]: Listening on Journal Sockets.1771lb # [ 1.861186] systemd[1]: Starting Load Kernel Modules...1772worker1 # [ 1.860451] systemd[1]: Listening on udev Control Socket.1773worker1 # [ 1.861544] systemd[1]: Listening on udev Kernel Socket.1774lb # [ 1.862360] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1775worker2 # [ 1.863430] systemd[1]: Created slice Slice /system/modprobe.1776worker2 # [ 1.864759] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1777worker1 # [ 1.862479] systemd[1]: Reached target Socket Units.1778worker2 # [ 1.865973] systemd[1]: Expecting device /dev/disk/by-label/nixos...1779worker2 # [ 1.866947] systemd[1]: Reached target Path Units.1780worker1 # [ 1.865293] systemd[1]: Starting Create List of Static Device Nodes...1781worker2 # [ 1.867728] systemd[1]: Reached target Slice Units.1782worker2 # [ 1.868559] systemd[1]: Reached target Swaps.1783worker1 # [ 1.866529] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1784worker2 # [ 1.869239] systemd[1]: Reached target Timer Units.1785lb # [ 1.867663] systemd[1]: Starting Coldplug All udev Devices...1786worker2 # [ 1.870181] systemd[1]: Listening on D-Bus System Message Bus Socket.1787worker2 # [ 1.871311] systemd[1]: Listening on Journal Socket (/dev/log).1788worker2 # [ 1.872441] systemd[1]: Listening on Journal Sockets.1789worker2 # [ 1.873346] systemd[1]: Listening on udev Control Socket.1790worker2 # [ 1.874289] systemd[1]: Listening on udev Kernel Socket.1791worker2 # [ 1.875224] systemd[1]: Reached target Socket Units.1792worker2 # [ 1.877838] systemd[1]: Starting Create List of Static Device Nodes...1793worker1 # [ 1.878271] systemd[1]: Mounting Kernel Configuration File System...1794client # [ 1.881289] systemd[1]: Starting Load Kernel Modules...1795worker2 # [ 1.877921] systemd[1]: Load Kernel Module configfs skipped, unmet condition check ConditionKernelModuleLoaded=!configfs1796client # [ 1.881405] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1797worker1 # [ 1.882540] systemd[1]: Starting Journal Service...1798lb # [ 1.887345] systemd[1]: Finished Create List of Static Device Nodes.1799worker2 # [ 1.888089] systemd[1]: Mounting Kernel Configuration File System...1800client # [ 1.893229] systemd[1]: Starting Coldplug All udev Devices...1801worker2 # [ 1.896156] systemd[1]: Starting Journal Service...1802client # [ 1.900965] systemd-journald[72]: Collecting audit messages is disabled.1803worker1 # [ 1.898269] systemd[1]: Starting Load Kernel Modules...1804worker1 # [ 1.899138] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1805lb # [ 1.903468] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1806lb # [ 1.904065] systemd[1]: Mounted Kernel Configuration File System.1807client # [ 1.905870] systemd[1]: Finished Create List of Static Device Nodes.1808client # [ 1.917327] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1809client # [ 1.917821] systemd[1]: Mounted Kernel Configuration File System.1810worker1 # [ 1.916462] systemd[1]: Starting Coldplug All udev Devices...1811lb # [ 1.919414] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1812worker2 # [ 1.921211] systemd[1]: Starting Load Kernel Modules...1813worker2 # [ 1.922090] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1814worker1 # [ 1.923120] systemd[1]: Finished Create List of Static Device Nodes.1815lb # [ 1.934484] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1816worker2 # [ 1.942566] systemd-journald[72]: Collecting audit messages is disabled.1817worker2 # [ 1.943605] systemd[1]: Starting Coldplug All udev Devices...1818lb # [ 1.943689] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01819lb # [ 1.943927] [drm] features: -virgl +edid -resource_blob -host_visible1820lb # [ 1.943935] [drm] features: -context_init1821lb # [ 1.944699] [drm] number of scanouts: 11822lb # [ 1.944716] [drm] number of cap sets: 01823worker1 # [ 1.946297] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1824worker1 # [ 1.950810] systemd[1]: Mounted Kernel Configuration File System.1825lb # [ 1.960147] systemd-journald[72]: Collecting audit messages is disabled.1826client # [ 1.965846] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1827client # [ 1.969429] systemd[1]: Starting Create Static Device Nodes in /dev...1828lb # [ 1.967052] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1829worker2 # [ 1.967989] systemd[1]: Finished Create List of Static Device Nodes.1830lb # [ 1.970461] systemd[1]: Starting Create Static Device Nodes in /dev...1831worker2 # [ 1.969131] systemd[1]: Mounted Kernel Configuration File System.1832worker1 # [ 1.970544] systemd-journald[72]: Collecting audit messages is disabled.1833worker2 # [ 1.980325] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1834client # [ 1.985919] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1835lb # [ 1.978606] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1836lb # [ 1.978628] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01837worker2 # [ 1.989444] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1838client # [ 1.997219] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1839worker1 # [ 1.987876] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1840worker1 # [ 1.988764] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1841worker1 # [ 1.990414] systemd[1]: Starting Create Static Device Nodes in /dev...1842client # [ 2.005026] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01843worker2 # [ 2.003974] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1844lb # [ 1.998394] Console: switching to colour frame buffer device 160x501845worker2 # [ 2.013680] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01846worker2 # [ 2.013928] [drm] features: -virgl +edid -resource_blob -host_visible1847worker2 # [ 2.013940] [drm] features: -context_init1848worker2 # [ 2.014706] [drm] number of scanouts: 11849worker2 # [ 2.014724] [drm] number of cap sets: 01850worker1 # [ 2.014209] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1851worker1 # [ 2.020645] [drm] pci: virtio-gpu-pci detected at 0000:00:05.01852client # [ 2.026444] [drm] features: -virgl +edid -resource_blob -host_visible1853worker1 # [ 2.020906] [drm] features: -virgl +edid -resource_blob -host_visible1854client # [ 2.026464] [drm] features: -context_init1855worker1 # [ 2.020916] [drm] features: -context_init1856client # [ 2.027253] [drm] number of scanouts: 11857worker1 # [ 2.021652] [drm] number of scanouts: 11858worker1 # [ 2.021669] [drm] number of cap sets: 01859client # [ 2.027271] [drm] number of cap sets: 01860lb # [ 2.026829] systemd[1]: Finished Create Static Device Nodes in /dev.1861lb # [ 2.027021] systemd[1]: Reached target Preparation for Local File Systems.1862lb # [ 2.027048] systemd[1]: Reached target Local File Systems.1863lb # [ 2.031437] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1864client # [ 2.033505] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1865client # [ 2.033533] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01866lb # [ 2.039425] systemd[1]: Starting Rule-based Manager for Device Events and Files...1867worker2 # [ 2.040742] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1868worker2 # [ 2.043543] systemd[1]: Starting Create Static Device Nodes in /dev...1869worker1 # [ 2.046750] systemd[1]: Finished Create Static Device Nodes in /dev.1870worker1 # [ 2.046973] systemd[1]: Reached target Preparation for Local File Systems.1871worker1 # [ 2.047004] systemd[1]: Reached target Local File Systems.1872client # [ 2.053002] systemd[1]: Finished Create Static Device Nodes in /dev.1873client # [ 2.053277] systemd[1]: Reached target Preparation for Local File Systems.1874client # [ 2.053310] systemd[1]: Reached target Local File Systems.1875worker2 # [ 2.048337] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1876lb # [ 2.054287] systemd[1]: Finished Load Kernel Modules.1877worker2 # [ 2.048359] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01878worker1 # [ 2.051184] virtio-pci 0000:00:05.0: [drm] Registered 1 planes with drm panic1879worker1 # [ 2.051202] [drm] Initialized virtio_gpu 0.1.0 for 0000:00:05.0 on minor 01880client # [ 2.061572] systemd[1]: Starting Rule-based Manager for Device Events and Files...1881lb # [ 2.062543] systemd[1]: Starting Apply Kernel Variables...1882worker1 # [ 2.059540] systemd[1]: Starting Rule-based Manager for Device Events and Files...1883client # [ 2.069360] Console: switching to colour frame buffer device 160x501884client # [ 2.075901] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1885worker2 # [ 2.072207] Console: switching to colour frame buffer device 160x501886worker2 # [ 2.096591] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1887client # [ 2.089618] systemd-modules-load[73]: Inserted module 'dm_mod'1888client # [ 2.103852] systemd[1]: Finished Load Kernel Modules.1889client # [ 2.106042] systemd[1]: Started Journal Service.1890worker1 # [ 2.102132] Console: switching to colour frame buffer device 160x501891worker1 # [ 2.102834] virtio-pci 0000:00:05.0: [drm] fb0: virtio_gpudrmfb frame buffer device1892client # [ 2.096415] systemd-modules-load[73]: Module 'virtio_balloon' is built in1893client # [ 2.097696] systemd-modules-load[73]: Module 'virtio_console' is built in1894client # [ 2.098811] systemd-modules-load[73]: Inserted module 'virtio_gpu'1895worker2 # [ 2.108597] systemd[1]: Finished Load Kernel Modules.1896client # [ 2.099883] systemd-modules-load[73]: Module 'virtio_rng' is built in1897worker2 # [ 2.112868] systemd[1]: Starting Apply Kernel Variables...1898lb # [ 2.119022] systemd[1]: Finished Apply Kernel Variables.1899worker1 # [ 2.118882] systemd[1]: Finished Load Kernel Modules.1900worker2 # [ 2.128326] systemd[1]: Finished Create Static Device Nodes in /dev.1901worker2 # [ 2.128607] systemd[1]: Reached target Preparation for Local File Systems.1902client # [ 2.120511] systemd[1]: Starting Apply Kernel Variables...1903worker2 # [ 2.128648] systemd[1]: Reached target Local File Systems.1904worker1 # [ 2.126505] systemd[1]: Starting Apply Kernel Variables...1905lb # [ 2.142806] systemd[1]: Started Journal Service.1906client # [ 2.135915] systemd[1]: Starting Create System Files and Directories...1907worker2 # [ 2.144331] systemd[1]: Starting Rule-based Manager for Device Events and Files...1908lb # [ 2.140342] systemd-modules-load[73]: Inserted module 'dm_mod'1909worker2 # [ 2.146653] systemd-modules-load[74]: Inserted module 'dm_mod'1910lb # [ 2.144263] systemd-modules-load[73]: Module 'virtio_balloon' is built in1911lb # [ 2.145420] systemd-modules-load[73]: Module 'virtio_console' is built in1912worker2 # [ 2.159784] systemd[1]: Started Journal Service.1913lb # [ 2.146498] systemd-modules-load[73]: Inserted module 'virtio_gpu'1914lb # [ 2.147499] systemd-modules-load[73]: Module 'virtio_rng' is built in1915worker2 # [ 2.151826] systemd-modules-load[74]: Module 'virtio_balloon' is built in1916worker2 # [ 2.153334] systemd-modules-load[74]: Module 'virtio_console' is built in1917worker2 # [ 2.157104] systemd-modules-load[74]: Inserted module 'virtio_gpu'1918lb # [ 2.157734] systemd[1]: Starting Create System Files and Directories...1919worker2 # [ 2.164408] systemd-modules-load[74]: Module 'virtio_rng' is built in1920worker2 # [ 2.168343] systemd[1]: Finished Apply Kernel Variables.1921lb # [ 2.165724] systemd-udevd[79]: Using default interface naming scheme 'v261'.1922client # [ 2.171183] systemd-udevd[79]: Using default interface naming scheme 'v261'.1923worker1 # [ 2.184640] systemd[1]: Started Journal Service.1924worker2 # [ 2.177241] systemd[1]: Starting Create System Files and Directories...1925client # [ 2.184353] systemd[1]: Finished Create System Files and Directories.1926worker1 # [ 2.180366] systemd-modules-load[73]: Inserted module 'dm_mod'1927worker1 # [ 2.181635] systemd-modules-load[73]: Module 'virtio_balloon' is built in1928worker1 # [ 2.184313] systemd-modules-load[73]: Module 'virtio_console' is built in1929lb # [ 2.189031] systemd[1]: Finished Create System Files and Directories.1930worker1 # [ 2.188396] systemd-modules-load[73]: Inserted module 'virtio_gpu'1931worker1 # [ 2.190529] systemd-modules-load[73]: Module 'virtio_rng' is built in1932client # [ 2.197379] systemd[1]: Finished Apply Kernel Variables.1933worker1 # [ 2.197049] systemd-udevd[79]: Using default interface naming scheme 'v261'.1934worker1 # [ 2.199550] systemd[1]: Starting Create System Files and Directories...1935worker1 # [ 2.202316] systemd[1]: Finished Apply Kernel Variables.1936lb # [ 2.213715] systemd[1]: Started Rule-based Manager for Device Events and Files.1937worker2 # [ 2.217029] systemd[1]: Finished Create System Files and Directories.1938client # [ 2.221992] systemd[1]: Started Rule-based Manager for Device Events and Files.1939worker2 # [ 2.230553] systemd-udevd[80]: Using default interface naming scheme 'v261'.1940worker1 # [ 2.242221] systemd[1]: Finished Create System Files and Directories.1941worker1 # [ 2.244868] systemd[1]: Started Rule-based Manager for Device Events and Files.1942worker2 # [ 2.267259] systemd[1]: Started Rule-based Manager for Device Events and Files.1943lb # [ 2.278001] systemd[1]: Starting Virtual Console Setup...1944client # [ 2.288113] systemd[1]: Starting Virtual Console Setup...1945worker1 # [ 2.304934] systemd[1]: Starting Virtual Console Setup...1946lb # [ 2.332493] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1947worker2 # [ 2.336860] systemd[1]: Starting Virtual Console Setup...1948client # [ 2.338087] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1949lb # [ 2.335924] systemd[1]: Finished Virtual Console Setup.1950client # [ 2.339993] systemd[1]: Finished Virtual Console Setup.1951worker1 # [ 2.373462] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1952worker1 # [ 2.376248] systemd[1]: Finished Virtual Console Setup.1953worker2 # [ 2.396464] systemd-vconsole-setup[101]: Configuration of first virtual console was skipped, ignoring remaining ones.1954worker2 # [ 2.399996] systemd[1]: Finished Virtual Console Setup.1955lb # [ 3.025809] systemd[1]: Finished Coldplug All udev Devices.1956lb # [ 3.026779] systemd[1]: Reached target System Initialization.1957lb # [ 3.027609] systemd[1]: Reached target Basic System.1958client # [ 3.039521] systemd[1]: Finished Coldplug All udev Devices.1959client # [ 3.044087] systemd[1]: Reached target System Initialization.1960client # [ 3.044984] systemd[1]: Reached target Basic System.1961worker1 # [ 3.053495] systemd[1]: Finished Coldplug All udev Devices.1962worker1 # [ 3.054506] systemd[1]: Reached target System Initialization.1963worker1 # [ 3.055314] systemd[1]: Reached target Basic System.1964worker2 # [ 3.141206] systemd[1]: Finished Coldplug All udev Devices.1965worker2 # [ 3.142209] systemd[1]: Reached target System Initialization.1966worker2 # [ 3.144185] systemd[1]: Reached target Basic System.1967lb # [ 3.187973] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1968client # [ 3.235150] (udev-worker)[92]: Network interface NamePolicy= disabled on kernel command line.1969lb # [ 3.230897] (udev-worker)[95]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1970lb # [ 3.234276] (udev-worker)[95]: Network interface NamePolicy= disabled on kernel command line.1971worker1 # [ 3.243574] (udev-worker)[97]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1972client # [ 3.249329] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1973client # [ 3.254731] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1974worker1 # [ 3.257873] (udev-worker)[93]: Network interface NamePolicy= disabled on kernel command line.1975worker1 # [ 3.265757] (udev-worker)[97]: Network interface NamePolicy= disabled on kernel command line.1976lb # [ 3.308614] systemd[1]: Found device /dev/disk/by-label/nixos.1977worker2 # [ 3.316064] (udev-worker)[91]: Network interface NamePolicy= disabled on kernel command line.1978lb # [ 3.315378] systemd[1]: Reached target Initrd Root Device.1979lb # [ 3.323931] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1980worker1 # [ 3.328803] systemd[1]: Found device /dev/disk/by-label/nixos.1981worker1 # [ 3.332639] systemd[1]: Reached target Initrd Root Device.1982worker2 # [ 3.335031] (udev-worker)[90]: eth1: Config file /etc/systemd/network/40-eth1.link is applied to device based on potentially unpredictable interface name.1983worker1 # [ 3.335474] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1984worker2 # [ 3.339960] (udev-worker)[90]: Network interface NamePolicy= disabled on kernel command line.1985client # [ 3.348130] systemd[1]: Found device /dev/disk/by-label/nixos.1986client # [ 3.353590] systemd[1]: Reached target Initrd Root Device.1987client # [ 3.357424] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1988lb # [ 3.372273] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1989lb # [ 3.378680] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1990lb # [ 3.391759] systemd[1]: Mounting /sysroot...1991worker1 # [ 3.391530] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1992worker2 # [ 3.399055] systemd[1]: Found device /dev/disk/by-label/nixos.1993worker2 # [ 3.400810] systemd[1]: Reached target Initrd Root Device.1994worker1 # [ 3.397157] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1995client # [ 3.404102] systemd-fsck[110]: nixos: clean, 12/65536 files, 13019/262144 blocks1996worker2 # [ 3.405104] systemd[1]: Starting File System Check on /dev/disk/by-label/nixos...1997client # [ 3.412189] systemd[1]: Finished File System Check on /dev/disk/by-label/nixos.1998worker1 # [ 3.408441] systemd[1]: Mounting /sysroot...1999client # [ 3.416138] systemd[1]: Mounting /sysroot...2000lb # [ 3.444489] EXT4-fs (vda): mounted filesystem 8ad90e9a-c614-4d36-9ae9-6371c810d3cf r/w with ordered data mode. Quota mode: none.
loading 2,315 hidden lines…
4316worker1 # [ 49.254836] nix-grpc-daemon[766]: ts=2026-09-23T04:12:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524317worker1 # [ 49.260292] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4318worker1 # [ 49.264539] nix-daemon[849]: reaped child process 900, status = succeeded4319client # building '/nix/store/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv' on lb:50051...4320worker1 # [ 49.275754] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4321client # farm-a> node-a: building 3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4322lb # [ 49.883912] envoy[906]: conn peer=[2001:db8:1::1]:40790 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9536 tx=4444 ms=62724323lb # [ 49.888885] envoy[906]: conn peer=[2001:db8:1::1]:43026 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=993 tx=481 ms=564324lb # [ 49.893776] envoy[906]: conn peer=[2001:db8:1::1]:43040 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4198 tx=1920 ms=17734325client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4326client # farm-a> LOG-farm-a-sig4327worker1 # [ 50.552401] nix-daemon[849]: reaped child process 922, status = succeeded4328worker1 # [ 50.651494] nix-daemon[849]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4329worker1 # [ 50.668602] nix-daemon[849]: reaped child process 941, status = succeeded4330lb # [ 50.676869] niks3-server[962]: 2026/09/23 04:12:05 INFO Received uploads request method=POST path=/api/pending_closures4331worker1 # [ 50.687191] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.287Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4332worker1 # [ 50.689990] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.287Z level=INFO msg="Uploading 2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a (128B)"4333lb # [ 50.720884] niks3-server[962]: 2026/09/23 04:12:05 INFO Registered completed upload object_key=log/3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv4334worker1 # [ 50.729692] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.331Z level=INFO msg="Uploading 1 narinfos"4335lb # [ 50.734757] niks3-server[962]: 2026/09/23 04:12:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4336lb # [ 50.742546] niks3-server[962]: 2026/09/23 04:12:05 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4337lb # [ 50.752535] niks3-server[962]: 2026/09/23 04:12:05 INFO Signed narinfos id=10 count=14338lb # [ 50.756591] niks3-server[962]: 2026/09/23 04:12:05 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.ls4339worker1 # [ 50.755309] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.357Z level=INFO msg="Upload complete. (269ms)"4340lb # [ 50.760196] niks3-server[962]: 2026/09/23 04:12:05 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4341lb # [ 50.765361] niks3-server[962]: 2026/09/23 04:12:05 INFO Completed upload id=104342lb # [ 50.767487] niks3-server[962]: 2026/09/23 04:12:05 INFO Registered completed upload object_key=2ykckkv44ccpw95ad66fm21ggp6krwj9.narinfo4343worker1 # [ 50.769134] nix-grpc-daemon[766]: ts=2026-09-23T04:12:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=1 drv=3m2hhq3bz9jazpcvgg4g03aa5wnlj12y-farm-a.drv assign_id=5 outputs=14344worker1 # [ 50.790522] nix-grpc-daemon[766]: ts=2026-09-23T04:12:05Z level=debug event=want client=4 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv cached=04345worker1 # [ 50.795634] nix-grpc-daemon[766]: ts=2026-09-23T04:12:05Z level=debug event=assigned drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv worker=192.168.1.3:50051 clients=14346worker1 # [ 50.800647] nix-daemon[849]: reaped child process 920, status = succeeded4347worker1 # [ 50.835575] nix-grpc-daemon[766]: ts=2026-09-23T04:12:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524348worker1 # [ 50.850969] nix-grpc-daemon[766]: ts=2026-09-23T04:12:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524349worker1 # [ 50.864401] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.466Z level=INFO msg="All 1 paths already cached"4350client # copying 1 paths to 'lb:50051'...4351client # copying path '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' to 'lb:50051'...4352worker1 # [ 50.875117] nix-grpc-daemon[766]: ts=2026-09-23T04:12:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550524353worker1 # [ 50.889953] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4354worker1 # [ 51.044378] nix-daemon[849]: accepted connection from pid 947, user nix-grpc-daemon (trusted)4355worker1 # [ 51.067109] nix-daemon[849]: reaped child process 950, status = succeeded4356lb # [ 51.066977] niks3-server[962]: 2026/09/23 04:12:05 INFO Received uploads request method=POST path=/api/pending_closures4357worker1 # [ 51.074883] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.676Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4358worker1 # [ 51.076656] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.677Z level=INFO msg="Uploading z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv (624B)"4359lb # [ 51.096731] niks3-server[962]: 2026/09/23 04:12:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4360worker1 # [ 51.105382] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.707Z level=INFO msg="Uploading 1 narinfos"4361lb # [ 51.111652] niks3-server[962]: 2026/09/23 04:12:05 INFO Signed narinfos id=11 count=14362lb # [ 51.119964] niks3-server[962]: 2026/09/23 04:12:05 INFO Registered completed upload object_key=nar/0cnhkgsbmxv5hj7zw3fa59qfxikw6251mlhz2gz1v6pgn8c8shwp.nar.zst4363lb # [ 51.133003] niks3-server[962]: 2026/09/23 04:12:05 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.ls4364worker1 # [ 51.139311] nix-grpc-daemon[902]: time=2026-09-23T04:12:05.741Z level=INFO msg="Upload complete. (232ms)"4365lb # [ 51.141078] niks3-server[962]: 2026/09/23 04:12:05 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4366lb # [ 51.146443] niks3-server[962]: 2026/09/23 04:12:05 INFO Completed upload id=114367worker1 # [ 51.146558] nix-grpc-daemon[766]: ts=2026-09-23T04:12:05Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_in=6244368lb # [ 51.151221] niks3-server[962]: 2026/09/23 04:12:05 INFO Registered completed upload object_key=z8kadp39h0myij9ln0hpxb61ldn71h4i.narinfo4369worker1 # [ 51.162648] nix-grpc-daemon[766]: ts=2026-09-23T04:12:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524370worker1 # [ 51.169719] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4371worker1 # [ 51.174677] nix-daemon[849]: reaped child process 945, status = succeeded4372client # building '/nix/store/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv' on lb:50051...4373worker1 # [ 51.185269] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4374client # farm-b> node-a: building z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4375client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4376client # farm-b> LOG-farm-b-sig4377worker1 # [ 52.456915] nix-daemon[849]: reaped child process 954, status = succeeded4378worker1 # [ 52.568104] nix-daemon[849]: accepted connection from pid 970, user nix-grpc-daemon (trusted)4379worker1 # [ 52.583664] nix-daemon[849]: reaped child process 973, status = succeeded4380lb # [ 52.593963] niks3-server[962]: 2026/09/23 04:12:07 INFO Received uploads request method=POST path=/api/pending_closures4381worker1 # [ 52.601213] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.202Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4382worker1 # [ 52.602937] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.202Z level=INFO msg="Uploading h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b (176B)"4383worker1 # [ 52.642537] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.244Z level=INFO msg="Uploading 1 narinfos"4384lb # [ 52.632157] niks3-server[962]: 2026/09/23 04:12:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4385lb # [ 52.653772] niks3-server[962]: 2026/09/23 04:12:07 INFO Signed narinfos id=12 count=14386lb # [ 52.661025] niks3-server[962]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.ls4387lb # [ 52.674652] niks3-server[962]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=nar/0f75gpfkjj0g9xi2113yvvpdw0l16spc2f1xik1ifnzfbrvx4pfk.nar.zst4388worker1 # [ 52.682313] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.284Z level=INFO msg="Upload complete. (284ms)"4389lb # [ 52.688906] niks3-server[962]: 2026/09/23 04:12:07 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete4390lb # [ 52.695477] niks3-server[962]: 2026/09/23 04:12:07 INFO Completed upload id=124391lb # [ 52.701837] niks3-server[962]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=log/z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv4392worker1 # [ 52.698554] nix-grpc-daemon[766]: ts=2026-09-23T04:12:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=1 drv=z8kadp39h0myij9ln0hpxb61ldn71h4i-farm-b.drv assign_id=6 outputs=14393lb # [ 52.707457] niks3-server[962]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=h1r17djqpb9gp4y7v8nzx65ns9vdr50c.narinfo4394worker1 # [ 52.719788] nix-grpc-daemon[766]: ts=2026-09-23T04:12:07Z level=debug event=want client=4 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv cached=04395worker1 # [ 52.725055] nix-grpc-daemon[766]: ts=2026-09-23T04:12:07Z level=debug event=assigned drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv worker=192.168.1.3:50051 clients=14396worker1 # [ 52.730386] nix-daemon[849]: reaped child process 952, status = succeeded4397worker1 # [ 52.760641] nix-grpc-daemon[766]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524398worker1 # [ 52.772322] nix-grpc-daemon[766]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524399worker1 # [ 52.786618] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.389Z level=INFO msg="All 2 paths already cached"4400client # copying 1 paths to 'lb:50051'...4401client # copying path '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' to 'lb:50051'...4402worker1 # [ 52.798009] nix-grpc-daemon[766]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550524403worker1 # [ 52.814675] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4404worker1 # [ 52.971285] nix-daemon[849]: accepted connection from pid 979, user nix-grpc-daemon (trusted)4405worker1 # [ 52.986163] nix-daemon[849]: reaped child process 982, status = succeeded4406lb # [ 52.996253] niks3-server[962]: 2026/09/23 04:12:07 INFO Received uploads request method=POST path=/api/pending_closures4407worker1 # [ 53.006008] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.608Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4408worker1 # [ 53.007637] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.608Z level=INFO msg="Uploading knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv (752B)"4409lb # [ 53.036214] niks3-server[962]: 2026/09/23 04:12:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign4410worker1 # [ 53.041566] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.643Z level=INFO msg="Uploading 1 narinfos"4411lb # [ 53.045884] niks3-server[962]: 2026/09/23 04:12:07 INFO Signed narinfos id=13 count=14412lb # [ 53.051915] niks3-server[962]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.ls4413lb # [ 53.060219] niks3-server[962]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=nar/0lpagyrngav67x80d1hq0njjlkmqdkvqawraz8dsp36ag50v6iwf.nar.zst4414lb # [ 53.070152] niks3-server[962]: 2026/09/23 04:12:07 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete4415worker1 # [ 53.075664] nix-grpc-daemon[902]: time=2026-09-23T04:12:07.678Z level=INFO msg="Upload complete. (244ms)"4416lb # [ 53.078795] niks3-server[962]: 2026/09/23 04:12:07 INFO Completed upload id=134417lb # [ 53.082451] niks3-server[962]: 2026/09/23 04:12:07 INFO Registered completed upload object_key=knvx2sz45j0sdqm3fnbdrv7m887k7a6x.narinfo4418worker1 # [ 53.085557] nix-grpc-daemon[766]: ts=2026-09-23T04:12:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_in=7524419worker1 # [ 53.103550] nix-grpc-daemon[766]: ts=2026-09-23T04:12:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524420worker1 # [ 53.110378] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4421worker1 # [ 53.115454] nix-daemon[849]: reaped child process 977, status = succeeded4422client # building '/nix/store/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv' on lb:50051...4423worker1 # [ 53.128159] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4424client # farm-top> node-a: building knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4425client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4426client # farm-top> LOG-farm-top-sig4427worker1 # [ 54.615717] nix-daemon[849]: reaped child process 986, status = succeeded4428worker1 # [ 54.682361] nix-daemon[849]: accepted connection from pid 1002, user nix-grpc-daemon (trusted)4429worker1 # [ 54.698152] nix-daemon[849]: reaped child process 1005, status = succeeded4430lb # [ 54.706835] niks3-server[962]: 2026/09/23 04:12:09 INFO Received uploads request method=POST path=/api/pending_closures4431worker1 # [ 54.716486] nix-grpc-daemon[902]: time=2026-09-23T04:12:09.317Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4432worker1 # [ 54.719128] nix-grpc-daemon[902]: time=2026-09-23T04:12:09.317Z level=INFO msg="Uploading 4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top (232B)"4433lb # [ 54.741864] niks3-server[962]: 2026/09/23 04:12:09 INFO Registered completed upload object_key=nar/07ar364rwqqniwi2nkdpqlgv7ws7702n5lqz703ackfmssrh40v8.nar.zst4434lb # [ 54.756615] niks3-server[962]: 2026/09/23 04:12:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign4435worker1 # [ 54.761794] nix-grpc-daemon[902]: time=2026-09-23T04:12:09.364Z level=INFO msg="Uploading 1 narinfos"4436lb # [ 54.767464] niks3-server[962]: 2026/09/23 04:12:09 INFO Signed narinfos id=14 count=14437lb # [ 54.773628] niks3-server[962]: 2026/09/23 04:12:09 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.ls4438lb # [ 54.780635] niks3-server[962]: 2026/09/23 04:12:09 INFO Registered completed upload object_key=log/knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv4439lb # [ 54.789462] niks3-server[962]: 2026/09/23 04:12:09 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete4440lb # [ 54.795055] niks3-server[962]: 2026/09/23 04:12:09 INFO Completed upload id=144441worker1 # [ 54.793708] nix-grpc-daemon[902]: time=2026-09-23T04:12:09.396Z level=INFO msg="Upload complete. (252ms)"4442lb # [ 54.798349] niks3-server[962]: 2026/09/23 04:12:09 INFO Registered completed upload object_key=4s2wbz2hg55lpb7s5vn3ynd342bs09lw.narinfo4443worker1 # [ 54.809004] nix-grpc-daemon[766]: ts=2026-09-23T04:12:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=1 drv=knvx2sz45j0sdqm3fnbdrv7m887k7a6x-farm-top.drv assign_id=7 outputs=14444worker1 # [ 54.822337] nix-daemon[849]: reaped child process 984, status = succeeded4445client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig >&2, in 6.32 seconds)4446client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath4447client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag sig outPath, in 0.18 seconds)4448client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4449worker1 # [ 55.159148] nix-grpc-daemon[766]: ts=2026-09-23T04:12:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:388704450worker1 # [ 55.166623] nix-grpc-daemon[766]: ts=2026-09-23T04:12:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:388704451client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.19 seconds)4452client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4453worker2 # [ 55.372628] nix-grpc-daemon[771]: ts=2026-09-23T04:12:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:364904454client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4455client # /nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a4456client # /nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b4457client # /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4458worker2 # [ 55.420594] nix-grpc-daemon[771]: ts=2026-09-23T04:12:09Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:364904459worker2 # [ 55.429613] nix-grpc-daemon[771]: copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'http://lb:5751'...4460worker2 # [ 55.446848] nix-grpc-daemon[771]: copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'http://lb:5751'...4461worker2 # [ 55.461512] nix-grpc-daemon[771]: copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'http://lb:5751'...4462client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&ca-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ca.pem' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.28 seconds)4463client: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top4464client # finding garbage collector roots...4465client # removing stale temporary roots file "/nix/var/nix/temproots/863"4466client # removing stale temporary roots file "/nix/var/nix/temproots/897"4467client # removing stale temporary roots file "/nix/var/nix/temproots/941"4468client # removing stale temporary roots file "/nix/var/nix/temproots/905"4469client # removing stale temporary roots file "/nix/var/nix/temproots/926"4470client # removing stale temporary roots file "/nix/var/nix/temproots/885"4471client # removing stale temporary roots file "/nix/var/nix/temproots/846"4472client # removing stale temporary roots file "/nix/var/nix/temproots/918"4473client # deleting specified paths...4474client # deleting '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top'4475client # deleting unused links...4476client # note: hard linking is currently saving 0.0 KiB4477worker1 # [ 55.841107] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550524478worker1 # [ 55.847826] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524479worker1 # [ 55.858603] nix-grpc-daemon[902]: time=2026-09-23T04:12:10.460Z level=INFO msg="All 1 paths already cached"4480worker2 # [ 55.871189] nix-grpc-daemon[771]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:493704481worker1 # [ 55.871869] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550524482client # copying 3 paths...4483worker1 # [ 55.885526] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550524484client # copying path '/nix/store/2ykckkv44ccpw95ad66fm21ggp6krwj9-farm-a' from 'grpc://lb:50051'...4485worker2 # [ 55.918673] nix-grpc-daemon[771]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:493704486worker2 # [ 55.926703] nix-grpc-daemon[771]: ts=2026-09-23T04:12:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_out=2324487worker1 # [ 55.930089] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550524488worker1 # [ 55.937438] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_out=1764489worker2 # [ 55.946981] nix-grpc-daemon[771]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:493704490worker2 # [ 55.953807] nix-grpc-daemon[771]: ts=2026-09-23T04:12:10Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_out=1284491client # copying path '/nix/store/h1r17djqpb9gp4y7v8nzx65ns9vdr50c-farm-b' from 'grpc://lb:50051'...4492client # copying path '/nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top' from 'grpc://lb:50051'...4493client: (finished: must succeed: nix-store --delete /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/4s2wbz2hg55lpb7s5vn3ynd342bs09lw-farm-top, in 0.55 seconds)4494(finished: subtest: outputs read back through any worker carry the cache signature, in 7.52 seconds)4495subtest: two clients wanting the same drv build it once4496client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4497client # [ 56.106223] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4498client # Running as unit: dedup1.service; invocation ID: 3db2f523220d4e318f9f6513015ddee44499client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 0.07 seconds)4500client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup4501worker1 # [ 56.289157] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550524502client # [ 56.300981] nix[994]: don't know how to build these paths:4503client # [ 56.302626] nix[994]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4504worker1 # [ 56.306751] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=debug event=want client=5 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04505worker1 # [ 56.313038] nix-grpc-daemon[766]: ts=2026-09-23T04:12:10Z level=debug event=assigned drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv worker=192.168.1.4:50051 clients=14506worker2 # [ 56.354986] nix-grpc-daemon[771]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704507worker2 # [ 56.370370] nix-grpc-daemon[771]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:493704508client # [ 56.384294] nix[994]: copying 1 paths to 'lb:50051'...4509worker2 # [ 56.386164] nix-grpc-daemon[771]: ts=2026-09-23T04:12:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:493704510client # [ 56.389433] nix[994]: copying path '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' to 'lb:50051'...4511worker2 # [ 56.401673] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4512worker2 # [ 56.564426] nix-daemon[840]: accepted connection from pid 1004, user nix-grpc-daemon (trusted)4513worker2 # [ 56.584700] nix-daemon[840]: reaped child process 1007, status = succeeded4514lb # [ 56.588646] niks3-server[962]: 2026/09/23 04:12:11 INFO Received uploads request method=POST path=/api/pending_closures4515worker2 # [ 56.600416] nix-grpc-daemon[853]: time=2026-09-23T04:12:11.179Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4516worker2 # [ 56.601991] nix-grpc-daemon[853]: time=2026-09-23T04:12:11.179Z level=INFO msg="Uploading cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv (432B)"4517lb # [ 56.623612] niks3-server[962]: 2026/09/23 04:12:11 INFO Registered completed upload object_key=nar/0l7vpaziqxpw0srrsgimqvaa0dyxa9glicn3ilkwk06l64hn837w.nar.zst4518worker2 # [ 56.639077] nix-grpc-daemon[853]: time=2026-09-23T04:12:11.218Z level=INFO msg="Uploading 1 narinfos"4519lb # [ 56.638296] niks3-server[962]: 2026/09/23 04:12:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign4520lb # [ 56.649928] niks3-server[962]: 2026/09/23 04:12:11 INFO Signed narinfos id=15 count=14521lb # [ 56.658145] niks3-server[962]: 2026/09/23 04:12:11 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.ls4522worker2 # [ 56.666496] nix-grpc-daemon[853]: time=2026-09-23T04:12:11.245Z level=INFO msg="Upload complete. (247ms)"4523lb # [ 56.665378] niks3-server[962]: 2026/09/23 04:12:11 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete4524lb # [ 56.673689] niks3-server[962]: 2026/09/23 04:12:11 INFO Completed upload id=154525lb # [ 56.675684] niks3-server[962]: 2026/09/23 04:12:11 INFO Registered completed upload object_key=cp23830c44gs2dw4xy80s5wqyjm7vw84.narinfo4526worker2 # [ 56.676581] nix-grpc-daemon[771]: ts=2026-09-23T04:12:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_in=4324527worker2 # [ 56.692641] nix-grpc-daemon[771]: ts=2026-09-23T04:12:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704528worker2 # [ 56.700400] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4529worker2 # [ 56.708786] nix-daemon[840]: reaped child process 1002, status = succeeded4530worker2 # [ 56.717690] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4531client # [ 56.717598] nix[994]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4532client # [ 57.197725] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup.4533client # Running as unit: dedup2.service; invocation ID: cf90519e5b0c4d7087d757bd65aee4d94534client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag dedup, in 1.10 seconds)4535worker1 # [ 57.422189] nix-grpc-daemon[766]: ts=2026-09-23T04:12:12Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550524536client # [ 57.434489] nix[1010]: don't know how to build these paths:4537client # [ 57.435707] nix[1010]: /nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4538worker1 # [ 57.441702] nix-grpc-daemon[766]: ts=2026-09-23T04:12:12Z level=debug event=want client=6 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv cached=04539client # [ 57.454694] nix[1010]: building '/nix/store/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv' on lb:50051...4540worker2 # [ 57.453227] nix-grpc-daemon[771]: ts=2026-09-23T04:12:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704541client: must succeed: sleep 24542lb # [ 59.886740] envoy[906]: conn peer=[2001:db8:1::1]:43052 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9571 tx=4368 ms=60734543lb # [ 59.891769] envoy[906]: conn peer=[2001:db8:1::1]:52350 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2889 ms=1864544lb # [ 59.896647] envoy[906]: conn peer=[2001:db8:1::1]:52384 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=329 ms=834545lb # [ 59.901334] envoy[906]: conn peer=[2001:db8:1::1]:52368 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=408 tx=375 ms=984546lb # [ 59.905996] envoy[906]: conn peer=[2001:db8:1::1]:52360 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=409 ms=1134547client: (finished: must succeed: sleep 2, in 2.05 seconds)4548client: waiting for success: ! systemctl is-active dedup1 dedup24549worker2 # [ 60.749266] nix-daemon[840]: reaped child process 1011, status = succeeded4550worker2 # [ 60.849679] nix-daemon[840]: accepted connection from pid 1041, user nix-grpc-daemon (trusted)4551worker2 # [ 60.865483] nix-daemon[840]: reaped child process 1044, status = succeeded4552lb # [ 60.870756] niks3-server[962]: 2026/09/23 04:12:15 INFO Received uploads request method=POST path=/api/pending_closures4553worker2 # [ 60.884376] nix-grpc-daemon[853]: time=2026-09-23T04:12:15.462Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4554worker2 # [ 60.885840] nix-grpc-daemon[853]: time=2026-09-23T04:12:15.462Z level=INFO msg="Uploading vamz940my4dma9a2daklr9pw9x9pp9hp-slow-dedup (120B)"4555lb # [ 60.921930] niks3-server[962]: 2026/09/23 04:12:15 INFO Registered completed upload object_key=log/cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv4556worker2 # [ 60.942340] nix-grpc-daemon[853]: time=2026-09-23T04:12:15.521Z level=INFO msg="Uploading 1 narinfos"4557lb # [ 60.931668] niks3-server[962]: 2026/09/23 04:12:15 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4558lb # [ 60.945133] niks3-server[962]: 2026/09/23 04:12:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign4559lb # [ 60.951796] niks3-server[962]: 2026/09/23 04:12:15 INFO Signed narinfos id=16 count=14560lb # [ 60.956878] niks3-server[962]: 2026/09/23 04:12:15 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.ls4561lb # [ 60.967237] niks3-server[962]: 2026/09/23 04:12:15 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete4562lb # [ 60.972835] niks3-server[962]: 2026/09/23 04:12:15 INFO Completed upload id=164563worker2 # [ 60.974762] nix-grpc-daemon[853]: time=2026-09-23T04:12:15.553Z level=INFO msg="Upload complete. (242ms)"4564lb # [ 60.976234] niks3-server[962]: 2026/09/23 04:12:15 INFO Registered completed upload object_key=vamz940my4dma9a2daklr9pw9x9pp9hp.narinfo4565worker2 # [ 60.989103] nix-grpc-daemon[771]: ts=2026-09-23T04:12:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=4 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14566worker2 # [ 61.005852] nix-grpc-daemon[771]: ts=2026-09-23T04:12:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=3 drv=cp23830c44gs2dw4xy80s5wqyjm7vw84-slow-dedup.drv assign_id=8 outputs=14567worker2 # [ 61.012880] nix-daemon[840]: reaped child process 1009, status = succeeded4568client # [ 61.023179] systemd[1]: dedup2.service: Deactivated successfully.4569client # [ 61.029122] systemd[1]: dedup2.service: Consumed 205ms CPU time over 3.826s wall clock time, 33.2M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4570client # [ 61.067235] systemd[1]: dedup1.service: Deactivated successfully.4571client # [ 61.070369] systemd[1]: dedup1.service: Consumed 168ms CPU time over 4.963s wall clock time, 41.5M memory peak, 7.4K incoming IP traffic, 8.4K outgoing IP traffic.4572client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.12 seconds)4573client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4574client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.06 seconds)4575worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4576worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4577worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4578worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.07 seconds)4579worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4580worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)4581worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4582worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)4583(finished: subtest: two clients wanting the same drv build it once, in 6.11 seconds)4584subtest: --timeout and --max-silent-time reach the builder4585worker1 # [ 62.385771] nix-grpc-daemon[766]: ts=2026-09-23T04:12:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550524586worker1 # [ 62.402328] nix-grpc-daemon[766]: ts=2026-09-23T04:12:17Z level=debug event=want client=7 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv cached=04587worker1 # [ 62.409449] nix-grpc-daemon[766]: ts=2026-09-23T04:12:17Z level=debug event=assigned drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv worker=192.168.1.3:50051 clients=14588worker1 # [ 62.447288] nix-grpc-daemon[766]: ts=2026-09-23T04:12:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524589worker1 # [ 62.462596] nix-grpc-daemon[766]: ts=2026-09-23T04:12:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524590worker1 # [ 62.477605] nix-grpc-daemon[766]: ts=2026-09-23T04:12:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550524591worker1 # [ 62.492452] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4592worker1 # [ 62.654105] nix-daemon[849]: accepted connection from pid 1036, user nix-grpc-daemon (trusted)4593worker1 # [ 62.674226] nix-daemon[849]: reaped child process 1039, status = succeeded4594lb # [ 62.674416] niks3-server[962]: 2026/09/23 04:12:17 INFO Received uploads request method=POST path=/api/pending_closures4595worker1 # [ 62.683173] nix-grpc-daemon[902]: time=2026-09-23T04:12:17.285Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4596worker1 # [ 62.684792] nix-grpc-daemon[902]: time=2026-09-23T04:12:17.285Z level=INFO msg="Uploading sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv (424B)"4597lb # [ 62.706282] niks3-server[962]: 2026/09/23 04:12:17 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.ls4598worker1 # [ 62.715183] nix-grpc-daemon[902]: time=2026-09-23T04:12:17.317Z level=INFO msg="Uploading 1 narinfos"4599lb # [ 62.719834] niks3-server[962]: 2026/09/23 04:12:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign4600lb # [ 62.731285] niks3-server[962]: 2026/09/23 04:12:17 INFO Signed narinfos id=17 count=14601lb # [ 62.740386] niks3-server[962]: 2026/09/23 04:12:17 INFO Registered completed upload object_key=nar/1m12xmj6yzmb9y2j8ly0yvh5fyzy1nqk7ajhqwr4wn3b5h0bsch5.nar.zst4602worker1 # [ 62.746925] nix-grpc-daemon[902]: time=2026-09-23T04:12:17.349Z level=INFO msg="Upload complete. (237ms)"4603lb # [ 62.752584] niks3-server[962]: 2026/09/23 04:12:17 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete4604lb # [ 62.758877] niks3-server[962]: 2026/09/23 04:12:17 INFO Completed upload id=174605worker1 # [ 62.756465] nix-grpc-daemon[766]: ts=2026-09-23T04:12:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_in=4244606lb # [ 62.763720] niks3-server[962]: 2026/09/23 04:12:17 INFO Registered completed upload object_key=sfkglm39i7dsf4k91pn9ij8xz5mkgirx.narinfo4607worker1 # [ 62.774043] nix-grpc-daemon[766]: ts=2026-09-23T04:12:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524608worker1 # [ 62.783406] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4609worker1 # [ 62.787185] nix-daemon[849]: reaped child process 1034, status = succeeded4610worker1 # [ 62.799111] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4611worker1 # [ 68.617060] nix-grpc-daemon[766]: ts=2026-09-23T04:12:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=5 drv=sfkglm39i7dsf4k91pn9ij8xz5mkgirx-slow-tmot.drv assign_id=9 outputs=04612worker1 # [ 68.634302] nix-daemon[849]: reaped child process 1041, status = succeeded4613worker1 # [ 68.635448] nix-daemon[849]: reaped child process 1043, status = succeeded4614worker1 # [ 68.890527] nix-grpc-daemon[766]: ts=2026-09-23T04:12:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550524615worker1 # [ 68.907516] nix-grpc-daemon[766]: ts=2026-09-23T04:12:23Z level=debug event=want client=8 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv cached=04616worker2 # [ 68.917472] nix-grpc-daemon[771]: ts=2026-09-23T04:12:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704617worker1 # [ 68.913981] nix-grpc-daemon[766]: ts=2026-09-23T04:12:23Z level=debug event=assigned drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv worker=192.168.1.4:50051 clients=14618worker2 # [ 68.932718] nix-grpc-daemon[771]: ts=2026-09-23T04:12:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:493704619worker2 # [ 68.946921] nix-grpc-daemon[771]: ts=2026-09-23T04:12:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:493704620worker2 # [ 68.961811] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4621worker2 # [ 69.126145] nix-daemon[840]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)4622worker2 # [ 69.146439] nix-daemon[840]: reaped child process 1060, status = succeeded4623lb # [ 69.149872] niks3-server[962]: 2026/09/23 04:12:23 INFO Received uploads request method=POST path=/api/pending_closures4624worker2 # [ 69.160390] nix-grpc-daemon[853]: time=2026-09-23T04:12:23.739Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4625worker2 # [ 69.161947] nix-grpc-daemon[853]: time=2026-09-23T04:12:23.739Z level=INFO msg="Uploading pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv (424B)"4626lb # [ 69.178642] niks3-server[962]: 2026/09/23 04:12:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign4627worker2 # [ 69.190607] nix-grpc-daemon[853]: time=2026-09-23T04:12:23.769Z level=INFO msg="Uploading 1 narinfos"4628lb # [ 69.191408] niks3-server[962]: 2026/09/23 04:12:23 INFO Signed narinfos id=18 count=14629lb # [ 69.201157] niks3-server[962]: 2026/09/23 04:12:23 INFO Registered completed upload object_key=nar/14f5yrnllfav3svmw4v4svw15r4zlf5yv1p972jy14mmq6gdw943.nar.zst4630lb # [ 69.212797] niks3-server[962]: 2026/09/23 04:12:23 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.ls4631lb # [ 69.223293] niks3-server[962]: 2026/09/23 04:12:23 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete4632worker2 # [ 69.225531] nix-grpc-daemon[853]: time=2026-09-23T04:12:23.804Z level=INFO msg="Upload complete. (246ms)"4633lb # [ 69.232515] niks3-server[962]: 2026/09/23 04:12:23 INFO Completed upload id=184634lb # [ 69.235150] niks3-server[962]: 2026/09/23 04:12:23 INFO Registered completed upload object_key=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s.narinfo4635worker2 # [ 69.235355] nix-grpc-daemon[771]: ts=2026-09-23T04:12:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_in=4244636worker2 # [ 69.252968] nix-grpc-daemon[771]: ts=2026-09-23T04:12:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704637worker2 # [ 69.262538] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4638worker2 # [ 69.268451] nix-daemon[840]: reaped child process 1055, status = succeeded4639worker2 # [ 69.282893] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4640lb # [ 69.883724] envoy[906]: conn peer=[2001:db8:1::1]:52396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1430 tx=1220 ms=35804641lb # [ 69.888920] envoy[906]: conn peer=[2001:db8:1::1]:52392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3044 tx=1565 ms=47544642lb # [ 69.893993] envoy[906]: conn peer=[2001:db8:1::1]:52528 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3027 tx=976 ms=62944643worker2 # [ 75.700667] nix-grpc-daemon[771]: ts=2026-09-23T04:12:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=6 drv=pqqkii3gblyblsbkdbaa4hlzm7wlfq9s-slow-tmom.drv assign_id=10 outputs=04644worker2 # [ 75.733916] nix-daemon[840]: reaped child process 1062, status = succeeded4645worker2 # [ 75.737013] nix-daemon[840]: reaped child process 1064, status = succeeded4646(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.77 seconds)4647subtest: the first client leaving does not fail the second4648client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4649client # [ 75.994943] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4650client # Running as unit: share1.service; invocation ID: d22e07145bbe411484c84dedfe503edb4651client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 0.08 seconds)4652client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share4653worker2 # [ 76.204860] nix-grpc-daemon[771]: ts=2026-09-23T04:12:30Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:493704654client # [ 76.212908] nix[1082]: don't know how to build these paths:4655client # [ 76.213942] nix[1082]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4656worker1 # [ 76.225119] nix-grpc-daemon[766]: ts=2026-09-23T04:12:30Z level=debug event=want client=9 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04657worker1 # [ 76.231601] nix-grpc-daemon[766]: ts=2026-09-23T04:12:30Z level=debug event=assigned drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv worker=192.168.1.3:50051 clients=14658worker1 # [ 76.238478] nix-grpc-daemon[766]: ts=2026-09-23T04:12:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524659worker1 # [ 76.249384] nix-grpc-daemon[766]: ts=2026-09-23T04:12:30Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524660client # [ 76.265515] nix[1082]: copying 1 paths to 'lb:50051'...4661client # [ 76.271799] nix[1082]: copying path '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' to 'lb:50051'...4662worker1 # [ 76.265096] nix-grpc-daemon[766]: ts=2026-09-23T04:12:30Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550524663worker1 # [ 76.279966] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4664worker1 # [ 76.443244] nix-daemon[849]: accepted connection from pid 1070, user nix-grpc-daemon (trusted)4665worker1 # [ 76.461695] nix-daemon[849]: reaped child process 1073, status = succeeded4666lb # [ 76.469097] niks3-server[962]: 2026/09/23 04:12:31 INFO Received uploads request method=POST path=/api/pending_closures4667worker1 # [ 76.479761] nix-grpc-daemon[902]: time=2026-09-23T04:12:31.081Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4668worker1 # [ 76.481402] nix-grpc-daemon[902]: time=2026-09-23T04:12:31.081Z level=INFO msg="Uploading rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv (432B)"4669lb # [ 76.502201] niks3-server[962]: 2026/09/23 04:12:31 INFO Registered completed upload object_key=nar/0chdjpc9an2ckdblh08lnyrmzvw3h8qs76cn5ydwjwvgwkrm731l.nar.zst4670lb # [ 76.517973] niks3-server[962]: 2026/09/23 04:12:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign4671worker1 # [ 76.523273] nix-grpc-daemon[902]: time=2026-09-23T04:12:31.125Z level=INFO msg="Uploading 1 narinfos"4672lb # [ 76.529752] niks3-server[962]: 2026/09/23 04:12:31 INFO Signed narinfos id=19 count=14673lb # [ 76.538465] niks3-server[962]: 2026/09/23 04:12:31 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.ls4674lb # [ 76.550392] niks3-server[962]: 2026/09/23 04:12:31 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete4675lb # [ 76.560291] niks3-server[962]: 2026/09/23 04:12:31 INFO Completed upload id=194676worker1 # [ 76.558470] nix-grpc-daemon[902]: time=2026-09-23T04:12:31.160Z level=INFO msg="Upload complete. (259ms)"4677lb # [ 76.566405] niks3-server[962]: 2026/09/23 04:12:31 INFO Registered completed upload object_key=rxafkggp1xca5h7p0r6gwpq3qqp2kn63.narinfo4678worker1 # [ 76.568396] nix-grpc-daemon[766]: ts=2026-09-23T04:12:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_in=4324679worker1 # [ 76.587216] nix-grpc-daemon[766]: ts=2026-09-23T04:12:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524680worker1 # [ 76.597357] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4681worker1 # [ 76.600452] nix-daemon[849]: reaped child process 1068, status = succeeded4682client # [ 76.616151] nix[1082]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4683worker1 # [ 76.614760] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4684client # [ 77.093141] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share.4685client # Running as unit: share2.service; invocation ID: 01527c429db64ab8a320d2719cbff8314686client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag share, in 1.11 seconds)4687worker2 # [ 77.312898] nix-grpc-daemon[771]: ts=2026-09-23T04:12:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:493704688client # [ 77.321037] nix[1098]: don't know how to build these paths:4689client # [ 77.322463] nix[1098]: /nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4690worker1 # [ 77.328579] nix-grpc-daemon[766]: ts=2026-09-23T04:12:31Z level=debug event=want client=10 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv cached=04691client # [ 77.341391] nix[1098]: building '/nix/store/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv' on lb:50051...4692worker1 # [ 77.336097] nix-grpc-daemon[766]: ts=2026-09-23T04:12:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524693client: must succeed: sleep 2; systemctl kill -s INT share14694lb # [ 79.888202] envoy[906]: conn peer=[2001:db8:1::1]:52534 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2959 tx=937 ms=68254695client # [ 80.479399] systemd[1]: share1.service: Sent signal SIGINT to main process 1082 (nix) on client request.4696client # [ 80.485834] nix[1082]: shutting downrxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)4697client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.09 seconds)4698client: waiting for success: ! systemctl is-active share14699client # [ 80.499620] nix[1082]: error: interrupted by the user4700client # [ 80.564870] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE4701client # [ 80.568959] systemd[1]: share1.service: Failed with result 'exit-code'.4702client # [ 80.570808] systemd[1]: share1.service: Consumed 185ms CPU time over 4.573s wall clock time, 41.4M memory peak, 6.3K incoming IP traffic, 8.2K outgoing IP traffic.4703client: (finished: waiting for success: ! systemctl is-active share1, in 1.12 seconds)4704worker1 # [ 81.839087] nix-daemon[849]: reaped child process 1077, status = succeeded4705client: waiting for success: ! systemctl is-active share24706worker1 # [ 81.935050] nix-daemon[849]: accepted connection from pid 1113, user nix-grpc-daemon (trusted)4707worker1 # [ 81.953743] nix-daemon[849]: reaped child process 1116, status = succeeded4708lb # [ 81.961007] niks3-server[962]: 2026/09/23 04:12:36 INFO Received uploads request method=POST path=/api/pending_closures4709worker1 # [ 81.974138] nix-grpc-daemon[902]: time=2026-09-23T04:12:36.574Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4710worker1 # [ 81.975590] nix-grpc-daemon[902]: time=2026-09-23T04:12:36.574Z level=INFO msg="Uploading fjww0fd4rwk1s35jav3cyal62l24n356-slow-share (120B)"4711lb # [ 82.001595] niks3-server[962]: 2026/09/23 04:12:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign4712worker1 # [ 82.006995] nix-grpc-daemon[902]: time=2026-09-23T04:12:36.609Z level=INFO msg="Uploading 1 narinfos"4713lb # [ 82.011664] niks3-server[962]: 2026/09/23 04:12:36 INFO Signed narinfos id=20 count=14714lb # [ 82.018850] niks3-server[962]: 2026/09/23 04:12:36 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.ls4715lb # [ 82.027138] niks3-server[962]: 2026/09/23 04:12:36 INFO Registered completed upload object_key=log/rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv4716lb # [ 82.034122] niks3-server[962]: 2026/09/23 04:12:36 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst4717lb # [ 82.041857] niks3-server[962]: 2026/09/23 04:12:36 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete4718worker1 # [ 82.047125] nix-grpc-daemon[902]: time=2026-09-23T04:12:36.649Z level=INFO msg="Upload complete. (214ms)"4719lb # [ 82.051825] niks3-server[962]: 2026/09/23 04:12:36 INFO Completed upload id=204720lb # [ 82.056404] niks3-server[962]: 2026/09/23 04:12:36 INFO Registered completed upload object_key=fjww0fd4rwk1s35jav3cyal62l24n356.narinfo4721worker1 # [ 82.061033] nix-grpc-daemon[766]: ts=2026-09-23T04:12:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=5 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14722worker1 # [ 82.072400] nix-grpc-daemon[766]: ts=2026-09-23T04:12:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=4 drv=rxafkggp1xca5h7p0r6gwpq3qqp2kn63-slow-share.drv assign_id=11 outputs=14723worker1 # [ 82.079301] nix-daemon[849]: reaped child process 1075, status = succeeded4724client # [ 82.138586] systemd[1]: share2.service: Deactivated successfully.4725client # [ 82.143509] systemd[1]: share2.service: Consumed 200ms CPU time over 5.049s wall clock time, 33.3M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.4726client: (finished: waiting for success: ! systemctl is-active share2, in 1.12 seconds)4727client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }4728client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.05 seconds)4729(finished: subtest: the first client leaving does not fail the second, in 7.11 seconds)4730subtest: build hook: nix-daemon with builders = grpc://lb4731client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log4732client # [ 83.175088] systemd[1]: Started Nix Daemon.4733client # [ 83.292179] nix-daemon[1150]: accepted connection from pid 1147, user root (trusted)4734worker2 # [ 83.434738] nix-grpc-daemon[771]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:493704735worker2 # [ 83.442373] nix-grpc-daemon[771]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:493704736worker1 # [ 83.451201] nix-grpc-daemon[766]: ts=2026-09-23T04:12:38Z level=debug event=want client=11 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv cached=04737worker1 # [ 83.457594] nix-grpc-daemon[766]: ts=2026-09-23T04:12:38Z level=debug event=assigned drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv worker=192.168.1.4:50051 clients=14738worker2 # [ 83.464875] nix-grpc-daemon[771]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704739worker2 # [ 83.480857] nix-grpc-daemon[771]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:493704740worker2 # [ 83.495715] nix-grpc-daemon[771]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:493704741worker2 # [ 83.510964] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4742worker2 # [ 83.674725] nix-daemon[840]: accepted connection from pid 1103, user nix-grpc-daemon (trusted)4743worker2 # [ 83.695065] nix-daemon[840]: reaped child process 1106, status = succeeded4744lb # [ 83.701578] niks3-server[962]: 2026/09/23 04:12:38 INFO Received uploads request method=POST path=/api/pending_closures4745worker2 # [ 83.715616] nix-grpc-daemon[853]: time=2026-09-23T04:12:38.293Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4746worker2 # [ 83.717397] nix-grpc-daemon[853]: time=2026-09-23T04:12:38.293Z level=INFO msg="Uploading xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv (512B)"4747lb # [ 83.733240] niks3-server[962]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=nar/1f1i5kab3qhx7pg1k0ma5r1m7ipfazkfkfb944dm7qcjr466pwij.nar.zst4748worker2 # [ 83.748845] nix-grpc-daemon[853]: time=2026-09-23T04:12:38.327Z level=INFO msg="Uploading 1 narinfos"4749lb # [ 83.749524] niks3-server[962]: 2026/09/23 04:12:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign4750lb # [ 83.759251] niks3-server[962]: 2026/09/23 04:12:38 INFO Signed narinfos id=21 count=14751lb # [ 83.764841] niks3-server[962]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.ls4752worker2 # [ 83.776502] nix-grpc-daemon[853]: time=2026-09-23T04:12:38.355Z level=INFO msg="Upload complete. (246ms)"4753lb # [ 83.775204] niks3-server[962]: 2026/09/23 04:12:38 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete4754lb # [ 83.779816] niks3-server[962]: 2026/09/23 04:12:38 INFO Completed upload id=214755lb # [ 83.783785] niks3-server[962]: 2026/09/23 04:12:38 INFO Registered completed upload object_key=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv.narinfo4756worker2 # [ 83.786453] nix-grpc-daemon[771]: ts=2026-09-23T04:12:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_in=5124757worker2 # [ 83.801383] nix-grpc-daemon[771]: ts=2026-09-23T04:12:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704758worker2 # [ 83.807456] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4759worker2 # [ 83.812526] nix-daemon[840]: reaped child process 1101, status = succeeded4760worker2 # [ 83.823103] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4761worker2 # [ 85.288368] nix-daemon[840]: reaped child process 1110, status = succeeded4762worker2 # [ 85.381833] nix-daemon[840]: accepted connection from pid 1126, user nix-grpc-daemon (trusted)4763worker2 # [ 85.401917] nix-daemon[840]: reaped child process 1129, status = succeeded4764lb # [ 85.398334] niks3-server[962]: 2026/09/23 04:12:40 INFO Received uploads request method=POST path=/api/pending_closures4765worker2 # [ 85.411557] nix-grpc-daemon[853]: time=2026-09-23T04:12:39.989Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4766worker2 # [ 85.415060] nix-grpc-daemon[853]: time=2026-09-23T04:12:39.990Z level=INFO msg="Uploading yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a (128B)"4767lb # [ 85.443768] niks3-server[962]: 2026/09/23 04:12:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign4768worker2 # [ 85.454024] nix-grpc-daemon[853]: time=2026-09-23T04:12:40.033Z level=INFO msg="Uploading 1 narinfos"4769lb # [ 85.459438] niks3-server[962]: 2026/09/23 04:12:40 INFO Signed narinfos id=22 count=14770lb # [ 85.466082] niks3-server[962]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.ls4771lb # [ 85.472251] niks3-server[962]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=log/xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv4772lb # [ 85.476562] niks3-server[962]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst4773lb # [ 85.483611] niks3-server[962]: 2026/09/23 04:12:40 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete4774worker2 # [ 85.490900] nix-grpc-daemon[853]: time=2026-09-23T04:12:40.070Z level=INFO msg="Upload complete. (254ms)"4775lb # [ 85.491357] niks3-server[962]: 2026/09/23 04:12:40 INFO Completed upload id=224776lb # [ 85.493499] niks3-server[962]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=yfs64sxmsg77fnzczdvc3g3yfdwh9fks.narinfo4777worker2 # [ 85.501083] nix-grpc-daemon[771]: ts=2026-09-23T04:12:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=1 drv=xs7jpxrx1nnx7zbhs6p2v76pjq3964kv-farm-a.drv assign_id=12 outputs=14778worker1 # [ 85.506237] nix-grpc-daemon[766]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550524779worker2 # [ 85.519464] nix-daemon[840]: reaped child process 1108, status = succeeded4780worker1 # [ 85.519732] nix-grpc-daemon[766]: copying path '/nix/store/yfs64sxmsg77fnzczdvc3g3yfdwh9fks-farm-a' from 'http://lb:5751'...4781worker1 # [ 85.559728] nix-grpc-daemon[766]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550524782worker1 # [ 85.566667] nix-grpc-daemon[766]: ts=2026-09-23T04:12:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_out=1284783worker2 # [ 85.850007] nix-grpc-daemon[771]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:493704784worker1 # [ 85.850622] nix-grpc-daemon[766]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524785worker1 # [ 85.861232] nix-grpc-daemon[902]: time=2026-09-23T04:12:40.462Z level=INFO msg="All 1 paths already cached"4786worker1 # [ 85.879007] nix-grpc-daemon[766]: ts=2026-09-23T04:12:40Z level=debug event=want client=12 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv cached=04787worker1 # [ 85.885199] nix-grpc-daemon[766]: ts=2026-09-23T04:12:40Z level=debug event=assigned drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv worker=192.168.1.4:50051 clients=14788worker2 # [ 85.931302] nix-grpc-daemon[771]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704789worker2 # [ 85.946914] nix-grpc-daemon[771]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:493704790worker2 # [ 85.961312] nix-grpc-daemon[853]: time=2026-09-23T04:12:40.540Z level=INFO msg="All 1 paths already cached"4791worker2 # [ 85.970766] nix-grpc-daemon[771]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:493704792worker2 # [ 85.984785] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4793worker2 # [ 86.147379] nix-daemon[840]: accepted connection from pid 1133, user nix-grpc-daemon (trusted)4794worker2 # [ 86.166685] nix-daemon[840]: reaped child process 1136, status = succeeded4795lb # [ 86.167285] niks3-server[962]: 2026/09/23 04:12:40 INFO Received uploads request method=POST path=/api/pending_closures4796worker2 # [ 86.178829] nix-grpc-daemon[853]: time=2026-09-23T04:12:40.757Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4797worker2 # [ 86.180432] nix-grpc-daemon[853]: time=2026-09-23T04:12:40.757Z level=INFO msg="Uploading r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv (624B)"4798lb # [ 86.197457] niks3-server[962]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.ls4799worker2 # [ 86.211284] nix-grpc-daemon[853]: time=2026-09-23T04:12:40.790Z level=INFO msg="Uploading 1 narinfos"4800lb # [ 86.212289] niks3-server[962]: 2026/09/23 04:12:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign4801lb # [ 86.223460] niks3-server[962]: 2026/09/23 04:12:40 INFO Signed narinfos id=23 count=14802lb # [ 86.231094] niks3-server[962]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=nar/15k66v3mh3v6p5vymv4pv3v3sfl56i2b1ia2rf093c37w6bhc04m.nar.zst4803worker2 # [ 86.241127] nix-grpc-daemon[853]: time=2026-09-23T04:12:40.820Z level=INFO msg="Upload complete. (238ms)"4804lb # [ 86.240883] niks3-server[962]: 2026/09/23 04:12:40 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete4805lb # [ 86.248809] niks3-server[962]: 2026/09/23 04:12:40 INFO Completed upload id=234806lb # [ 86.251133] niks3-server[962]: 2026/09/23 04:12:40 INFO Registered completed upload object_key=r1jsr4b2g1qwly395z6idx3z2362ih8i.narinfo4807worker2 # [ 86.251679] nix-grpc-daemon[771]: ts=2026-09-23T04:12:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_in=6244808worker2 # [ 86.266851] nix-grpc-daemon[771]: ts=2026-09-23T04:12:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704809worker2 # [ 86.275557] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4810worker2 # [ 86.278958] nix-daemon[840]: reaped child process 1131, status = succeeded4811worker2 # [ 86.289482] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4812worker2 # [ 87.763962] nix-daemon[840]: reaped child process 1140, status = succeeded4813worker2 # [ 87.869999] nix-daemon[840]: accepted connection from pid 1156, user nix-grpc-daemon (trusted)4814worker2 # [ 87.886867] nix-daemon[840]: reaped child process 1159, status = succeeded4815lb # [ 87.890449] niks3-server[962]: 2026/09/23 04:12:42 INFO Received uploads request method=POST path=/api/pending_closures4816worker2 # [ 87.904426] nix-grpc-daemon[853]: time=2026-09-23T04:12:42.481Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4817worker2 # [ 87.906139] nix-grpc-daemon[853]: time=2026-09-23T04:12:42.481Z level=INFO msg="Uploading la6ryzsihag4zpxgxls54fsg34dvlalk-farm-b (176B)"4818worker2 # [ 87.942581] nix-grpc-daemon[853]: time=2026-09-23T04:12:42.521Z level=INFO msg="Uploading 1 narinfos"4819lb # [ 87.932501] niks3-server[962]: 2026/09/23 04:12:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign4820lb # [ 87.945057] niks3-server[962]: 2026/09/23 04:12:42 INFO Signed narinfos id=24 count=14821lb # [ 87.954304] niks3-server[962]: 2026/09/23 04:12:42 INFO Registered completed upload object_key=nar/069d0xnyw2q345iqzx6agxc5fxifc6q36bkkklw2aa4qwanxdd1g.nar.zst4822lb # [ 87.965863] niks3-server[962]: 2026/09/23 04:12:42 INFO Registered completed upload object_key=log/r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv4823lb # [ 87.972182] niks3-server[962]: 2026/09/23 04:12:42 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.ls4824lb # [ 87.982800] niks3-server[962]: 2026/09/23 04:12:42 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete4825worker2 # [ 87.992392] nix-grpc-daemon[853]: time=2026-09-23T04:12:42.571Z level=INFO msg="Upload complete. (291ms)"4826lb # [ 87.990410] niks3-server[962]: 2026/09/23 04:12:42 INFO Registered completed upload object_key=la6ryzsihag4zpxgxls54fsg34dvlalk.narinfo4827lb # [ 87.994716] niks3-server[962]: 2026/09/23 04:12:42 INFO Completed upload id=244828worker2 # [ 88.005213] nix-grpc-daemon[771]: ts=2026-09-23T04:12:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=1 drv=r1jsr4b2g1qwly395z6idx3z2362ih8i-farm-b.drv assign_id=13 outputs=14829worker2 # [ 88.024814] nix-grpc-daemon[771]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:493704830worker2 # [ 88.030167] nix-daemon[840]: reaped child process 1138, status = succeeded4831worker2 # [ 88.041106] nix-grpc-daemon[771]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:493704832worker2 # [ 88.048240] nix-grpc-daemon[771]: ts=2026-09-23T04:12:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_out=1764833worker1 # [ 88.320356] nix-grpc-daemon[766]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:550524834worker1 # [ 88.326802] nix-grpc-daemon[766]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524835worker1 # [ 88.343836] nix-grpc-daemon[902]: time=2026-09-23T04:12:42.945Z level=INFO msg="All 1 paths already cached"4836worker1 # [ 88.361980] nix-grpc-daemon[766]: ts=2026-09-23T04:12:42Z level=debug event=want client=13 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv cached=04837worker2 # [ 88.369771] nix-grpc-daemon[771]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704838worker1 # [ 88.368425] nix-grpc-daemon[766]: ts=2026-09-23T04:12:42Z level=debug event=assigned drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv worker=192.168.1.4:50051 clients=14839worker2 # [ 88.385622] nix-grpc-daemon[771]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:493704840worker2 # [ 88.401903] nix-grpc-daemon[853]: time=2026-09-23T04:12:42.981Z level=INFO msg="All 2 paths already cached"4841worker2 # [ 88.412124] nix-grpc-daemon[771]: ts=2026-09-23T04:12:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:493704842worker2 # [ 88.426209] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4843worker2 # [ 88.593344] nix-daemon[840]: accepted connection from pid 1163, user nix-grpc-daemon (trusted)4844worker2 # [ 88.610091] nix-daemon[840]: reaped child process 1166, status = succeeded4845lb # [ 88.613746] niks3-server[962]: 2026/09/23 04:12:43 INFO Received uploads request method=POST path=/api/pending_closures4846worker2 # [ 88.626339] nix-grpc-daemon[853]: time=2026-09-23T04:12:43.205Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4847worker2 # [ 88.627880] nix-grpc-daemon[853]: time=2026-09-23T04:12:43.205Z level=INFO msg="Uploading aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv (752B)"4848lb # [ 88.643158] niks3-server[962]: 2026/09/23 04:12:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign4849worker2 # [ 88.654307] nix-grpc-daemon[853]: time=2026-09-23T04:12:43.233Z level=INFO msg="Uploading 1 narinfos"4850lb # [ 88.657963] niks3-server[962]: 2026/09/23 04:12:43 INFO Signed narinfos id=25 count=14851lb # [ 88.665899] niks3-server[962]: 2026/09/23 04:12:43 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.ls4852lb # [ 88.677397] niks3-server[962]: 2026/09/23 04:12:43 INFO Registered completed upload object_key=nar/1ahq7sp1sl2scxqzfjd8gqimpqfkjbyi9ivxdhdwx0jzh2gbv74b.nar.zst4853worker2 # [ 88.684606] nix-grpc-daemon[853]: time=2026-09-23T04:12:43.263Z level=INFO msg="Upload complete. (238ms)"4854lb # [ 88.685254] niks3-server[962]: 2026/09/23 04:12:43 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete4855lb # [ 88.689795] niks3-server[962]: 2026/09/23 04:12:43 INFO Completed upload id=254856worker2 # [ 88.692198] nix-grpc-daemon[771]: ts=2026-09-23T04:12:43Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_in=7524857lb # [ 88.694796] niks3-server[962]: 2026/09/23 04:12:43 INFO Registered completed upload object_key=aa4abikjj4w30bfx1p8hxmq0916qbz1c.narinfo4858worker2 # [ 88.706099] nix-grpc-daemon[771]: ts=2026-09-23T04:12:43Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493704859worker2 # [ 88.714676] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4860worker2 # [ 88.717820] nix-daemon[840]: reaped child process 1161, status = succeeded4861worker2 # [ 88.729299] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)4862lb # [ 89.890931] envoy[906]: conn peer=[2001:db8:1::1]:35780 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2990 tx=812 ms=42994863lb # [ 89.895849] envoy[906]: conn peer=[2001:db8:1::1]:35786 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1443 tx=1197 ms=48054864lb # [ 89.901036] envoy[906]: conn peer=[2001:db8:1::1]:44670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=333 ms=664865lb # [ 89.905728] envoy[906]: conn peer=[2001:db8:1::1]:44668 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3440 tx=1968 ms=21934866lb # [ 89.910467] envoy[906]: conn peer=[2001:db8:1::1]:44672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3888 tx=2158 ms=22574867lb # [ 89.915989] envoy[906]: conn peer=[2001:db8:1::1]:44688 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=436 tx=375 ms=704868worker2 # [ 90.396979] nix-daemon[840]: reaped child process 1170, status = succeeded4869worker2 # [ 90.511043] nix-daemon[840]: accepted connection from pid 1186, user nix-grpc-daemon (trusted)4870worker2 # [ 90.529902] nix-daemon[840]: reaped child process 1189, status = succeeded4871lb # [ 90.533473] niks3-server[962]: 2026/09/23 04:12:45 INFO Received uploads request method=POST path=/api/pending_closures4872worker2 # [ 90.547841] nix-grpc-daemon[853]: time=2026-09-23T04:12:45.125Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4873worker2 # [ 90.549722] nix-grpc-daemon[853]: time=2026-09-23T04:12:45.125Z level=INFO msg="Uploading 39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top (232B)"4874lb # [ 90.575528] niks3-server[962]: 2026/09/23 04:12:45 INFO Registered completed upload object_key=nar/0vq3rhfhhbqyx5pbvaa9ddj9wbw9arph334y3bp3idfrikwnjc92.nar.zst4875worker2 # [ 90.587782] nix-grpc-daemon[853]: time=2026-09-23T04:12:45.166Z level=INFO msg="Uploading 1 narinfos"4876lb # [ 90.592200] niks3-server[962]: 2026/09/23 04:12:45 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign4877lb # [ 90.604123] niks3-server[962]: 2026/09/23 04:12:45 INFO Signed narinfos id=26 count=14878lb # [ 90.606609] niks3-server[962]: 2026/09/23 04:12:45 INFO Registered completed upload object_key=log/aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv4879lb # [ 90.619464] niks3-server[962]: 2026/09/23 04:12:45 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.ls4880lb # [ 90.624938] niks3-server[962]: 2026/09/23 04:12:45 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete4881worker2 # [ 90.627294] nix-grpc-daemon[853]: time=2026-09-23T04:12:45.206Z level=INFO msg="Upload complete. (284ms)"4882lb # [ 90.628900] niks3-server[962]: 2026/09/23 04:12:45 INFO Completed upload id=264883lb # [ 90.632675] niks3-server[962]: 2026/09/23 04:12:45 INFO Registered completed upload object_key=39qxwzy2zkij70y9zvbr59pb0df38w5h.narinfo4884worker2 # [ 90.641152] nix-grpc-daemon[771]: ts=2026-09-23T04:12:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=1 drv=aa4abikjj4w30bfx1p8hxmq0916qbz1c-farm-top.drv assign_id=14 outputs=14885worker2 # [ 90.659353] nix-grpc-daemon[771]: ts=2026-09-23T04:12:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:493704886worker2 # [ 90.665540] nix-daemon[840]: reaped child process 1168, status = succeeded4887worker2 # [ 90.689786] nix-grpc-daemon[771]: ts=2026-09-23T04:12:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:493704888worker2 # [ 90.696345] nix-grpc-daemon[771]: ts=2026-09-23T04:12:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_out=2324889client # [ 90.821824] nix-daemon[1150]: reaped child process 1152, status = succeeded4890client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --print-out-paths --no-link -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag hook 2>/tmp/hook.log, in 7.80 seconds)4891client: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top4892client: (finished: must succeed: grep farm-top-hook /nix/store/39qxwzy2zkij70y9zvbr59pb0df38w5h-farm-top, in 0.04 seconds)4893client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }4894client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.05 seconds)4895client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }4896client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.03 seconds)4897(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 7.92 seconds)4898subtest: an input only one worker has still reaches the builder4899client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only4900client # this derivation will be built:4901client # /nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv4902client # building '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv'...4903client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag w1only, in 0.29 seconds)4904client: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure4905client: (finished: must succeed: nix-store --export /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only > /tmp/shared/inp.closure, in 0.12 seconds)4906worker1: must succeed: nix-store --import < /tmp/shared/inp.closure4907worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.19 seconds)4908worker2: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only4909worker2: (finished: must fail: test -e /nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only, in 0.03 seconds)4910client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&24911client # [ 91.779458] nix-daemon[1150]: accepted connection from pid 1232, user root (trusted)4912client # this derivation will be built:4913client # /nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4914client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...4915worker2 # [ 91.926413] nix-grpc-daemon[771]: ts=2026-09-23T04:12:46Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:493704916worker1 # [ 91.926580] nix-grpc-daemon[766]: ts=2026-09-23T04:12:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524917worker1 # [ 92.094874] nix-daemon[849]: accepted connection from pid 1130, user nix-grpc-daemon (trusted)4918worker1 # [ 92.114446] nix-daemon[849]: reaped child process 1133, status = succeeded4919lb # [ 92.115462] niks3-server[962]: 2026/09/23 04:12:46 INFO Received uploads request method=POST path=/api/pending_closures4920worker1 # [ 92.125564] nix-grpc-daemon[902]: time=2026-09-23T04:12:46.727Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4921worker1 # [ 92.127748] nix-grpc-daemon[902]: time=2026-09-23T04:12:46.727Z level=INFO msg="Uploading dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only (120B)"4922lb # [ 92.151669] niks3-server[962]: 2026/09/23 04:12:46 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign4923worker1 # [ 92.158277] nix-grpc-daemon[902]: time=2026-09-23T04:12:46.760Z level=INFO msg="Uploading 1 narinfos"4924lb # [ 92.165535] niks3-server[962]: 2026/09/23 04:12:46 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst4925lb # [ 92.175435] niks3-server[962]: 2026/09/23 04:12:46 INFO Signed narinfos id=27 count=14926lb # [ 92.182180] niks3-server[962]: 2026/09/23 04:12:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.ls4927worker1 # [ 92.186360] nix-grpc-daemon[902]: time=2026-09-23T04:12:46.788Z level=INFO msg="Upload complete. (254ms)"4928lb # [ 92.190666] niks3-server[962]: 2026/09/23 04:12:46 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete4929lb # [ 92.195877] niks3-server[962]: 2026/09/23 04:12:46 INFO Completed upload id=274930lb # [ 92.199923] niks3-server[962]: 2026/09/23 04:12:46 INFO Registered completed upload object_key=dymj0d1vj8q0ns6p096qk7fii766dap6.narinfo4931worker1 # [ 92.210131] nix-grpc-daemon[766]: ts=2026-09-23T04:12:46Z level=debug event=want client=14 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv cached=04932worker1 # [ 92.217508] nix-grpc-daemon[766]: ts=2026-09-23T04:12:46Z level=debug event=assigned drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv worker=192.168.1.3:50051 clients=14933worker1 # [ 92.255255] nix-grpc-daemon[766]: ts=2026-09-23T04:12:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524934worker1 # [ 92.270640] nix-grpc-daemon[766]: ts=2026-09-23T04:12:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550524935client # copying 2 paths to 'lb:50051'...4936client # copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' to 'lb:50051'...4937worker1 # [ 92.290253] nix-grpc-daemon[766]: ts=2026-09-23T04:12:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550524938client # copying path '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' to 'lb:50051'...4939worker1 # [ 92.304129] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4940worker1 # [ 92.460206] nix-daemon[849]: accepted connection from pid 1139, user nix-grpc-daemon (trusted)4941worker1 # [ 92.479062] nix-daemon[849]: reaped child process 1142, status = succeeded4942lb # [ 92.478141] niks3-server[962]: 2026/09/23 04:12:47 INFO Received uploads request method=POST path=/api/pending_closures4943lb # [ 92.488267] niks3-server[962]: 2026/09/23 04:12:47 INFO Received uploads request method=POST path=/api/pending_closures4944worker1 # [ 92.500076] nix-grpc-daemon[902]: time=2026-09-23T04:12:47.100Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"4945worker1 # [ 92.501813] nix-grpc-daemon[902]: time=2026-09-23T04:12:47.100Z level=INFO msg="Uploading 4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv (416B)"4946worker1 # [ 92.503899] nix-grpc-daemon[902]: time=2026-09-23T04:12:47.100Z level=INFO msg="Uploading 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv (568B)"4947lb # [ 92.528632] niks3-server[962]: 2026/09/23 04:12:47 INFO Registered completed upload object_key=nar/0i2vbprwp0wxa61nkss61gzivqmmjh6klkgv0rlfnxk46rcaba1v.nar.zst4948lb # [ 92.539230] niks3-server[962]: 2026/09/23 04:12:47 INFO Registered completed upload object_key=nar/0p6zgq1gdngshl62q0hv3lslqwv90s7fmflwc7wgnycf4ka6ik69.nar.zst4949lb # [ 92.551233] niks3-server[962]: 2026/09/23 04:12:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign4950lb # [ 92.557695] niks3-server[962]: 2026/09/23 04:12:47 INFO Signed narinfos id=29 count=24951worker1 # [ 92.555885] nix-grpc-daemon[902]: time=2026-09-23T04:12:47.157Z level=INFO msg="Uploading 3 narinfos"4952lb # [ 92.562918] niks3-server[962]: 2026/09/23 04:12:47 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign4953lb # [ 92.569341] niks3-server[962]: 2026/09/23 04:12:47 INFO Signed narinfos id=28 count=14954lb # [ 92.573240] niks3-server[962]: 2026/09/23 04:12:47 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.ls4955lb # [ 92.575792] niks3-server[962]: 2026/09/23 04:12:47 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.ls4956lb # [ 92.593010] niks3-server[962]: 2026/09/23 04:12:47 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete4957lb # [ 92.600749] niks3-server[962]: 2026/09/23 04:12:47 INFO Completed upload id=284958lb # [ 92.604469] niks3-server[962]: 2026/09/23 04:12:47 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete4959lb # [ 92.608863] niks3-server[962]: 2026/09/23 04:12:47 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4960worker1 # [ 92.606372] nix-grpc-daemon[902]: time=2026-09-23T04:12:47.208Z level=INFO msg="Upload complete. (284ms)"4961lb # [ 92.611903] niks3-server[962]: 2026/09/23 04:12:47 INFO Completed upload id=294962lb # [ 92.614665] niks3-server[962]: 2026/09/23 04:12:47 INFO Registered completed upload object_key=4llc9br53475v1z1bdqrjyfi2sjvwnvr.narinfo4963lb # [ 92.618184] niks3-server[962]: 2026/09/23 04:12:47 INFO Registered completed upload object_key=71l2x9mybp0i90r6pdnglwjnayj4xpl8.narinfo4964worker1 # [ 92.617562] nix-grpc-daemon[766]: ts=2026-09-23T04:12:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=2 nar_bytes_in=9844965worker1 # [ 92.635517] nix-grpc-daemon[766]: ts=2026-09-23T04:12:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550524966worker1 # [ 92.646592] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4967worker1 # [ 92.654572] nix-daemon[849]: reaped child process 1137, status = succeeded4968client # building '/nix/store/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv' on lb:50051...4969worker1 # [ 92.666858] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)4970client # uses-input-w1onlya> node-a: building 71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4971client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds4972worker1 # [ 94.170445] nix-daemon[849]: reaped child process 1146, status = succeeded4973worker1 # [ 94.254371] nix-daemon[849]: accepted connection from pid 1162, user nix-grpc-daemon (trusted)4974worker1 # [ 94.272102] nix-daemon[849]: reaped child process 1165, status = succeeded4975lb # [ 94.279341] niks3-server[962]: 2026/09/23 04:12:48 INFO Received uploads request method=POST path=/api/pending_closures4976worker1 # [ 94.288172] nix-grpc-daemon[902]: time=2026-09-23T04:12:48.889Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"4977lb # [ 94.309665] niks3-server[962]: 2026/09/23 04:12:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign4978worker1 # [ 94.316463] nix-grpc-daemon[902]: time=2026-09-23T04:12:48.918Z level=INFO msg="Uploading 1 narinfos"4979lb # [ 94.324220] niks3-server[962]: 2026/09/23 04:12:48 INFO Signed narinfos id=30 count=14980lb # [ 94.331339] niks3-server[962]: 2026/09/23 04:12:48 INFO Registered completed upload object_key=log/71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv4981lb # [ 94.345649] niks3-server[962]: 2026/09/23 04:12:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.ls4982worker1 # [ 94.349571] nix-grpc-daemon[902]: time=2026-09-23T04:12:48.952Z level=INFO msg="Upload complete. (266ms)"4983lb # [ 94.351270] niks3-server[962]: 2026/09/23 04:12:48 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete4984lb # [ 94.355583] niks3-server[962]: 2026/09/23 04:12:48 INFO Completed upload id=304985lb # [ 94.362362] niks3-server[962]: 2026/09/23 04:12:48 INFO Registered completed upload object_key=2i7aa0cdpp60vpdm180r56gqfh25m7hc.narinfo4986worker1 # [ 94.360916] nix-grpc-daemon[766]: ts=2026-09-23T04:12:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=1 drv=71l2x9mybp0i90r6pdnglwjnayj4xpl8-uses-input-w1onlya.drv assign_id=15 outputs=14987worker1 # [ 94.376771] nix-daemon[849]: reaped child process 1144, status = succeeded4988client # copying 1 paths...4989worker1 # [ 94.406980] nix-grpc-daemon[766]: ts=2026-09-23T04:12:49Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550524990client # copying path '/nix/store/2i7aa0cdpp60vpdm180r56gqfh25m7hc-uses-input-w1onlya' from 'grpc://lb:50051'...4991worker1 # [ 94.440209] nix-grpc-daemon[766]: ts=2026-09-23T04:12:49Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550524992worker1 # [ 94.447035] nix-grpc-daemon[766]: ts=2026-09-23T04:12:49Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_out=1204993client # [ 94.578707] nix-daemon[1150]: reaped child process 1235, status = succeeded4994client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt a >&2, in 3.00 seconds)4995client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&24996client # [ 94.739731] nix-daemon[1150]: accepted connection from pid 1253, user root (trusted)4997client # this derivation will be built:4998client # /nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv4999client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5000worker2 # [ 94.887836] nix-grpc-daemon[771]: ts=2026-09-23T04:12:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:493705001worker1 # [ 94.886842] nix-grpc-daemon[766]: ts=2026-09-23T04:12:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550525002worker1 # [ 94.896667] nix-grpc-daemon[902]: time=2026-09-23T04:12:49.499Z level=INFO msg="All 1 paths already cached"5003worker1 # [ 94.913462] nix-grpc-daemon[766]: ts=2026-09-23T04:12:49Z level=debug event=want client=15 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv cached=05004worker1 # [ 94.920450] nix-grpc-daemon[766]: ts=2026-09-23T04:12:49Z level=debug event=assigned drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv worker=192.168.1.4:50051 clients=15005worker2 # [ 94.927118] nix-grpc-daemon[771]: ts=2026-09-23T04:12:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493705006worker2 # [ 94.942682] nix-grpc-daemon[771]: ts=2026-09-23T04:12:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:493705007client # copying 1 paths to 'lb:50051'...5008client # copying path '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' to 'lb:50051'...5009worker2 # [ 94.962366] nix-grpc-daemon[771]: ts=2026-09-23T04:12:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:493705010worker2 # [ 94.977998] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5011worker2 # [ 95.000817] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)5012worker2 # [ 95.060882] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 163 ms (attempt 2/5)5013worker2 # [ 95.225976] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 343 ms (attempt 3/5)5014worker2 # [ 95.570844] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 332 ms (attempt 4/5)5015worker2 # [ 95.914511] nix-grpc-daemon[771]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5016worker2 # [ 95.921507] nix-grpc-daemon[771]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5017worker2 # [ 95.928899] nix-grpc-daemon[771]: copying path '/nix/store/4llc9br53475v1z1bdqrjyfi2sjvwnvr-local-input-w1only.drv' from 'http://lb:5751'...5018worker2 # [ 96.107111] nix-daemon[840]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5019worker2 # [ 96.126204] nix-daemon[840]: reaped child process 1204, status = succeeded5020lb # [ 96.131983] niks3-server[962]: 2026/09/23 04:12:50 INFO Received uploads request method=POST path=/api/pending_closures5021worker2 # [ 96.146615] nix-grpc-daemon[853]: time=2026-09-23T04:12:50.725Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5022worker2 # [ 96.148491] nix-grpc-daemon[853]: time=2026-09-23T04:12:50.725Z level=INFO msg="Uploading 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv (568B)"5023lb # [ 96.166788] niks3-server[962]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.ls5024worker2 # [ 96.178581] nix-grpc-daemon[853]: time=2026-09-23T04:12:50.757Z level=INFO msg="Uploading 1 narinfos"5025lb # [ 96.178059] niks3-server[962]: 2026/09/23 04:12:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5026lb # [ 96.189925] niks3-server[962]: 2026/09/23 04:12:50 INFO Signed narinfos id=31 count=15027lb # [ 96.195961] niks3-server[962]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=nar/14jznd749vfl63s1gv7i29r5z6jcai2byp2nz83dr5gib6z2cyaz.nar.zst5028worker2 # [ 96.208679] nix-grpc-daemon[853]: time=2026-09-23T04:12:50.787Z level=INFO msg="Upload complete. (260ms)"5029lb # [ 96.206481] niks3-server[962]: 2026/09/23 04:12:50 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5030lb # [ 96.211528] niks3-server[962]: 2026/09/23 04:12:50 INFO Completed upload id=315031lb # [ 96.214330] niks3-server[962]: 2026/09/23 04:12:50 INFO Registered completed upload object_key=12ialcyr3l4m98rzfgk6apzimy55z3j8.narinfo5032worker2 # [ 96.220326] nix-grpc-daemon[771]: ts=2026-09-23T04:12:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=1 paths=1 nar_bytes_in=5685033worker2 # [ 96.232747] nix-grpc-daemon[771]: ts=2026-09-23T04:12:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:493705034worker2 # [ 96.245491] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5035worker2 # [ 96.251483] nix-daemon[840]: reaped child process 1194, status = succeeded5036worker2 # [ 96.266376] nix-grpc-daemon[771]: copying path '/nix/store/dymj0d1vj8q0ns6p096qk7fii766dap6-local-input-w1only' from 'http://lb:5751'...5037client # building '/nix/store/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv' on lb:50051...5038worker2 # [ 96.282938] nix-daemon[840]: accepted connection from pid 771, user nix-grpc-daemon (trusted)5039client # uses-input-w1onlyb> worker2: building 12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5040client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5041worker2 # [ 97.533767] nix-daemon[840]: reaped child process 1209, status = succeeded5042worker2 # [ 97.650392] nix-daemon[840]: accepted connection from pid 1225, user nix-grpc-daemon (trusted)5043worker2 # [ 97.667206] nix-daemon[840]: reaped child process 1228, status = succeeded5044lb # [ 97.674367] niks3-server[962]: 2026/09/23 04:12:52 INFO Received uploads request method=POST path=/api/pending_closures5045worker2 # [ 97.688385] nix-grpc-daemon[853]: time=2026-09-23T04:12:52.266Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5046lb # [ 97.704556] niks3-server[962]: 2026/09/23 04:12:52 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.ls5047lb # [ 97.716384] niks3-server[962]: 2026/09/23 04:12:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5048worker2 # [ 97.724723] nix-grpc-daemon[853]: time=2026-09-23T04:12:52.303Z level=INFO msg="Uploading 1 narinfos"5049lb # [ 97.728265] niks3-server[962]: 2026/09/23 04:12:52 INFO Signed narinfos id=32 count=15050lb # [ 97.734192] niks3-server[962]: 2026/09/23 04:12:52 INFO Registered completed upload object_key=log/12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv5051worker2 # [ 97.749607] nix-grpc-daemon[853]: time=2026-09-23T04:12:52.328Z level=INFO msg="Upload complete. (251ms)"5052lb # [ 97.747012] niks3-server[962]: 2026/09/23 04:12:52 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5053lb # [ 97.751024] niks3-server[962]: 2026/09/23 04:12:52 INFO Registered completed upload object_key=jfy62nvxcd0b9rkhly152q21ddbrcn28.narinfo5054lb # [ 97.756643] niks3-server[962]: 2026/09/23 04:12:52 INFO Completed upload id=325055client # copying 1 paths...5056worker1 # [ 97.766172] nix-grpc-daemon[766]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550525057worker2 # [ 97.761028] nix-grpc-daemon[771]: ts=2026-09-23T04:12:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=1 drv=12ialcyr3l4m98rzfgk6apzimy55z3j8-uses-input-w1onlyb.drv assign_id=16 outputs=15058worker2 # [ 97.777206] nix-daemon[840]: reaped child process 1206, status = succeeded5059worker1 # [ 97.778446] nix-grpc-daemon[766]: copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'http://lb:5751'...5060client # copying path '/nix/store/jfy62nvxcd0b9rkhly152q21ddbrcn28-uses-input-w1onlyb' from 'grpc://lb:50051'...5061worker2 # [ 97.827332] nix-grpc-daemon[771]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:493705062worker2 # [ 97.832921] nix-grpc-daemon[771]: ts=2026-09-23T04:12:52Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:49370 duration_s=0 paths=1 nar_bytes_out=1205063client # [ 97.908918] nix-daemon[1150]: reaped child process 1256, status = succeeded5064client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt b >&2, in 3.33 seconds)5065client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&25066client # [ 98.080176] nix-daemon[1150]: accepted connection from pid 1275, user root (trusted)5067client # this derivation will be built:5068client # /nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5069worker1 # [ 98.218764] nix-grpc-daemon[766]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:550525070client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux'...5071worker1 # [ 98.226497] nix-grpc-daemon[766]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550525072worker1 # [ 98.234472] nix-grpc-daemon[902]: time=2026-09-23T04:12:52.834Z level=INFO msg="All 1 paths already cached"5073worker1 # [ 98.245942] nix-grpc-daemon[766]: ts=2026-09-23T04:12:52Z level=debug event=want client=16 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv cached=05074worker1 # [ 98.255289] nix-grpc-daemon[766]: ts=2026-09-23T04:12:52Z level=debug event=assigned drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv worker=192.168.1.3:50051 clients=15075worker1 # [ 98.265181] nix-grpc-daemon[766]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550525076worker1 # [ 98.272680] nix-grpc-daemon[766]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:550525077worker1 # [ 98.282526] nix-grpc-daemon[902]: time=2026-09-23T04:12:52.885Z level=INFO msg="All 1 paths already cached"5078client # copying 1 paths to 'lb:50051'...5079client # copying path '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' to 'lb:50051'...5080worker1 # [ 98.293336] nix-grpc-daemon[766]: ts=2026-09-23T04:12:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:550525081worker1 # [ 98.308120] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5082worker1 # [ 98.455543] nix-daemon[849]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5083worker1 # [ 98.472994] nix-daemon[849]: reaped child process 1177, status = succeeded5084lb # [ 98.478923] niks3-server[962]: 2026/09/23 04:12:53 INFO Received uploads request method=POST path=/api/pending_closures5085worker1 # [ 98.487031] nix-grpc-daemon[902]: time=2026-09-23T04:12:53.089Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5086worker1 # [ 98.488562] nix-grpc-daemon[902]: time=2026-09-23T04:12:53.089Z level=INFO msg="Uploading n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv (568B)"5087lb # [ 98.508849] niks3-server[962]: 2026/09/23 04:12:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5088worker1 # [ 98.513635] nix-grpc-daemon[902]: time=2026-09-23T04:12:53.116Z level=INFO msg="Uploading 1 narinfos"5089lb # [ 98.521571] niks3-server[962]: 2026/09/23 04:12:53 INFO Signed narinfos id=33 count=15090lb # [ 98.530094] niks3-server[962]: 2026/09/23 04:12:53 INFO Registered completed upload object_key=nar/1qd3ps964395j2cc3rlqd58m21s7f6na9s0k2z5kclx0c3l4sl40.nar.zst5091lb # [ 98.538117] niks3-server[962]: 2026/09/23 04:12:53 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.ls5092worker1 # [ 98.546366] nix-grpc-daemon[902]: time=2026-09-23T04:12:53.148Z level=INFO msg="Upload complete. (224ms)"5093lb # [ 98.548180] niks3-server[962]: 2026/09/23 04:12:53 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5094lb # [ 98.553270] niks3-server[962]: 2026/09/23 04:12:53 INFO Completed upload id=335095worker1 # [ 98.556270] nix-grpc-daemon[766]: ts=2026-09-23T04:12:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_in=5685096lb # [ 98.560426] niks3-server[962]: 2026/09/23 04:12:53 INFO Registered completed upload object_key=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr.narinfo5097worker1 # [ 98.567940] nix-grpc-daemon[766]: ts=2026-09-23T04:12:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:550525098worker1 # [ 98.577444] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5099worker1 # [ 98.580355] nix-daemon[849]: reaped child process 1172, status = succeeded5100client # building '/nix/store/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv' on lb:50051...5101worker1 # [ 98.592949] nix-daemon[849]: accepted connection from pid 766, user nix-grpc-daemon (trusted)5102client # uses-input-w1onlyc> node-a: building n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5103client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5104lb # [ 99.897365] envoy[906]: conn peer=[2001:db8:1::1]:44710 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=438 tx=406 ms=715105lb # [ 99.902226] envoy[906]: conn peer=[2001:db8:1::1]:44704 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4352 tx=2416 ms=24325106lb # [ 99.907292] envoy[906]: conn peer=[2001:db8:1::1]:58078 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3918 tx=2084 ms=25815107lb # [ 99.912278] envoy[906]: conn peer=[2001:db8:1::1]:58080 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=328 ms=675108lb # [ 99.916980] envoy[906]: conn peer=[2001:db8:1::1]:58096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2124 ms=29575109lb # [ 99.922582] envoy[906]: conn peer=[2001:db8:1::1]:58102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=422 tx=328 ms=235110worker1 # [ 100.029201] nix-daemon[849]: reaped child process 1181, status = succeeded5111worker1 # [ 100.149482] nix-daemon[849]: accepted connection from pid 1197, user nix-grpc-daemon (trusted)5112worker1 # [ 100.169502] nix-daemon[849]: reaped child process 1200, status = succeeded5113lb # [ 100.175933] niks3-server[962]: 2026/09/23 04:12:54 INFO Received uploads request method=POST path=/api/pending_closures5114worker1 # [ 100.184410] nix-grpc-daemon[902]: time=2026-09-23T04:12:54.785Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5115lb # [ 100.207734] niks3-server[962]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.ls5116worker1 # [ 100.219028] nix-grpc-daemon[902]: time=2026-09-23T04:12:54.821Z level=INFO msg="Uploading 1 narinfos"5117lb # [ 100.222089] niks3-server[962]: 2026/09/23 04:12:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5118lb # [ 100.231943] niks3-server[962]: 2026/09/23 04:12:54 INFO Signed narinfos id=34 count=15119worker1 # [ 100.245283] nix-grpc-daemon[902]: time=2026-09-23T04:12:54.847Z level=INFO msg="Upload complete. (243ms)"5120lb # [ 100.241105] niks3-server[962]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=log/n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv5121lb # [ 100.252939] niks3-server[962]: 2026/09/23 04:12:54 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5122lb # [ 100.257918] niks3-server[962]: 2026/09/23 04:12:54 INFO Completed upload id=345123worker1 # [ 100.256939] nix-grpc-daemon[766]: ts=2026-09-23T04:12:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=1 drv=n7pcsnwv1a8b1wwcq715iprr5f6c4mcr-uses-input-w1onlyc.drv assign_id=17 outputs=15124lb # [ 100.260974] niks3-server[962]: 2026/09/23 04:12:54 INFO Registered completed upload object_key=13r5ipj38sk1irdi724wm1mbl9lw720b.narinfo5125client # copying 1 paths...5126worker1 # [ 100.269771] nix-daemon[849]: reaped child process 1179, status = succeeded5127worker2 # [ 100.275479] nix-grpc-daemon[771]: ts=2026-09-23T04:12:54Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:493705128worker2 # [ 100.288827] nix-grpc-daemon[771]: copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'http://lb:5751'...5129client # copying path '/nix/store/13r5ipj38sk1irdi724wm1mbl9lw720b-uses-input-w1onlyc' from 'grpc://lb:50051'...5130worker1 # [ 100.326856] nix-grpc-daemon[766]: ts=2026-09-23T04:12:54Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:550525131worker1 # [ 100.334223] nix-grpc-daemon[766]: ts=2026-09-23T04:12:54Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:55052 duration_s=0 paths=1 nar_bytes_out=1205132client # [ 100.421642] nix-daemon[1150]: reaped child process 1278, status = succeeded5133client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&system=aarch64-linux aarch64-linux - 4' --no-link -f /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix job --argstr tag w1only --argstr salt c >&2, in 2.51 seconds)5134(finished: subtest: an input only one worker has still reaches the builder, in 9.47 seconds)5135subtest: upload whose reference the worker lacks is completed from the cache5136client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache5137client # this derivation will be built:5138client # /nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv5139client # building '/nix/store/x1rrnjihpa1xvpyxxxdrjzp3c8z8nayq-local-input-viacache.drv'...5140client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A input --argstr tag viacache, in 0.33 seconds)5141client: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache5142client # this derivation will be built:5143client # /nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv5144client # building '/nix/store/1m587dk4visgyryf6n0mgp2rlp7pbjd1-refers-to-input-viacache.drv'...5145client: (finished: must succeed: nix-build --no-out-link /nix/store/xwnww2wldfcdimh1fkyvzhwrccjdhpmk-dep.nix -A referrer --argstr tag viacache, in 0.27 seconds)5146client: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure5147client: (finished: must succeed: nix-store --export /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache > /tmp/shared/ref.closure, in 0.12 seconds)5148worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5149worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.19 seconds)5150worker2 # [ 101.407128] systemd[1]: Stopping Nix worker-protocol over gRPC...5151worker2 # [ 102.267054] nix-grpc-daemon[771]: ts=2026-09-23T04:12:56Z level=info event=signal_exit5152worker2 # [ 103.441240] nix-grpc-daemon[771]: ts=2026-09-23T04:12:58Z level=info event=scheduler_restarting5153worker1 # [ 103.448468] nix-grpc-daemon[766]: ts=2026-09-23T04:12:58Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:550385154worker1 # [ 103.448758] nix-grpc-daemon[766]: ts=2026-09-23T04:12:58Z level=info event=worker_gone addr=192.168.1.4:500515155worker2 # [ 103.478681] nix-grpc-daemon[771]: ts=2026-09-23T04:12:58Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5156worker2 # [ 103.508524] nix-daemon[840]: reaped child process 842, status = succeeded5157worker2 # [ 104.234945] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5158worker2 # [ 104.239861] systemd[1]: Stopped Nix worker-protocol over gRPC.5159worker2 # [ 104.247872] systemd[1]: nix-grpc-daemon.service: Consumed 3.602s CPU time over 1min 24.207s wall clock time, 65.4M memory peak, 99K incoming IP traffic, 219.3K outgoing IP traffic.5160worker2 # [ 104.261803] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5161worker2 # [ 104.272173] systemd[1]: Closed Nix worker-protocol over gRPC.5162worker2 # [ 104.273974] systemd[1]: nix-grpc-daemon.socket: Consumed 557us CPU time over 1min 34.158s wall clock time, 256K memory peak, 98.7K incoming IP traffic, 87.8K outgoing IP traffic.5163lb: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy5164lb: must succeed: curl -sf localhost:9901/clusters5165lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5166lb: (finished: waiting until exactly ['worker2'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5167client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&25168worker1 # [ 104.555457] nix-grpc-daemon[766]: ts=2026-09-23T04:12:59Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:550525169worker1 # [ 104.564468] nix-grpc-daemon[766]: ts=2026-09-23T04:12:59Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:550525170client # /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5171client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache >&2, in 0.25 seconds)5172worker2: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache5173worker2: (finished: must fail: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache, in 0.03 seconds)5174worker2 # [ 104.695026] systemd[1]: Listening on Nix worker-protocol over gRPC.5175worker1 # [ 104.762005] systemd[1]: Stopping Nix worker-protocol over gRPC...5176worker1 # [ 105.143210] nix-grpc-daemon[766]: ts=2026-09-23T04:12:59Z level=info event=signal_exit5177worker2 # [ 105.433111] systemd[1]: Starting scheduler-token.service...5178lb # [ 105.443224] niks3-server[962]: 2026/09/23 04:13:00 INFO lead: released remote=192.168.1.3:570685179worker2 # [ 105.522774] systemd[1]: scheduler-token.service: Deactivated successfully.5180worker2 # [ 105.529459] systemd[1]: Finished scheduler-token.service.5181worker2 # [ 105.531623] systemd[1]: scheduler-token.service: Consumed 61ms CPU time over 94ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5182worker2 # [ 105.556237] systemd[1]: Starting Nix worker-protocol over gRPC...5183worker2 # [ 105.670721] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5184worker2 # [ 105.673108] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5185worker2 # [ 105.677082] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15186worker2 # [ 105.684048] nix-grpc-daemon[1253]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5187worker2 # [ 105.685535] nix-grpc-daemon[1253]: E0000 00:00:1790136780.260981 1253 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5188worker2 # [ 105.687858] nix-grpc-daemon[1253]: E0000 00:00:1790136780.263127 1253 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5189lb # [ 105.700191] niks3-server[962]: 2026/09/23 04:13:00 INFO lead: acquired remote=192.168.1.4:538945190worker2 # [ 105.735130] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5191worker2 # [ 105.737434] nix-grpc-daemon[1253]: E0000 00:00:1790136780.280850 1296 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5192worker2 # [ 105.739989] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_take_over5193worker2 # [ 105.741159] nix-grpc-daemon[1253]: E0000 00:00:1790136780.298995 1296 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5194worker2 # [ 105.746182] nix-grpc-daemon[1253]: E0000 00:00:1790136780.299154 1296 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5195worker2 # [ 105.751007] nix-grpc-daemon[1253]: E0000 00:00:1790136780.313499 1296 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5196worker2 # [ 105.753551] systemd[1]: Started Nix worker-protocol over gRPC.5197worker2 # [ 105.754369] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500515198worker2 # [ 105.755789] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5199worker2 # [ 105.952576] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500515200worker2 # [ 105.957591] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5201worker2 # [ 106.158631] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500515202worker2 # [ 106.164107] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5203worker2 # [ 106.364701] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_connected addr=lb:500515204worker2 # [ 106.369765] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5205worker1 # [ 106.445409] nix-grpc-daemon[766]: ts=2026-09-23T04:13:01Z level=info event=scheduler_restarting5206worker1 # [ 106.452377] nix-grpc-daemon[766]: ts=2026-09-23T04:13:01Z level=info event=scheduler_restarting addr=lb:500515207worker1 # [ 106.456978] nix-grpc-daemon[766]: ts=2026-09-23T04:13:01Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:550385208worker1 # [ 106.462198] nix-grpc-daemon[766]: ts=2026-09-23T04:13:01Z level=info event=worker_gone addr=192.168.1.3:500515209worker1 # [ 106.490444] nix-daemon[849]: reaped child process 851, status = succeeded5210worker2 # [ 106.570753] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:01Z level=info event=scheduler_connected addr=lb:500515211worker2 # [ 106.576277] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5212worker1 # [ 106.668608] nix-grpc-daemon[766]: ts=2026-09-23T04:13:01Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5213worker2 # [ 106.778253] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:01Z level=info event=scheduler_connected addr=lb:500515214worker2 # [ 106.795180] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:01Z level=info event=oidc_jwks provider=mock keys=15215worker2 # [ 106.799829] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:01Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:499545216worker2 # [ 106.805351] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:01Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05217worker1 # [ 107.257678] nix-grpc-daemon[766]: E0000 00:00:1790136781.859908 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5218worker1 # [ 107.286563] nix-grpc-daemon[766]: E0000 00:00:1790136781.888994 812 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5219worker1 # [ 108.178553] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5220worker1 # [ 108.184960] systemd[1]: Stopped Nix worker-protocol over gRPC.5221worker1 # [ 108.187611] systemd[1]: nix-grpc-daemon.service: Consumed 2.897s CPU time over 1min 28.292s wall clock time, 65M memory peak, 95.2K incoming IP traffic, 184.3K outgoing IP traffic.5222lb: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy5223lb: must succeed: curl -sf localhost:9901/clusters5224worker1 # [ 108.210804] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5225worker1 # [ 108.212974] systemd[1]: Closed Nix worker-protocol over gRPC.5226worker1 # [ 108.214466] systemd[1]: nix-grpc-daemon.socket: Consumed 544us CPU time over 1min 38.097s wall clock time, 256K memory peak, 120.1K incoming IP traffic, 104.3K outgoing IP traffic.5227lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5228lb: (finished: waiting until exactly ['worker1'] endpoints of aarch64-linux are unhealthy, in 0.07 seconds)5229client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&25230worker2 # [ 108.506843] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499645231worker2 # [ 108.522842] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5232worker2 # [ 108.549057] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 72 ms (attempt 1/5)5233worker2 # [ 108.629350] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 173 ms (attempt 2/5)5234worker2 # [ 108.804215] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 331 ms (attempt 3/5)5235worker2 # [ 109.137350] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 379 ms (attempt 4/5)5236worker2 # [ 109.522928] nix-grpc-daemon[1253]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5237worker2 # [ 109.529945] nix-grpc-daemon[1253]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5238client # copying 2 paths...5239client # copying path '/nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache' to 'grpc://lb:50051'...5240worker2 # [ 109.555415] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499645241client # copying path '/nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache' to 'grpc://lb:50051'...5242worker2 # [ 109.570552] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5243worker2 # [ 109.610358] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:04Z level=info event=niks3_push_started pid=13145244worker2 # [ 109.763228] nix-grpc-daemon[1314]: time=2026-09-23T04:13:04.342Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5245worker2 # [ 109.874499] nix-daemon[840]: accepted connection from pid 1326, user nix-grpc-daemon (trusted)5246worker2 # [ 109.893750] nix-daemon[840]: reaped child process 1329, status = succeeded5247lb # [ 109.897188] envoy[906]: conn peer=[2001:db8:1::1]:58114 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3724 tx=2107 ms=21315248lb # [ 109.902798] envoy[906]: conn peer=[2001:db8:1::1]:58118 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=422 tx=328 ms=265249lb # [ 109.910308] envoy[906]: conn peer=[2001:db8:1::4]:52356 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=51034 tx=3229 ms=831715250lb # [ 109.919583] envoy[906]: conn peer=[2001:db8:1::1]:60398 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=305251lb # [ 109.925004] envoy[906]: conn peer=[2001:db8:1::3]:43230 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=10634 tx=3071 ms=864765252worker2 # [ 109.936068] nix-grpc-daemon[1314]: time=2026-09-23T04:13:04.514Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5253worker2 # [ 109.937832] nix-grpc-daemon[1314]: time=2026-09-23T04:13:04.514Z level=INFO msg="Uploading syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache (128B)"5254lb # [ 109.934266] niks3-server[962]: 2026/09/23 04:13:04 INFO Received uploads request method=POST path=/api/pending_closures5255worker2 # [ 109.939820] nix-grpc-daemon[1314]: time=2026-09-23T04:13:04.514Z level=INFO msg="Uploading 4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache (184B)"5256lb # [ 109.938745] niks3-server[962]: 2026/09/23 04:13:04 INFO Received uploads request method=POST path=/api/pending_closures5257lb # [ 109.976939] niks3-server[962]: 2026/09/23 04:13:04 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5258lb # [ 109.988394] niks3-server[962]: 2026/09/23 04:13:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5259lb # [ 110.002069] niks3-server[962]: 2026/09/23 04:13:04 INFO Signed narinfos id=35 count=15260worker2 # [ 110.004109] nix-grpc-daemon[1314]: time=2026-09-23T04:13:04.580Z level=INFO msg="Uploading 3 narinfos"5261lb # [ 110.003716] niks3-server[962]: 2026/09/23 04:13:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5262lb # [ 110.013475] niks3-server[962]: 2026/09/23 04:13:04 INFO Registered completed upload object_key=nar/1rf1rg73pvnaa9kk6sy9r1ydj24b4g0kb0wnx0z5hxl1nvkmlavq.nar.zst5263lb # [ 110.022756] niks3-server[962]: 2026/09/23 04:13:04 INFO Signed narinfos id=36 count=25264lb # [ 110.028757] niks3-server[962]: 2026/09/23 04:13:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.ls5265lb # [ 110.035821] niks3-server[962]: 2026/09/23 04:13:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.ls5266lb # [ 110.043191] niks3-server[962]: 2026/09/23 04:13:04 INFO Registered completed upload object_key=4rig83zwv4xj1zajs08ima13w0yh676v.narinfo5267lb # [ 110.054897] niks3-server[962]: 2026/09/23 04:13:04 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5268lb # [ 110.061781] niks3-server[962]: 2026/09/23 04:13:04 INFO Completed upload id=355269lb # [ 110.065948] niks3-server[962]: 2026/09/23 04:13:04 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5270lb # [ 110.070505] niks3-server[962]: 2026/09/23 04:13:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5271lb # [ 110.072981] niks3-server[962]: 2026/09/23 04:13:04 INFO Registered completed upload object_key=syvivri0wisb7fd33rxcg9kz63kc2bv7.narinfo5272lb # [ 110.075987] niks3-server[962]: 2026/09/23 04:13:04 INFO Completed upload id=365273worker2 # [ 110.078541] nix-grpc-daemon[1314]: time=2026-09-23T04:13:04.657Z level=INFO msg="Upload complete. (311ms)"5274worker2 # [ 110.089153] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49964 duration_s=0 paths=2 nar_bytes_in=3125275worker2 # [ 110.108421] nix-daemon[840]: reaped child process 1312, status = succeeded5276client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache >&2, in 1.84 seconds)5277worker2: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache5278worker2: (finished: must succeed: test -e /nix/store/syvivri0wisb7fd33rxcg9kz63kc2bv7-local-input-viacache && test -e /nix/store/4rig83zwv4xj1zajs08ima13w0yh676v-refers-to-input-viacache, in 0.03 seconds)5279worker1 # [ 110.235255] systemd[1]: Listening on Nix worker-protocol over gRPC.5280worker1 # [ 110.245925] systemd[1]: Starting Nix worker-protocol over gRPC...5281worker1 # [ 110.399053] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5282worker1 # [ 110.404611] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5283worker1 # [ 110.406196] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15284worker1 # [ 110.412053] nix-grpc-daemon[1223]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5285worker1 # [ 110.413575] nix-grpc-daemon[1223]: E0000 00:00:1790136785.013074 1223 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5286lb: waiting until exactly no endpoints of aarch64-linux are unhealthy5287lb: must succeed: curl -sf localhost:9901/clusters5288worker1 # [ 110.456128] nix-grpc-daemon[1223]: E0000 00:00:1790136785.019850 1223 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5289worker1 # [ 110.458546] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5290worker1 # [ 110.465335] nix-grpc-daemon[1223]: E0000 00:00:1790136785.029913 1263 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5291worker1 # [ 110.468265] nix-grpc-daemon[1223]: E0000 00:00:1790136785.046621 1263 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5292worker1 # [ 110.470693] nix-grpc-daemon[1223]: E0000 00:00:1790136785.047814 1263 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5293worker1 # [ 110.473137] systemd[1]: Started Nix worker-protocol over gRPC.5294worker1 # [ 110.473933] nix-grpc-daemon[1223]: E0000 00:00:1790136785.066901 1263 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5295worker1 # [ 110.482190] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:05Z level=info event=scheduler_connected addr=lb:500515296worker2 # [ 110.487086] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:499545297worker2 # [ 110.494076] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05298lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5299lb: must succeed: curl -sf localhost:9901/clusters5300lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5301lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)5302(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.15 seconds)5303subtest: a niks3 restart does not cost the leader its role5304lb: must succeed: curl -sf localhost:9901/clusters5305lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5306lb: must succeed: curl -sf localhost:9901/clusters5307lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5308lb # [ 111.700399] systemd[1]: Stopping niks3 server...5309worker1 # [ 111.700262] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:06Z level=info event=scheduler_lock_lost status=2005310lb # [ 111.706504] niks3-server[962]: 2026/09/23 04:13:06 INFO Shutdown signal received, draining in-flight requests timeout=10s5311worker2 # [ 111.713871] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:06Z level=info event=scheduler_lock_lost status=2005312lb # [ 111.724132] niks3-server[962]: 2026/09/23 04:13:06 INFO lead: released remote=192.168.1.4:538945313lb # [ 111.730519] systemd[1]: niks3.service: Deactivated successfully.5314lb # [ 111.733066] systemd[1]: Stopped niks3 server.5315lb # [ 111.733785] systemd[1]: niks3.service: Consumed 639ms CPU time over 1min 29.521s wall clock time, 27.4M memory peak, 93.6K incoming IP traffic, 136.3K outgoing IP traffic.5316lb # [ 111.743296] systemd[1]: Starting niks3 server...5317lb # [ 111.802653] niks3-server[1102]: 2026/09/23 04:13:06 goose: no migrations to run. current version: 202609200000005318lb # [ 111.807465] niks3-server[1102]: 2026/09/23 04:13:06 OK 1_commit_pending_closure.sql (6.22ms)5319lb # [ 111.811145] niks3-server[1102]: 2026/09/23 04:13:06 OK 2_object_stats_trigger.sql (3.32ms)5320lb # [ 111.812915] niks3-server[1102]: 2026/09/23 04:13:06 goose: up to current file version: 25321lb # [ 111.814583] niks3-server[1102]: 2026/09/23 04:13:06 INFO Loaded signing key name=farm-test-1 path=/nix/store/3qiyvklx8qgcqqsb1swilx6drs1cdj8m-key5322lb # [ 111.824580] niks3-server[1102]: 2026/09/23 04:13:06 INFO Read proxy enabled — serving cache objects from S35323lb # [ 111.825910] niks3-server[1102]: 2026/09/23 04:13:06 INFO Using socket-activated listener address=0.0.0.0:57515324lb # [ 111.830600] niks3-server[1102]: 2026/09/23 04:13:06 INFO systemd watchdog enabled interval=15s5325lb # [ 111.831993] niks3-server[1102]: 2026/09/23 04:13:06 INFO Starting HTTP server address=0.0.0.0:57515326lb # [ 111.834317] systemd[1]: Started niks3 server.5327lb: waiting for unit niks3.service5328lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5329lb: must succeed: curl -sf localhost:9901/clusters5330lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5331worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5332worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5333lb: must succeed: curl -sf localhost:9901/clusters5334lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5335lb: must succeed: curl -sf localhost:9901/clusters5336lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5337worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5338worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.06 seconds)5339client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&25340worker2 # [ 112.309230] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:499645341client # don't know how to build these paths:5342client # /nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5343worker2 # [ 112.319319] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499645344worker2 # [ 112.350212] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:06Z level=debug event=want client=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv cached=05345worker2 # [ 112.357661] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:06Z level=debug event=assigned drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv worker=192.168.1.4:50051 clients=15346worker2 # [ 112.395243] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499645347worker2 # [ 112.410799] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499645348client # copying 1 paths to 'lb:50051'...5349client # copying path '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' to 'lb:50051'...5350worker2 # [ 112.427047] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499645351worker2 # [ 112.442168] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5352worker2 # [ 112.601275] nix-daemon[840]: accepted connection from pid 1346, user nix-grpc-daemon (trusted)5353worker2 # [ 112.619946] nix-daemon[840]: reaped child process 1349, status = succeeded5354lb # [ 112.620612] niks3-server[1102]: 2026/09/23 04:13:07 INFO Received uploads request method=POST path=/api/pending_closures5355worker2 # [ 112.637089] nix-grpc-daemon[1314]: time=2026-09-23T04:13:07.215Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5356worker2 # [ 112.638847] nix-grpc-daemon[1314]: time=2026-09-23T04:13:07.215Z level=INFO msg="Uploading mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv (544B)"5357lb # [ 112.663082] niks3-server[1102]: 2026/09/23 04:13:07 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.ls5358lb # [ 112.675123] niks3-server[1102]: 2026/09/23 04:13:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign5359lb # [ 112.690255] niks3-server[1102]: 2026/09/23 04:13:07 INFO Signed narinfos id=37 count=15360worker2 # [ 112.693619] nix-grpc-daemon[1314]: time=2026-09-23T04:13:07.271Z level=INFO msg="Uploading 1 narinfos"5361lb # [ 112.705005] niks3-server[1102]: 2026/09/23 04:13:07 INFO Registered completed upload object_key=nar/0hn539w6achi7azzyhpihsi61g85x9f535dxv438yq5y50ksrp1d.nar.zst5362lb # [ 112.719566] niks3-server[1102]: 2026/09/23 04:13:07 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete5363lb # [ 112.733518] niks3-server[1102]: 2026/09/23 04:13:07 INFO lead: acquired remote=192.168.1.4:565665364lb # [ 112.763873] niks3-server[1102]: 2026/09/23 04:13:07 INFO Completed upload id=375365worker2 # [ 112.766930] nix-grpc-daemon[1314]: time=2026-09-23T04:13:07.346Z level=INFO msg="Upload complete. (307ms)"5366lb # [ 112.769107] niks3-server[1102]: 2026/09/23 04:13:07 INFO Registered completed upload object_key=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z.narinfo5367worker2 # [ 112.778464] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49964 duration_s=0 paths=1 nar_bytes_in=5445368worker2 # [ 112.796435] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499645369worker2 # [ 112.809147] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5370worker2 # [ 112.812543] nix-daemon[840]: reaped child process 1344, status = succeeded5371client # building '/nix/store/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv' on lb:50051...5372worker2 # [ 112.826173] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5373client # farm-a> worker2: building mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5374client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5375client # farm-a> LOG-farm-a-after-niks3-restart5376worker2 # [ 114.499538] nix-daemon[840]: reaped child process 1355, status = succeeded5377worker2 # [ 114.616990] nix-daemon[840]: accepted connection from pid 1371, user nix-grpc-daemon (trusted)5378worker2 # [ 114.635390] nix-daemon[840]: reaped child process 1374, status = succeeded5379lb # [ 114.637196] niks3-server[1102]: 2026/09/23 04:13:09 INFO Received uploads request method=POST path=/api/pending_closures5380worker2 # [ 114.654859] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.231Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5381worker2 # [ 114.656833] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.231Z level=INFO msg="Uploading j3iswx3ynilvcycpyw0z667966c9kpzx-farm-a (144B)"5382lb # [ 114.684762] niks3-server[1102]: 2026/09/23 04:13:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign5383worker2 # [ 114.698068] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.276Z level=INFO msg="Uploading 1 narinfos"5384lb # [ 114.699315] niks3-server[1102]: 2026/09/23 04:13:09 INFO Signed narinfos id=38 count=15385lb # [ 114.710961] niks3-server[1102]: 2026/09/23 04:13:09 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5386lb # [ 114.724969] niks3-server[1102]: 2026/09/23 04:13:09 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.ls5387lb # [ 114.732156] niks3-server[1102]: 2026/09/23 04:13:09 INFO Registered completed upload object_key=log/mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv5388lb # [ 114.745971] niks3-server[1102]: 2026/09/23 04:13:09 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete5389lb # [ 114.753391] niks3-server[1102]: 2026/09/23 04:13:09 INFO Completed upload id=385390worker2 # [ 114.755653] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.334Z level=INFO msg="Upload complete. (313ms)"5391lb # [ 114.758323] niks3-server[1102]: 2026/09/23 04:13:09 INFO Registered completed upload object_key=j3iswx3ynilvcycpyw0z667966c9kpzx.narinfo5392worker2 # [ 114.778412] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49964 duration_s=1 drv=mxnzwd2qgmj6r8zhjk6jlcxv8j46h04z-farm-a.drv assign_id=1 outputs=15393worker2 # [ 114.798026] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:09Z level=debug event=want client=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv cached=05394worker2 # [ 114.802802] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:09Z level=debug event=assigned drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv worker=192.168.1.4:50051 clients=15395worker2 # [ 114.807792] nix-daemon[840]: reaped child process 1353, status = succeeded5396worker2 # [ 114.822990] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499645397worker2 # [ 114.838300] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499645398worker2 # [ 114.853676] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.432Z level=INFO msg="All 1 paths already cached"5399client # copying 1 paths to 'lb:50051'...5400client # copying path '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' to 'lb:50051'...5401worker2 # [ 114.864321] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499645402worker2 # [ 114.877593] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5403worker2 # [ 115.030370] nix-daemon[840]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5404worker2 # [ 115.049209] nix-daemon[840]: reaped child process 1383, status = succeeded5405lb # [ 115.052781] niks3-server[1102]: 2026/09/23 04:13:09 INFO Received uploads request method=POST path=/api/pending_closures5406worker2 # [ 115.074418] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.653Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5407worker2 # [ 115.076466] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.653Z level=INFO msg="Uploading 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv (656B)"5408lb # [ 115.109144] niks3-server[1102]: 2026/09/23 04:13:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign5409lb # [ 115.120775] niks3-server[1102]: 2026/09/23 04:13:09 INFO Registered completed upload object_key=nar/1xpcisa10g5by3yknnrs1xwhxhf4s243h7gwzjii0799pwbhfjfa.nar.zst5410lb # [ 115.132428] niks3-server[1102]: 2026/09/23 04:13:09 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.ls5411lb # [ 115.145669] niks3-server[1102]: 2026/09/23 04:13:09 INFO Signed narinfos id=39 count=15412worker2 # [ 115.149286] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.728Z level=INFO msg="Uploading 1 narinfos"5413lb # [ 115.157914] niks3-server[1102]: 2026/09/23 04:13:09 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete5414lb # [ 115.168411] niks3-server[1102]: 2026/09/23 04:13:09 INFO Registered completed upload object_key=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch.narinfo5415lb # [ 115.181378] niks3-server[1102]: 2026/09/23 04:13:09 INFO Completed upload id=395416worker2 # [ 115.184393] nix-grpc-daemon[1314]: time=2026-09-23T04:13:09.763Z level=INFO msg="Upload complete. (290ms)"5417worker2 # [ 115.196309] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49964 duration_s=0 paths=1 nar_bytes_in=6565418worker2 # [ 115.209058] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499645419worker2 # [ 115.223592] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5420worker2 # [ 115.233404] nix-daemon[840]: reaped child process 1378, status = succeeded5421client # building '/nix/store/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv' on lb:50051...5422client # farm-b> worker2: building 5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5423worker2 # [ 115.249940] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5424client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5425client # farm-b> LOG-farm-b-after-niks3-restart5426worker2 # [ 116.282078] nix-daemon[840]: reaped child process 1387, status = succeeded5427worker2 # [ 116.380239] nix-daemon[840]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)5428worker2 # [ 116.399321] nix-daemon[840]: reaped child process 1406, status = succeeded5429lb # [ 116.403745] niks3-server[1102]: 2026/09/23 04:13:11 INFO Received uploads request method=POST path=/api/pending_closures5430worker2 # [ 116.419378] nix-grpc-daemon[1314]: time=2026-09-23T04:13:10.997Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5431worker2 # [ 116.421072] nix-grpc-daemon[1314]: time=2026-09-23T04:13:10.997Z level=INFO msg="Uploading myar0ljvsqm9h7fxyh6qrqkg0175r3xn-farm-b (192B)"5432lb # [ 116.442761] niks3-server[1102]: 2026/09/23 04:13:11 INFO Registered completed upload object_key=log/5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv5433lb # [ 116.461694] niks3-server[1102]: 2026/09/23 04:13:11 INFO Registered completed upload object_key=nar/0zvv5rxmyk9afay83j52m7ipyhrqivhg6h6a0ksjd658pxzwi78w.nar.zst5434worker2 # [ 116.474250] nix-grpc-daemon[1314]: time=2026-09-23T04:13:11.053Z level=INFO msg="Uploading 1 narinfos"5435lb # [ 116.473525] niks3-server[1102]: 2026/09/23 04:13:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign5436lb # [ 116.480378] niks3-server[1102]: 2026/09/23 04:13:11 INFO Signed narinfos id=40 count=15437lb # [ 116.485404] niks3-server[1102]: 2026/09/23 04:13:11 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.ls5438lb # [ 116.494523] niks3-server[1102]: 2026/09/23 04:13:11 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete5439lb # [ 116.498546] niks3-server[1102]: 2026/09/23 04:13:11 INFO Completed upload id=405440worker2 # [ 116.501513] nix-grpc-daemon[1314]: time=2026-09-23T04:13:11.080Z level=INFO msg="Upload complete. (236ms)"5441lb # [ 116.502849] niks3-server[1102]: 2026/09/23 04:13:11 INFO Registered completed upload object_key=myar0ljvsqm9h7fxyh6qrqkg0175r3xn.narinfo5442worker2 # [ 116.520448] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49964 duration_s=1 drv=5ml8wmsr73y0fh0dflaimhsh2xbcd3ch-farm-b.drv assign_id=2 outputs=15443worker2 # [ 116.536574] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:11Z level=debug event=want client=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv cached=05444worker2 # [ 116.541279] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:11Z level=debug event=assigned drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv worker=192.168.1.4:50051 clients=15445worker2 # [ 116.545776] nix-daemon[840]: reaped child process 1385, status = succeeded5446worker2 # [ 116.563013] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499645447worker2 # [ 116.577381] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:499645448worker2 # [ 116.591421] nix-grpc-daemon[1314]: time=2026-09-23T04:13:11.170Z level=INFO msg="All 2 paths already cached"5449client # copying 1 paths to 'lb:50051'...5450client # copying path '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' to 'lb:50051'...5451worker2 # [ 116.602305] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:499645452worker2 # [ 116.616688] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5453worker2 # [ 116.768113] nix-daemon[840]: accepted connection from pid 1412, user nix-grpc-daemon (trusted)5454worker2 # [ 116.787575] nix-daemon[840]: reaped child process 1415, status = succeeded5455lb # [ 116.788994] niks3-server[1102]: 2026/09/23 04:13:11 INFO Received uploads request method=POST path=/api/pending_closures5456worker2 # [ 116.801340] nix-grpc-daemon[1314]: time=2026-09-23T04:13:11.380Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5457worker2 # [ 116.802944] nix-grpc-daemon[1314]: time=2026-09-23T04:13:11.380Z level=INFO msg="Uploading 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv (784B)"5458lb # [ 116.818331] niks3-server[1102]: 2026/09/23 04:13:11 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign5459worker2 # [ 116.830909] nix-grpc-daemon[1314]: time=2026-09-23T04:13:11.409Z level=INFO msg="Uploading 1 narinfos"5460lb # [ 116.834122] niks3-server[1102]: 2026/09/23 04:13:11 INFO Signed narinfos id=41 count=15461lb # [ 116.841014] niks3-server[1102]: 2026/09/23 04:13:11 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.ls5462lb # [ 116.852889] niks3-server[1102]: 2026/09/23 04:13:11 INFO Registered completed upload object_key=nar/1fnvw3c0v6n1j60nd5iagl3bggzcvixsw103w98mwhnpmnmgb248.nar.zst5463lb # [ 116.859576] niks3-server[1102]: 2026/09/23 04:13:11 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete5464worker2 # [ 116.861190] nix-grpc-daemon[1314]: time=2026-09-23T04:13:11.440Z level=INFO msg="Upload complete. (229ms)"5465lb # [ 116.868261] niks3-server[1102]: 2026/09/23 04:13:11 INFO Completed upload id=415466worker2 # [ 116.870744] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:49964 duration_s=0 paths=1 nar_bytes_in=7845467lb # [ 116.870310] niks3-server[1102]: 2026/09/23 04:13:11 INFO Registered completed upload object_key=6j7gr603xfslr5lxcbh72430f5q1r4b0.narinfo5468worker2 # [ 116.880701] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:499645469worker2 # [ 116.891163] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5470worker2 # [ 116.895701] nix-daemon[840]: reaped child process 1410, status = succeeded5471client # building '/nix/store/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv' on lb:50051...5472worker2 # [ 116.908228] nix-daemon[840]: accepted connection from pid 1253, user nix-grpc-daemon (trusted)5473client # farm-top> worker2: building 6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5474client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5475client # farm-top> LOG-farm-top-after-niks3-restart5476worker2 # [ 118.159518] nix-daemon[840]: reaped child process 1419, status = succeeded5477worker2 # [ 118.286759] nix-daemon[840]: accepted connection from pid 1435, user nix-grpc-daemon (trusted)5478worker2 # [ 118.306829] nix-daemon[840]: reaped child process 1438, status = succeeded5479lb # [ 118.308681] niks3-server[1102]: 2026/09/23 04:13:12 INFO Received uploads request method=POST path=/api/pending_closures5480worker2 # [ 118.325521] nix-grpc-daemon[1314]: time=2026-09-23T04:13:12.903Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5481worker2 # [ 118.327323] nix-grpc-daemon[1314]: time=2026-09-23T04:13:12.903Z level=INFO msg="Uploading z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc-farm-top (248B)"5482lb # [ 118.358624] niks3-server[1102]: 2026/09/23 04:13:12 INFO Registered completed upload object_key=nar/0cypkbh15calmx90jgh2waxpq661jalf4c99dgb7j606z61xh0ld.nar.zst5483worker2 # [ 118.374024] nix-grpc-daemon[1314]: time=2026-09-23T04:13:12.952Z level=INFO msg="Uploading 1 narinfos"5484lb # [ 118.372343] niks3-server[1102]: 2026/09/23 04:13:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign5485lb # [ 118.379799] niks3-server[1102]: 2026/09/23 04:13:12 INFO Registered completed upload object_key=log/6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv5486lb # [ 118.386340] niks3-server[1102]: 2026/09/23 04:13:12 INFO Signed narinfos id=42 count=15487lb # [ 118.393596] niks3-server[1102]: 2026/09/23 04:13:12 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.ls5488lb # [ 118.397671] niks3-server[1102]: 2026/09/23 04:13:13 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete5489worker2 # [ 118.400186] nix-grpc-daemon[1314]: time=2026-09-23T04:13:12.979Z level=INFO msg="Upload complete. (280ms)"5490lb # [ 118.401063] niks3-server[1102]: 2026/09/23 04:13:13 INFO Completed upload id=425491lb # [ 118.405382] niks3-server[1102]: 2026/09/23 04:13:13 INFO Registered completed upload object_key=z8mx9hni3m7cymmz0ddfhg3jhqvi9mlc.narinfo5492worker2 # [ 118.413129] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:49964 duration_s=1 drv=6j7gr603xfslr5lxcbh72430f5q1r4b0-farm-top.drv assign_id=3 outputs=15493worker2 # [ 118.433549] nix-daemon[840]: reaped child process 1417, status = succeeded5494client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart >&2, in 6.38 seconds)5495client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath5496client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag after-niks3-restart outPath, in 0.17 seconds)5497(finished: subtest: a niks3 restart does not cost the leader its role, in 7.04 seconds)5498subtest: scheduler down: the other node takes the lock and keeps it5499lb: must succeed: curl -sf localhost:9901/clusters5500lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5501lb: must succeed: curl -sf localhost:9901/clusters5502lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5503lb: must succeed: curl -sf localhost:9901/clusters5504lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5505lb: must succeed: curl -sf localhost:9901/clusters5506lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5507worker2 # [ 118.879557] systemd[1]: Stopping Nix worker-protocol over gRPC...5508worker2 # [ 119.702664] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:14Z level=info event=signal_exit5509lb # [ 119.722132] niks3-server[1102]: 2026/09/23 04:13:14 INFO lead: released remote=192.168.1.4:565665510lb # [ 119.901083] envoy[906]: conn peer=[2001:db8:1::1]:60410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1006 tx=304 ms=15995511lb # [ 119.906014] envoy[906]: conn peer=[2001:db8:1::1]:35226 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9749 tx=4449 ms=61195512worker1 # [ 120.721401] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:15Z level=info event=scheduler_restarting addr=lb:500515513worker2 # [ 120.729997] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:15Z level=info event=scheduler_restarting5514worker2 # [ 120.733922] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:15Z level=info event=scheduler_restarting addr=lb:500515515worker2 # [ 120.738725] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:15Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:499545516worker2 # [ 120.743764] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:15Z level=info event=worker_gone addr=192.168.1.3:500515517worker2 # [ 120.749585] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:15Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:499545518worker2 # [ 120.760474] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:15Z level=info event=worker_gone addr=192.168.1.4:500515519worker2 # [ 120.768900] nix-grpc-daemon[1253]: E0000 00:00:1790136795.347534 1299 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5520worker2 # [ 120.782700] nix-grpc-daemon[1253]: E0000 00:00:1790136795.347717 1299 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5521worker2 # [ 120.790505] nix-daemon[840]: reaped child process 1306, status = succeeded5522worker1 # [ 120.920930] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:15Z level=info event=scheduler_connected addr=lb:500515523worker1 # [ 120.926302] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5524worker2 # [ 120.945189] nix-grpc-daemon[1253]: ts=2026-09-23T04:13:15Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5525worker1 # [ 121.127078] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:15Z level=info event=scheduler_connected addr=lb:500515526worker1 # [ 121.132326] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5527worker1 # [ 121.332981] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:15Z level=info event=scheduler_connected addr=lb:500515528worker1 # [ 121.337994] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:15Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5529worker1 # [ 121.538452] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_connected addr=lb:500515530worker1 # [ 121.543504] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5531worker1 # [ 121.745854] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_connected addr=lb:500515532worker1 # [ 121.749975] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5533worker1 # [ 121.946287] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_connected addr=lb:500515534worker1 # [ 121.951438] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5535worker2 # [ 121.993559] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5536worker2 # [ 122.000572] systemd[1]: Stopped Nix worker-protocol over gRPC.5537worker2 # [ 122.003151] systemd[1]: nix-grpc-daemon.service: Consumed 1.373s CPU time over 16.451s wall clock time, 39M memory peak, 47.3K incoming IP traffic, 83.1K outgoing IP traffic.5538worker2 # [ 122.018384] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5539worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5540worker2 # [ 122.030134] systemd[1]: Closed Nix worker-protocol over gRPC.5541worker2 # [ 122.031691] systemd[1]: nix-grpc-daemon.socket: Consumed 867us CPU time over 17.336s wall clock time, 256K memory peak, 38.6K incoming IP traffic, 29.3K outgoing IP traffic.5542worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.08 seconds)5543worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5544worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5545worker1 # [ 122.151929] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_connected addr=lb:500515546worker1 # [ 122.155307] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5547worker1 # [ 122.356119] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_connected addr=lb:500515548worker1 # [ 122.361483] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:16Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5549worker1 # [ 122.562051] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_connected addr=lb:500515550worker1 # [ 122.567264] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5551worker1 # [ 122.705547] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_take_over5552lb # [ 122.707429] niks3-server[1102]: 2026/09/23 04:13:17 INFO lead: acquired remote=192.168.1.3:498125553worker1 # [ 122.769769] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_connected addr=lb:500515554worker1 # [ 122.773961] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5555worker1 # [ 122.970460] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_connected addr=lb:500515556worker1 # [ 122.975939] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5557worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5558worker1 # [ 123.177284] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_connected addr=lb:500515559worker1 # [ 123.188482] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5560worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5561worker1 # [ 123.384620] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=scheduler_connected addr=lb:500515562worker1 # [ 123.393969] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:526865563worker1 # [ 123.399253] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:17Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05564worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5565worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5566client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&25567worker1 # [ 124.517409] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:19Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:526985568worker1 # [ 124.530674] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5569client # don't know how to build these paths:5570client # /nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5571worker1 # [ 124.555201] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526985572worker1 # [ 124.571420] nix-grpc-daemon[1223]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)5573worker1 # [ 124.627735] nix-grpc-daemon[1223]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 115 ms (attempt 2/5)5574worker1 # [ 124.744573] nix-grpc-daemon[1223]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 120 ms (attempt 3/5)5575worker1 # [ 124.866459] nix-grpc-daemon[1223]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 270 ms (attempt 4/5)5576worker1 # [ 125.147773] nix-grpc-daemon[1223]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5577worker1 # [ 125.153884] nix-grpc-daemon[1223]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5578worker1 # [ 125.184240] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:19Z level=debug event=want client=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv cached=05579worker1 # [ 125.190127] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:19Z level=debug event=assigned drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv worker=192.168.1.3:50051 clients=15580worker1 # [ 125.235310] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526985581worker1 # [ 125.251027] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526985582client # copying 1 paths to 'lb:50051'...5583client # copying path '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' to 'lb:50051'...5584worker1 # [ 125.265245] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526985585worker1 # [ 125.279294] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5586worker1 # [ 125.306064] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:19Z level=info event=niks3_push_started pid=13035587worker1 # [ 125.456547] nix-grpc-daemon[1303]: time=2026-09-23T04:13:20.058Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""5588worker1 # [ 125.561831] nix-daemon[849]: accepted connection from pid 1316, user nix-grpc-daemon (trusted)5589worker1 # [ 125.581117] nix-daemon[849]: reaped child process 1319, status = succeeded5590lb # [ 125.583456] niks3-server[1102]: 2026/09/23 04:13:20 INFO Received uploads request method=POST path=/api/pending_closures5591worker1 # [ 125.592136] nix-grpc-daemon[1303]: time=2026-09-23T04:13:20.194Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5592worker1 # [ 125.593590] nix-grpc-daemon[1303]: time=2026-09-23T04:13:20.194Z level=INFO msg="Uploading adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv (520B)"5593lb # [ 125.629659] niks3-server[1102]: 2026/09/23 04:13:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign5594worker1 # [ 125.642826] nix-grpc-daemon[1303]: time=2026-09-23T04:13:20.244Z level=INFO msg="Uploading 1 narinfos"5595lb # [ 125.643177] niks3-server[1102]: 2026/09/23 04:13:20 INFO Signed narinfos id=43 count=15596lb # [ 125.652790] niks3-server[1102]: 2026/09/23 04:13:20 INFO Registered completed upload object_key=nar/0scd0jwin78ibwq92wkajw2zj7adk8bih7yhx8bxz1706hh53507.nar.zst5597lb # [ 125.663094] niks3-server[1102]: 2026/09/23 04:13:20 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.ls5598worker1 # [ 125.680087] nix-grpc-daemon[1303]: time=2026-09-23T04:13:20.282Z level=INFO msg="Upload complete. (219ms)"5599lb # [ 125.681353] niks3-server[1102]: 2026/09/23 04:13:20 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete5600lb # [ 125.691693] niks3-server[1102]: 2026/09/23 04:13:20 INFO Completed upload id=435601worker1 # [ 125.690336] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52698 duration_s=0 paths=1 nar_bytes_in=5205602lb # [ 125.696374] niks3-server[1102]: 2026/09/23 04:13:20 INFO Registered completed upload object_key=adj3g5w8jh83wl959p7daq6igl0gaay3.narinfo5603worker1 # [ 125.704432] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526985604worker1 # [ 125.712666] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5605worker1 # [ 125.716200] nix-daemon[849]: reaped child process 1301, status = succeeded5606client # building '/nix/store/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv' on lb:50051...5607client # farm-a> node-a: building adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5608worker1 # [ 125.729026] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5609client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5610client # farm-a> LOG-farm-a-on-standby5611worker1 # [ 127.437486] nix-daemon[849]: reaped child process 1323, status = succeeded5612worker1 # [ 127.515826] nix-daemon[849]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)5613worker1 # [ 127.531496] nix-daemon[849]: reaped child process 1342, status = succeeded5614lb # [ 127.540749] niks3-server[1102]: 2026/09/23 04:13:22 INFO Received uploads request method=POST path=/api/pending_closures5615worker1 # [ 127.551086] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.151Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5616worker1 # [ 127.553886] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.151Z level=INFO msg="Uploading 7qyaw17jjsjxkkgb1kfm3wxvczk56aan-farm-a (136B)"5617lb # [ 127.580477] niks3-server[1102]: 2026/09/23 04:13:22 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5618lb # [ 127.592778] niks3-server[1102]: 2026/09/23 04:13:22 INFO Registered completed upload object_key=log/adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv5619worker1 # [ 127.601253] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.203Z level=INFO msg="Uploading 1 narinfos"5620lb # [ 127.605224] niks3-server[1102]: 2026/09/23 04:13:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign5621lb # [ 127.612694] niks3-server[1102]: 2026/09/23 04:13:22 INFO Signed narinfos id=44 count=15622lb # [ 127.614698] niks3-server[1102]: 2026/09/23 04:13:22 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.ls5623lb # [ 127.624592] niks3-server[1102]: 2026/09/23 04:13:22 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete5624lb # [ 127.629355] niks3-server[1102]: 2026/09/23 04:13:22 INFO Registered completed upload object_key=7qyaw17jjsjxkkgb1kfm3wxvczk56aan.narinfo5625lb # [ 127.634023] niks3-server[1102]: 2026/09/23 04:13:22 INFO Completed upload id=445626worker1 # [ 127.631784] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.234Z level=INFO msg="Upload complete. (291ms)"5627worker1 # [ 127.653113] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52698 duration_s=1 drv=adj3g5w8jh83wl959p7daq6igl0gaay3-farm-a.drv assign_id=1 outputs=15628worker1 # [ 127.667970] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:22Z level=debug event=want client=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv cached=05629worker1 # [ 127.671595] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:22Z level=debug event=assigned drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv worker=192.168.1.3:50051 clients=15630worker1 # [ 127.675863] nix-daemon[849]: reaped child process 1321, status = succeeded5631worker1 # [ 127.700653] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526985632worker1 # [ 127.711559] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:22Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526985633worker1 # [ 127.724914] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.327Z level=INFO msg="All 1 paths already cached"5634client # copying 1 paths to 'lb:50051'...5635client # copying path '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' to 'lb:50051'...5636worker1 # [ 127.735293] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:22Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526985637worker1 # [ 127.748867] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5638worker1 # [ 127.894995] nix-daemon[849]: accepted connection from pid 1348, user nix-grpc-daemon (trusted)5639worker1 # [ 127.913838] nix-daemon[849]: reaped child process 1351, status = succeeded5640lb # [ 127.913884] niks3-server[1102]: 2026/09/23 04:13:22 INFO Received uploads request method=POST path=/api/pending_closures5641worker1 # [ 127.921642] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.523Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5642worker1 # [ 127.923182] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.523Z level=INFO msg="Uploading s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv (640B)"5643worker1 # [ 127.951220] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.553Z level=INFO msg="Uploading 1 narinfos"5644lb # [ 127.946257] niks3-server[1102]: 2026/09/23 04:13:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign5645lb # [ 127.960486] niks3-server[1102]: 2026/09/23 04:13:22 INFO Signed narinfos id=45 count=15646lb # [ 127.963622] niks3-server[1102]: 2026/09/23 04:13:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.ls5647worker1 # [ 127.979612] nix-grpc-daemon[1303]: time=2026-09-23T04:13:22.582Z level=INFO msg="Upload complete. (215ms)"5648lb # [ 127.980464] niks3-server[1102]: 2026/09/23 04:13:22 INFO Registered completed upload object_key=nar/0g5yl4v9pymang94vyp5ib5wi5jwf0bpp184y8qjavkpag12wmlv.nar.zst5649lb # [ 127.987214] niks3-server[1102]: 2026/09/23 04:13:22 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete5650worker1 # [ 127.989959] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52698 duration_s=0 paths=1 nar_bytes_in=6405651lb # [ 127.995274] niks3-server[1102]: 2026/09/23 04:13:22 INFO Completed upload id=455652lb # [ 127.997866] niks3-server[1102]: 2026/09/23 04:13:22 INFO Registered completed upload object_key=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g.narinfo5653worker1 # [ 128.003398] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526985654worker1 # [ 128.012318] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5655worker1 # [ 128.015527] nix-daemon[849]: reaped child process 1346, status = succeeded5656client # building '/nix/store/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv' on lb:50051...5657client # farm-b> node-a: building s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5658worker1 # [ 128.028483] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5659client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5660client # farm-b> LOG-farm-b-on-standby5661worker1 # [ 129.100740] nix-daemon[849]: reaped child process 1355, status = succeeded5662worker1 # [ 129.211256] nix-daemon[849]: accepted connection from pid 1371, user nix-grpc-daemon (trusted)5663worker1 # [ 129.227448] nix-daemon[849]: reaped child process 1374, status = succeeded5664lb # [ 129.236744] niks3-server[1102]: 2026/09/23 04:13:23 INFO Received uploads request method=POST path=/api/pending_closures5665worker1 # [ 129.246943] nix-grpc-daemon[1303]: time=2026-09-23T04:13:23.848Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5666worker1 # [ 129.248668] nix-grpc-daemon[1303]: time=2026-09-23T04:13:23.848Z level=INFO msg="Uploading 6hsxxv65wr2jz8cgv2v66iq16niq1gsv-farm-b (184B)"5667lb # [ 129.275286] niks3-server[1102]: 2026/09/23 04:13:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign5668worker1 # [ 129.288245] nix-grpc-daemon[1303]: time=2026-09-23T04:13:23.890Z level=INFO msg="Uploading 1 narinfos"5669lb # [ 129.293838] niks3-server[1102]: 2026/09/23 04:13:23 INFO Signed narinfos id=46 count=15670lb # [ 129.302658] niks3-server[1102]: 2026/09/23 04:13:23 INFO Registered completed upload object_key=log/s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv5671lb # [ 129.312242] niks3-server[1102]: 2026/09/23 04:13:23 INFO Registered completed upload object_key=nar/1grn64wcl2mc78k1jvrz5mcrk1ld70lhf8b9jpiqy8mvs6gfzq0v.nar.zst5672worker1 # [ 129.322741] nix-grpc-daemon[1303]: time=2026-09-23T04:13:23.925Z level=INFO msg="Upload complete. (283ms)"5673lb # [ 129.324416] niks3-server[1102]: 2026/09/23 04:13:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.ls5674lb # [ 129.327722] niks3-server[1102]: 2026/09/23 04:13:23 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete5675lb # [ 129.334518] niks3-server[1102]: 2026/09/23 04:13:23 INFO Completed upload id=465676lb # [ 129.337376] niks3-server[1102]: 2026/09/23 04:13:23 INFO Registered completed upload object_key=6hsxxv65wr2jz8cgv2v66iq16niq1gsv.narinfo5677worker1 # [ 129.340703] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52698 duration_s=1 drv=s5xf4hr9bjv2jdhhqy1ih0c1ld822d2g-farm-b.drv assign_id=2 outputs=15678worker1 # [ 129.355828] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:23Z level=debug event=want client=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv cached=05679worker1 # [ 129.360625] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:23Z level=debug event=assigned drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv worker=192.168.1.3:50051 clients=15680worker1 # [ 129.365246] nix-daemon[849]: reaped child process 1353, status = succeeded5681worker1 # [ 129.395221] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526985682worker1 # [ 129.410364] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526985683worker1 # [ 129.424271] nix-grpc-daemon[1303]: time=2026-09-23T04:13:24.026Z level=INFO msg="All 2 paths already cached"5684client # copying 1 paths to 'lb:50051'...5685client # copying path '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' to 'lb:50051'...5686worker1 # [ 129.435608] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526985687worker1 # [ 129.450282] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5688worker1 # [ 129.603126] nix-daemon[849]: accepted connection from pid 1380, user nix-grpc-daemon (trusted)5689worker1 # [ 129.619814] nix-daemon[849]: reaped child process 1383, status = succeeded5690lb # [ 129.626200] niks3-server[1102]: 2026/09/23 04:13:24 INFO Received uploads request method=POST path=/api/pending_closures5691worker1 # [ 129.633969] nix-grpc-daemon[1303]: time=2026-09-23T04:13:24.236Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5692worker1 # [ 129.635495] nix-grpc-daemon[1303]: time=2026-09-23T04:13:24.236Z level=INFO msg="Uploading 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv (768B)"5693worker1 # [ 129.654682] nix-grpc-daemon[1303]: time=2026-09-23T04:13:24.257Z level=INFO msg="Uploading 1 narinfos"5694lb # [ 129.655505] niks3-server[1102]: 2026/09/23 04:13:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign5695lb # [ 129.668783] niks3-server[1102]: 2026/09/23 04:13:24 INFO Signed narinfos id=47 count=15696lb # [ 129.677191] niks3-server[1102]: 2026/09/23 04:13:24 INFO Registered completed upload object_key=nar/1zyx46mv29cfygdhc5ai40qr4vl88lkr1lilm0h72j23w6wakgmc.nar.zst5697lb # [ 129.691531] niks3-server[1102]: 2026/09/23 04:13:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.ls5698worker1 # [ 129.696338] nix-grpc-daemon[1303]: time=2026-09-23T04:13:24.298Z level=INFO msg="Upload complete. (231ms)"5699lb # [ 129.697902] niks3-server[1102]: 2026/09/23 04:13:24 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete5700lb # [ 129.704968] niks3-server[1102]: 2026/09/23 04:13:24 INFO Registered completed upload object_key=7s5szig4ffga53df6rls98r9lhxsrsc5.narinfo5701lb # [ 129.710404] niks3-server[1102]: 2026/09/23 04:13:24 INFO Completed upload id=475702worker1 # [ 129.708138] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52698 duration_s=0 paths=1 nar_bytes_in=7685703worker1 # [ 129.720476] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526985704worker1 # [ 129.731195] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5705worker1 # [ 129.734709] nix-daemon[849]: reaped child process 1378, status = succeeded5706client # building '/nix/store/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv' on lb:50051...5707worker1 # [ 129.746186] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5708client # farm-top> node-a: building 7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5709lb # [ 129.899179] envoy[906]: conn peer=[2001:db8:1::4]:58660 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5513 tx=665 ms=151885710client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5711client # farm-top> LOG-farm-top-on-standby5712worker1 # [ 131.424941] nix-daemon[849]: reaped child process 1387, status = succeeded5713worker1 # [ 131.527364] nix-daemon[849]: accepted connection from pid 1403, user nix-grpc-daemon (trusted)5714worker1 # [ 131.544400] nix-daemon[849]: reaped child process 1406, status = succeeded5715lb # [ 131.551198] niks3-server[1102]: 2026/09/23 04:13:26 INFO Received uploads request method=POST path=/api/pending_closures5716worker1 # [ 131.562931] nix-grpc-daemon[1303]: time=2026-09-23T04:13:26.164Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5717worker1 # [ 131.565500] nix-grpc-daemon[1303]: time=2026-09-23T04:13:26.164Z level=INFO msg="Uploading 1ini8mfd1rzqyp8jgqihp4k889588mpx-farm-top (240B)"5718lb # [ 131.594427] niks3-server[1102]: 2026/09/23 04:13:26 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.ls5719worker1 # [ 131.602685] nix-grpc-daemon[1303]: time=2026-09-23T04:13:26.204Z level=INFO msg="Uploading 1 narinfos"5720lb # [ 131.604901] niks3-server[1102]: 2026/09/23 04:13:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign5721lb # [ 131.615561] niks3-server[1102]: 2026/09/23 04:13:26 INFO Signed narinfos id=48 count=15722lb # [ 131.620423] niks3-server[1102]: 2026/09/23 04:13:26 INFO Registered completed upload object_key=nar/1mncazpqn39nifc2fwbx80la5i264r8xc1lj8fnlbx006znc1925.nar.zst5723lb # [ 131.623726] niks3-server[1102]: 2026/09/23 04:13:26 INFO Registered completed upload object_key=log/7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv5724lb # [ 131.634511] niks3-server[1102]: 2026/09/23 04:13:26 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete5725lb # [ 131.638240] niks3-server[1102]: 2026/09/23 04:13:26 INFO Completed upload id=485726lb # [ 131.640906] niks3-server[1102]: 2026/09/23 04:13:26 INFO Registered completed upload object_key=1ini8mfd1rzqyp8jgqihp4k889588mpx.narinfo5727worker1 # [ 131.632770] nix-grpc-daemon[1303]: time=2026-09-23T04:13:26.235Z level=INFO msg="Upload complete. (272ms)"5728worker1 # [ 131.645908] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52698 duration_s=1 drv=7s5szig4ffga53df6rls98r9lhxsrsc5-farm-top.drv assign_id=3 outputs=15729worker1 # [ 131.664806] nix-daemon[849]: reaped child process 1385, status = succeeded5730client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby >&2, in 7.40 seconds)5731client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath5732client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag on-standby outPath, in 0.18 seconds)5733worker2 # [ 131.955374] systemd[1]: Listening on Nix worker-protocol over gRPC.5734worker2 # [ 131.964355] systemd[1]: Starting scheduler-token.service...5735worker2 # [ 132.043891] systemd[1]: scheduler-token.service: Deactivated successfully.5736worker2 # [ 132.052205] systemd[1]: Finished scheduler-token.service.5737worker2 # [ 132.053727] systemd[1]: scheduler-token.service: Consumed 46ms CPU time over 81ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5738worker2 # [ 132.059291] systemd[1]: Starting Nix worker-protocol over gRPC...5739worker2 # [ 132.181907] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:26Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5740worker2 # [ 132.184318] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:26Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5741worker2 # [ 132.188429] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:26Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15742worker2 # [ 132.195226] nix-grpc-daemon[1459]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5743worker2 # [ 132.212427] nix-grpc-daemon[1459]: E0000 00:00:1790136806.772153 1459 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5744lb: must succeed: curl -sf localhost:9901/clusters5745worker2 # [ 132.244273] nix-grpc-daemon[1459]: E0000 00:00:1790136806.774372 1459 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5746worker2 # [ 132.252831] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:26Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5747lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5748worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5749worker2 # [ 132.255125] nix-grpc-daemon[1459]: E0000 00:00:1790136806.797692 1500 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5750worker2 # [ 132.260518] nix-grpc-daemon[1459]: E0000 00:00:1790136806.812933 1500 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5751worker2 # [ 132.264896] nix-grpc-daemon[1459]: E0000 00:00:1790136806.813067 1500 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5752worker2 # [ 132.267722] nix-grpc-daemon[1459]: E0000 00:00:1790136806.831231 1500 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5753worker2 # [ 132.270676] systemd[1]: Started Nix worker-protocol over gRPC.5754worker2 # [ 132.271492] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:26Z level=info event=scheduler_connected addr=lb:500515755worker1 # [ 132.269973] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:26Z level=info event=oidc_jwks provider=mock keys=15756worker1 # [ 132.278879] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:26Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:526865757worker1 # [ 132.290397] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:26Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05758worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)5759lb: must succeed: curl -sf localhost:9901/clusters5760lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)5761lb: must succeed: curl -sf localhost:9901/clusters5762lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5763worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5764worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5765(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 13.87 seconds)5766subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise5767client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs5768client # [ 132.561241] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs.5769client # Running as unit: rs.service; invocation ID: 638ba5feb21a47e097990df0096f5f6b5770client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag rs, in 0.09 seconds)5771worker1 # [ 132.755819] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:526985772client # [ 132.768490] nix[1402]: don't know how to build these paths:5773client # [ 132.769481] nix[1402]: /nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv5774worker1 # [ 132.771532] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:27Z level=debug event=want client=2 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv cached=05775worker1 # [ 132.778369] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:27Z level=debug event=assigned drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv worker=192.168.1.3:50051 clients=15776worker1 # [ 132.818833] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526985777worker1 # [ 132.834592] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:526985778client # [ 132.852237] nix[1402]: copying 1 paths to 'lb:50051'...5779worker1 # [ 132.848780] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:526985780client # [ 132.856683] nix[1402]: copying path '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' to 'lb:50051'...5781worker1 # [ 132.863762] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5782worker1 # [ 133.019237] nix-daemon[849]: accepted connection from pid 1419, user nix-grpc-daemon (trusted)5783worker1 # [ 133.039188] nix-daemon[849]: reaped child process 1422, status = succeeded5784lb # [ 133.044828] niks3-server[1102]: 2026/09/23 04:13:27 INFO Received uploads request method=POST path=/api/pending_closures5785worker1 # [ 133.052910] nix-grpc-daemon[1303]: time=2026-09-23T04:13:27.655Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5786worker1 # [ 133.054992] nix-grpc-daemon[1303]: time=2026-09-23T04:13:27.657Z level=INFO msg="Uploading ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv (416B)"5787lb # [ 133.078223] niks3-server[1102]: 2026/09/23 04:13:27 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.ls5788worker1 # [ 133.086895] nix-grpc-daemon[1303]: time=2026-09-23T04:13:27.689Z level=INFO msg="Uploading 1 narinfos"5789lb # [ 133.091004] niks3-server[1102]: 2026/09/23 04:13:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign5790lb # [ 133.101324] niks3-server[1102]: 2026/09/23 04:13:27 INFO Signed narinfos id=49 count=15791lb # [ 133.107929] niks3-server[1102]: 2026/09/23 04:13:27 INFO Registered completed upload object_key=nar/0lynldv3n40qnhi1516wgiry3i3pja3sanwyyd8q4nslr915d7r1.nar.zst5792lb # [ 133.116144] niks3-server[1102]: 2026/09/23 04:13:27 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete5793lb # [ 133.123990] niks3-server[1102]: 2026/09/23 04:13:27 INFO Completed upload id=495794worker1 # [ 133.122065] nix-grpc-daemon[1303]: time=2026-09-23T04:13:27.724Z level=INFO msg="Upload complete. (242ms)"5795lb # [ 133.126347] niks3-server[1102]: 2026/09/23 04:13:27 INFO Registered completed upload object_key=ajji8m1zd5q5wi64gp7vclb5dd8hr19m.narinfo5796worker1 # [ 133.132979] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52698 duration_s=0 paths=1 nar_bytes_in=4165797worker1 # [ 133.152885] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:526985798worker1 # [ 133.156879] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5799worker1 # [ 133.162879] nix-daemon[849]: reaped child process 1417, status = succeeded5800client # [ 133.177600] nix[1402]: building '/nix/store/ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv' on lb:50051...5801worker1 # [ 133.174222] nix-daemon[849]: accepted connection from pid 1223, user nix-grpc-daemon (trusted)5802lb: must succeed: curl -sf localhost:9901/clusters5803lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5804lb: must succeed: curl -sf localhost:9901/clusters5805lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5806lb: must succeed: curl -sf localhost:9901/clusters5807lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5808lb: must succeed: curl -sf localhost:9901/clusters5809lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5810worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true5811worker2 # [ 135.369101] systemd-journald[271]: Received client request to rotate journal, rotating.5812worker2 # [ 135.389832] systemd-journald[271]: Vacuuming done, freed 0B of archived journals from /var/log/journal/98a9a6e0b8854ff2a37e9e979230b133.5813worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.09 seconds)5814worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service5815worker1 # [ 135.444360] systemd[1]: Stopping Nix worker-protocol over gRPC...5816worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.05 seconds)5817client: must succeed: sleep 5.05818worker1 # [ 136.433122] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:31Z level=info event=draining5819lb # [ 139.900576] envoy[906]: conn peer=[2001:db8:1::1]:45396 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9649 tx=4457 ms=71475820client: (finished: must succeed: sleep 5.0, in 5.05 seconds)5821worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='5822worker1 # [ 140.812557] nix-daemon[849]: reaped child process 1426, status = succeeded5823worker1 # [ 140.919380] nix-daemon[849]: accepted connection from pid 1467, user nix-grpc-daemon (trusted)5824worker1 # [ 140.936463] nix-daemon[849]: reaped child process 1470, status = succeeded5825lb # [ 140.945925] niks3-server[1102]: 2026/09/23 04:13:35 INFO Received uploads request method=POST path=/api/pending_closures5826worker1 # [ 140.956407] nix-grpc-daemon[1303]: time=2026-09-23T04:13:35.556Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5827worker1 # [ 140.957906] nix-grpc-daemon[1303]: time=2026-09-23T04:13:35.556Z level=INFO msg="Uploading k48d52qj0czpjmzj15yq3fd14zfw5iq6-slow-rs (120B)"5828lb # [ 140.985391] niks3-server[1102]: 2026/09/23 04:13:35 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst5829worker1 # [ 141.000461] nix-grpc-daemon[1303]: time=2026-09-23T04:13:35.602Z level=INFO msg="Uploading 1 narinfos"5830lb # [ 141.004486] niks3-server[1102]: 2026/09/23 04:13:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign5831lb # [ 141.015823] niks3-server[1102]: 2026/09/23 04:13:35 INFO Signed narinfos id=50 count=15832lb # [ 141.025050] niks3-server[1102]: 2026/09/23 04:13:35 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.ls5833worker1 # [ 141.023833] nix-grpc-daemon[1303]: time=2026-09-23T04:13:35.626Z level=INFO msg="Upload complete. (231ms)"5834lb # [ 141.027668] niks3-server[1102]: 2026/09/23 04:13:35 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete5835lb # [ 141.040973] niks3-server[1102]: 2026/09/23 04:13:35 INFO Completed upload id=505836worker1 # [ 141.034771] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52698 duration_s=7 drv=ajji8m1zd5q5wi64gp7vclb5dd8hr19m-slow-rs.drv assign_id=4 outputs=15837lb # [ 141.042609] niks3-server[1102]: 2026/09/23 04:13:35 INFO Registered completed upload object_key=k48d52qj0czpjmzj15yq3fd14zfw5iq6.narinfo5838worker1 # [ 141.053039] nix-daemon[849]: reaped child process 1424, status = succeeded5839client # [ 141.084774] systemd[1]: rs.service: Deactivated successfully.5840client # [ 141.089045] systemd[1]: rs.service: Consumed 192ms CPU time over 8.524s wall clock time, 41.2M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.5841worker1 # [ 141.437375] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:36Z level=info event=signal_exit5842lb # [ 141.719840] niks3-server[1102]: 2026/09/23 04:13:36 INFO lead: released remote=192.168.1.3:498125843lb # [ 142.211190] niks3-server[1102]: 2026/09/23 04:13:36 INFO lead: acquired remote=192.168.1.4:568745844worker2 # [ 142.214318] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:36Z level=info event=scheduler_take_over5845worker2 # [ 142.716819] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:37Z level=info event=scheduler_restarting addr=lb:500515846worker1 # [ 142.717990] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:37Z level=info event=scheduler_restarting5847worker1 # [ 142.722606] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:37Z level=info event=scheduler_restarting addr=lb:500515848worker1 # [ 142.727703] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:526865849worker1 # [ 142.732594] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:37Z level=info event=worker_gone addr=192.168.1.3:500515850worker1 # [ 142.737237] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:37Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:526865851worker1 # [ 142.743076] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:37Z level=info event=worker_gone addr=192.168.1.4:500515852worker1 # [ 142.765897] nix-daemon[849]: reaped child process 1293, status = succeeded5853worker2 # [ 142.918256] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:37Z level=info event=scheduler_connected addr=lb:500515854worker2 # [ 142.933714] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:37Z level=info event=oidc_jwks provider=mock keys=15855worker1 # [ 142.936915] nix-grpc-daemon[1223]: ts=2026-09-23T04:13:37Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5856worker2 # [ 142.941480] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:37Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:520845857worker2 # [ 142.951566] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:37Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=05858worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.20 seconds)5859client: waiting for success: ! systemctl is-active rs5860client: (finished: waiting for success: ! systemctl is-active rs, in 0.06 seconds)5861client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }5862client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.05 seconds)5863client: must fail: journalctl -u rs | grep -q 'reconnecting'5864client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.03 seconds)5865lb: must succeed: curl -sf localhost:9901/clusters5866lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5867worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5868worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5869worker1 # [ 143.264103] nix-grpc-daemon[1223]: E0000 00:00:1790136817.866426 1269 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5870worker1 # [ 143.271656] nix-grpc-daemon[1223]: E0000 00:00:1790136817.873914 1269 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5871lb: must succeed: curl -sf localhost:9901/clusters5872lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5873worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5874worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)5875worker1 # [ 144.373509] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5876worker1 # [ 144.379906] systemd[1]: Stopped Nix worker-protocol over gRPC.5877worker1 # [ 144.384498] systemd[1]: nix-grpc-daemon.service: Consumed 1.591s CPU time over 34.132s wall clock time, 38.8M memory peak, 49K incoming IP traffic, 92.8K outgoing IP traffic.5878worker1 # [ 144.403320] systemd[1]: Starting Nix worker-protocol over gRPC...5879worker1 # [ 144.539719] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:39Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5880worker1 # [ 144.542142] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:39Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5881worker1 # [ 144.545974] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:39Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15882worker1 # [ 144.551378] nix-grpc-daemon[1477]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5883worker1 # [ 144.568159] nix-grpc-daemon[1477]: E0000 00:00:1790136819.153236 1477 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5884worker1 # [ 144.570604] nix-grpc-daemon[1477]: E0000 00:00:1790136819.156909 1477 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5885worker1 # [ 144.596433] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:39Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5886worker1 # [ 144.598668] nix-grpc-daemon[1477]: E0000 00:00:1790136819.163081 1520 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5887worker1 # [ 144.601365] systemd[1]: Started Nix worker-protocol over gRPC.5888worker1 # [ 144.602210] nix-grpc-daemon[1477]: E0000 00:00:1790136819.183484 1520 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5889worker1 # [ 144.607526] nix-grpc-daemon[1477]: E0000 00:00:1790136819.183655 1520 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5890worker1 # [ 144.612194] nix-grpc-daemon[1477]: E0000 00:00:1790136819.198017 1520 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5891worker1 # [ 144.614675] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:39Z level=info event=scheduler_connected addr=lb:500515892worker2 # [ 144.618668] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:39Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:520845893worker2 # [ 144.624146] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:39Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=05894lb: must succeed: curl -sf localhost:9901/clusters5895lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5896worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5897worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)5898(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 12.98 seconds)5899subtest: scheduler restart mid-build: build finishes, no double build5900client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid5901client # [ 145.539540] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid.5902client # Running as unit: mid.service; invocation ID: a81327ed3ff74f1ca390b124539c97cc5903client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag mid, in 0.09 seconds)5904worker2 # [ 145.748784] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:40Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520885905worker2 # [ 145.766720] nix-daemon[840]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5906client # [ 145.781057] nix[1438]: don't know how to build these paths:5907client # [ 145.782698] nix[1438]: /nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv5908worker2 # [ 145.793165] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:40Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=05909worker2 # [ 145.799338] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:40Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=15910worker2 # [ 145.847439] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520885911worker2 # [ 145.863135] nix-grpc-daemon[1459]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)5912worker2 # [ 145.889241] nix-grpc-daemon[1459]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 2/5)5913worker2 # [ 145.940917] nix-grpc-daemon[1459]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 3/5)5914worker2 # [ 145.992716] nix-grpc-daemon[1459]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 566 ms (attempt 4/5)5915worker2 # [ 146.568354] nix-grpc-daemon[1459]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5916worker2 # [ 146.574664] nix-grpc-daemon[1459]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5917worker2 # [ 146.593432] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520885918client # [ 146.607006] nix[1438]: copying 1 paths to 'lb:50051'...5919worker2 # [ 146.611966] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520885920worker2 # [ 146.626488] nix-daemon[840]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5921worker2 # [ 146.659787] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:41Z level=info event=niks3_push_started pid=15855922worker2 # [ 146.805894] nix-grpc-daemon[1585]: time=2026-09-23T04:13:41.381Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""5923worker2 # [ 146.928298] nix-daemon[840]: accepted connection from pid 1601, user nix-grpc-daemon (trusted)5924worker2 # [ 146.945514] nix-daemon[840]: reaped child process 1604, status = succeeded5925lb # [ 146.949849] niks3-server[1102]: 2026/09/23 04:13:41 INFO Received uploads request method=POST path=/api/pending_closures5926worker2 # [ 146.966982] nix-grpc-daemon[1585]: time=2026-09-23T04:13:41.545Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5927worker2 # [ 146.970873] nix-grpc-daemon[1585]: time=2026-09-23T04:13:41.545Z level=INFO msg="Uploading chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv (424B)"5928lb # [ 147.002448] niks3-server[1102]: 2026/09/23 04:13:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign5929worker2 # [ 147.014696] nix-grpc-daemon[1585]: time=2026-09-23T04:13:41.592Z level=INFO msg="Uploading 1 narinfos"5930lb # [ 147.013112] niks3-server[1102]: 2026/09/23 04:13:41 INFO Signed narinfos id=51 count=15931lb # [ 147.020788] niks3-server[1102]: 2026/09/23 04:13:41 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.ls5932lb # [ 147.027996] niks3-server[1102]: 2026/09/23 04:13:41 INFO Registered completed upload object_key=nar/14lxjrdcxwbhc1kgg39y29gnbx5cz80j4852d1i4w3rvq7g4b04g.nar.zst5933lb # [ 147.037274] niks3-server[1102]: 2026/09/23 04:13:41 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete5934worker2 # [ 147.048759] nix-grpc-daemon[1585]: time=2026-09-23T04:13:41.627Z level=INFO msg="Upload complete. (245ms)"5935lb # [ 147.046404] niks3-server[1102]: 2026/09/23 04:13:41 INFO Completed upload id=515936lb # [ 147.056569] niks3-server[1102]: 2026/09/23 04:13:41 INFO Registered completed upload object_key=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc.narinfo5937worker2 # [ 147.060155] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:41Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52088 duration_s=0 paths=1 nar_bytes_in=4245938worker2 # [ 147.074099] nix-grpc-daemon[1459]: ts=2026-09-23T04:13:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520885939worker2 # [ 147.082578] nix-daemon[840]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5940worker2 # [ 147.084769] nix-daemon[840]: reaped child process 1583, status = succeeded5941worker2 # [ 147.097435] nix-daemon[840]: accepted connection from pid 1459, user nix-grpc-daemon (trusted)5942client # [ 146.611775] nix[1438]: copying path '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' to 'lb:50051'...5943client # [ 147.096720] nix[1438]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...5944lb: must succeed: curl -sf localhost:9901/clusters5945lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)5946lb: must succeed: curl -sf localhost:9901/clusters5947lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5948worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service5949worker2 # [ 149.406751] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1459 (nix-grpc-daemon) on client request.5950worker2 # [ 149.408831] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.5951lb # [ 149.426861] niks3-server[1102]: 2026/09/23 04:13:44 INFO lead: released remote=192.168.1.4:568745952worker2 # [ 149.435706] systemd[1]: Starting scheduler-token.service...5953client # [ 149.431560] nix[1438]: chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again5954worker1 # [ 149.426362] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"5955client # [ 149.437995] nix[1438]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting5956worker2 # [ 149.448663] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL5957worker2 # [ 149.450140] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.5958worker2 # [ 149.451416] systemd[1]: nix-grpc-daemon.service: Consumed 489ms CPU time over 17.376s wall clock time, 35M memory peak, 15.5K incoming IP traffic, 27.5K outgoing IP traffic.5959worker2 # [ 149.487918] nix-daemon[840]: reaped child process 1575, status = succeeded5960worker2 # [ 149.490057] nix-daemon[840]: reaped child process 1606, status = succeeded5961worker2 # [ 149.496413] nix-daemon[1608]: unexpected Nix daemon error: error: interrupted by the user5962worker2 # [ 149.507265] nix-daemon[840]: reaped child process 1608, status = failed with exit code 15963worker2 # [ 149.525291] systemd[1]: scheduler-token.service: Deactivated successfully.5964worker2 # [ 149.526265] systemd[1]: Finished scheduler-token.service.5965worker2 # [ 149.527021] systemd[1]: scheduler-token.service: Consumed 24ms CPU time over 89ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5966worker2 # [ 149.532708] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.5967worker2 # [ 149.534197] systemd[1]: Starting scheduler-token.service...5968lb # [ 149.563182] niks3-server[1102]: 2026/09/23 04:13:44 INFO lead: acquired remote=192.168.1.3:389545969worker1 # [ 149.566869] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_take_over5970worker2 # [ 149.580792] systemd[1]: scheduler-token.service: Deactivated successfully.5971worker2 # [ 149.584136] systemd[1]: Finished scheduler-token.service.5972worker2 # [ 149.584926] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 50ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5973worker2 # [ 149.588876] systemd[1]: Starting Nix worker-protocol over gRPC...5974worker1 # [ 149.626337] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_connected addr=lb:500515975worker2 # [ 149.690641] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5976worker2 # [ 149.692867] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5977worker2 # [ 149.696641] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15978worker2 # [ 149.701895] nix-grpc-daemon[1651]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5979worker2 # [ 149.703349] nix-grpc-daemon[1651]: E0000 00:00:1790136824.280388 1651 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5980worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.34 seconds)5981client: must succeed: sleep 5.05982worker1 # [ 149.721079] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5983worker2 # [ 149.717832] nix-grpc-daemon[1651]: E0000 00:00:1790136824.286280 1651 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5984worker2 # [ 149.746307] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5985worker2 # [ 149.747978] systemd[1]: Started Nix worker-protocol over gRPC.5986worker2 # [ 149.754882] nix-grpc-daemon[1651]: E0000 00:00:1790136824.297462 1691 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5987worker2 # [ 149.757811] nix-grpc-daemon[1651]: E0000 00:00:1790136824.314153 1691 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5988worker2 # [ 149.760301] nix-grpc-daemon[1651]: E0000 00:00:1790136824.314288 1691 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5989worker2 # [ 149.765370] nix-grpc-daemon[1651]: E0000 00:00:1790136824.333484 1691 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5990worker2 # [ 149.769889] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=scheduler_connected addr=lb:500515991worker2 # [ 149.771763] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5992lb # [ 149.902840] envoy[906]: conn peer=[2001:db8:1::1]:39736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2979 tx=1516 ms=83065993lb # [ 149.908049] envoy[906]: conn peer=[2001:db8:1::3]:45056 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3126 tx=1050 ms=324475994lb # [ 149.912972] envoy[906]: conn peer=[2001:db8:1::4]:48748 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1935 tx=315 ms=171745995worker1 # [ 149.921885] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_connected addr=lb:500515996worker1 # [ 149.927292] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"5997client # [ 149.936086] nix[1438]: scheduler at lb:50051: scheduler passive, another node serves, reconnecting5998worker2 # [ 149.973029] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=scheduler_connected addr=lb:500515999worker2 # [ 149.979496] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6000worker1 # [ 150.128070] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_connected addr=lb:500516001worker1 # [ 150.133277] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6002worker2 # [ 150.180559] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=scheduler_connected addr=lb:500516003worker2 # [ 150.187127] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6004worker1 # [ 150.334059] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_connected addr=lb:500516005worker1 # [ 150.339420] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6006worker2 # [ 150.388225] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=scheduler_connected addr=lb:500516007worker2 # [ 150.393180] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:44Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6008worker1 # [ 150.541781] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516009worker1 # [ 150.545926] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6010worker2 # [ 150.594059] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516011worker2 # [ 150.599502] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6012worker1 # [ 150.742081] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516013worker1 # [ 150.747342] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6014worker2 # [ 150.802106] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516015worker2 # [ 150.806341] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6016client # [ 150.939634] nix[1438]: scheduler at lb:50051: no healthy upstream, reconnecting6017worker1 # [ 150.948073] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516018worker1 # [ 150.953138] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6019worker2 # [ 151.002733] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516020worker2 # [ 151.007944] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6021worker1 # [ 151.153892] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516022worker1 # [ 151.159034] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6023worker2 # [ 151.209259] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516024worker2 # [ 151.214421] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6025worker1 # [ 151.359767] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516026worker1 # [ 151.365101] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:45Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6027worker2 # [ 151.415391] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:45Z level=info event=scheduler_connected addr=lb:500516028worker1 # [ 151.433524] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:46Z level=info event=oidc_jwks provider=mock keys=16029worker1 # [ 151.438180] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:46Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:506086030worker1 # [ 151.444626] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:46Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06031worker1 # [ 151.568244] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:46Z level=info event=scheduler_connected addr=lb:500516032worker1 # [ 151.572346] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:46Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:506086033worker1 # [ 151.577134] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:46Z level=info event=worker_hello addr=192.168.1.3:50051 systems=aarch64-linux max_jobs=1 running=06034worker1 # [ 152.945830] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:47Z level=debug event=want client=1 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv cached=06035worker1 # [ 152.951847] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:47Z level=debug event=assigned drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv worker=192.168.1.4:50051 clients=16036worker2 # [ 153.001125] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566566037worker2 # [ 153.016425] nix-daemon[840]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)6038worker2 # [ 153.039274] nix-daemon[840]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)6039client # [ 153.056566] nix[1438]: building '/nix/store/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv' on lb:50051...6040worker2 # [ 153.058651] nix-daemon[840]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)6041client: (finished: must succeed: sleep 5.0, in 5.05 seconds)6042client: waiting for success: ! systemctl is-active mid6043worker2 # [ 155.074956] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:49Z level=info event=niks3_push_started pid=17336044worker2 # [ 155.108822] nix-daemon[840]: reaped child process 1708, status = succeeded6045worker2 # [ 155.207856] nix-grpc-daemon[1733]: time=2026-09-23T04:13:49.786Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6046worker2 # [ 155.318226] nix-daemon[840]: accepted connection from pid 1746, user nix-grpc-daemon (trusted)6047worker2 # [ 155.337781] nix-daemon[840]: reaped child process 1749, status = succeeded6048lb # [ 155.334409] niks3-server[1102]: 2026/09/23 04:13:49 INFO Received uploads request method=POST path=/api/pending_closures6049worker2 # [ 155.348162] nix-grpc-daemon[1733]: time=2026-09-23T04:13:49.927Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6050worker2 # [ 155.353156] nix-grpc-daemon[1733]: time=2026-09-23T04:13:49.929Z level=INFO msg="Uploading fk2vdwmn2msb3ja19s2jwak35ignz889-slow-mid (120B)"6051lb # [ 155.393801] niks3-server[1102]: 2026/09/23 04:13:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6052lb # [ 155.408456] niks3-server[1102]: 2026/09/23 04:13:50 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.ls6053lb # [ 155.421769] niks3-server[1102]: 2026/09/23 04:13:50 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6054lb # [ 155.444485] niks3-server[1102]: 2026/09/23 04:13:50 INFO Signed narinfos id=52 count=16055worker2 # [ 155.448151] nix-grpc-daemon[1733]: time=2026-09-23T04:13:50.026Z level=INFO msg="Uploading 1 narinfos"6056lb # [ 155.455470] niks3-server[1102]: 2026/09/23 04:13:50 INFO Registered completed upload object_key=log/chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv6057lb # [ 155.467568] niks3-server[1102]: 2026/09/23 04:13:50 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6058lb # [ 155.475725] niks3-server[1102]: 2026/09/23 04:13:50 INFO Completed upload id=526059worker2 # [ 155.478390] nix-grpc-daemon[1733]: time=2026-09-23T04:13:50.057Z level=INFO msg="Upload complete. (266ms)"6060lb # [ 155.483370] niks3-server[1102]: 2026/09/23 04:13:50 INFO Registered completed upload object_key=fk2vdwmn2msb3ja19s2jwak35ignz889.narinfo6061worker2 # [ 155.492362] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56656 duration_s=2 drv=chbb3scb8b4b0sd5wk0a0vcrl8x07pdc-slow-mid.drv assign_id=1 outputs=16062worker2 # [ 155.509988] nix-daemon[840]: reaped child process 1706, status = succeeded6063client # [ 155.525995] systemd[1]: mid.service: Deactivated successfully.6064client # [ 155.531023] systemd[1]: mid.service: Consumed 202ms CPU time over 9.987s wall clock time, 41.4M memory peak, 9.2K incoming IP traffic, 11.3K outgoing IP traffic.6065client: (finished: waiting for success: ! systemctl is-active mid, in 1.12 seconds)6066client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6067client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.06 seconds)6068(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.72 seconds)6069subtest: interrupting the client stops the build on the worker6070client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr6071client # [ 156.232467] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr.6072client # Running as unit: intr.service; invocation ID: 02485d03a7244f3b9112e24cf96b0a0c6073client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag intr, in 0.05 seconds)6074worker1 # [ 156.416434] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506186075worker1 # [ 156.429293] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6076client # [ 156.451976] nix[1474]: don't know how to build these paths:6077client # [ 156.454460] nix[1474]: /nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6078worker1 # [ 156.459858] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:51Z level=debug event=want client=2 drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv cached=06079worker1 # [ 156.466659] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:51Z level=debug event=assigned drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv worker=192.168.1.3:50051 clients=16080worker1 # [ 156.511536] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186081worker1 # [ 156.527237] nix-grpc-daemon[1477]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 19 ms (attempt 1/5)6082worker1 # [ 156.555621] nix-grpc-daemon[1477]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 2/5)6083worker1 # [ 156.626817] nix-grpc-daemon[1477]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 3/5)6084worker1 # [ 156.664505] nix-grpc-daemon[1477]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 769 ms (attempt 4/5)6085worker1 # [ 157.445622] nix-grpc-daemon[1477]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6086worker1 # [ 157.449005] nix-grpc-daemon[1477]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6087worker1 # [ 157.460092] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186088worker1 # [ 157.476889] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186089client # [ 157.481175] nix[1474]: copying 1 paths to 'lb:50051'...6090client # [ 157.486548] nix[1474]: copying path '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' to 'lb:50051'...6091worker1 # [ 157.488623] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6092worker1 # [ 157.512333] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:52Z level=info event=niks3_push_started pid=15786093worker1 # [ 157.638482] nix-grpc-daemon[1578]: time=2026-09-23T04:13:52.240Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker1.key ca=""6094worker1 # [ 157.745530] nix-daemon[849]: accepted connection from pid 1590, user nix-grpc-daemon (trusted)6095worker1 # [ 157.764688] nix-daemon[849]: reaped child process 1593, status = succeeded6096lb # [ 157.765622] niks3-server[1102]: 2026/09/23 04:13:52 INFO Received uploads request method=POST path=/api/pending_closures6097worker1 # [ 157.778017] nix-grpc-daemon[1578]: time=2026-09-23T04:13:52.379Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6098worker1 # [ 157.779509] nix-grpc-daemon[1578]: time=2026-09-23T04:13:52.379Z level=INFO msg="Uploading 85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv (424B)"6099lb # [ 157.810883] niks3-server[1102]: 2026/09/23 04:13:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6100worker1 # [ 157.822703] nix-grpc-daemon[1578]: time=2026-09-23T04:13:52.424Z level=INFO msg="Uploading 1 narinfos"6101lb # [ 157.823943] niks3-server[1102]: 2026/09/23 04:13:52 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.ls6102lb # [ 157.832257] niks3-server[1102]: 2026/09/23 04:13:52 INFO Signed narinfos id=53 count=16103lb # [ 157.834541] niks3-server[1102]: 2026/09/23 04:13:52 INFO Registered completed upload object_key=nar/0h460h76prjyvhqhyxlhz1xrmh3s520jbd19gaagy4hp4vc92r1p.nar.zst6104lb # [ 157.847455] niks3-server[1102]: 2026/09/23 04:13:52 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6105lb # [ 157.853348] niks3-server[1102]: 2026/09/23 04:13:52 INFO Completed upload id=536106worker1 # [ 157.852423] nix-grpc-daemon[1578]: time=2026-09-23T04:13:52.454Z level=INFO msg="Upload complete. (210ms)"6107lb # [ 157.855769] niks3-server[1102]: 2026/09/23 04:13:52 INFO Registered completed upload object_key=85wa5czf6m5ng93xyq44r1affb74csh8.narinfo6108worker1 # [ 157.860301] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=4246109worker1 # [ 157.868335] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186110worker1 # [ 157.877370] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6111worker1 # [ 157.881856] nix-daemon[849]: reaped child process 1576, status = succeeded6112worker1 # [ 157.892373] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6113client # [ 157.896662] nix[1474]: building '/nix/store/85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv' on lb:50051...6114client: must succeed: systemctl kill -s INT intr6115worker1 # [ 159.898716] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:54Z level=info event=revoked_running drv=85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv6116client # [ 159.901413] systemd[1]: intr.service: Sent signal SIGINT to main process 1474 (nix) on client request.6117lb # [ 159.899929] envoy[906]: conn peer=[2001:db8:1::1]:57108 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4314 tx=2093 ms=97706118client # [ 159.910804] nix[1474]: shutting down85wa5czf6m5ng93xyq44r1affb74csh8-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6119client # [ 159.922582] nix[1474]: error: interrupted by the user6120worker1 # [ 159.918759] nix-daemon[1597]: unexpected Nix daemon error: error: interrupted by the user6121client: (finished: must succeed: systemctl kill -s INT intr, in 0.08 seconds)6122worker1 # [ 159.928976] nix-daemon[849]: reaped child process 1595, status = succeeded6123worker1 # [ 159.930164] nix-daemon[849]: reaped child process 1597, status = failed with exit code 16124client # [ 159.949115] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6125client # [ 159.954234] systemd[1]: intr.service: Failed with result 'exit-code'.6126client # [ 159.956675] systemd[1]: intr.service: Consumed 166ms CPU time over 3.720s wall clock time, 41.2M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6127(finished: subtest: interrupting the client stops the build on the worker, in 3.84 seconds)6128subtest: a deploy mid-build drains: the build finishes, the new generation takes over6129client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw6130client # [ 160.093797] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw.6131client # Running as unit: sw.service; invocation ID: 29452575d82544269d6e2704848464556132client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix --argstr tag sw, in 0.08 seconds)6133client # [ 160.287682] nix[1495]: don't know how to build these paths:6134client # [ 160.289312] nix[1495]: /nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6135worker2 # [ 160.285980] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:566566136worker1 # [ 160.291424] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:54Z level=debug event=want client=3 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv cached=06137worker1 # [ 160.297310] nix-grpc-daemon[1477]: ts=2026-09-23T04:13:54Z level=debug event=assigned drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv worker=192.168.1.4:50051 clients=16138worker2 # [ 160.342683] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566566139worker2 # [ 160.358421] nix-grpc-daemon[1651]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)6140worker2 # [ 160.374551] nix-grpc-daemon[1651]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 60 ms (attempt 2/5)6141worker2 # [ 160.436496] nix-grpc-daemon[1651]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 219 ms (attempt 3/5)6142worker2 # [ 160.657592] nix-grpc-daemon[1651]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 135 ms (attempt 4/5)6143worker2 # [ 160.794853] nix-grpc-daemon[1651]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6144worker2 # [ 160.801224] nix-grpc-daemon[1651]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6145worker2 # [ 160.820068] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:566566146client # [ 160.836065] nix[1495]: copying 1 paths to 'lb:50051'...6147worker2 # [ 160.834879] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:566566148client # [ 160.838768] nix[1495]: copying path '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' to 'lb:50051'...6149worker2 # [ 160.850110] nix-daemon[840]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)6150worker2 # [ 161.011036] nix-daemon[840]: accepted connection from pid 1781, user nix-grpc-daemon (trusted)6151worker2 # [ 161.030694] nix-daemon[840]: reaped child process 1784, status = succeeded6152lb # [ 161.031578] niks3-server[1102]: 2026/09/23 04:13:55 INFO Received uploads request method=POST path=/api/pending_closures6153worker2 # [ 161.045363] nix-grpc-daemon[1733]: time=2026-09-23T04:13:55.623Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6154worker2 # [ 161.046931] nix-grpc-daemon[1733]: time=2026-09-23T04:13:55.623Z level=INFO msg="Uploading x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv (416B)"6155lb # [ 161.062644] niks3-server[1102]: 2026/09/23 04:13:55 INFO Registered completed upload object_key=nar/0jczp45bkr8vp1ymznxf6da1iv8dzh42pnl36wiprbzwhpi2ddia.nar.zst6156worker2 # [ 161.080091] nix-grpc-daemon[1733]: time=2026-09-23T04:13:55.657Z level=INFO msg="Uploading 1 narinfos"6157lb # [ 161.077336] niks3-server[1102]: 2026/09/23 04:13:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6158lb # [ 161.087992] niks3-server[1102]: 2026/09/23 04:13:55 INFO Signed narinfos id=54 count=16159worker2 # [ 161.105431] nix-grpc-daemon[1733]: time=2026-09-23T04:13:55.684Z level=INFO msg="Upload complete. (238ms)"6160lb # [ 161.103329] niks3-server[1102]: 2026/09/23 04:13:55 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.ls6161lb # [ 161.110770] niks3-server[1102]: 2026/09/23 04:13:55 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6162worker2 # [ 161.116347] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56656 duration_s=0 paths=1 nar_bytes_in=4166163lb # [ 161.117724] niks3-server[1102]: 2026/09/23 04:13:55 INFO Completed upload id=546164lb # [ 161.120331] niks3-server[1102]: 2026/09/23 04:13:55 INFO Registered completed upload object_key=x0m4asg5xx7vgj4dip2crwn6zhxqngvs.narinfo6165worker2 # [ 161.131963] nix-grpc-daemon[1651]: ts=2026-09-23T04:13:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:566566166worker2 # [ 161.138706] nix-daemon[840]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)6167worker2 # [ 161.143771] nix-daemon[840]: reaped child process 1779, status = succeeded6168worker2 # [ 161.155053] nix-daemon[840]: accepted connection from pid 1651, user nix-grpc-daemon (trusted)6169client # [ 161.155191] nix[1495]: building '/nix/store/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv' on lb:50051...6170worker2: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&26171worker2 # Checking switch inhibitors... done6172worker2 # [ 164.286416] nixos[1825]: switching to system configuration /nix/store/x8kjjmji2ck289pjncrbp4826lif72v0-nixos-system-worker2-test6173worker2 # activating the configuration...6174worker2 # [ 164.292453] systemd[1]: Stopped target Local File Systems.6175worker2 # [ 164.293289] systemd[1]: Stopped target Remote File Systems.6176worker2 # setting up /etc...6177worker2 # [ 164.865575] systemd[1]: Reload requested from client PID 1825 ('.switch-to-conf') (unit backdoor.service)...6178worker2 # [ 164.867488] systemd[1]: Reloading...6179worker2 # [ 169.775857] systemd[1]: Reloading finished in 4903 ms.6180worker2 # restarting sysinit-reactivation.target6181lb # [ 169.903924] envoy[906]: conn peer=[2001:db8:1::1]:57906 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3013 tx=851 ms=35276182worker2 # [ 169.912721] systemd[1]: Starting Re-setup tmpfiles on a system that is already running....6183worker2 # [ 170.038938] systemd[1]: Finished Re-setup tmpfiles on a system that is already running..6184worker2 # reloading the following units: dbus-broker.service, nix-grpc-daemon.service6185worker2 # [ 170.044299] systemd[1]: Reached target Reactivate sysinit units.6186worker2 # [ 170.045168] systemd[1]: Reloading D-Bus System Message Bus...6187worker2 # [ 170.046265] dbus-broker-launch[494]: Caught SIGHUP, trigger reload.6188worker2 # [ 170.061202] systemd[1]: Reloading Nix worker-protocol over gRPC...6189worker2 # [ 170.064948] dbus-broker-launch[494]: Looking up NSS user entry for 'systemd-timesync'...6190worker2 # [ 170.068851] dbus-broker-launch[494]: NSS returned no entry for 'systemd-timesync'6191worker2 # [ 170.070146] dbus-broker-launch[494]: Invalid user-name in /nix/store/ng2ydfkv03gvmdws4f4mjf9wj6qj3kq6-system-path/share/dbus-1/system.d/org.freedesktop.timesync1.conf +16: user="systemd-timesync"6192worker2 # [ 170.100953] systemd[1]: Reloaded Nix worker-protocol over gRPC.6193worker2 # [ 170.113508] systemd[1]: Reloaded D-Bus System Message Bus.6194worker2 # [ 170.114354] systemd[1]: Reached target Local File Systems.6195worker2 # [ 170.115137] systemd[1]: Load Kernel Module efi_pstore skipped, unmet condition check ConditionKernelModuleLoaded=!efi_pstore6196worker2 # [ 170.129989] systemd[1]: Starting Create SUID/SGID Wrappers...6197worker2 # [ 170.130850] systemd[1]: Update Boot Loader Random Seed skipped, no trigger condition checks were met.6198worker2 # [ 170.140096] systemd[1]: Clear Stale Hibernate Storage Info skipped, unmet condition check ConditionPathExists=/sys/firmware/efi/efivars/HibernateLocation-8cf2644b-4b0b-428f-9387-6d876050dc676199worker2 # [ 170.142205] systemd[1]: Platform Persistent Storage Archival skipped, unmet condition check ConditionDirectoryNotEmpty=/sys/fs/pstore6200worker2 # [ 170.143704] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6201worker2 # [ 170.152151] systemd[1]: TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os6202worker2 # [ 170.153376] systemd[1]: Reached target Remote File Systems.6203worker2 # [ 170.156914] systemd[1]: TPM PCR Measurements skipped, unmet condition check ConditionSecurity=measured-os6204worker2 # [ 170.158984] systemd[1]: Make TPM PCR Policy skipped, unmet condition check ConditionSecurity=measured-uki6205worker2 # [ 170.464488] systemd[1]: suid-sgid-wrappers.service: Deactivated successfully.6206worker2 # [ 170.465549] systemd[1]: Finished Create SUID/SGID Wrappers.6207worker2 # [ 170.475361] systemd[1]: Starting linger-users.service...6208worker2 # [ 170.517912] systemd[1]: linger-users.service: Deactivated successfully.6209worker2 # [ 170.518888] systemd[1]: Finished linger-users.service.6210worker2 # [ 170.715125] nix-grpc-daemon[1651]: ts=2026-09-23T04:14:05Z level=info event=draining6211worker2 # the following new units were started: sysinit-reactivation.target, systemd-tmpfiles-resetup.service6212worker2 # [ 170.776335] nixos[1825]: finished switching to system configuration /nix/store/x8kjjmji2ck289pjncrbp4826lif72v0-nixos-system-worker2-test6213worker2: (finished: must succeed: timeout 20 /run/current-system/specialisation/next/bin/switch-to-configuration test >&2, in 6.96 seconds)6214worker2: must succeed: pgrep -f 'read -t [9]0 x'6215worker2: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.05 seconds)6216client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&26217client # don't know how to build these paths:6218client # /nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6219worker1 # [ 171.051715] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:506186220worker1 # [ 171.059662] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186221worker1 # [ 171.085769] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:05Z level=debug event=want client=4 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv cached=06222worker1 # [ 171.092731] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:05Z level=debug event=assigned drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv worker=192.168.1.3:50051 clients=16223worker1 # [ 171.136959] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186224worker1 # [ 171.151889] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186225client # copying 1 paths to 'lb:50051'...6226worker1 # [ 171.167720] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186227client # copying path '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' to 'lb:50051'...6228worker1 # [ 171.190542] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6229worker1 # [ 171.347969] nix-daemon[849]: accepted connection from pid 1652, user nix-grpc-daemon (trusted)6230worker1 # [ 171.366074] nix-daemon[849]: reaped child process 1655, status = succeeded6231lb # [ 171.375496] niks3-server[1102]: 2026/09/23 04:14:05 INFO Received uploads request method=POST path=/api/pending_closures6232worker1 # [ 171.384604] nix-grpc-daemon[1578]: time=2026-09-23T04:14:05.986Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6233worker1 # [ 171.387028] nix-grpc-daemon[1578]: time=2026-09-23T04:14:05.988Z level=INFO msg="Uploading k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv (528B)"6234lb # [ 171.410911] niks3-server[1102]: 2026/09/23 04:14:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6235worker1 # [ 171.419771] nix-grpc-daemon[1578]: time=2026-09-23T04:14:06.021Z level=INFO msg="Uploading 1 narinfos"6236lb # [ 171.423312] niks3-server[1102]: 2026/09/23 04:14:06 INFO Signed narinfos id=55 count=16237lb # [ 171.432861] niks3-server[1102]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.ls6238lb # [ 171.441909] niks3-server[1102]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=nar/1h3fzlmfdwdiqr732wwrq7jnpw7n6p8f9qx1089c6g6x2i4csv75.nar.zst6239lb # [ 171.453160] niks3-server[1102]: 2026/09/23 04:14:06 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6240lb # [ 171.461458] niks3-server[1102]: 2026/09/23 04:14:06 INFO Completed upload id=556241worker1 # [ 171.453109] nix-grpc-daemon[1578]: time=2026-09-23T04:14:06.055Z level=INFO msg="Upload complete. (245ms)"6242worker1 # [ 171.464518] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=5286243lb # [ 171.469525] niks3-server[1102]: 2026/09/23 04:14:06 INFO Registered completed upload object_key=k3l35rf796jrnjns08398p1l8fcx178a.narinfo6244worker1 # [ 171.480802] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186245worker1 # [ 171.490906] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6246worker1 # [ 171.497625] nix-daemon[849]: reaped child process 1650, status = succeeded6247client # building '/nix/store/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv' on lb:50051...6248worker1 # [ 171.509470] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6249client # farm-a> node-a: building k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6250client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6251client # farm-a> LOG-farm-a-during-drain6252worker1 # [ 172.345179] nix-daemon[849]: reaped child process 1659, status = succeeded6253worker1 # [ 172.458912] nix-daemon[849]: accepted connection from pid 1675, user nix-grpc-daemon (trusted)6254worker1 # [ 172.475934] nix-daemon[849]: reaped child process 1678, status = succeeded6255lb # [ 172.484981] niks3-server[1102]: 2026/09/23 04:14:07 INFO Received uploads request method=POST path=/api/pending_closures6256worker1 # [ 172.493625] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.094Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6257worker1 # [ 172.496347] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.094Z level=INFO msg="Uploading 42c24pndjip9nvdj8j25mkv0650griqf-farm-a (136B)"6258lb # [ 172.518259] niks3-server[1102]: 2026/09/23 04:14:07 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6259lb # [ 172.536102] niks3-server[1102]: 2026/09/23 04:14:07 INFO Registered completed upload object_key=log/k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv6260lb # [ 172.549890] niks3-server[1102]: 2026/09/23 04:14:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6261worker1 # [ 172.553614] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.155Z level=INFO msg="Uploading 1 narinfos"6262lb # [ 172.557579] niks3-server[1102]: 2026/09/23 04:14:07 INFO Signed narinfos id=56 count=16263lb # [ 172.563072] niks3-server[1102]: 2026/09/23 04:14:07 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.ls6264lb # [ 172.570861] niks3-server[1102]: 2026/09/23 04:14:07 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6265lb # [ 172.575856] niks3-server[1102]: 2026/09/23 04:14:07 INFO Completed upload id=566266lb # [ 172.580300] niks3-server[1102]: 2026/09/23 04:14:07 INFO Registered completed upload object_key=42c24pndjip9nvdj8j25mkv0650griqf.narinfo6267worker1 # [ 172.569811] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.172Z level=INFO msg="Upload complete. (247ms)"6268worker1 # [ 172.587442] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=k3l35rf796jrnjns08398p1l8fcx178a-farm-a.drv assign_id=4 outputs=16269worker1 # [ 172.601571] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:07Z level=debug event=want client=4 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv cached=06270worker1 # [ 172.605229] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:07Z level=debug event=assigned drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv worker=192.168.1.3:50051 clients=16271worker1 # [ 172.609061] nix-daemon[849]: reaped child process 1657, status = succeeded6272worker1 # [ 172.632601] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186273worker1 # [ 172.642795] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186274worker1 # [ 172.658294] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.260Z level=INFO msg="All 1 paths already cached"6275client # copying 1 paths to 'lb:50051'...6276worker1 # [ 172.669165] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186277client # copying path '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' to 'lb:50051'...6278worker1 # [ 172.690564] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6279worker1 # [ 172.845889] nix-daemon[849]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)6280worker1 # [ 172.866463] nix-daemon[849]: reaped child process 1687, status = succeeded6281lb # [ 172.868609] niks3-server[1102]: 2026/09/23 04:14:07 INFO Received uploads request method=POST path=/api/pending_closures6282worker1 # [ 172.876482] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.479Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6283worker1 # [ 172.878494] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.480Z level=INFO msg="Uploading im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv (640B)"6284lb # [ 172.897487] niks3-server[1102]: 2026/09/23 04:14:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6285worker1 # [ 172.905399] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.507Z level=INFO msg="Uploading 1 narinfos"6286lb # [ 172.912815] niks3-server[1102]: 2026/09/23 04:14:07 INFO Signed narinfos id=57 count=16287lb # [ 172.921532] niks3-server[1102]: 2026/09/23 04:14:07 INFO Registered completed upload object_key=nar/0rxmdnq61ka29n0b2as2hdbz91wi2ap17db890js8r5plhpg436g.nar.zst6288lb # [ 172.936231] niks3-server[1102]: 2026/09/23 04:14:07 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.ls6289worker1 # [ 172.944280] nix-grpc-daemon[1578]: time=2026-09-23T04:14:07.546Z level=INFO msg="Upload complete. (236ms)"6290lb # [ 172.942412] niks3-server[1102]: 2026/09/23 04:14:07 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6291lb # [ 172.951994] niks3-server[1102]: 2026/09/23 04:14:07 INFO Completed upload id=576292lb # [ 172.955510] niks3-server[1102]: 2026/09/23 04:14:07 INFO Registered completed upload object_key=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l.narinfo6293worker1 # [ 172.954872] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=6406294worker1 # [ 172.969634] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186295worker1 # [ 172.978271] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6296worker1 # [ 172.982363] nix-daemon[849]: reaped child process 1682, status = succeeded6297client # building '/nix/store/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv' on lb:50051...6298worker1 # [ 172.994459] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6299client # farm-b> node-a: building im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6300client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6301client # farm-b> LOG-farm-b-during-drain6302worker1 # [ 174.105667] nix-daemon[849]: reaped child process 1691, status = succeeded6303worker1 # [ 174.178712] nix-daemon[849]: accepted connection from pid 1707, user nix-grpc-daemon (trusted)6304worker1 # [ 174.198061] nix-daemon[849]: reaped child process 1710, status = succeeded6305lb # [ 174.201450] niks3-server[1102]: 2026/09/23 04:14:08 INFO Received uploads request method=POST path=/api/pending_closures6306worker1 # [ 174.211576] nix-grpc-daemon[1578]: time=2026-09-23T04:14:08.813Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6307worker1 # [ 174.213314] nix-grpc-daemon[1578]: time=2026-09-23T04:14:08.813Z level=INFO msg="Uploading daks7y2c761g04qpzsjx25629wigr8js-farm-b (184B)"6308lb # [ 174.240099] niks3-server[1102]: 2026/09/23 04:14:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6309worker1 # [ 174.248989] nix-grpc-daemon[1578]: time=2026-09-23T04:14:08.851Z level=INFO msg="Uploading 1 narinfos"6310lb # [ 174.254544] niks3-server[1102]: 2026/09/23 04:14:08 INFO Signed narinfos id=58 count=16311lb # [ 174.266570] niks3-server[1102]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.ls6312lb # [ 174.275931] niks3-server[1102]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=log/im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv6313worker1 # [ 174.285975] nix-grpc-daemon[1578]: time=2026-09-23T04:14:08.888Z level=INFO msg="Upload complete. (277ms)"6314lb # [ 174.287434] niks3-server[1102]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=nar/0lnpxl34nrzp147jy5bafczkvxcl00hcf69qjhsljmv2ip9jisfh.nar.zst6315lb # [ 174.293593] niks3-server[1102]: 2026/09/23 04:14:08 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6316lb # [ 174.300547] niks3-server[1102]: 2026/09/23 04:14:08 INFO Completed upload id=586317lb # [ 174.302819] niks3-server[1102]: 2026/09/23 04:14:08 INFO Registered completed upload object_key=daks7y2c761g04qpzsjx25629wigr8js.narinfo6318worker1 # [ 174.301844] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=im7gh7nqqi41f48wvwjqkgv3nm7w5b4l-farm-b.drv assign_id=5 outputs=16319worker1 # [ 174.320821] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:08Z level=debug event=want client=4 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv cached=06320worker1 # [ 174.325449] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:08Z level=debug event=assigned drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv worker=192.168.1.3:50051 clients=16321worker1 # [ 174.330317] nix-daemon[849]: reaped child process 1689, status = succeeded6322worker1 # [ 174.359566] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186323worker1 # [ 174.374632] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186324worker1 # [ 174.388310] nix-grpc-daemon[1578]: time=2026-09-23T04:14:08.990Z level=INFO msg="All 2 paths already cached"6325client # copying 1 paths to 'lb:50051'...6326worker1 # [ 174.399203] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186327client # copying path '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' to 'lb:50051'...6328worker1 # [ 174.426377] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6329worker1 # [ 174.589157] nix-daemon[849]: accepted connection from pid 1716, user nix-grpc-daemon (trusted)6330worker1 # [ 174.605413] nix-daemon[849]: reaped child process 1719, status = succeeded6331lb # [ 174.613790] niks3-server[1102]: 2026/09/23 04:14:09 INFO Received uploads request method=POST path=/api/pending_closures6332worker1 # [ 174.621539] nix-grpc-daemon[1578]: time=2026-09-23T04:14:09.223Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6333worker1 # [ 174.623091] nix-grpc-daemon[1578]: time=2026-09-23T04:14:09.223Z level=INFO msg="Uploading lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv (768B)"6334lb # [ 174.640101] niks3-server[1102]: 2026/09/23 04:14:09 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.ls6335worker1 # [ 174.650655] nix-grpc-daemon[1578]: time=2026-09-23T04:14:09.253Z level=INFO msg="Uploading 1 narinfos"6336lb # [ 174.655325] niks3-server[1102]: 2026/09/23 04:14:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6337lb # [ 174.666672] niks3-server[1102]: 2026/09/23 04:14:09 INFO Signed narinfos id=59 count=16338lb # [ 174.675795] niks3-server[1102]: 2026/09/23 04:14:09 INFO Registered completed upload object_key=nar/066jhl91shf8ar6h3cqqb855j5cswl7170wrg5lzf3fhpka0xywn.nar.zst6339worker1 # [ 174.681961] nix-grpc-daemon[1578]: time=2026-09-23T04:14:09.284Z level=INFO msg="Upload complete. (238ms)"6340lb # [ 174.684862] niks3-server[1102]: 2026/09/23 04:14:09 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6341lb # [ 174.689299] niks3-server[1102]: 2026/09/23 04:14:09 INFO Completed upload id=596342lb # [ 174.692704] niks3-server[1102]: 2026/09/23 04:14:09 INFO Registered completed upload object_key=lbk8wy49c881163x07gb8z669j2fg1ff.narinfo6343worker1 # [ 174.692060] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=7686344worker1 # [ 174.700617] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186345worker1 # [ 174.712259] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6346worker1 # [ 174.720292] nix-daemon[849]: reaped child process 1714, status = succeeded6347client # building '/nix/store/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv' on lb:50051...6348worker1 # [ 174.725418] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6349client # farm-top> node-a: building lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6350client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6351client # farm-top> LOG-farm-top-during-drain6352worker1 # [ 175.604185] nix-daemon[849]: reaped child process 1723, status = succeeded6353worker1 # [ 175.699583] nix-daemon[849]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6354worker1 # [ 175.714507] nix-daemon[849]: reaped child process 1742, status = succeeded6355lb # [ 175.725785] niks3-server[1102]: 2026/09/23 04:14:10 INFO Received uploads request method=POST path=/api/pending_closures6356worker1 # [ 175.734425] nix-grpc-daemon[1578]: time=2026-09-23T04:14:10.335Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6357worker1 # [ 175.736243] nix-grpc-daemon[1578]: time=2026-09-23T04:14:10.335Z level=INFO msg="Uploading xdwmx2f4n5n8bggglniz2n35any0xz4w-farm-top (240B)"6358lb # [ 175.765351] niks3-server[1102]: 2026/09/23 04:14:10 INFO Registered completed upload object_key=log/lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv6359worker1 # [ 175.774165] nix-grpc-daemon[1578]: time=2026-09-23T04:14:10.376Z level=INFO msg="Uploading 1 narinfos"6360lb # [ 175.778964] niks3-server[1102]: 2026/09/23 04:14:10 INFO Registered completed upload object_key=nar/1bpy9ga4n2446pary6msidbs8mnil1a34jn8p8yl2scvdg691mz2.nar.zst6361lb # [ 175.789790] niks3-server[1102]: 2026/09/23 04:14:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6362worker1 # [ 175.800155] nix-grpc-daemon[1578]: time=2026-09-23T04:14:10.402Z level=INFO msg="Upload complete. (264ms)"6363lb # [ 175.802133] niks3-server[1102]: 2026/09/23 04:14:10 INFO Signed narinfos id=60 count=16364lb # [ 175.805922] niks3-server[1102]: 2026/09/23 04:14:10 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.ls6365lb # [ 175.814442] niks3-server[1102]: 2026/09/23 04:14:10 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6366lb # [ 175.821280] niks3-server[1102]: 2026/09/23 04:14:10 INFO Completed upload id=606367worker1 # [ 175.816216] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=lbk8wy49c881163x07gb8z669j2fg1ff-farm-top.drv assign_id=6 outputs=16368lb # [ 175.823999] niks3-server[1102]: 2026/09/23 04:14:10 INFO Registered completed upload object_key=xdwmx2f4n5n8bggglniz2n35any0xz4w.narinfo6369worker1 # [ 175.830672] nix-daemon[849]: reaped child process 1721, status = succeeded6370client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain >&2, in 5.03 seconds)6371client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath6372client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag during-drain outPath, in 0.18 seconds)6373worker2: must succeed: pkill -f 'read -t [9]0 x'6374worker2: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.05 seconds)6375client: waiting for success: ! systemctl is-active sw6376worker2 # [ 176.204450] nix-daemon[840]: reaped child process 1788, status = succeeded6377worker2 # [ 176.319240] nix-daemon[840]: accepted connection from pid 1965, user nix-grpc-daemon (trusted)6378worker2 # [ 176.336393] nix-daemon[840]: reaped child process 1968, status = succeeded6379lb # [ 176.339730] niks3-server[1102]: 2026/09/23 04:14:10 INFO Received uploads request method=POST path=/api/pending_closures6380worker2 # [ 176.352165] nix-grpc-daemon[1733]: time=2026-09-23T04:14:10.930Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6381worker2 # [ 176.353962] nix-grpc-daemon[1733]: time=2026-09-23T04:14:10.930Z level=INFO msg="Uploading qr8vyl2zl3nxnin3vw41c6z35vj7087f-slow-sw (120B)"6382lb # [ 176.384673] niks3-server[1102]: 2026/09/23 04:14:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6383worker2 # [ 176.396594] nix-grpc-daemon[1733]: time=2026-09-23T04:14:10.975Z level=INFO msg="Uploading 1 narinfos"6384lb # [ 176.398464] niks3-server[1102]: 2026/09/23 04:14:11 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.ls6385lb # [ 176.406201] niks3-server[1102]: 2026/09/23 04:14:11 INFO Signed narinfos id=61 count=16386lb # [ 176.410905] niks3-server[1102]: 2026/09/23 04:14:11 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6387lb # [ 176.418885] niks3-server[1102]: 2026/09/23 04:14:11 INFO Registered completed upload object_key=log/x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv6388lb # [ 176.422782] niks3-server[1102]: 2026/09/23 04:14:11 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6389worker2 # [ 176.426202] nix-grpc-daemon[1733]: time=2026-09-23T04:14:11.005Z level=INFO msg="Upload complete. (242ms)"6390lb # [ 176.427249] niks3-server[1102]: 2026/09/23 04:14:11 INFO Completed upload id=616391lb # [ 176.431385] niks3-server[1102]: 2026/09/23 04:14:11 INFO Registered completed upload object_key=qr8vyl2zl3nxnin3vw41c6z35vj7087f.narinfo6392worker2 # [ 176.441066] nix-grpc-daemon[1651]: ts=2026-09-23T04:14:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56656 duration_s=15 drv=x0m4asg5xx7vgj4dip2crwn6zhxqngvs-slow-sw.drv assign_id=3 outputs=16393worker2 # [ 176.455768] nix-daemon[840]: reaped child process 1786, status = succeeded6394client # [ 176.474737] systemd[1]: sw.service: Deactivated successfully.6395client # [ 176.480275] systemd[1]: sw.service: Consumed 192ms CPU time over 16.381s wall clock time, 41.2M memory peak, 7.7K incoming IP traffic, 8.7K outgoing IP traffic.6396worker2 # [ 176.717909] nix-grpc-daemon[1651]: ts=2026-09-23T04:14:11Z level=info event=signal_exit6397client: (finished: waiting for success: ! systemctl is-active sw, in 1.10 seconds)6398client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6399client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.05 seconds)6400worker2: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6401worker2 # [ 177.325794] systemd[1]: Starting scheduler-token.service...6402worker2 # [ 177.398830] systemd[1]: scheduler-token.service: Deactivated successfully.6403worker2 # [ 177.403559] systemd[1]: Finished scheduler-token.service.6404worker2 # [ 177.405276] systemd[1]: scheduler-token.service: Consumed 41ms CPU time over 76ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6405worker1 # [ 177.725212] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:12Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:506086406worker2 # [ 177.733021] nix-grpc-daemon[1651]: ts=2026-09-23T04:14:12Z level=info event=scheduler_restarting6407worker1 # [ 177.729917] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:12Z level=info event=worker_gone addr=192.168.1.4:500516408worker2 # [ 177.754445] nix-grpc-daemon[1651]: ts=2026-09-23T04:14:12Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6409worker2 # [ 177.781826] nix-daemon[840]: reaped child process 1704, status = succeeded6410worker2 # [ 178.306959] nix-grpc-daemon[1651]: E0000 00:00:1790136852.885855 1697 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6411worker2 # [ 178.313667] nix-grpc-daemon[1651]: E0000 00:00:1790136852.886193 1697 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6412worker2 # [ 179.750443] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6413worker2 # [ 179.752354] systemd[1]: nix-grpc-daemon.service: Consumed 773ms CPU time over 30.314s wall clock time, 35.9M memory peak, 22.7K incoming IP traffic, 36.8K outgoing IP traffic.6414worker2 # [ 179.778431] systemd[1]: Starting Nix worker-protocol over gRPC...6415lb # [ 179.899716] envoy[906]: conn peer=[2001:db8:1::1]:52318 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9608 tx=4427 ms=47826416lb # [ 179.904759] envoy[906]: conn peer=[2001:db8:1::1]:57912 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3064 tx=1578 ms=161686417lb # [ 179.909575] envoy[906]: conn peer=[2001:db8:1::4]:36050 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=7599 tx=679 ms=279696418worker2 # [ 179.926039] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:14Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6419worker2 # [ 179.928475] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6420worker2 # [ 179.932428] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16421worker2 # [ 179.938957] nix-grpc-daemon[1985]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6422worker2 # [ 179.948078] nix-grpc-daemon[1985]: E0000 00:00:1790136854.516051 1985 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6423worker2 # [ 179.950440] nix-grpc-daemon[1985]: E0000 00:00:1790136854.518120 1985 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6424worker2 # [ 179.955732] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:14Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6425worker2 # [ 179.980255] nix-grpc-daemon[1985]: E0000 00:00:1790136854.533070 2025 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6426worker2 # [ 179.982656] systemd[1]: Started Nix worker-protocol over gRPC.6427worker2 # [ 179.983465] nix-grpc-daemon[1985]: E0000 00:00:1790136854.550957 2025 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6428worker2: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 2.75 seconds)6429lb: waiting until exactly no endpoints of aarch64-linux are unhealthy6430worker2 # [ 179.996994] nix-grpc-daemon[1985]: E0000 00:00:1790136854.551200 2025 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6431lb: must succeed: curl -sf localhost:9901/clusters6432worker2 # [ 179.999433] nix-grpc-daemon[1985]: E0000 00:00:1790136854.574108 2025 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6433worker2 # [ 180.008483] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:14Z level=info event=scheduler_connected addr=lb:500516434worker1 # [ 180.008370] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:14Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:506086435worker1 # [ 180.008823] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:14Z level=info event=worker_hello addr=192.168.1.4:50051 systems=aarch64-linux max_jobs=1 running=06436lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6437lb: must succeed: curl -sf localhost:9901/clusters6438lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)6439lb: (finished: waiting until exactly no endpoints of aarch64-linux are unhealthy, in 1.14 seconds)6440lb: must succeed: curl -sf localhost:9901/clusters6441lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6442worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6443worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)6444(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 21.22 seconds)6445subtest: queue gauges show a burst while every build blocks6446lb: must succeed: curl -sf localhost:9901/clusters6447lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6448lb: must succeed: curl -sf localhost:9901/clusters6449lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6450worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6451worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6452client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6453client # [ 181.492738] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6454client # Running as unit: burst.service; invocation ID: 8405c488ba6448a5b1dcbbfcfa4741b06455client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/bzzab28rfcdvvdzhgyw7sy9agp2z9pdd-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.09 seconds)6456lb: must succeed: curl -sf localhost:9901/clusters6457lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6458lb: must succeed: curl -sf localhost:9901/clusters6459lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6460worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6461worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6462worker2 # [ 181.695656] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520706463worker2 # [ 181.710809] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6464client # [ 181.730296] nix[1551]: don't know how to build these paths:6465client # [ 181.731751] nix[1551]: /nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6466client # [ 181.733965] nix[1551]: /nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6467client # [ 181.735476] nix[1551]: /nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6468client # [ 181.737973] nix[1551]: /nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6469worker1 # [ 181.745097] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=want client=5 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv cached=06470worker1 # [ 181.745438] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=assigned drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv worker=192.168.1.3:50051 clients=16471worker1 # [ 181.745564] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=want client=5 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv cached=06472worker1 # [ 181.745693] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=assigned drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv worker=192.168.1.4:50051 clients=16473worker1 # [ 181.745832] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=want client=5 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv cached=06474worker1 # [ 181.745961] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=want client=5 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv cached=06475worker2 # [ 181.792125] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520706476worker1 # [ 181.788990] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186477worker1 # [ 181.804613] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186478worker2 # [ 181.808604] nix-grpc-daemon[1985]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)6479client # [ 181.823764] nix[1551]: copying 1 paths to 'lb:50051'...6480worker1 # [ 181.818815] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186481client # [ 181.826444] nix[1551]: copying path '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' to 'lb:50051'...6482worker1 # [ 181.833593] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6483worker2 # [ 181.893925] nix-grpc-daemon[1985]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 99 ms (attempt 2/5)6484worker1 # [ 181.987137] nix-daemon[849]: accepted connection from pid 1757, user nix-grpc-daemon (trusted)6485worker2 # [ 181.994914] nix-grpc-daemon[1985]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 264 ms (attempt 3/5)6486worker1 # [ 182.005921] nix-daemon[849]: reaped child process 1760, status = succeeded6487lb # [ 182.010756] niks3-server[1102]: 2026/09/23 04:14:16 INFO Received uploads request method=POST path=/api/pending_closures6488worker1 # [ 182.019339] nix-grpc-daemon[1578]: time=2026-09-23T04:14:16.621Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6489worker1 # [ 182.020974] nix-grpc-daemon[1578]: time=2026-09-23T04:14:16.621Z level=INFO msg="Uploading d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv (416B)"6490lb # [ 182.042228] niks3-server[1102]: 2026/09/23 04:14:16 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.ls6491worker1 # [ 182.059843] nix-grpc-daemon[1578]: time=2026-09-23T04:14:16.662Z level=INFO msg="Uploading 1 narinfos"6492lb # [ 182.060621] niks3-server[1102]: 2026/09/23 04:14:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6493lb # [ 182.071834] niks3-server[1102]: 2026/09/23 04:14:16 INFO Signed narinfos id=62 count=16494lb # [ 182.087122] niks3-server[1102]: 2026/09/23 04:14:16 INFO Registered completed upload object_key=nar/06ismykzsix0w29371dm1hbzzf3ljf9jn7dyh8skp5wqfkkw0vlk.nar.zst6495lb # [ 182.101022] niks3-server[1102]: 2026/09/23 04:14:16 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6496worker1 # [ 182.104447] nix-grpc-daemon[1578]: time=2026-09-23T04:14:16.706Z level=INFO msg="Upload complete. (255ms)"6497lb # [ 182.108888] niks3-server[1102]: 2026/09/23 04:14:16 INFO Completed upload id=626498lb # [ 182.115236] niks3-server[1102]: 2026/09/23 04:14:16 INFO Registered completed upload object_key=d4rrlfallgfbcpmjnsn604l2mgff4hgx.narinfo6499worker1 # [ 182.116304] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=4166500worker1 # [ 182.135012] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186501worker1 # [ 182.141593] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6502worker1 # [ 182.146775] nix-daemon[849]: reaped child process 1755, status = succeeded6503client # [ 182.162501] nix[1551]: building '/nix/store/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv' on lb:50051...6504worker1 # [ 182.159190] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6505worker2 # [ 182.260796] nix-grpc-daemon[1985]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 591 ms (attempt 4/5)6506lb: must succeed: curl -sf localhost:9901/clusters6507lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6508lb: must succeed: curl -sf localhost:9901/clusters6509lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6510worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6511worker2 # [ 182.856835] nix-grpc-daemon[1985]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6512worker2 # [ 182.864292] nix-grpc-daemon[1985]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6513worker2 # [ 182.879272] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520706514client # [ 182.896976] nix[1551]: copying 1 paths to 'lb:50051'...6515worker2 # [ 182.894069] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520706516worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.08 seconds)6517client # [ 182.897237] nix[1551]: copying path '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' to 'lb:50051'...6518worker2 # [ 182.909876] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6519worker2 # [ 182.937362] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:17Z level=info event=niks3_push_started pid=20466520worker2 # [ 183.105247] nix-grpc-daemon[2046]: time=2026-09-23T04:14:17.683Z level=INFO msg="Configuring client TLS" cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.pem key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/worker2.key ca=""6521worker2 # [ 183.215776] nix-daemon[840]: accepted connection from pid 2063, user nix-grpc-daemon (trusted)6522worker2 # [ 183.234062] nix-daemon[840]: reaped child process 2066, status = succeeded6523lb # [ 183.242596] niks3-server[1102]: 2026/09/23 04:14:17 INFO Received uploads request method=POST path=/api/pending_closures6524worker2 # [ 183.258905] nix-grpc-daemon[2046]: time=2026-09-23T04:14:17.837Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6525worker2 # [ 183.262813] nix-grpc-daemon[2046]: time=2026-09-23T04:14:17.837Z level=INFO msg="Uploading dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv (416B)"6526lb # [ 183.288277] niks3-server[1102]: 2026/09/23 04:14:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6527worker2 # [ 183.300103] nix-grpc-daemon[2046]: time=2026-09-23T04:14:17.878Z level=INFO msg="Uploading 1 narinfos"6528lb # [ 183.301163] niks3-server[1102]: 2026/09/23 04:14:17 INFO Signed narinfos id=63 count=16529lb # [ 183.303530] niks3-server[1102]: 2026/09/23 04:14:17 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.ls6530worker2 # [ 183.318011] nix-grpc-daemon[2046]: time=2026-09-23T04:14:17.897Z level=INFO msg="Upload complete. (209ms)"6531lb # [ 183.316659] niks3-server[1102]: 2026/09/23 04:14:17 INFO Registered completed upload object_key=nar/06mdwsnf16qmfi86y8r33asi4vcrp5z7kdxqki87psgfpgvvpdki.nar.zst6532worker2 # [ 183.324213] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=0 paths=1 nar_bytes_in=4166533lb # [ 183.323033] niks3-server[1102]: 2026/09/23 04:14:17 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6534lb # [ 183.327335] niks3-server[1102]: 2026/09/23 04:14:17 INFO Completed upload id=636535lb # [ 183.329598] niks3-server[1102]: 2026/09/23 04:14:17 INFO Registered completed upload object_key=dspy0axfmddglppg9zj9blmxlzhfs40x.narinfo6536worker2 # [ 183.335064] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520706537worker2 # [ 183.341058] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6538worker2 # [ 183.345445] nix-daemon[840]: reaped child process 2044, status = succeeded6539client # [ 183.355321] nix[1551]: building '/nix/store/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv' on lb:50051...6540worker2 # [ 183.356887] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6541worker1 # [ 184.459401] nix-daemon[849]: reaped child process 1764, status = succeeded6542worker1 # [ 184.540106] nix-daemon[849]: accepted connection from pid 1792, user nix-grpc-daemon (trusted)6543worker1 # [ 184.558476] nix-daemon[849]: reaped child process 1795, status = succeeded6544lb # [ 184.566111] niks3-server[1102]: 2026/09/23 04:14:19 INFO Received uploads request method=POST path=/api/pending_closures6545worker1 # [ 184.572437] nix-grpc-daemon[1578]: time=2026-09-23T04:14:19.174Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6546worker1 # [ 184.574110] nix-grpc-daemon[1578]: time=2026-09-23T04:14:19.174Z level=INFO msg="Uploading k6zivhvzgmlyxfmy64fzd759bxlhf9b5-slow-q0 (120B)"6547lb # [ 184.601970] niks3-server[1102]: 2026/09/23 04:14:19 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6548worker1 # [ 184.619966] nix-grpc-daemon[1578]: time=2026-09-23T04:14:19.222Z level=INFO msg="Uploading 1 narinfos"6549lb # [ 184.620774] niks3-server[1102]: 2026/09/23 04:14:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign6550lb # [ 184.629934] niks3-server[1102]: 2026/09/23 04:14:19 INFO Signed narinfos id=64 count=16551lb # [ 184.639398] niks3-server[1102]: 2026/09/23 04:14:19 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete6552worker1 # [ 184.647294] nix-grpc-daemon[1578]: time=2026-09-23T04:14:19.249Z level=INFO msg="Upload complete. (272ms)"6553lb # [ 184.649313] niks3-server[1102]: 2026/09/23 04:14:19 INFO Registered completed upload object_key=log/d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv6554lb # [ 184.660945] niks3-server[1102]: 2026/09/23 04:14:19 INFO Completed upload id=646555lb # [ 184.664622] niks3-server[1102]: 2026/09/23 04:14:19 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.narinfo6556worker1 # [ 184.663375] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=2 drv=d4rrlfallgfbcpmjnsn604l2mgff4hgx-slow-q0.drv assign_id=7 outputs=16557lb # [ 184.668798] niks3-server[1102]: 2026/09/23 04:14:19 INFO Registered completed upload object_key=k6zivhvzgmlyxfmy64fzd759bxlhf9b5.ls6558worker1 # [ 184.676356] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:19Z level=debug event=assigned drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv worker=192.168.1.3:50051 clients=16559worker1 # [ 184.682059] nix-daemon[849]: reaped child process 1762, status = succeeded6560worker1 # [ 184.703912] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186561worker1 # [ 184.718640] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186562client # [ 184.737288] nix[1551]: copying 1 paths to 'lb:50051'...6563worker1 # [ 184.733161] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186564client # [ 184.742367] nix[1551]: copying path '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' to 'lb:50051'...6565worker1 # [ 184.746180] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6566worker1 # [ 184.898540] nix-daemon[849]: accepted connection from pid 1799, user nix-grpc-daemon (trusted)6567worker1 # [ 184.913870] nix-daemon[849]: reaped child process 1802, status = succeeded6568lb # [ 184.921132] niks3-server[1102]: 2026/09/23 04:14:19 INFO Received uploads request method=POST path=/api/pending_closures6569worker1 # [ 184.929012] nix-grpc-daemon[1578]: time=2026-09-23T04:14:19.531Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6570worker1 # [ 184.930557] nix-grpc-daemon[1578]: time=2026-09-23T04:14:19.531Z level=INFO msg="Uploading jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv (416B)"6571worker1 # [ 184.957122] nix-grpc-daemon[1578]: time=2026-09-23T04:14:19.559Z level=INFO msg="Uploading 1 narinfos"6572lb # [ 184.950952] niks3-server[1102]: 2026/09/23 04:14:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign6573lb # [ 184.966390] niks3-server[1102]: 2026/09/23 04:14:19 INFO Signed narinfos id=65 count=16574lb # [ 184.974507] niks3-server[1102]: 2026/09/23 04:14:19 INFO Registered completed upload object_key=nar/1fyfjzlx8vymsfmdv5cv8ymgjkr18fwm737qvys29plwwiwmyvwr.nar.zst6575lb # [ 184.991041] niks3-server[1102]: 2026/09/23 04:14:19 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.ls6576worker1 # [ 184.998510] nix-grpc-daemon[1578]: time=2026-09-23T04:14:19.601Z level=INFO msg="Upload complete. (237ms)"6577lb # [ 185.000683] niks3-server[1102]: 2026/09/23 04:14:19 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete6578lb # [ 185.006457] niks3-server[1102]: 2026/09/23 04:14:19 INFO Registered completed upload object_key=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy.narinfo6579lb # [ 185.012336] niks3-server[1102]: 2026/09/23 04:14:19 INFO Completed upload id=656580worker1 # [ 185.008296] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=4166581worker1 # [ 185.024634] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186582worker1 # [ 185.034420] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6583worker1 # [ 185.038302] nix-daemon[849]: reaped child process 1797, status = succeeded6584worker1 # [ 185.049511] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6585client # [ 185.053153] nix[1551]: building '/nix/store/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv' on lb:50051...6586worker2 # [ 185.586728] nix-daemon[840]: reaped child process 2070, status = succeeded6587worker2 # [ 185.686435] nix-daemon[840]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)6588worker2 # [ 185.707305] nix-daemon[840]: reaped child process 2098, status = succeeded6589lb # [ 185.704748] niks3-server[1102]: 2026/09/23 04:14:20 INFO Received uploads request method=POST path=/api/pending_closures6590worker2 # [ 185.719145] nix-grpc-daemon[2046]: time=2026-09-23T04:14:20.296Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6591worker2 # [ 185.721619] nix-grpc-daemon[2046]: time=2026-09-23T04:14:20.296Z level=INFO msg="Uploading y3qkgrgq2zzpgvi2rf5chq94i627wkdb-slow-q3 (120B)"6592worker2 # [ 185.758164] nix-grpc-daemon[2046]: time=2026-09-23T04:14:20.336Z level=INFO msg="Uploading 1 narinfos"6593lb # [ 185.747071] niks3-server[1102]: 2026/09/23 04:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign6594lb # [ 185.762061] niks3-server[1102]: 2026/09/23 04:14:20 INFO Signed narinfos id=66 count=16595lb # [ 185.771172] niks3-server[1102]: 2026/09/23 04:14:20 INFO Registered completed upload object_key=log/dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv6596lb # [ 185.780708] niks3-server[1102]: 2026/09/23 04:14:20 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6597lb # [ 185.790391] niks3-server[1102]: 2026/09/23 04:14:20 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.ls6598worker2 # [ 185.798173] nix-grpc-daemon[2046]: time=2026-09-23T04:14:20.377Z level=INFO msg="Upload complete. (230ms)"6599lb # [ 185.796419] niks3-server[1102]: 2026/09/23 04:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete6600lb # [ 185.800287] niks3-server[1102]: 2026/09/23 04:14:20 INFO Completed upload id=666601lb # [ 185.804768] niks3-server[1102]: 2026/09/23 04:14:20 INFO Registered completed upload object_key=y3qkgrgq2zzpgvi2rf5chq94i627wkdb.narinfo6602worker2 # [ 185.806974] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=2 drv=dspy0axfmddglppg9zj9blmxlzhfs40x-slow-q3.drv assign_id=8 outputs=16603worker1 # [ 185.804889] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:20Z level=debug event=assigned drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv worker=192.168.1.4:50051 clients=16604worker2 # [ 185.825955] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520706605worker2 # [ 185.830503] nix-daemon[840]: reaped child process 2068, status = succeeded6606worker2 # [ 185.837476] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:20Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520706607client # [ 185.849302] nix[1551]: copying 1 paths to 'lb:50051'...6608worker2 # [ 185.848651] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:20Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520706609client # [ 185.853344] nix[1551]: copying path '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' to 'lb:50051'...6610worker2 # [ 185.859494] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6611worker2 # [ 185.987273] nix-daemon[840]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)6612worker2 # [ 186.005527] nix-daemon[840]: reaped child process 2105, status = succeeded6613lb # [ 186.007680] niks3-server[1102]: 2026/09/23 04:14:20 INFO Received uploads request method=POST path=/api/pending_closures6614worker2 # [ 186.018649] nix-grpc-daemon[2046]: time=2026-09-23T04:14:20.597Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6615worker2 # [ 186.020203] nix-grpc-daemon[2046]: time=2026-09-23T04:14:20.597Z level=INFO msg="Uploading pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv (416B)"6616lb # [ 186.039381] niks3-server[1102]: 2026/09/23 04:14:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign6617worker2 # [ 186.051280] nix-grpc-daemon[2046]: time=2026-09-23T04:14:20.630Z level=INFO msg="Uploading 1 narinfos"6618lb # [ 186.050347] niks3-server[1102]: 2026/09/23 04:14:20 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.ls6619lb # [ 186.059337] niks3-server[1102]: 2026/09/23 04:14:20 INFO Signed narinfos id=67 count=16620worker2 # [ 186.070239] nix-grpc-daemon[2046]: time=2026-09-23T04:14:20.649Z level=INFO msg="Upload complete. (199ms)"6621lb # [ 186.068348] niks3-server[1102]: 2026/09/23 04:14:20 INFO Registered completed upload object_key=nar/1k21jrz411xc9gkca2pnr9mrjq1s3bg3zhsam3nnv4j066dgzgm7.nar.zst6622lb # [ 186.077168] niks3-server[1102]: 2026/09/23 04:14:20 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete6623worker2 # [ 186.078369] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:20Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=0 paths=1 nar_bytes_in=4166624lb # [ 186.082560] niks3-server[1102]: 2026/09/23 04:14:20 INFO Completed upload id=676625lb # [ 186.084945] niks3-server[1102]: 2026/09/23 04:14:20 INFO Registered completed upload object_key=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd.narinfo6626worker2 # [ 186.091553] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:20Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520706627worker2 # [ 186.100327] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6628worker2 # [ 186.103687] nix-daemon[840]: reaped child process 2100, status = succeeded6629client # [ 186.115068] nix[1551]: building '/nix/store/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv' on lb:50051...6630worker2 # [ 186.116154] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)6631worker1 # [ 186.688465] nix-daemon[849]: reaped child process 1806, status = succeeded6632worker1 # [ 186.782372] nix-daemon[849]: accepted connection from pid 1831, user nix-grpc-daemon (trusted)6633worker1 # [ 186.797398] nix-daemon[849]: reaped child process 1834, status = succeeded6634lb # [ 186.806823] niks3-server[1102]: 2026/09/23 04:14:21 INFO Received uploads request method=POST path=/api/pending_closures6635worker1 # [ 186.816369] nix-grpc-daemon[1578]: time=2026-09-23T04:14:21.417Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6636worker1 # [ 186.817828] nix-grpc-daemon[1578]: time=2026-09-23T04:14:21.417Z level=INFO msg="Uploading gjyi8j90pk5flrsqqdhj6h132pcb7n33-slow-q2 (120B)"6637lb # [ 186.848935] niks3-server[1102]: 2026/09/23 04:14:21 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.ls6638lb # [ 186.860877] niks3-server[1102]: 2026/09/23 04:14:21 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6639worker1 # [ 186.870247] nix-grpc-daemon[1578]: time=2026-09-23T04:14:21.472Z level=INFO msg="Uploading 1 narinfos"6640lb # [ 186.874020] niks3-server[1102]: 2026/09/23 04:14:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign6641lb # [ 186.882590] niks3-server[1102]: 2026/09/23 04:14:21 INFO Signed narinfos id=68 count=16642lb # [ 186.887163] niks3-server[1102]: 2026/09/23 04:14:21 INFO Registered completed upload object_key=log/jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv6643worker1 # [ 186.897012] nix-grpc-daemon[1578]: time=2026-09-23T04:14:21.499Z level=INFO msg="Upload complete. (234ms)"6644lb # [ 186.899010] niks3-server[1102]: 2026/09/23 04:14:21 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete6645lb # [ 186.905363] niks3-server[1102]: 2026/09/23 04:14:21 INFO Completed upload id=686646lb # [ 186.907775] niks3-server[1102]: 2026/09/23 04:14:21 INFO Registered completed upload object_key=gjyi8j90pk5flrsqqdhj6h132pcb7n33.narinfo6647worker1 # [ 186.909087] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=jgwqjvc2sfm39r12g67n9dhiqhi4z7xy-slow-q2.drv assign_id=9 outputs=16648worker1 # [ 186.924167] nix-daemon[849]: reaped child process 1804, status = succeeded6649worker2 # [ 187.950620] nix-daemon[840]: reaped child process 2109, status = succeeded6650worker2 # [ 188.051276] nix-daemon[840]: accepted connection from pid 2134, user nix-grpc-daemon (trusted)6651worker2 # [ 188.070389] nix-daemon[840]: reaped child process 2137, status = succeeded6652lb # [ 188.073000] niks3-server[1102]: 2026/09/23 04:14:22 INFO Received uploads request method=POST path=/api/pending_closures6653worker2 # [ 188.086380] nix-grpc-daemon[2046]: time=2026-09-23T04:14:22.664Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6654worker2 # [ 188.087842] nix-grpc-daemon[2046]: time=2026-09-23T04:14:22.664Z level=INFO msg="Uploading l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx-slow-q1 (120B)"6655lb # [ 188.113515] niks3-server[1102]: 2026/09/23 04:14:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign6656worker2 # [ 188.124539] nix-grpc-daemon[2046]: time=2026-09-23T04:14:22.703Z level=INFO msg="Uploading 1 narinfos"6657lb # [ 188.128457] niks3-server[1102]: 2026/09/23 04:14:22 INFO Signed narinfos id=69 count=16658lb # [ 188.131212] niks3-server[1102]: 2026/09/23 04:14:22 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.ls6659lb # [ 188.145808] niks3-server[1102]: 2026/09/23 04:14:22 INFO Registered completed upload object_key=log/pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv6660worker2 # [ 188.152806] nix-grpc-daemon[2046]: time=2026-09-23T04:14:22.732Z level=INFO msg="Upload complete. (225ms)"6661lb # [ 188.152413] niks3-server[1102]: 2026/09/23 04:14:22 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6662lb # [ 188.157394] niks3-server[1102]: 2026/09/23 04:14:22 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete6663lb # [ 188.160836] niks3-server[1102]: 2026/09/23 04:14:22 INFO Completed upload id=696664lb # [ 188.163562] niks3-server[1102]: 2026/09/23 04:14:22 INFO Registered completed upload object_key=l3s7m1wb77d0cgdpbn5bcb47ibrwjvmx.narinfo6665worker2 # [ 188.164737] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=2 drv=pgfqjrvqz7k2z3ab3gk4xjy7130b5jcd-slow-q1.drv assign_id=10 outputs=16666worker2 # [ 188.176074] nix-daemon[840]: reaped child process 2107, status = succeeded6667client # [ 188.206949] systemd[1]: burst.service: Deactivated successfully.6668client # [ 188.211952] systemd[1]: burst.service: Consumed 230ms CPU time over 6.715s wall clock time, 51.7M memory peak, 14K incoming IP traffic, 18.9K outgoing IP traffic.6669client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6670client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.07 seconds)6671lb: must succeed: curl -sf localhost:9901/clusters6672lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.08 seconds)6673lb: must succeed: curl -sf localhost:9901/clusters6674lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6675worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6676worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)6677(finished: subtest: queue gauges show a burst while every build blocks, in 8.14 seconds)6678subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6679client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16680worker2 # [ 189.617013] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520706681worker1 # [ 189.617916] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186682worker1 # [ 189.661062] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=want client=6 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv cached=06683worker1 # [ 189.668124] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=assigned drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv worker=192.168.1.3:50051 clients=16684worker1 # [ 189.674047] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=want client=6 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv cached=06685worker1 # [ 189.679219] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=want client=6 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv cached=06686worker1 # [ 189.712585] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186687worker1 # [ 189.727860] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186688worker1 # [ 189.752422] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186689worker1 # [ 189.767319] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6690lb # [ 189.901448] envoy[906]: conn peer=[2001:db8:1::1]:36102 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9714 tx=5223 ms=64926691worker1 # [ 189.924919] nix-daemon[849]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)6692worker1 # [ 189.942723] nix-daemon[849]: reaped child process 1854, status = succeeded6693lb # [ 189.951387] niks3-server[1102]: 2026/09/23 04:14:24 INFO Received uploads request method=POST path=/api/pending_closures6694worker1 # [ 189.961977] nix-grpc-daemon[1578]: time=2026-09-23T04:14:24.563Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6695worker1 # [ 189.963534] nix-grpc-daemon[1578]: time=2026-09-23T04:14:24.563Z level=INFO msg="Uploading 3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv (512B)"6696lb # [ 189.992613] niks3-server[1102]: 2026/09/23 04:14:24 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.ls6697worker1 # [ 190.002158] nix-grpc-daemon[1578]: time=2026-09-23T04:14:24.604Z level=INFO msg="Uploading 1 narinfos"6698lb # [ 190.004322] niks3-server[1102]: 2026/09/23 04:14:24 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign6699lb # [ 190.013403] niks3-server[1102]: 2026/09/23 04:14:24 INFO Signed narinfos id=70 count=16700lb # [ 190.023975] niks3-server[1102]: 2026/09/23 04:14:24 INFO Registered completed upload object_key=nar/0zxfsmqijfc2dkglls3yp1x3w0w1na7vx39fxc124zddjzpaygd8.nar.zst6701lb # [ 190.039545] niks3-server[1102]: 2026/09/23 04:14:24 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete6702worker1 # [ 190.043598] nix-grpc-daemon[1578]: time=2026-09-23T04:14:24.646Z level=INFO msg="Upload complete. (259ms)"6703lb # [ 190.046689] niks3-server[1102]: 2026/09/23 04:14:24 INFO Completed upload id=706704lb # [ 190.050290] niks3-server[1102]: 2026/09/23 04:14:24 INFO Registered completed upload object_key=3223c4sl4ncpldbvw558vqwddijwzfpd.narinfo6705worker1 # [ 190.056619] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=5126706worker1 # [ 190.072179] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186707worker1 # [ 190.079155] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6708worker1 # [ 190.085235] nix-daemon[849]: reaped child process 1849, status = succeeded6709worker1 # [ 190.101128] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6710worker1 # [ 191.533525] nix-daemon[849]: reaped child process 1858, status = succeeded6711worker1 # [ 191.635942] nix-daemon[849]: accepted connection from pid 1874, user nix-grpc-daemon (trusted)6712worker1 # [ 191.651711] nix-daemon[849]: reaped child process 1877, status = succeeded6713lb # [ 191.662274] niks3-server[1102]: 2026/09/23 04:14:26 INFO Received uploads request method=POST path=/api/pending_closures6714worker1 # [ 191.668381] nix-grpc-daemon[1578]: time=2026-09-23T04:14:26.270Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6715worker1 # [ 191.669835] nix-grpc-daemon[1578]: time=2026-09-23T04:14:26.270Z level=INFO msg="Uploading dla3w0zf02z5jhccwa27i85bbgzrrsvb-farm-a (128B)"6716lb # [ 191.705583] niks3-server[1102]: 2026/09/23 04:14:26 INFO Registered completed upload object_key=log/3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv6717worker1 # [ 191.714928] nix-grpc-daemon[1578]: time=2026-09-23T04:14:26.317Z level=INFO msg="Uploading 1 narinfos"6718lb # [ 191.720355] niks3-server[1102]: 2026/09/23 04:14:26 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6719lb # [ 191.728573] niks3-server[1102]: 2026/09/23 04:14:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign6720lb # [ 191.731727] niks3-server[1102]: 2026/09/23 04:14:26 INFO Signed narinfos id=71 count=16721lb # [ 191.736987] niks3-server[1102]: 2026/09/23 04:14:26 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.ls6722worker1 # [ 191.743310] nix-grpc-daemon[1578]: time=2026-09-23T04:14:26.345Z level=INFO msg="Upload complete. (230ms)"6723lb # [ 191.744979] niks3-server[1102]: 2026/09/23 04:14:26 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete6724lb # [ 191.748861] niks3-server[1102]: 2026/09/23 04:14:26 INFO Completed upload id=716725lb # [ 191.752278] niks3-server[1102]: 2026/09/23 04:14:26 INFO Registered completed upload object_key=dla3w0zf02z5jhccwa27i85bbgzrrsvb.narinfo6726worker1 # [ 191.759378] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=3223c4sl4ncpldbvw558vqwddijwzfpd-farm-a.drv assign_id=11 outputs=16727worker1 # [ 191.773639] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:26Z level=debug event=assigned drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv worker=192.168.1.3:50051 clients=16728worker1 # [ 191.783887] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186729worker1 # [ 191.789950] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:26Z level=debug event=want client=6 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv cached=06730worker1 # [ 191.795558] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186731worker1 # [ 191.812266] nix-daemon[849]: reaped child process 1856, status = succeeded6732worker1 # [ 191.814034] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6733worker1 # [ 191.818638] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:26Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186734worker1 # [ 191.946257] nix-daemon[849]: accepted connection from pid 1883, user nix-grpc-daemon (trusted)6735worker1 # [ 191.961569] nix-daemon[849]: reaped child process 1886, status = succeeded6736lb # [ 191.969486] niks3-server[1102]: 2026/09/23 04:14:26 INFO Received uploads request method=POST path=/api/pending_closures6737worker1 # [ 191.976689] nix-grpc-daemon[1578]: time=2026-09-23T04:14:26.578Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6738worker1 # [ 191.978174] nix-grpc-daemon[1578]: time=2026-09-23T04:14:26.578Z level=INFO msg="Uploading pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv (512B)"6739lb # [ 191.998523] niks3-server[1102]: 2026/09/23 04:14:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign6740worker1 # [ 192.004703] nix-grpc-daemon[1578]: time=2026-09-23T04:14:26.607Z level=INFO msg="Uploading 1 narinfos"6741lb # [ 192.012931] niks3-server[1102]: 2026/09/23 04:14:26 INFO Signed narinfos id=72 count=16742lb # [ 192.020656] niks3-server[1102]: 2026/09/23 04:14:26 INFO Registered completed upload object_key=nar/0g5bxr683biqh358mkd9xrclwllan4cgnv4pzijas586xj68wwx0.nar.zst6743lb # [ 192.036110] niks3-server[1102]: 2026/09/23 04:14:26 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.ls6744worker1 # [ 192.039650] nix-grpc-daemon[1578]: time=2026-09-23T04:14:26.642Z level=INFO msg="Upload complete. (217ms)"6745lb # [ 192.039973] niks3-server[1102]: 2026/09/23 04:14:26 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete6746lb # [ 192.049331] niks3-server[1102]: 2026/09/23 04:14:26 INFO Completed upload id=726747worker1 # [ 192.049020] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=5126748lb # [ 192.052793] niks3-server[1102]: 2026/09/23 04:14:26 INFO Registered completed upload object_key=pplpzcnjfn295nm16cg0iihpfc9iffry.narinfo6749worker1 # [ 192.062025] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186750worker1 # [ 192.067645] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6751worker1 # [ 192.072293] nix-daemon[849]: reaped child process 1881, status = succeeded6752worker1 # [ 192.083988] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6753worker1 # [ 193.111476] nix-daemon[849]: reaped child process 1890, status = succeeded6754worker1 # [ 193.203233] nix-daemon[849]: accepted connection from pid 1906, user nix-grpc-daemon (trusted)6755worker1 # [ 193.220311] nix-daemon[849]: reaped child process 1909, status = succeeded6756lb # [ 193.229713] niks3-server[1102]: 2026/09/23 04:14:27 INFO Received uploads request method=POST path=/api/pending_closures6757worker1 # [ 193.236531] nix-grpc-daemon[1578]: time=2026-09-23T04:14:27.838Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6758worker1 # [ 193.238005] nix-grpc-daemon[1578]: time=2026-09-23T04:14:27.838Z level=INFO msg="Uploading fhc1zkcygpzppf0bj5fr2k85lf7p87v9-farm-a (128B)"6759lb # [ 193.271030] niks3-server[1102]: 2026/09/23 04:14:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign6760worker1 # [ 193.278834] nix-grpc-daemon[1578]: time=2026-09-23T04:14:27.881Z level=INFO msg="Uploading 1 narinfos"6761lb # [ 193.285009] niks3-server[1102]: 2026/09/23 04:14:27 INFO Signed narinfos id=73 count=16762lb # [ 193.287976] niks3-server[1102]: 2026/09/23 04:14:27 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6763worker1 # [ 193.304310] nix-grpc-daemon[1578]: time=2026-09-23T04:14:27.906Z level=INFO msg="Upload complete. (211ms)"6764lb # [ 193.308504] niks3-server[1102]: 2026/09/23 04:14:27 INFO Registered completed upload object_key=log/pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv6765worker1 # [ 193.312730] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=pplpzcnjfn295nm16cg0iihpfc9iffry-farm-a.drv assign_id=12 outputs=16766lb # [ 193.315396] niks3-server[1102]: 2026/09/23 04:14:27 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete6767lb # [ 193.324954] niks3-server[1102]: 2026/09/23 04:14:27 INFO Completed upload id=736768lb # [ 193.328330] niks3-server[1102]: 2026/09/23 04:14:27 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.ls6769worker1 # [ 193.328457] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:27Z level=debug event=assigned drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv worker=192.168.1.3:50051 clients=16770lb # [ 193.335013] niks3-server[1102]: 2026/09/23 04:14:27 INFO Registered completed upload object_key=fhc1zkcygpzppf0bj5fr2k85lf7p87v9.narinfo6771worker1 # [ 193.335189] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:27Z level=debug event=want client=6 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv cached=06772worker1 # [ 193.338840] nix-daemon[849]: reaped child process 1888, status = succeeded6773worker1 # [ 193.366940] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186774worker1 # [ 193.381668] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186775worker1 # [ 193.395497] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186776worker1 # [ 193.409396] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6777worker1 # [ 193.555189] nix-daemon[849]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6778worker1 # [ 193.573067] nix-daemon[849]: reaped child process 1918, status = succeeded6779lb # [ 193.577341] niks3-server[1102]: 2026/09/23 04:14:28 INFO Received uploads request method=POST path=/api/pending_closures6780worker1 # [ 193.584741] nix-grpc-daemon[1578]: time=2026-09-23T04:14:28.187Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6781worker1 # [ 193.586510] nix-grpc-daemon[1578]: time=2026-09-23T04:14:28.187Z level=INFO msg="Uploading sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv (512B)"6782worker1 # [ 193.606156] nix-grpc-daemon[1578]: time=2026-09-23T04:14:28.208Z level=INFO msg="Uploading 1 narinfos"6783lb # [ 193.607402] niks3-server[1102]: 2026/09/23 04:14:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign6784lb # [ 193.620164] niks3-server[1102]: 2026/09/23 04:14:28 INFO Signed narinfos id=74 count=16785lb # [ 193.623345] niks3-server[1102]: 2026/09/23 04:14:28 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.ls6786worker1 # [ 193.637053] nix-grpc-daemon[1578]: time=2026-09-23T04:14:28.239Z level=INFO msg="Upload complete. (216ms)"6787lb # [ 193.639213] niks3-server[1102]: 2026/09/23 04:14:28 INFO Registered completed upload object_key=nar/1fxz200fmpbckafrn03b2snx9knqpbwf8rwqlbvg68356qj7aj5w.nar.zst6788worker1 # [ 193.644146] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:28Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=5126789lb # [ 193.648125] niks3-server[1102]: 2026/09/23 04:14:28 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete6790lb # [ 193.651667] niks3-server[1102]: 2026/09/23 04:14:28 INFO Completed upload id=746791lb # [ 193.654384] niks3-server[1102]: 2026/09/23 04:14:28 INFO Registered completed upload object_key=sy43wp9v52pni1wms6q0p3qybpc2pwad.narinfo6792worker1 # [ 193.657008] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186793worker1 # [ 193.664684] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6794worker1 # [ 193.667307] nix-daemon[849]: reaped child process 1913, status = succeeded6795worker1 # [ 193.677006] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6796worker1 # [ 194.781248] nix-daemon[849]: reaped child process 1922, status = succeeded6797worker1 # [ 194.844634] nix-daemon[849]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)6798worker1 # [ 194.862164] nix-daemon[849]: reaped child process 1941, status = succeeded6799lb # [ 194.871409] niks3-server[1102]: 2026/09/23 04:14:29 INFO Received uploads request method=POST path=/api/pending_closures6800worker1 # [ 194.880478] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.481Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6801worker1 # [ 194.882530] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.481Z level=INFO msg="Uploading g5laypvfk2469p8binw3mksk2hv7zhnh-farm-a (128B)"6802lb # [ 194.909940] niks3-server[1102]: 2026/09/23 04:14:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign6803worker1 # [ 194.916864] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.519Z level=INFO msg="Uploading 1 narinfos"6804lb # [ 194.924574] niks3-server[1102]: 2026/09/23 04:14:29 INFO Signed narinfos id=75 count=16805lb # [ 194.927836] niks3-server[1102]: 2026/09/23 04:14:29 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6806lb # [ 194.950330] niks3-server[1102]: 2026/09/23 04:14:29 INFO Registered completed upload object_key=log/sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv6807worker1 # [ 194.955195] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.557Z level=INFO msg="Upload complete. (266ms)"6808lb # [ 194.958510] niks3-server[1102]: 2026/09/23 04:14:29 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.ls6809lb # [ 194.962152] niks3-server[1102]: 2026/09/23 04:14:29 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete6810lb # [ 194.966614] niks3-server[1102]: 2026/09/23 04:14:29 INFO Registered completed upload object_key=g5laypvfk2469p8binw3mksk2hv7zhnh.narinfo6811lb # [ 194.970797] niks3-server[1102]: 2026/09/23 04:14:29 INFO Completed upload id=756812worker1 # [ 194.966938] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=sy43wp9v52pni1wms6q0p3qybpc2pwad-farm-a.drv assign_id=13 outputs=16813worker1 # [ 194.987297] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:29Z level=debug event=assigned drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv worker=192.168.1.3:50051 clients=16814worker1 # [ 194.992955] nix-daemon[849]: reaped child process 1920, status = succeeded6815worker1 # [ 195.014454] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186816worker1 # [ 195.022563] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:29Z level=debug event=want client=6 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv cached=06817worker1 # [ 195.034794] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186818worker1 # [ 195.052922] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.655Z level=INFO msg="All 1 paths already cached"6819worker1 # [ 195.064898] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186820worker1 # [ 195.077451] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6821worker1 # [ 195.225365] nix-daemon[849]: accepted connection from pid 1947, user nix-grpc-daemon (trusted)6822worker1 # [ 195.241250] nix-daemon[849]: reaped child process 1950, status = succeeded6823lb # [ 195.248881] niks3-server[1102]: 2026/09/23 04:14:29 INFO Received uploads request method=POST path=/api/pending_closures6824worker1 # [ 195.257362] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.859Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6825worker1 # [ 195.258934] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.859Z level=INFO msg="Uploading cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv (624B)"6826lb # [ 195.280622] niks3-server[1102]: 2026/09/23 04:14:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign6827worker1 # [ 195.285939] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.888Z level=INFO msg="Uploading 1 narinfos"6828lb # [ 195.294989] niks3-server[1102]: 2026/09/23 04:14:29 INFO Signed narinfos id=76 count=16829lb # [ 195.302315] niks3-server[1102]: 2026/09/23 04:14:29 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.ls6830lb # [ 195.312050] niks3-server[1102]: 2026/09/23 04:14:29 INFO Registered completed upload object_key=nar/1w4sykcjji9zlw0a5xibxycy6x3jyb4drd29x2k9g7x5whfwwpsq.nar.zst6831lb # [ 195.316651] niks3-server[1102]: 2026/09/23 04:14:29 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete6832worker1 # [ 195.319910] nix-grpc-daemon[1578]: time=2026-09-23T04:14:29.922Z level=INFO msg="Upload complete. (228ms)"6833lb # [ 195.322820] niks3-server[1102]: 2026/09/23 04:14:29 INFO Completed upload id=766834lb # [ 195.326037] niks3-server[1102]: 2026/09/23 04:14:29 INFO Registered completed upload object_key=cnf7j5vjqyj799isbg26wgjnrlk208gi.narinfo6835worker1 # [ 195.328325] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=6246836worker1 # [ 195.343671] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186837worker1 # [ 195.354602] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6838worker1 # [ 195.357325] nix-daemon[849]: reaped child process 1945, status = succeeded6839worker1 # [ 195.370827] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6840worker1 # [ 196.440849] nix-daemon[849]: reaped child process 1954, status = succeeded6841worker1 # [ 196.544291] nix-daemon[849]: accepted connection from pid 1970, user nix-grpc-daemon (trusted)6842worker1 # [ 196.564173] nix-daemon[849]: reaped child process 1973, status = succeeded6843lb # [ 196.571452] niks3-server[1102]: 2026/09/23 04:14:31 INFO Received uploads request method=POST path=/api/pending_closures6844worker1 # [ 196.586141] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.186Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6845worker1 # [ 196.588269] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.186Z level=INFO msg="Uploading p1334cl91hzsgv8bmiqzkwvhbwq24vn1-farm-b (176B)"6846lb # [ 196.619394] niks3-server[1102]: 2026/09/23 04:14:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign6847worker1 # [ 196.623750] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.226Z level=INFO msg="Uploading 1 narinfos"6848lb # [ 196.630044] niks3-server[1102]: 2026/09/23 04:14:31 INFO Signed narinfos id=77 count=16849lb # [ 196.637003] niks3-server[1102]: 2026/09/23 04:14:31 INFO Registered completed upload object_key=nar/0pa7j07n0n5p9wcfihhx3357xrxs4h8s57kncplymh51lvzvz2kh.nar.zst6850lb # [ 196.644822] niks3-server[1102]: 2026/09/23 04:14:31 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.ls6851lb # [ 196.651223] niks3-server[1102]: 2026/09/23 04:14:31 INFO Registered completed upload object_key=log/cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv6852lb # [ 196.655727] niks3-server[1102]: 2026/09/23 04:14:31 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete6853worker1 # [ 196.653504] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.255Z level=INFO msg="Upload complete. (272ms)"6854lb # [ 196.658710] niks3-server[1102]: 2026/09/23 04:14:31 INFO Completed upload id=776855lb # [ 196.663558] niks3-server[1102]: 2026/09/23 04:14:31 INFO Registered completed upload object_key=p1334cl91hzsgv8bmiqzkwvhbwq24vn1.narinfo6856worker1 # [ 196.670974] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=cnf7j5vjqyj799isbg26wgjnrlk208gi-farm-b.drv assign_id=14 outputs=16857worker1 # [ 196.688265] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:31Z level=debug event=assigned drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv worker=192.168.1.3:50051 clients=16858worker1 # [ 196.694691] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186859worker1 # [ 196.699810] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:31Z level=debug event=want client=6 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv cached=06860worker1 # [ 196.704594] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186861worker1 # [ 196.709135] nix-daemon[849]: reaped child process 1952, status = succeeded6862worker1 # [ 196.710874] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.310Z level=INFO msg="All 1 paths already cached"6863worker1 # [ 196.714275] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186864worker1 # [ 196.728471] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6865worker1 # [ 196.851214] nix-daemon[849]: accepted connection from pid 1979, user nix-grpc-daemon (trusted)6866worker1 # [ 196.868275] nix-daemon[849]: reaped child process 1982, status = succeeded6867lb # [ 196.875762] niks3-server[1102]: 2026/09/23 04:14:31 INFO Received uploads request method=POST path=/api/pending_closures6868worker1 # [ 196.884061] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.486Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6869worker1 # [ 196.885547] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.486Z level=INFO msg="Uploading 7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv (624B)"6870lb # [ 196.908136] niks3-server[1102]: 2026/09/23 04:14:31 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.ls6871worker1 # [ 196.921918] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.524Z level=INFO msg="Uploading 1 narinfos"6872lb # [ 196.922747] niks3-server[1102]: 2026/09/23 04:14:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign6873lb # [ 196.934833] niks3-server[1102]: 2026/09/23 04:14:31 INFO Signed narinfos id=78 count=16874lb # [ 196.942044] niks3-server[1102]: 2026/09/23 04:14:31 INFO Registered completed upload object_key=nar/0l0zss9ls4bhz18slrrmz7l1j93xz7424q7314p64rpgrpqf6b7k.nar.zst6875lb # [ 196.954818] niks3-server[1102]: 2026/09/23 04:14:31 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete6876lb # [ 196.960559] niks3-server[1102]: 2026/09/23 04:14:31 INFO Completed upload id=786877worker1 # [ 196.958764] nix-grpc-daemon[1578]: time=2026-09-23T04:14:31.561Z level=INFO msg="Upload complete. (224ms)"6878lb # [ 196.963562] niks3-server[1102]: 2026/09/23 04:14:31 INFO Registered completed upload object_key=7vjzkx8444zm2h7882vsj6bmj73zqjq3.narinfo6879worker1 # [ 196.968330] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=6246880worker1 # [ 196.986284] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186881worker1 # [ 196.993487] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6882worker1 # [ 196.998873] nix-daemon[849]: reaped child process 1977, status = succeeded6883worker1 # [ 197.010211] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6884worker1 # [ 197.636994] nix-daemon[849]: reaped child process 1986, status = succeeded6885worker1 # [ 197.738496] nix-daemon[849]: accepted connection from pid 2002, user nix-grpc-daemon (trusted)6886worker1 # [ 197.758618] nix-daemon[849]: reaped child process 2005, status = succeeded6887lb # [ 197.759142] niks3-server[1102]: 2026/09/23 04:14:32 INFO Received uploads request method=POST path=/api/pending_closures6888worker1 # [ 197.770317] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.371Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6889worker1 # [ 197.771801] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.371Z level=INFO msg="Uploading nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8-farm-b (176B)"6890lb # [ 197.800097] niks3-server[1102]: 2026/09/23 04:14:32 INFO Registered completed upload object_key=nar/0kqa5m59sp6aj1ixqj6q54wwqxkcn98fw1rc9hfkcdk460wjdl0l.nar.zst6891lb # [ 197.812378] niks3-server[1102]: 2026/09/23 04:14:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign6892worker1 # [ 197.815562] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.417Z level=INFO msg="Uploading 1 narinfos"6893lb # [ 197.821055] niks3-server[1102]: 2026/09/23 04:14:32 INFO Signed narinfos id=79 count=16894lb # [ 197.826139] niks3-server[1102]: 2026/09/23 04:14:32 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.ls6895lb # [ 197.833189] niks3-server[1102]: 2026/09/23 04:14:32 INFO Registered completed upload object_key=log/7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv6896lb # [ 197.841589] niks3-server[1102]: 2026/09/23 04:14:32 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete6897worker1 # [ 197.845046] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.447Z level=INFO msg="Upload complete. (220ms)"6898lb # [ 197.847236] niks3-server[1102]: 2026/09/23 04:14:32 INFO Completed upload id=796899lb # [ 197.851010] niks3-server[1102]: 2026/09/23 04:14:32 INFO Registered completed upload object_key=nrnwwsxw1z9dz81vri9gl1dkhkrc6vh8.narinfo6900worker1 # [ 197.860191] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 drv=7vjzkx8444zm2h7882vsj6bmj73zqjq3-farm-b.drv assign_id=15 outputs=16901worker1 # [ 197.876791] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:32Z level=debug event=assigned drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv worker=192.168.1.3:50051 clients=16902worker1 # [ 197.882774] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186903worker1 # [ 197.888559] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:32Z level=debug event=want client=6 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv cached=06904worker1 # [ 197.892831] nix-daemon[849]: reaped child process 1984, status = succeeded6905worker1 # [ 197.911237] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:32Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186906worker1 # [ 197.924831] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.527Z level=INFO msg="All 1 paths already cached"6907worker1 # [ 197.933533] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:32Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186908worker1 # [ 197.947815] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6909worker1 # [ 198.087801] nix-daemon[849]: accepted connection from pid 2011, user nix-grpc-daemon (trusted)6910worker1 # [ 198.107590] nix-daemon[849]: reaped child process 2014, status = succeeded6911lb # [ 198.112149] niks3-server[1102]: 2026/09/23 04:14:32 INFO Received uploads request method=POST path=/api/pending_closures6912worker1 # [ 198.120155] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.722Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6913worker1 # [ 198.122272] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.724Z level=INFO msg="Uploading cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv (624B)"6914lb # [ 198.142937] niks3-server[1102]: 2026/09/23 04:14:32 INFO Registered completed upload object_key=nar/0f6g0vggy0qbqsj64l1nivyqdh6q12kh8620ysnf210ani0727vf.nar.zst6915worker1 # [ 198.154174] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.756Z level=INFO msg="Uploading 1 narinfos"6916lb # [ 198.157245] niks3-server[1102]: 2026/09/23 04:14:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign6917lb # [ 198.167211] niks3-server[1102]: 2026/09/23 04:14:32 INFO Signed narinfos id=80 count=16918worker1 # [ 198.176103] nix-grpc-daemon[1578]: time=2026-09-23T04:14:32.778Z level=INFO msg="Upload complete. (216ms)"6919lb # [ 198.177136] niks3-server[1102]: 2026/09/23 04:14:32 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.ls6920lb # [ 198.181483] niks3-server[1102]: 2026/09/23 04:14:32 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete6921lb # [ 198.188559] niks3-server[1102]: 2026/09/23 04:14:32 INFO Completed upload id=806922worker1 # [ 198.184135] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=6246923lb # [ 198.190446] niks3-server[1102]: 2026/09/23 04:14:32 INFO Registered completed upload object_key=cgwyij19i3fl7sqlyz54vzfr4zajg077.narinfo6924worker1 # [ 198.199395] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186925worker1 # [ 198.206125] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6926worker1 # [ 198.210755] nix-daemon[849]: reaped child process 2009, status = succeeded6927worker1 # [ 198.221477] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6928worker1 # [ 198.871192] nix-daemon[849]: reaped child process 2018, status = succeeded6929worker1 # [ 198.968563] nix-daemon[849]: accepted connection from pid 2034, user nix-grpc-daemon (trusted)6930worker1 # [ 198.985700] nix-daemon[849]: reaped child process 2037, status = succeeded6931lb # [ 198.997818] niks3-server[1102]: 2026/09/23 04:14:33 INFO Received uploads request method=POST path=/api/pending_closures6932worker1 # [ 199.006192] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.606Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6933worker1 # [ 199.007758] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.606Z level=INFO msg="Uploading hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5-farm-b (176B)"6934lb # [ 199.033937] niks3-server[1102]: 2026/09/23 04:14:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign6935worker1 # [ 199.039785] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.642Z level=INFO msg="Uploading 1 narinfos"6936lb # [ 199.050268] niks3-server[1102]: 2026/09/23 04:14:33 INFO Signed narinfos id=81 count=16937lb # [ 199.056842] niks3-server[1102]: 2026/09/23 04:14:33 INFO Registered completed upload object_key=nar/0x964sxcxsdh3nfjarc1ib52kr50kl8zxl7h34jq5ydx763drras.nar.zst6938worker1 # [ 199.065303] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.667Z level=INFO msg="Upload complete. (232ms)"6939lb # [ 199.068240] niks3-server[1102]: 2026/09/23 04:14:33 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.ls6940lb # [ 199.073140] niks3-server[1102]: 2026/09/23 04:14:33 INFO Registered completed upload object_key=log/cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv6941lb # [ 199.076789] niks3-server[1102]: 2026/09/23 04:14:33 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete6942lb # [ 199.082546] niks3-server[1102]: 2026/09/23 04:14:33 INFO Completed upload id=816943worker1 # [ 199.078609] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 drv=cgwyij19i3fl7sqlyz54vzfr4zajg077-farm-b.drv assign_id=16 outputs=16944lb # [ 199.086084] niks3-server[1102]: 2026/09/23 04:14:33 INFO Registered completed upload object_key=hfsg1k13f2w2fjgkzxk1qi2fh35fdlk5.narinfo6945worker1 # [ 199.092585] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:33Z level=debug event=assigned drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv worker=192.168.1.3:50051 clients=16946worker1 # [ 199.099557] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186947worker1 # [ 199.105292] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:33Z level=debug event=want client=6 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv cached=06948worker1 # [ 199.111085] nix-daemon[849]: reaped child process 2016, status = succeeded6949worker1 # [ 199.114478] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186950worker1 # [ 199.119550] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.720Z level=INFO msg="All 2 paths already cached"6951worker1 # [ 199.123121] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186952worker1 # [ 199.136215] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6953worker1 # [ 199.256094] nix-daemon[849]: accepted connection from pid 2043, user nix-grpc-daemon (trusted)6954worker1 # [ 199.273068] nix-daemon[849]: reaped child process 2046, status = succeeded6955lb # [ 199.279385] niks3-server[1102]: 2026/09/23 04:14:33 INFO Received uploads request method=POST path=/api/pending_closures6956worker1 # [ 199.286905] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.889Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6957worker1 # [ 199.288513] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.889Z level=INFO msg="Uploading dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv (752B)"6958lb # [ 199.308555] niks3-server[1102]: 2026/09/23 04:14:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign6959worker1 # [ 199.313273] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.915Z level=INFO msg="Uploading 1 narinfos"6960lb # [ 199.321775] niks3-server[1102]: 2026/09/23 04:14:33 INFO Signed narinfos id=82 count=16961lb # [ 199.330466] niks3-server[1102]: 2026/09/23 04:14:33 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.ls6962lb # [ 199.337644] niks3-server[1102]: 2026/09/23 04:14:33 INFO Registered completed upload object_key=nar/0s7ff7rwqx5p9mdzddqgzd9bbd5j8njfc62k07mgnpy19810bs86.nar.zst6963worker1 # [ 199.344496] nix-grpc-daemon[1578]: time=2026-09-23T04:14:33.947Z level=INFO msg="Upload complete. (202ms)"6964lb # [ 199.346387] niks3-server[1102]: 2026/09/23 04:14:33 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete6965worker1 # [ 199.352137] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=7526966lb # [ 199.355597] niks3-server[1102]: 2026/09/23 04:14:33 INFO Completed upload id=826967lb # [ 199.358134] niks3-server[1102]: 2026/09/23 04:14:33 INFO Registered completed upload object_key=dca2k4dvnk52m7gijqfq8q99ih3ni8ny.narinfo6968worker1 # [ 199.363722] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186969worker1 # [ 199.370075] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6970worker1 # [ 199.374661] nix-daemon[849]: reaped child process 2041, status = succeeded6971worker1 # [ 199.385568] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6972worker1 # [ 200.651463] nix-daemon[849]: reaped child process 2050, status = succeeded6973worker1 # [ 200.772833] nix-daemon[849]: accepted connection from pid 2066, user nix-grpc-daemon (trusted)6974worker1 # [ 200.789997] nix-daemon[849]: reaped child process 2069, status = succeeded6975lb # [ 200.798824] niks3-server[1102]: 2026/09/23 04:14:35 INFO Received uploads request method=POST path=/api/pending_closures6976worker1 # [ 200.811375] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.412Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6977worker1 # [ 200.813458] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.412Z level=INFO msg="Uploading d88xgckxx0s0n5ly2d0jw1qy1641z8pq-farm-top (232B)"6978lb # [ 200.838954] niks3-server[1102]: 2026/09/23 04:14:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign6979worker1 # [ 200.846694] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.448Z level=INFO msg="Uploading 1 narinfos"6980lb # [ 200.855317] niks3-server[1102]: 2026/09/23 04:14:35 INFO Signed narinfos id=83 count=16981lb # [ 200.865067] niks3-server[1102]: 2026/09/23 04:14:35 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.ls6982lb # [ 200.874416] niks3-server[1102]: 2026/09/23 04:14:35 INFO Registered completed upload object_key=nar/0knxhkhz3x04vk8as5wv5ar2d0h02prv7q5f42qm1xjgr70ancd0.nar.zst6983worker1 # [ 200.887574] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.490Z level=INFO msg="Upload complete. (290ms)"6984lb # [ 200.889010] niks3-server[1102]: 2026/09/23 04:14:35 INFO Registered completed upload object_key=log/dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv6985lb # [ 200.894681] niks3-server[1102]: 2026/09/23 04:14:35 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete6986lb # [ 200.899561] niks3-server[1102]: 2026/09/23 04:14:35 INFO Completed upload id=836987lb # [ 200.903447] niks3-server[1102]: 2026/09/23 04:14:35 INFO Registered completed upload object_key=d88xgckxx0s0n5ly2d0jw1qy1641z8pq.narinfo6988worker1 # [ 200.901952] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=dca2k4dvnk52m7gijqfq8q99ih3ni8ny-farm-top.drv assign_id=17 outputs=16989worker1 # [ 200.920534] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:35Z level=debug event=assigned drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv worker=192.168.1.3:50051 clients=16990worker1 # [ 200.926453] nix-daemon[849]: reaped child process 2048, status = succeeded6991worker1 # [ 200.947522] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506186992worker1 # [ 200.963671] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506186993worker1 # [ 200.978330] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.580Z level=INFO msg="All 2 paths already cached"6994worker1 # [ 200.988111] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506186995worker1 # [ 201.001261] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)6996worker1 # [ 201.148283] nix-daemon[849]: accepted connection from pid 2073, user nix-grpc-daemon (trusted)6997worker1 # [ 201.165803] nix-daemon[849]: reaped child process 2076, status = succeeded6998lb # [ 201.173722] niks3-server[1102]: 2026/09/23 04:14:35 INFO Received uploads request method=POST path=/api/pending_closures6999worker1 # [ 201.181404] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.783Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7000worker1 # [ 201.182930] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.783Z level=INFO msg="Uploading 2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv (752B)"7001lb # [ 201.200671] niks3-server[1102]: 2026/09/23 04:14:35 INFO Registered completed upload object_key=nar/0cpiyiv7frlhngqhhc703bglz5q2cpg4n8sgds4sfg7nvw0fcdn5.nar.zst7002worker1 # [ 201.210624] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.813Z level=INFO msg="Uploading 1 narinfos"7003lb # [ 201.213861] niks3-server[1102]: 2026/09/23 04:14:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7004lb # [ 201.227033] niks3-server[1102]: 2026/09/23 04:14:35 INFO Signed narinfos id=84 count=17005lb # [ 201.232286] niks3-server[1102]: 2026/09/23 04:14:35 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.ls7006worker1 # [ 201.242634] nix-grpc-daemon[1578]: time=2026-09-23T04:14:35.845Z level=INFO msg="Upload complete. (226ms)"7007lb # [ 201.244344] niks3-server[1102]: 2026/09/23 04:14:35 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7008lb # [ 201.248434] niks3-server[1102]: 2026/09/23 04:14:35 INFO Completed upload id=847009lb # [ 201.251898] niks3-server[1102]: 2026/09/23 04:14:35 INFO Registered completed upload object_key=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9.narinfo7010worker1 # [ 201.252283] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=7527011worker1 # [ 201.267552] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506187012worker1 # [ 201.276510] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)7013worker1 # [ 201.279520] nix-daemon[849]: reaped child process 2071, status = succeeded7014worker1 # [ 201.290643] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)7015worker1 # [ 202.144353] nix-daemon[849]: reaped child process 2080, status = succeeded7016worker1 # [ 202.270497] nix-daemon[849]: accepted connection from pid 2096, user nix-grpc-daemon (trusted)7017worker1 # [ 202.285590] nix-daemon[849]: reaped child process 2099, status = succeeded7018lb # [ 202.295255] niks3-server[1102]: 2026/09/23 04:14:36 INFO Received uploads request method=POST path=/api/pending_closures7019worker1 # [ 202.308718] nix-grpc-daemon[1578]: time=2026-09-23T04:14:36.907Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7020worker1 # [ 202.310569] nix-grpc-daemon[1578]: time=2026-09-23T04:14:36.907Z level=INFO msg="Uploading vz84kyfcw804d0iyx7nj61j47ms9zxmn-farm-top (232B)"7021lb # [ 202.342072] niks3-server[1102]: 2026/09/23 04:14:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7022worker1 # [ 202.348857] nix-grpc-daemon[1578]: time=2026-09-23T04:14:36.951Z level=INFO msg="Uploading 1 narinfos"7023lb # [ 202.356400] niks3-server[1102]: 2026/09/23 04:14:36 INFO Signed narinfos id=85 count=17024lb # [ 202.363730] niks3-server[1102]: 2026/09/23 04:14:36 INFO Registered completed upload object_key=log/2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv7025lb # [ 202.376264] niks3-server[1102]: 2026/09/23 04:14:36 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.ls7026lb # [ 202.379807] niks3-server[1102]: 2026/09/23 04:14:36 INFO Registered completed upload object_key=nar/0aaz4iim7fa45zws98gbhp73l23kbm0fvp4anssnxa96v666ljhs.nar.zst7027lb # [ 202.390698] niks3-server[1102]: 2026/09/23 04:14:36 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7028lb # [ 202.397818] niks3-server[1102]: 2026/09/23 04:14:37 INFO Completed upload id=857029worker1 # [ 202.386810] nix-grpc-daemon[1578]: time=2026-09-23T04:14:36.989Z level=INFO msg="Upload complete. (284ms)"7030lb # [ 202.401479] niks3-server[1102]: 2026/09/23 04:14:37 INFO Registered completed upload object_key=vz84kyfcw804d0iyx7nj61j47ms9zxmn.narinfo7031worker1 # [ 202.402066] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=2bdcl1cxw40gyh0sjgz98vb0jixgnrd9-farm-top.drv assign_id=18 outputs=17032worker1 # [ 202.420293] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:36Z level=debug event=assigned drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv worker=192.168.1.3:50051 clients=17033worker1 # [ 202.426990] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506187034worker1 # [ 202.432665] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:506187035worker1 # [ 202.437632] nix-daemon[849]: reaped child process 2078, status = succeeded7036worker1 # [ 202.440642] nix-grpc-daemon[1578]: time=2026-09-23T04:14:37.042Z level=INFO msg="All 2 paths already cached"7037worker1 # [ 202.449220] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:506187038worker1 # [ 202.462152] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)7039worker1 # [ 202.594138] nix-daemon[849]: accepted connection from pid 2103, user nix-grpc-daemon (trusted)7040worker1 # [ 202.612659] nix-daemon[849]: reaped child process 2106, status = succeeded7041lb # [ 202.618120] niks3-server[1102]: 2026/09/23 04:14:37 INFO Received uploads request method=POST path=/api/pending_closures7042worker1 # [ 202.625566] nix-grpc-daemon[1578]: time=2026-09-23T04:14:37.227Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7043worker1 # [ 202.627075] nix-grpc-daemon[1578]: time=2026-09-23T04:14:37.227Z level=INFO msg="Uploading qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv (752B)"7044lb # [ 202.650555] niks3-server[1102]: 2026/09/23 04:14:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/86/sign7045worker1 # [ 202.655311] nix-grpc-daemon[1578]: time=2026-09-23T04:14:37.257Z level=INFO msg="Uploading 1 narinfos"7046lb # [ 202.663846] niks3-server[1102]: 2026/09/23 04:14:37 INFO Signed narinfos id=86 count=17047lb # [ 202.670025] niks3-server[1102]: 2026/09/23 04:14:37 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.ls7048worker1 # [ 202.679643] nix-grpc-daemon[1578]: time=2026-09-23T04:14:37.282Z level=INFO msg="Upload complete. (207ms)"7049lb # [ 202.681176] niks3-server[1102]: 2026/09/23 04:14:37 INFO Registered completed upload object_key=nar/05zmsnqx4aj55yncx917mz6d3hzrvz2rx2x75313vvy1kq1zv9fv.nar.zst7050lb # [ 202.689258] niks3-server[1102]: 2026/09/23 04:14:37 INFO Received complete upload request method=POST path=/api/pending_closures/86/complete7051worker1 # [ 202.688185] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=0 paths=1 nar_bytes_in=7527052lb # [ 202.692729] niks3-server[1102]: 2026/09/23 04:14:37 INFO Completed upload id=867053lb # [ 202.694862] niks3-server[1102]: 2026/09/23 04:14:37 INFO Registered completed upload object_key=qx10pa00pilyp5xdzd48l88d5jz143mr.narinfo7054worker1 # [ 202.697929] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:506187055worker1 # [ 202.705615] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)7056worker1 # [ 202.708447] nix-daemon[849]: reaped child process 2101, status = succeeded7057worker1 # [ 202.717780] nix-daemon[849]: accepted connection from pid 1477, user nix-grpc-daemon (trusted)7058worker1 # [ 203.611507] nix-daemon[849]: reaped child process 2110, status = succeeded7059worker1 # [ 203.696511] nix-daemon[849]: accepted connection from pid 2126, user nix-grpc-daemon (trusted)7060worker1 # [ 203.712897] nix-daemon[849]: reaped child process 2129, status = succeeded7061lb # [ 203.720861] niks3-server[1102]: 2026/09/23 04:14:38 INFO Received uploads request method=POST path=/api/pending_closures7062worker1 # [ 203.730848] nix-grpc-daemon[1578]: time=2026-09-23T04:14:38.332Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7063worker1 # [ 203.732624] nix-grpc-daemon[1578]: time=2026-09-23T04:14:38.332Z level=INFO msg="Uploading iagwbb072hnm79qb9wsrjp5907zc7aa2-farm-top (232B)"7064lb # [ 203.761913] niks3-server[1102]: 2026/09/23 04:14:38 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.ls7065worker1 # [ 203.771164] nix-grpc-daemon[1578]: time=2026-09-23T04:14:38.373Z level=INFO msg="Uploading 1 narinfos"7066lb # [ 203.774962] niks3-server[1102]: 2026/09/23 04:14:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/87/sign7067lb # [ 203.785422] niks3-server[1102]: 2026/09/23 04:14:38 INFO Signed narinfos id=87 count=17068lb # [ 203.791206] niks3-server[1102]: 2026/09/23 04:14:38 INFO Registered completed upload object_key=log/qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv7069lb # [ 203.798185] niks3-server[1102]: 2026/09/23 04:14:38 INFO Registered completed upload object_key=nar/053jd22j3kv7v6j0q7cspdpc0h1c1xmywmajl6sd5hzwg50z2yj1.nar.zst7070lb # [ 203.808285] niks3-server[1102]: 2026/09/23 04:14:38 INFO Received complete upload request method=POST path=/api/pending_closures/87/complete7071lb # [ 203.813366] niks3-server[1102]: 2026/09/23 04:14:38 INFO Completed upload id=877072worker1 # [ 203.804579] nix-grpc-daemon[1578]: time=2026-09-23T04:14:38.407Z level=INFO msg="Upload complete. (276ms)"7073lb # [ 203.817197] niks3-server[1102]: 2026/09/23 04:14:38 INFO Registered completed upload object_key=iagwbb072hnm79qb9wsrjp5907zc7aa2.narinfo7074worker1 # [ 203.817730] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:50618 duration_s=1 drv=qx10pa00pilyp5xdzd48l88d5jz143mr-farm-top.drv assign_id=19 outputs=17075worker1 # [ 203.837031] nix-daemon[849]: reaped child process 2108, status = succeeded7076client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.49 seconds)7077worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7078worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7079client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]'7080client # [ 204.013996] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7081client # Running as unit: gpu.service; invocation ID: 22268d7f132d4580b40413ca5015c79d7082client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.09 seconds)7083lb: must succeed: curl -sf localhost:9901/clusters7084lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7085lb: must succeed: curl -sf localhost:9901/clusters7086lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7087worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7088client # [ 204.219059] nix[1623]: don't know how to build these paths:7089worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7090client # [ 204.219997] nix[1623]: /nix/store/3i1nziwpdqq19lzpqy4rywzxnm8dlps6-farm-top.drv7091worker2 # [ 204.217211] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:38Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520707092worker2 # [ 204.225245] nix-grpc-daemon[1985]: ts=2026-09-23T04:14:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520707093client # [ 204.250686] nix[1623]: warning: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}, waiting up to 30s for a worker7094worker1 # [ 204.245008] nix-grpc-daemon[1477]: ts=2026-09-23T04:14:38Z level=debug event=want client=7 drv=1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv cached=07095lb: must succeed: curl -sf localhost:9901/clusters7096lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7097lb: must succeed: curl -sf localhost:9901/clusters7098lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7099worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7100worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.06 seconds)7101client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7102lb # [ 209.902119] envoy[906]: conn peer=[2001:db8:1::1]:36116 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27080 tx=12299 ms=142247103client # [ 234.256637] nix[1623]: 1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv: no connected, non-draining worker for system 'aarch64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,gccarch-armv8-a,kvm,nixos-test,vip}7104client # [ 234.269609] nix[1623]: error: dependency '1jn0w6x679ijjxwcfb2gxc6lw2fxim2k-farm-a.drv' failed7105client # [ 234.311691] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7106client # [ 234.319486] systemd[1]: gpu.service: Failed with result 'exit-code'.7107client # [ 234.323997] systemd[1]: gpu.service: Consumed 200ms CPU time over 30.303s wall clock time, 37.4M memory peak, 5.6K incoming IP traffic, 5.2K outgoing IP traffic.7108client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.03 seconds)7109client: must succeed: journalctl -u gpu -o cat7110client: (finished: must succeed: journalctl -u gpu -o cat, in 0.03 seconds)7111lb: must succeed: curl -sf localhost:9901/clusters7112lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)7113lb: must succeed: curl -sf localhost:9901/clusters7114lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7115worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7116worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7117(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.24 seconds)7118subtest: low disk drains a worker and builds go to the other7119worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7120worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 1.16 seconds)7121worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7122worker1 # [ 236.607904] nix-grpc-daemon[1477]: ts=2026-09-23T04:15:11Z level=info event=unhealthy reason=min_free7123worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.11 seconds)7124client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&27125client # don't know how to build these paths:7126client # /nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7127worker2 # [ 237.136113] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:520707128worker2 # [ 237.144711] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520707129worker1 # [ 237.168479] nix-grpc-daemon[1477]: ts=2026-09-23T04:15:11Z level=debug event=want client=8 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv cached=07130worker1 # [ 237.174365] nix-grpc-daemon[1477]: ts=2026-09-23T04:15:11Z level=debug event=assigned drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv worker=192.168.1.4:50051 clients=17131worker2 # [ 237.219355] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520707132worker2 # [ 237.236398] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520707133client # copying 1 paths to 'lb:50051'...7134client # copying path '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' to 'lb:50051'...7135worker2 # [ 237.254899] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520707136worker2 # [ 237.271667] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7137worker2 # [ 237.439740] nix-daemon[840]: accepted connection from pid 2145, user nix-grpc-daemon (trusted)7138worker2 # [ 237.460394] nix-daemon[840]: reaped child process 2148, status = succeeded7139lb # [ 237.462359] niks3-server[1102]: 2026/09/23 04:15:12 INFO Received uploads request method=POST path=/api/pending_closures7140worker2 # [ 237.480313] nix-grpc-daemon[2046]: time=2026-09-23T04:15:12.058Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7141worker2 # [ 237.482391] nix-grpc-daemon[2046]: time=2026-09-23T04:15:12.058Z level=INFO msg="Uploading kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv (512B)"7142lb # [ 237.512582] niks3-server[1102]: 2026/09/23 04:15:12 INFO Registered completed upload object_key=nar/1sfy0gcq4znfcj1f68c749r5x2jwmwsfbfp6xdflz5xkndjwjjm3.nar.zst7143worker2 # [ 237.529391] nix-grpc-daemon[2046]: time=2026-09-23T04:15:12.108Z level=INFO msg="Uploading 1 narinfos"7144lb # [ 237.526184] niks3-server[1102]: 2026/09/23 04:15:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/88/sign7145lb # [ 237.536395] niks3-server[1102]: 2026/09/23 04:15:12 INFO Signed narinfos id=88 count=17146lb # [ 237.554565] niks3-server[1102]: 2026/09/23 04:15:12 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.ls7147lb # [ 237.568671] niks3-server[1102]: 2026/09/23 04:15:12 INFO Received complete upload request method=POST path=/api/pending_closures/88/complete7148lb # [ 237.576366] niks3-server[1102]: 2026/09/23 04:15:12 INFO Completed upload id=887149worker2 # [ 237.577747] nix-grpc-daemon[2046]: time=2026-09-23T04:15:12.156Z level=INFO msg="Upload complete. (286ms)"7150lb # [ 237.580506] niks3-server[1102]: 2026/09/23 04:15:12 INFO Registered completed upload object_key=kksl1jqfg56vabgqvrkz5vki6nq7r59r.narinfo7151worker2 # [ 237.588383] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=0 paths=1 nar_bytes_in=5127152worker2 # [ 237.609115] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520707153worker2 # [ 237.622665] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7154worker2 # [ 237.627308] nix-daemon[840]: reaped child process 2143, status = succeeded7155client # building '/nix/store/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv' on lb:50051...7156worker2 # [ 237.643506] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7157client # farm-a> worker2-next: building kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7158client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7159client # farm-a> LOG-farm-a-drain7160worker2 # [ 238.543422] nix-daemon[840]: reaped child process 2152, status = succeeded7161worker2 # [ 238.612897] nix-daemon[840]: accepted connection from pid 2168, user nix-grpc-daemon (trusted)7162worker2 # [ 238.632786] nix-daemon[840]: reaped child process 2171, status = succeeded7163lb # [ 238.634440] niks3-server[1102]: 2026/09/23 04:15:13 INFO Received uploads request method=POST path=/api/pending_closures7164worker2 # [ 238.651728] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.228Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7165worker2 # [ 238.653678] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.229Z level=INFO msg="Uploading 272b39wn7cbnhkcc2rrz2f6kmhhmcwnr-farm-a (128B)"7166lb # [ 238.688094] niks3-server[1102]: 2026/09/23 04:15:13 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7167lb # [ 238.697223] niks3-server[1102]: 2026/09/23 04:15:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/89/sign7168worker2 # [ 238.708205] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.286Z level=INFO msg="Uploading 1 narinfos"7169lb # [ 238.708571] niks3-server[1102]: 2026/09/23 04:15:13 INFO Signed narinfos id=89 count=17170lb # [ 238.715437] niks3-server[1102]: 2026/09/23 04:15:13 INFO Registered completed upload object_key=log/kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv7171lb # [ 238.723603] niks3-server[1102]: 2026/09/23 04:15:13 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.ls7172lb # [ 238.730736] niks3-server[1102]: 2026/09/23 04:15:13 INFO Received complete upload request method=POST path=/api/pending_closures/89/complete7173lb # [ 238.737764] niks3-server[1102]: 2026/09/23 04:15:13 INFO Completed upload id=897174worker2 # [ 238.739588] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.318Z level=INFO msg="Upload complete. (280ms)"7175lb # [ 238.739953] niks3-server[1102]: 2026/09/23 04:15:13 INFO Registered completed upload object_key=272b39wn7cbnhkcc2rrz2f6kmhhmcwnr.narinfo7176worker2 # [ 238.753325] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=1 drv=kksl1jqfg56vabgqvrkz5vki6nq7r59r-farm-a.drv assign_id=20 outputs=17177worker1 # [ 238.754303] nix-grpc-daemon[1477]: ts=2026-09-23T04:15:13Z level=debug event=want client=8 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv cached=07178worker1 # [ 238.760668] nix-grpc-daemon[1477]: ts=2026-09-23T04:15:13Z level=debug event=assigned drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv worker=192.168.1.4:50051 clients=17179worker2 # [ 238.776246] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520707180worker2 # [ 238.783096] nix-daemon[840]: reaped child process 2150, status = succeeded7181worker2 # [ 238.791432] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520707182worker2 # [ 238.805681] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.384Z level=INFO msg="All 1 paths already cached"7183client # copying 1 paths to 'lb:50051'...7184client # copying path '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' to 'lb:50051'...7185worker2 # [ 238.816921] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520707186worker2 # [ 238.832572] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7187worker2 # [ 238.981602] nix-daemon[840]: accepted connection from pid 2175, user nix-grpc-daemon (trusted)7188worker2 # [ 239.004586] nix-daemon[840]: reaped child process 2178, status = succeeded7189lb # [ 239.000639] niks3-server[1102]: 2026/09/23 04:15:13 INFO Received uploads request method=POST path=/api/pending_closures7190worker2 # [ 239.015678] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.594Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7191worker2 # [ 239.017429] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.594Z level=INFO msg="Uploading fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv (632B)"7192lb # [ 239.036720] niks3-server[1102]: 2026/09/23 04:15:13 INFO Registered completed upload object_key=nar/1ax4z88sb8d24gsvydxvgdy2lijkmhnycxhr5war1j3sq97cyzqf.nar.zst7193worker2 # [ 239.049339] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.628Z level=INFO msg="Uploading 1 narinfos"7194lb # [ 239.053302] niks3-server[1102]: 2026/09/23 04:15:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/90/sign7195lb # [ 239.060729] niks3-server[1102]: 2026/09/23 04:15:13 INFO Signed narinfos id=90 count=17196worker2 # [ 239.073203] nix-grpc-daemon[2046]: time=2026-09-23T04:15:13.652Z level=INFO msg="Upload complete. (226ms)"7197lb # [ 239.071516] niks3-server[1102]: 2026/09/23 04:15:13 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.ls7198lb # [ 239.078199] niks3-server[1102]: 2026/09/23 04:15:13 INFO Received complete upload request method=POST path=/api/pending_closures/90/complete7199worker2 # [ 239.084133] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=0 paths=1 nar_bytes_in=6327200lb # [ 239.085048] niks3-server[1102]: 2026/09/23 04:15:13 INFO Completed upload id=907201lb # [ 239.087033] niks3-server[1102]: 2026/09/23 04:15:13 INFO Registered completed upload object_key=fa7adcm0x8sbgmrsfx912fp8xakk99g6.narinfo7202worker2 # [ 239.095007] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520707203worker2 # [ 239.104737] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7204worker2 # [ 239.107664] nix-daemon[840]: reaped child process 2173, status = succeeded7205client # building '/nix/store/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv' on lb:50051...7206worker2 # [ 239.121276] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7207client # farm-b> worker2-next: building fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7208lb # [ 239.910213] envoy[906]: conn peer=[2001:db8:1::1]:57642 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=931 tx=613 ms=300717209client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7210client # farm-b> LOG-farm-b-drain7211worker2 # [ 240.385827] nix-daemon[840]: reaped child process 2182, status = succeeded7212worker2 # [ 240.517824] nix-daemon[840]: accepted connection from pid 2198, user nix-grpc-daemon (trusted)7213worker2 # [ 240.538317] nix-daemon[840]: reaped child process 2201, status = succeeded7214lb # [ 240.535553] niks3-server[1102]: 2026/09/23 04:15:15 INFO Received uploads request method=POST path=/api/pending_closures7215worker2 # [ 240.551793] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.128Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7216worker2 # [ 240.553636] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.128Z level=INFO msg="Uploading g774lks418zc45bd9hwhinm61pwcwnr9-farm-b (176B)"7217lb # [ 240.584623] niks3-server[1102]: 2026/09/23 04:15:15 INFO Registered completed upload object_key=nar/1ammzz4zx9sggy8v73zxk3j08vbqxgpw14n8lx2fh57bx24yzvnj.nar.zst7218worker2 # [ 240.599460] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.178Z level=INFO msg="Uploading 1 narinfos"7219lb # [ 240.600075] niks3-server[1102]: 2026/09/23 04:15:15 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.ls7220lb # [ 240.603492] niks3-server[1102]: 2026/09/23 04:15:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/91/sign7221lb # [ 240.616750] niks3-server[1102]: 2026/09/23 04:15:15 INFO Signed narinfos id=91 count=17222lb # [ 240.618703] niks3-server[1102]: 2026/09/23 04:15:15 INFO Received complete upload request method=POST path=/api/pending_closures/91/complete7223worker2 # [ 240.619514] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.198Z level=INFO msg="Upload complete. (287ms)"7224lb # [ 240.627489] niks3-server[1102]: 2026/09/23 04:15:15 INFO Completed upload id=917225lb # [ 240.629390] niks3-server[1102]: 2026/09/23 04:15:15 INFO Registered completed upload object_key=g774lks418zc45bd9hwhinm61pwcwnr9.narinfo7226worker2 # [ 240.631844] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:15Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=1 drv=fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv assign_id=21 outputs=17227lb # [ 240.635045] niks3-server[1102]: 2026/09/23 04:15:15 INFO Registered completed upload object_key=log/fa7adcm0x8sbgmrsfx912fp8xakk99g6-farm-b.drv7228worker1 # [ 240.636618] nix-grpc-daemon[1477]: ts=2026-09-23T04:15:15Z level=debug event=want client=8 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv cached=07229worker1 # [ 240.642988] nix-grpc-daemon[1477]: ts=2026-09-23T04:15:15Z level=debug event=assigned drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv worker=192.168.1.4:50051 clients=17230worker2 # [ 240.650953] nix-daemon[840]: reaped child process 2180, status = succeeded7231worker2 # [ 240.687084] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520707232worker2 # [ 240.702727] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:520707233worker2 # [ 240.718401] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.297Z level=INFO msg="All 2 paths already cached"7234client # copying 1 paths to 'lb:50051'...7235client # copying path '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' to 'lb:50051'...7236worker2 # [ 240.729987] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:520707237worker2 # [ 240.744578] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7238worker2 # [ 240.909720] nix-daemon[840]: accepted connection from pid 2205, user nix-grpc-daemon (trusted)7239worker2 # [ 240.933068] nix-daemon[840]: reaped child process 2208, status = succeeded7240lb # [ 240.934526] niks3-server[1102]: 2026/09/23 04:15:15 INFO Received uploads request method=POST path=/api/pending_closures7241worker2 # [ 240.949414] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.528Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7242worker2 # [ 240.951396] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.528Z level=INFO msg="Uploading 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv (760B)"7243worker2 # [ 240.984294] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.563Z level=INFO msg="Uploading 1 narinfos"7244lb # [ 240.972304] niks3-server[1102]: 2026/09/23 04:15:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/92/sign7245lb # [ 240.989385] niks3-server[1102]: 2026/09/23 04:15:15 INFO Signed narinfos id=92 count=17246lb # [ 240.996521] niks3-server[1102]: 2026/09/23 04:15:15 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.ls7247lb # [ 241.005901] niks3-server[1102]: 2026/09/23 04:15:15 INFO Registered completed upload object_key=nar/1hrr94bvb37dwmf1giypzl9qnff72xm540ak7q49j34fyzia9ngb.nar.zst7248worker2 # [ 241.023472] nix-grpc-daemon[2046]: time=2026-09-23T04:15:15.602Z level=INFO msg="Upload complete. (261ms)"7249lb # [ 241.021542] niks3-server[1102]: 2026/09/23 04:15:15 INFO Received complete upload request method=POST path=/api/pending_closures/92/complete7250lb # [ 241.026574] niks3-server[1102]: 2026/09/23 04:15:15 INFO Completed upload id=927251lb # [ 241.030072] niks3-server[1102]: 2026/09/23 04:15:15 INFO Registered completed upload object_key=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l.narinfo7252worker2 # [ 241.036426] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=0 paths=1 nar_bytes_in=7607253worker2 # [ 241.056737] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:520707254worker2 # [ 241.065109] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7255worker2 # [ 241.071718] nix-daemon[840]: reaped child process 2203, status = succeeded7256client # building '/nix/store/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv' on lb:50051...7257client # farm-top> worker2-next: building 77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7258worker2 # [ 241.083031] nix-daemon[840]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7259client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7260client # farm-top> LOG-farm-top-drain7261worker2 # [ 242.188693] nix-daemon[840]: reaped child process 2212, status = succeeded7262worker2 # [ 242.267629] nix-daemon[840]: accepted connection from pid 2228, user nix-grpc-daemon (trusted)7263worker2 # [ 242.290616] nix-daemon[840]: reaped child process 2231, status = succeeded7264lb # [ 242.287767] niks3-server[1102]: 2026/09/23 04:15:16 INFO Received uploads request method=POST path=/api/pending_closures7265worker2 # [ 242.306813] nix-grpc-daemon[2046]: time=2026-09-23T04:15:16.884Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7266worker2 # [ 242.308700] nix-grpc-daemon[2046]: time=2026-09-23T04:15:16.884Z level=INFO msg="Uploading bzkv277k4v9r9h4aq5h13ddcavmp36fg-farm-top (232B)"7267worker2 # [ 242.344421] nix-grpc-daemon[2046]: time=2026-09-23T04:15:16.923Z level=INFO msg="Uploading 1 narinfos"7268lb # [ 242.331973] niks3-server[1102]: 2026/09/23 04:15:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/93/sign7269lb # [ 242.347258] niks3-server[1102]: 2026/09/23 04:15:16 INFO Signed narinfos id=93 count=17270lb # [ 242.355751] niks3-server[1102]: 2026/09/23 04:15:16 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.ls7271lb # [ 242.364089] niks3-server[1102]: 2026/09/23 04:15:16 INFO Registered completed upload object_key=log/77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv7272lb # [ 242.367388] niks3-server[1102]: 2026/09/23 04:15:16 INFO Registered completed upload object_key=nar/0l8sy8grn9pmq2n58ag0hx791zdfyxwk9319ynjqmn7yy7wl64s7.nar.zst7273worker2 # [ 242.381267] nix-grpc-daemon[2046]: time=2026-09-23T04:15:16.960Z level=INFO msg="Upload complete. (284ms)"7274lb # [ 242.379477] niks3-server[1102]: 2026/09/23 04:15:16 INFO Received complete upload request method=POST path=/api/pending_closures/93/complete7275lb # [ 242.383076] niks3-server[1102]: 2026/09/23 04:15:16 INFO Completed upload id=937276lb # [ 242.386573] niks3-server[1102]: 2026/09/23 04:15:16 INFO Registered completed upload object_key=bzkv277k4v9r9h4aq5h13ddcavmp36fg.narinfo7277worker2 # [ 242.393185] nix-grpc-daemon[1985]: ts=2026-09-23T04:15:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:52070 duration_s=1 drv=77925dzx1nz9jvb0jf5cxdn4b8gwrv5l-farm-top.drv assign_id=22 outputs=17278worker2 # [ 242.414427] nix-daemon[840]: reaped child process 2210, status = succeeded7279client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.pem&client-key=/nix/store/hshhzk1ggy1xxf00y8n24b7qg938ckjz-farm-certs/ci.key' --eval-store auto -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain >&2, in 5.53 seconds)7280client: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath7281client: (finished: must succeed: nix eval --raw -f /nix/store/vr4gwv4y4d4pl04zngqql7lcz1lvgab1-job.nix --argstr tag drain outPath, in 0.19 seconds)7282worker1: must succeed: rm /nix/.rw-store/fill7283worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.16 seconds)7284worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7285worker1 # [ 243.612703] nix-grpc-daemon[1477]: ts=2026-09-23T04:15:18Z level=info event=healthy reason=min_free7286worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.11 seconds)7287(finished: subtest: low disk drains a worker and builds go to the other, in 9.35 seconds)7288(finished: run the VM test script, in 244.90 seconds)7289test script finished in 245.26s7290cleanup7291kill QemuMachine (pid 49)7292client # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7293client # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7294client # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7295client # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7296kill QemuMachine (pid 50)7297lb # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7298lb # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7299lb # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7300lb # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7301kill QemuMachine (pid 51)7302worker1 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7303worker1 # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7304worker1 # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7305worker1 # [2026-09-23T04:15:19Z INFO virtiofsd] Client disconnected, shutting down7306kill QemuMachine (pid 52)7307worker2 # qemu-system-aarch64: terminating on signal 15 from pid 39 (/nix/store/3fl7bdkdxk2k4nssy1d1161isbzn6bsr-python3-3.14.7/bin/python3.14)7308worker2 # [2026-09-23T04:15:20Z INFO virtiofsd] Client disconnected, shutting down7309worker2 # [2026-09-23T04:15:20Z INFO virtiofsd] Client disconnected, shutting down7310worker2 # [2026-09-23T04:15:20Z INFO virtiofsd] Client disconnected, shutting down7311(finished: cleanup, in 1.08 seconds)7312additionally exposed symbols:7313 client, lb, worker1, worker2,7314 vlan1,7315 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh