vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #502
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1: QEMU running (pid 52)13worker1 # Disk image does not exist, creating the virtualisation disk image...14worker1 # Formatting '/build/vm-state-worker1/tmp.Nbr6SOfDu0', fmt=raw size=107374182415worker1 # mke2fs 1.47.4 (6-Mar-2025)16worker1 # Discarding device blocks: 0/262144 done17worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes18worker1 # Filesystem UUID: 7d823774-0814-4db3-bca3-e30f19ea8fc019worker1 # Superblock backups stored on blocks:20worker1 # 32768, 98304, 163840, 22937621worker1 # 22worker1 # Allocating group tables: 0/8 done23worker1 # Writing inode tables: 0/8 done24worker1 # Creating journal (8192 blocks): done25worker1 # Writing superblocks and filesystem accounting information: 0/8 done26worker1 # 27worker1 # Virtualisation disk image created.28worker1 # Starting virtiofs daemons...29worker1 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)30worker1 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether31worker1 # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...32worker1 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)33worker1 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether34worker1 # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...35worker1 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)36worker2 # Disk image does not exist, creating the virtualisation disk image...37worker1 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether38worker2 # Formatting '/build/vm-state-worker2/tmp.OSiLaam6a0', fmt=raw size=107374182439worker1 # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...40worker2 # mke2fs 1.47.4 (6-Mar-2025)41client: QEMU running (pid 49)42worker2: QEMU running (pid 51)43worker2 # Discarding device blocks: 0/262144 done44client # Disk image does not exist, creating the virtualisation disk image...45worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes46client # Formatting '/build/vm-state-client/tmp.w5YXlg8wic', fmt=raw size=107374182447worker2 # Filesystem UUID: 39edbccd-c281-4673-885f-74cac8e156c648client # mke2fs 1.47.4 (6-Mar-2025)49worker2 # Superblock backups stored on blocks:50client # Discarding device blocks: 0/262144 done51worker2 # 32768, 98304, 163840, 22937652client # Creating filesystem with 262144 4k blocks and 65536 inodes53worker2 # 54client # Filesystem UUID: 8c7b3614-db1c-412b-887e-f6c723c095b255worker2 # Allocating group tables: 0/8 done56worker2 # Writing inode tables: 0/8 done57worker2 # Creating journal (8192 blocks): done58lb # Disk image does not exist, creating the virtualisation disk image...59lb: QEMU running (pid 50)60client # Superblock backups stored on blocks:61worker1 # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests62worker1 # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests63client # 32768, 98304, 163840, 22937664worker2 # Writing superblocks and filesystem accounting information: 0/8 done65worker1 # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests66client # 67client # Allocating group tables: 0/8 done68worker2 # 69(finished: start all VMs, in 0.25 seconds)70lb # Formatting '/build/vm-state-lb/tmp.nHTMfWawhd', fmt=raw size=107374182471client # Writing inode tables: 0/8 done72worker2 # Virtualisation disk image created.73worker2 # Starting virtiofs daemons...74client: waiting for unit network-addresses-eth1.service75lb # mke2fs 1.47.4 (6-Mar-2025)76client # Creating journal (8192 blocks): done77worker2 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)78client: waiting for the VM to finish booting79lb # Discarding device blocks: 0/262144 done80client # Writing superblocks and filesystem accounting information: 0/8 done81client # 82lb # Creating filesystem with 262144 4k blocks and 65536 inodes83worker2 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84worker2 # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...85worker2 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)86worker2 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether87client # Virtualisation disk image created.88client # Starting virtiofs daemons...89lb # Filesystem UUID: 1cb8b1e1-3655-44cd-a136-c6cc505c204890client # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)91client # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether92worker2 # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...93client # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...94worker2 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)95client # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)96worker2 # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether97client # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether98worker2 # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...99lb # Superblock backups stored on blocks:100worker2 # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests101worker2 # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests102client # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...103worker2 # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests104client # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)105client # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether106client # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...107client # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests108client # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests109client # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests110lb # 32768, 98304, 163840, 229376111lb # 112lb # Allocating group tables: 0/8 done113lb # Writing inode tables: 0/8 done114lb # Creating journal (8192 blocks): done115lb # Writing superblocks and filesystem accounting information: 0/8 done116lb # 117lb # Virtualisation disk image created.118lb # Starting virtiofs daemons...119lb # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120lb # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121lb # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...122lb # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123lb # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124lb # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...125lb # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126lb # [2026-09-22T15:03:11Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127lb # [2026-09-22T15:03:11Z INFO virtiofsd] Waiting for vhost-user socket connection...128lb # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests129lb # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests130lb # [2026-09-22T15:03:11Z INFO virtiofsd] Client connected, servicing requests131client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135client # 136client # 137worker2 # 138worker2 # 139worker1 # 140worker1 # 141client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00142worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00143worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00144client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...145client # 146client # 147client # 148client # 149worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...150worker2 # 151worker2 # 152worker2 # 153worker2 # 154worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...155worker1 # 156worker1 # 157worker1 # 158worker1 # 159client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00160worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00161worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00162client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...163client # 164client # 165worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...166worker2 # 167worker2 # 168worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169worker1 # 170worker1 # 171lb # 172lb # 173lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00174lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...175lb # 176lb # 177lb # 178lb # 179lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181lb # 182lb # 183client # Booting from ROM...184worker2 # Booting from ROM...185worker1 # Booting from ROM...186lb # Booting from ROM...187client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/dgyp9fws2n1q2rg73k4nrsyrfqvcsmf6-nixos-system-client-test/init regInfo=/nix/store/2jmcw81c7phijdhwbkm9pn7gpcs89hq7-closure-info/registration console=ttyS0,115200n8 console=tty0189client # [ 0.000000] BIOS-provided physical RAM map:190client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200client # [ 0.000000] NX (Execute Disable) protection: active201client # [ 0.000000] APIC: Static calls initialized202client # [ 0.000000] SMBIOS 2.8 present.203worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026204client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014205client # [ 0.000000] DMI: Memory slots populated: 1/1206client # [ 0.000000] Hypervisor detected: KVM207client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000208client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00209client # [ 0.000000] kvm-clock: using sched offset of 476344059 cycles210worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gcqx0ly4a11w34z8kprj4p7mv3aciczk-nixos-system-worker2-test/init regInfo=/nix/store/744ap97ay03z4ngf1q9ax96r0w4xa55l-closure-info/registration console=ttyS0,115200n8 console=tty0211worker2 # [ 0.000000] BIOS-provided physical RAM map:212client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns213worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026214worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable215client # [ 0.000005] tsc: Detected 2400.012 MHz processor216worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved217client # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000218worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved219client # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs220lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026221worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable222client # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT223worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved224client # [ 0.002725] found SMP MP-table at [mem 0x000f5450-0x000f545f]225worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qdm894g7fk24niwl4p4m0ddcdy1ggpcl-nixos-system-worker1-test/init regInfo=/nix/store/ydgn654pqammmz3kxba0bglp9b3f3vq8-closure-info/registration console=ttyS0,115200n8 console=tty0226client # [ 0.002736] Using GB pages for direct mapping227worker1 # [ 0.000000] BIOS-provided physical RAM map:228worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved229client # [ 0.002804] RAMDISK: [mem 0x5e370000-0x5ffcffff]230worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved231worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable232client # [ 0.002812] ACPI: Early table checksum verification disabled233lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qfr7fghmwm537hb3f4f5is3yb8phm6jx-nixos-system-lb-test/init regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration console=ttyS0,115200n8 console=tty0234client # [ 0.002815] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )235worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved236worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved237lb # [ 0.000000] BIOS-provided physical RAM map:238worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved239worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved240client # [ 0.002819] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)241lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable242worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved243worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable244client # [ 0.002824] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)245lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved246worker2 # [ 0.000000] NX (Execute Disable) protection: active247worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved248worker2 # [ 0.000000] APIC: Static calls initialized249lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved250client # [ 0.002831] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)251worker2 # [ 0.000000] SMBIOS 2.8 present.252worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved253client # [ 0.002833] ACPI: FACS 0x000000005FFE0000 000040254lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable255worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved256worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014257lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved258client # [ 0.002835] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)259worker2 # [ 0.000000] DMI: Memory slots populated: 1/1260worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved261worker2 # [ 0.000000] Hypervisor detected: KVM262lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved263client # [ 0.002836] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)264worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved265worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000266lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved267client # [ 0.002838] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)268worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00269worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved270lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved271worker1 # [ 0.000000] NX (Execute Disable) protection: active272worker2 # [ 0.000000] kvm-clock: using sched offset of 482679587 cycles273client # [ 0.002840] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)274worker1 # [ 0.000000] APIC: Static calls initialized275lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved276worker1 # [ 0.000000] SMBIOS 2.8 present.277client # [ 0.002841] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]278worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns279lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved280worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor281client # [ 0.002842] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]282lb # [ 0.000000] NX (Execute Disable) protection: active283worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014284worker1 # [ 0.000000] DMI: Memory slots populated: 1/1285lb # [ 0.000000] APIC: Static calls initialized286worker2 # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000287client # [ 0.002842] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]288lb # [ 0.000000] SMBIOS 2.8 present.289worker1 # [ 0.000000] Hypervisor detected: KVM290worker2 # [ 0.000840] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs291client # [ 0.002843] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]292worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000293lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014294worker2 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT295worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00296client # [ 0.002843] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]297lb # [ 0.000000] DMI: Memory slots populated: 1/1298worker2 # [ 0.002772] found SMP MP-table at [mem 0x000f5450-0x000f545f]299worker1 # [ 0.000001] kvm-clock: using sched offset of 495723932 cycles300lb # [ 0.000000] Hypervisor detected: KVM301client # [ 0.002844] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]302worker2 # [ 0.002783] Using GB pages for direct mapping303lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000304worker2 # [ 0.002942] RAMDISK: [mem 0x5e36f000-0x5ffcffff]305client # [ 0.002844] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]306worker1 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns307lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00308client # [ 0.003073] No NUMA configuration found309worker2 # [ 0.002951] ACPI: Early table checksum verification disabled310worker1 # [ 0.000006] tsc: Detected 2400.012 MHz processor311lb # [ 0.000000] kvm-clock: using sched offset of 466086719 cycles312worker2 # [ 0.002953] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )313client # [ 0.003074] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]314worker1 # [ 0.000813] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000315client # [ 0.003077] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]316lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns317worker2 # [ 0.002957] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)318worker1 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs319client # [ 0.005372] Zone ranges:320lb # [ 0.000005] tsc: Detected 2400.012 MHz processor321client # [ 0.005373] DMA [mem 0x0000000000001000-0x0000000000ffffff]322worker1 # [ 0.000843] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT323worker2 # [ 0.002961] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)324lb # [ 0.000810] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000325client # [ 0.005374] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]326worker1 # [ 0.002725] found SMP MP-table at [mem 0x000f5450-0x000f545f]327client # [ 0.005375] Normal empty328worker2 # [ 0.002968] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)329worker1 # [ 0.002737] Using GB pages for direct mapping330lb # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs331client # [ 0.005376] Device empty332worker2 # [ 0.002970] ACPI: FACS 0x000000005FFE0000 000040333worker1 # [ 0.002853] RAMDISK: [mem 0x5e370000-0x5ffcffff]334client # [ 0.005377] Movable zone start for each node335lb # [ 0.000839] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT336client # [ 0.005377] Early memory node ranges337worker1 # [ 0.002862] ACPI: Early table checksum verification disabled338worker2 # [ 0.002972] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)339lb # [ 0.002725] found SMP MP-table at [mem 0x000f5450-0x000f545f]340client # [ 0.005378] node 0: [mem 0x0000000000001000-0x000000000009efff]341worker1 # [ 0.002867] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )342lb # [ 0.002738] Using GB pages for direct mapping343client # [ 0.005379] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]344lb # [ 0.002822] RAMDISK: [mem 0x5e36e000-0x5ffcffff]345worker2 # [ 0.002973] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)346worker1 # [ 0.002871] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)347lb # [ 0.002830] ACPI: Early table checksum verification disabled348client # [ 0.005380] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]349worker2 # [ 0.002975] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)350worker1 # [ 0.002876] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)351lb # [ 0.002833] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )352client # [ 0.005399] On node 0, zone DMA: 1 pages in unavailable ranges353worker2 # [ 0.002976] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)354client # [ 0.005666] On node 0, zone DMA: 97 pages in unavailable ranges355worker1 # [ 0.002883] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)356lb # [ 0.002837] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)357worker2 # [ 0.002978] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]358client # [ 0.032687] On node 0, zone DMA32: 40 pages in unavailable ranges359worker1 # [ 0.002886] ACPI: FACS 0x000000005FFE0000 000040360client # [ 0.033147] ACPI: PM-Timer IO Port: 0x608361lb # [ 0.002842] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)362worker2 # [ 0.002979] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]363worker1 # [ 0.002887] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)364client # [ 0.033158] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])365lb # [ 0.002849] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)366worker2 # [ 0.002979] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]367client # [ 0.033182] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23368worker1 # [ 0.002889] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)369lb # [ 0.002851] ACPI: FACS 0x000000005FFE0000 000040370worker2 # [ 0.002980] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]371client # [ 0.033185] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)372worker1 # [ 0.002890] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)373lb # [ 0.002852] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)374worker2 # [ 0.002980] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]375client # [ 0.033186] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)376worker1 # [ 0.002892] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)377worker2 # [ 0.002981] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]378lb # [ 0.002854] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)379client # [ 0.033187] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)380worker2 # [ 0.002981] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]381worker1 # [ 0.002893] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]382client # [ 0.033189] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)383lb # [ 0.002855] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)384worker2 # [ 0.003215] No NUMA configuration found385worker1 # [ 0.002894] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]386client # [ 0.033189] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)387lb # [ 0.002857] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)388worker2 # [ 0.003216] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]389worker1 # [ 0.002895] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]390client # [ 0.033192] ACPI: Using ACPI (MADT) for SMP configuration information391lb # [ 0.002858] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]392worker2 # [ 0.003219] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]393worker2 # [ 0.005622] Zone ranges:394client # [ 0.033193] ACPI: HPET id: 0x8086a201 base: 0xfed00000395worker1 # [ 0.002895] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]396client # [ 0.033196] TSC deadline timer available397lb # [ 0.002859] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]398worker2 # [ 0.005622] DMA [mem 0x0000000000001000-0x0000000000ffffff]399client # [ 0.033200] CPU topo: Max. logical packages: 1400worker1 # [ 0.002896] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]401lb # [ 0.002860] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]402worker2 # [ 0.005624] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]403client # [ 0.033201] CPU topo: Max. logical dies: 1404worker2 # [ 0.005625] Normal empty405worker1 # [ 0.002896] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]406worker2 # [ 0.005626] Device empty407client # [ 0.033202] CPU topo: Max. dies per package: 1408lb # [ 0.002860] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]409worker2 # [ 0.005627] Movable zone start for each node410client # [ 0.033205] CPU topo: Max. threads per core: 1411worker1 # [ 0.002897] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]412worker2 # [ 0.005628] Early memory node ranges413lb # [ 0.002861] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]414worker1 # [ 0.003134] No NUMA configuration found415client # [ 0.033205] CPU topo: Num. cores per package: 1416worker2 # [ 0.005628] node 0: [mem 0x0000000000001000-0x000000000009efff]417client # [ 0.033206] CPU topo: Num. threads per package: 1418lb # [ 0.002861] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]419worker1 # [ 0.003134] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]420client # [ 0.033206] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs421worker2 # [ 0.005629] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]422worker1 # [ 0.003137] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]423lb # [ 0.002862] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]424worker1 # [ 0.005529] Zone ranges:425lb # [ 0.003083] No NUMA configuration found426worker2 # [ 0.005630] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]427client # [ 0.033225] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()428worker1 # [ 0.005530] DMA [mem 0x0000000000001000-0x0000000000ffffff]429worker2 # [ 0.005650] On node 0, zone DMA: 1 pages in unavailable ranges430lb # [ 0.003084] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]431client # [ 0.033254] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]432worker1 # [ 0.005531] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]433lb # [ 0.003086] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]434worker1 # [ 0.005533] Normal empty435worker2 # [ 0.005929] On node 0, zone DMA: 97 pages in unavailable ranges436lb # [ 0.006374] Zone ranges:437worker1 # [ 0.005534] Device empty438client # [ 0.033255] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]439worker2 # [ 0.038385] On node 0, zone DMA32: 40 pages in unavailable ranges440worker1 # [ 0.005534] Movable zone start for each node441lb # [ 0.006375] DMA [mem 0x0000000000001000-0x0000000000ffffff]442client # [ 0.033257] [mem 0x60000000-0xafffffff] available for PCI devices443worker2 # [ 0.038862] ACPI: PM-Timer IO Port: 0x608444worker1 # [ 0.005535] Early memory node ranges445lb # [ 0.006377] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]446client # [ 0.033258] Booting paravirtualized kernel on KVM447worker2 # [ 0.038875] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])448worker1 # [ 0.005535] node 0: [mem 0x0000000000001000-0x000000000009efff]449lb # [ 0.006378] Normal empty450lb # [ 0.006379] Device empty451worker2 # [ 0.038905] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23452worker1 # [ 0.005536] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]453client # [ 0.033261] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns454lb # [ 0.006380] Movable zone start for each node455lb # [ 0.006380] Early memory node ranges456worker2 # [ 0.038908] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)457worker1 # [ 0.005538] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]458client # [ 0.037735] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1459lb # [ 0.006381] node 0: [mem 0x0000000000001000-0x000000000009efff]460worker2 # [ 0.038910] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)461worker1 # [ 0.005561] On node 0, zone DMA: 1 pages in unavailable ranges462client # [ 0.039937] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152463lb # [ 0.006382] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]464worker2 # [ 0.038911] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)465worker1 # [ 0.005843] On node 0, zone DMA: 97 pages in unavailable ranges466client # [ 0.039986] kvm-guest: PV spinlocks disabled, single CPU467lb # [ 0.006383] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]468worker1 # [ 0.035562] On node 0, zone DMA32: 40 pages in unavailable ranges469worker2 # [ 0.038912] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)470worker1 # [ 0.036047] ACPI: PM-Timer IO Port: 0x608471lb # [ 0.006410] On node 0, zone DMA: 1 pages in unavailable ranges472worker2 # [ 0.038913] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)473worker1 # [ 0.036062] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])474lb # [ 0.006769] On node 0, zone DMA: 97 pages in unavailable ranges475worker2 # [ 0.038915] ACPI: Using ACPI (MADT) for SMP configuration information476worker1 # [ 0.036094] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23477lb # [ 0.038465] On node 0, zone DMA32: 40 pages in unavailable ranges478worker2 # [ 0.038917] ACPI: HPET id: 0x8086a201 base: 0xfed00000479lb # [ 0.038935] ACPI: PM-Timer IO Port: 0x608480worker1 # [ 0.036097] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)481worker2 # [ 0.038920] TSC deadline timer available482client # [ 0.039988] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/dgyp9fws2n1q2rg73k4nrsyrfqvcsmf6-nixos-system-client-test/init regInfo=/nix/store/2jmcw81c7phijdhwbkm9pn7gpcs89hq7-closure-info/registration console=ttyS0,115200n8 console=tty0483lb # [ 0.038948] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])484worker2 # [ 0.038924] CPU topo: Max. logical packages: 1485worker1 # [ 0.036099] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)486worker2 # [ 0.038925] CPU topo: Max. logical dies: 1487lb # [ 0.038973] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23488worker1 # [ 0.036100] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)489worker2 # [ 0.038926] CPU topo: Max. dies per package: 1490client # [ 0.040080] Unknown kernel command line parameters "regInfo=/nix/store/2jmcw81c7phijdhwbkm9pn7gpcs89hq7-closure-info/registration", will be passed to user space.491lb # [ 0.038977] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)492worker2 # [ 0.038929] CPU topo: Max. threads per core: 1493worker1 # [ 0.036101] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)494client # [ 0.040093] random: crng init done495worker2 # [ 0.038929] CPU topo: Num. cores per package: 1496lb # [ 0.038978] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)497worker1 # [ 0.036102] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)498client # [ 0.040093] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes499worker2 # [ 0.038930] CPU topo: Num. threads per package: 1500lb # [ 0.038979] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)501worker1 # [ 0.036104] ACPI: Using ACPI (MADT) for SMP configuration information502worker2 # [ 0.038930] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs503client # [ 0.042253] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)504lb # [ 0.038981] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)505worker1 # [ 0.036105] ACPI: HPET id: 0x8086a201 base: 0xfed00000506worker2 # [ 0.038950] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()507worker1 # [ 0.036110] TSC deadline timer available508lb # [ 0.038981] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)509client # [ 0.043332] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)510worker1 # [ 0.036114] CPU topo: Max. logical packages: 1511client # [ 0.043363] Fallback order for Node 0: 0512worker2 # [ 0.038986] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]513lb # [ 0.038984] ACPI: Using ACPI (MADT) for SMP configuration information514worker1 # [ 0.036114] CPU topo: Max. logical dies: 1515client # [ 0.043365] Built 1 zonelists, mobility grouping on. Total pages: 393078516lb # [ 0.038985] ACPI: HPET id: 0x8086a201 base: 0xfed00000517worker1 # [ 0.036115] CPU topo: Max. dies per package: 1518worker2 # [ 0.038987] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]519client # [ 0.043367] Policy zone: DMA32520lb # [ 0.038989] TSC deadline timer available521worker1 # [ 0.036118] CPU topo: Max. threads per core: 1522worker2 # [ 0.038989] [mem 0x60000000-0xafffffff] available for PCI devices523lb # [ 0.038994] CPU topo: Max. logical packages: 1524client # [ 0.046172] mem auto-init: stack:all(zero), heap alloc:on, heap free:off525worker1 # [ 0.036119] CPU topo: Num. cores per package: 1526worker2 # [ 0.038990] Booting paravirtualized kernel on KVM527lb # [ 0.038994] CPU topo: Max. logical dies: 1528client # [ 0.049270] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1529worker1 # [ 0.036119] CPU topo: Num. threads per package: 1530lb # [ 0.038995] CPU topo: Max. dies per package: 1531client # [ 0.052642] allocated 3145728 bytes of page_ext532worker1 # [ 0.036119] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs533lb # [ 0.038998] CPU topo: Max. threads per core: 1534worker2 # [ 0.038993] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns535client # [ 0.062416] ftrace: allocating 48787 entries in 192 pages536lb # [ 0.038999] CPU topo: Num. cores per package: 1537worker1 # [ 0.036143] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()538client # [ 0.062418] ftrace: allocated 192 pages with 2 groups539worker2 # [ 0.043504] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1540lb # [ 0.038999] CPU topo: Num. threads per package: 1541client # [ 0.063252] Dynamic Preempt: lazy542worker1 # [ 0.036179] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]543worker2 # [ 0.045811] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152544lb # [ 0.038999] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs545client # [ 0.063412] rcu: Preemptible hierarchical RCU implementation.546worker1 # [ 0.036180] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]547worker2 # [ 0.045856] kvm-guest: PV spinlocks disabled, single CPU548client # [ 0.063413] rcu: RCU event tracing is enabled.549lb # [ 0.039022] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()550worker1 # [ 0.036182] [mem 0x60000000-0xafffffff] available for PCI devices551client # [ 0.063413] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.552lb # [ 0.039058] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]553worker1 # [ 0.036183] Booting paravirtualized kernel on KVM554client # [ 0.063415] Trampoline variant of Tasks RCU enabled.555client # [ 0.063415] Rude variant of Tasks RCU enabled.556lb # [ 0.039060] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]557client # [ 0.063415] Tracing variant of Tasks RCU enabled.558worker1 # [ 0.036187] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns559lb # [ 0.039061] [mem 0x60000000-0xafffffff] available for PCI devices560lb # [ 0.039063] Booting paravirtualized kernel on KVM561worker1 # [ 0.040682] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1562client # [ 0.063416] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.563worker2 # [ 0.045857] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/gcqx0ly4a11w34z8kprj4p7mv3aciczk-nixos-system-worker2-test/init regInfo=/nix/store/744ap97ay03z4ngf1q9ax96r0w4xa55l-closure-info/registration console=ttyS0,115200n8 console=tty0564worker1 # [ 0.042987] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152565client # [ 0.063417] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1566lb # [ 0.039067] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns567worker1 # [ 0.043034] kvm-guest: PV spinlocks disabled, single CPU568client # [ 0.063432] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.569worker2 # [ 0.045955] Unknown kernel command line parameters "regInfo=/nix/store/744ap97ay03z4ngf1q9ax96r0w4xa55l-closure-info/registration", will be passed to user space.570lb # [ 0.043544] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1571worker2 # [ 0.045967] random: crng init done572lb # [ 0.045831] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152573client # [ 0.063433] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.574worker2 # [ 0.045968] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes575lb # [ 0.045873] kvm-guest: PV spinlocks disabled, single CPU576client # [ 0.063434] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.577worker2 # [ 0.048231] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)578client # [ 0.067816] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16579worker2 # [ 0.049660] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)580worker1 # [ 0.043036] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qdm894g7fk24niwl4p4m0ddcdy1ggpcl-nixos-system-worker1-test/init regInfo=/nix/store/ydgn654pqammmz3kxba0bglp9b3f3vq8-closure-info/registration console=ttyS0,115200n8 console=tty0581client # [ 0.068094] rcu: srcu_init: Setting srcu_struct sizes based on contention.582worker2 # [ 0.049697] Fallback order for Node 0: 0583worker2 # [ 0.049701] Built 1 zonelists, mobility grouping on. Total pages: 393078584client # [ 0.068100] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns585worker2 # [ 0.049702] Policy zone: DMA32586worker1 # [ 0.043137] Unknown kernel command line parameters "regInfo=/nix/store/ydgn654pqammmz3kxba0bglp9b3f3vq8-closure-info/registration", will be passed to user space.587lb # [ 0.045874] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qfr7fghmwm537hb3f4f5is3yb8phm6jx-nixos-system-lb-test/init regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration console=ttyS0,115200n8 console=tty0588worker1 # [ 0.043149] random: crng init done589worker2 # [ 0.052340] mem auto-init: stack:all(zero), heap alloc:on, heap free:off590client # [ 0.068202] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)591worker1 # [ 0.043150] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes592client # [ 0.071729] Console: colour VGA+ 80x25593worker2 # [ 0.055808] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1594worker2 # [ 0.059340] allocated 3145728 bytes of page_ext595lb # [ 0.045973] Unknown kernel command line parameters "regInfo=/nix/store/s1480xaqc00gh7hz31b4z2fwjpnxi3sa-closure-info/registration", will be passed to user space.596client # [ 0.071732] printk: legacy console [tty0] enabled597worker1 # [ 0.045395] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)598lb # [ 0.045986] random: crng init done599client # [ 0.101249] printk: legacy console [ttyS0] enabled600worker2 # [ 0.069766] ftrace: allocating 48787 entries in 192 pages601worker1 # [ 0.046801] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)602worker2 # [ 0.069768] ftrace: allocated 192 pages with 2 groups603lb # [ 0.045987] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes604client # [ 0.196601] ACPI: Core revision 20250807605worker1 # [ 0.046835] Fallback order for Node 0: 0606worker2 # [ 0.070615] Dynamic Preempt: lazy607lb # [ 0.048223] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)608worker2 # [ 0.070757] rcu: Preemptible hierarchical RCU implementation.609worker1 # [ 0.046838] Built 1 zonelists, mobility grouping on. Total pages: 393078610worker1 # [ 0.046840] Policy zone: DMA32611worker2 # [ 0.070758] rcu: RCU event tracing is enabled.612client # [ 0.197409] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns613lb # [ 0.049345] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)614worker1 # [ 0.049458] mem auto-init: stack:all(zero), heap alloc:on, heap free:off615worker2 # [ 0.070759] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.616lb # [ 0.049377] Fallback order for Node 0: 0617client # [ 0.198876] APIC: Switch to symmetric I/O mode setup618worker2 # [ 0.070760] Trampoline variant of Tasks RCU enabled.619worker1 # [ 0.053037] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1620lb # [ 0.049381] Built 1 zonelists, mobility grouping on. Total pages: 393078621client # [ 0.199802] x2apic enabled622worker2 # [ 0.070761] Rude variant of Tasks RCU enabled.623worker1 # [ 0.056611] allocated 3145728 bytes of page_ext624lb # [ 0.049382] Policy zone: DMA32625worker2 # [ 0.070761] Tracing variant of Tasks RCU enabled.626worker1 # [ 0.067004] ftrace: allocating 48787 entries in 192 pages627client # [ 0.200530] APIC: Switched APIC routing to: physical x2apic628lb # [ 0.052338] mem auto-init: stack:all(zero), heap alloc:on, heap free:off629worker1 # [ 0.067007] ftrace: allocated 192 pages with 2 groups630worker2 # [ 0.070762] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.631lb # [ 0.055933] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1632worker1 # [ 0.067911] Dynamic Preempt: lazy633lb # [ 0.059431] allocated 3145728 bytes of page_ext634worker2 # [ 0.070763] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1635worker1 # [ 0.068095] rcu: Preemptible hierarchical RCU implementation.636client # [ 0.202289] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1637lb # [ 0.069347] ftrace: allocating 48787 entries in 192 pages638worker1 # [ 0.068096] rcu: RCU event tracing is enabled.639lb # [ 0.069349] ftrace: allocated 192 pages with 2 groups640worker2 # [ 0.070784] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.641worker1 # [ 0.068097] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.642lb # [ 0.070215] Dynamic Preempt: lazy643worker1 # [ 0.068098] Trampoline variant of Tasks RCU enabled.644client # [ 0.203229] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns645worker2 # [ 0.070786] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.646lb # [ 0.070384] rcu: Preemptible hierarchical RCU implementation.647worker1 # [ 0.068099] Rude variant of Tasks RCU enabled.648lb # [ 0.070385] rcu: RCU event tracing is enabled.649worker1 # [ 0.068099] Tracing variant of Tasks RCU enabled.650worker2 # [ 0.070787] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.651client # [ 0.204796] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)652lb # [ 0.070386] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.653worker2 # [ 0.075601] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16654worker1 # [ 0.068100] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.655lb # [ 0.070387] Trampoline variant of Tasks RCU enabled.656client # [ 0.206879] x86/cpu: User Mode Instruction Prevention (UMIP) activated657lb # [ 0.070388] Rude variant of Tasks RCU enabled.658worker2 # [ 0.075890] rcu: srcu_init: Setting srcu_struct sizes based on contention.659worker1 # [ 0.068101] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1660lb # [ 0.070388] Tracing variant of Tasks RCU enabled.661client # [ 0.207970] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127662worker1 # [ 0.068123] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.663worker2 # [ 0.075897] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns664lb # [ 0.070389] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.665client # [ 0.208793] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0666lb # [ 0.070389] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1667worker1 # [ 0.068124] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.668worker2 # [ 0.076004] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)669worker2 # [ 0.079576] Console: colour VGA+ 80x25670lb # [ 0.070406] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.671worker1 # [ 0.068125] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.672worker2 # [ 0.079579] printk: legacy console [tty0] enabled673client # [ 0.209797] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto674worker2 # [ 0.109244] printk: legacy console [ttyS0] enabled675worker1 # [ 0.072861] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16676lb # [ 0.070408] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.677worker2 # [ 0.205034] ACPI: Core revision 20250807678client # [ 0.210793] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl679worker1 # [ 0.073155] rcu: srcu_init: Setting srcu_struct sizes based on contention.680lb # [ 0.070409] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.681client # [ 0.212793] Transient Scheduler Attacks: Vulnerable: No microcode682worker1 # [ 0.073164] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns683lb # [ 0.074924] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16684worker2 # [ 0.205892] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns685client # [ 0.213793] Spectre V2 : Mitigation: Enhanced / Automatic IBRS686lb # [ 0.075216] rcu: srcu_init: Setting srcu_struct sizes based on contention.687worker1 # [ 0.073279] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)688worker2 # [ 0.207381] APIC: Switch to symmetric I/O mode setup689client # [ 0.214793] Speculative Return Stack Overflow: Mitigation: Safe RET690worker1 # [ 0.076849] Console: colour VGA+ 80x25691worker2 # [ 0.208319] x2apic enabled692lb # [ 0.075221] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns693worker1 # [ 0.076852] printk: legacy console [tty0] enabled694worker1 # [ 0.106352] printk: legacy console [ttyS0] enabled695worker2 # [ 0.209059] APIC: Switched APIC routing to: physical x2apic696lb # [ 0.075336] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)697worker1 # [ 0.201961] ACPI: Core revision 20250807698lb # [ 0.078852] Console: colour VGA+ 80x25699lb # [ 0.078855] printk: legacy console [tty0] enabled700worker2 # [ 0.210848] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1701lb # [ 0.108309] printk: legacy console [ttyS0] enabled702worker1 # [ 0.202890] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns703client # [ 0.215793] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization704lb # [ 0.204024] ACPI: Core revision 20250807705worker1 # [ 0.204386] APIC: Switch to symmetric I/O mode setup706client # [ 0.217800] Spectre V2 : Enabling IBPB for BPF707worker2 # [ 0.211808] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns708worker1 # [ 0.205369] x2apic enabled709lb # [ 0.204863] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns710client # [ 0.219306] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier711worker2 # [ 0.213415] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)712worker1 # [ 0.206154] APIC: Switched APIC routing to: physical x2apic713client # [ 0.219794] active return thunk: srso_alias_return_thunk714lb # [ 0.206400] APIC: Switch to symmetric I/O mode setup715worker2 # [ 0.214502] x86/cpu: User Mode Instruction Prevention (UMIP) activated716lb # [ 0.207391] x2apic enabled717client # [ 0.221417] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'718worker1 # [ 0.207955] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1719worker2 # [ 0.215562] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127720lb # [ 0.208174] APIC: Switched APIC routing to: physical x2apic721client # [ 0.222660] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'722worker2 # [ 0.216412] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0723client # [ 0.223505] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'724worker1 # [ 0.208936] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns725lb # [ 0.209974] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1726client # [ 0.224519] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'727worker2 # [ 0.217417] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto728worker1 # [ 0.210613] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)729client # [ 0.225524] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'730lb # [ 0.210978] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns731worker1 # [ 0.211699] x86/cpu: User Mode Instruction Prevention (UMIP) activated732worker2 # [ 0.218412] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl733client # [ 0.225793] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'734lb # [ 0.212663] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)735worker1 # [ 0.212743] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127736worker2 # [ 0.219412] Transient Scheduler Attacks: Vulnerable: No microcode737client # [ 0.226793] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'738worker2 # [ 0.220412] Spectre V2 : Mitigation: Enhanced / Automatic IBRS739worker1 # [ 0.214292] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0740lb # [ 0.213745] x86/cpu: User Mode Instruction Prevention (UMIP) activated741client # [ 0.228672] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256742worker2 # [ 0.221412] Speculative Return Stack Overflow: Mitigation: Safe RET743lb # [ 0.214828] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127744client # [ 0.229487] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64745worker1 # [ 0.214614] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto746lb # [ 0.216336] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0747client # [ 0.230492] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512748worker1 # [ 0.216610] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl749client # [ 0.230793] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024750lb # [ 0.216664] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto751worker1 # [ 0.217610] Transient Scheduler Attacks: Vulnerable: No microcode752client # [ 0.231793] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8753worker1 # [ 0.219330] Spectre V2 : Mitigation: Enhanced / Automatic IBRS754worker2 # [ 0.222412] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization755lb # [ 0.218660] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl756client # [ 0.232793] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.757worker2 # [ 0.224420] Spectre V2 : Enabling IBPB for BPF758worker1 # [ 0.219610] Speculative Return Stack Overflow: Mitigation: Safe RET759lb # [ 0.219660] Transient Scheduler Attacks: Vulnerable: No microcode760worker2 # [ 0.225413] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier761lb # [ 0.221393] Spectre V2 : Mitigation: Enhanced / Automatic IBRS762worker2 # [ 0.226413] active return thunk: srso_alias_return_thunk763lb # [ 0.222401] Speculative Return Stack Overflow: Mitigation: Safe RET764worker2 # [ 0.227432] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'765worker1 # [ 0.220610] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization766worker2 # [ 0.228412] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'767worker1 # [ 0.222618] Spectre V2 : Enabling IBPB for BPF768worker2 # [ 0.229412] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'769worker1 # [ 0.223611] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier770lb # [ 0.223451] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization771worker2 # [ 0.231185] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'772worker1 # [ 0.224611] active return thunk: srso_alias_return_thunk773lb # [ 0.224667] Spectre V2 : Enabling IBPB for BPF774worker2 # [ 0.232176] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'775worker1 # [ 0.225630] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'776lb # [ 0.226189] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier777worker2 # [ 0.233187] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'778worker1 # [ 0.226610] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'779lb # [ 0.226660] active return thunk: srso_alias_return_thunk780worker2 # [ 0.234205] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'781worker1 # [ 0.228346] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'782lb # [ 0.228307] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'783worker2 # [ 0.235412] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256784worker1 # [ 0.229350] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'785lb # [ 0.228659] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'786worker2 # [ 0.236412] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64787worker1 # [ 0.230361] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'788worker2 # [ 0.237412] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512789lb # [ 0.229659] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'790worker1 # [ 0.231351] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'791worker2 # [ 0.238412] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024792lb # [ 0.230660] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'793worker1 # [ 0.232387] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'794worker2 # [ 0.239412] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8795lb # [ 0.231659] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'796worker1 # [ 0.233610] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256797lb # [ 0.232660] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'798worker2 # [ 0.240358] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.799worker1 # [ 0.234610] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64800lb # [ 0.233660] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'801worker1 # [ 0.235610] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512802lb # [ 0.234660] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256803worker1 # [ 0.236610] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024804lb # [ 0.235660] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64805worker1 # [ 0.237610] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8806lb # [ 0.236659] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512807worker1 # [ 0.238562] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.808lb # [ 0.237659] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809lb # [ 0.238659] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810lb # [ 0.239659] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811client # [ 0.266871] Freeing SMP alternatives memory: 44K812client # [ 0.267638] pid_max: default: 32768 minimum: 301813client # [ 0.268518] LSM: initializing lsm=capability,landlock,yama,bpf,ima814client # [ 0.269684] landlock: Up and running.815client # [ 0.270220] Yama: becoming mindful.816client # [ 0.271009] LSM support for eBPF active817client # [ 0.271959] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818client # [ 0.273190] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819client # [ 0.276053] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820client # [ 0.277398] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821client # [ 0.277803] ... version: 2822client # [ 0.278566] ... bit width: 48823client # [ 0.278796] ... generic counters: 6824client # [ 0.279520] ... generic bitmap: 000000000000003f825client # [ 0.279795] ... fixed-purpose counters: 0826client # [ 0.280556] ... fixed-purpose bitmap: 0000000000000000827worker2 # [ 0.275198] Freeing SMP alternatives memory: 44K828worker2 # [ 0.275974] pid_max: default: 32768 minimum: 301829client # [ 0.280796] ... value mask: 0000ffffffffffff830client # [ 0.281713] ... max period: 00007fffffffffff831worker2 # [ 0.276509] LSM: initializing lsm=capability,landlock,yama,bpf,ima832worker2 # [ 0.277520] landlock: Up and running.833client # [ 0.282505] ... global_ctrl mask: 000000000000003f834worker2 # [ 0.278413] Yama: becoming mindful.835client # [ 0.282893] signal: max sigframe size: 3376836worker2 # [ 0.279484] LSM support for eBPF active837client # [ 0.283731] rcu: Hierarchical SRCU implementation.838worker1 # [ 0.273572] Freeing SMP alternatives memory: 44K839client # [ 0.284389] rcu: Max phase no-delay instances is 400.840worker2 # [ 0.280248] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)841worker1 # [ 0.274183] pid_max: default: 32768 minimum: 301842worker2 # [ 0.281450] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)843worker1 # [ 0.275260] LSM: initializing lsm=capability,landlock,yama,bpf,ima844lb # [ 0.274658] Freeing SMP alternatives memory: 44K845worker1 # [ 0.275733] landlock: Up and running.846lb # [ 0.275231] pid_max: default: 32768 minimum: 301847worker1 # [ 0.277052] Yama: becoming mindful.848lb # [ 0.276320] LSM: initializing lsm=capability,landlock,yama,bpf,ima849worker1 # [ 0.277823] LSM support for eBPF active850worker2 # [ 0.283873] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)851lb # [ 0.276766] landlock: Up and running.852lb # [ 0.277660] Yama: becoming mindful.853worker1 # [ 0.278666] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)854client # [ 0.289357] smp: Bringing up secondary CPUs ...855worker2 # [ 0.284987] Performance Events: Fam17h+ core perfctr, AMD PMU driver.856lb # [ 0.278874] LSM support for eBPF active857client # [ 0.289810] smp: Brought up 1 node, 1 CPU858worker2 # [ 0.285421] ... version: 2859worker1 # [ 0.279651] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)860client # [ 0.290444] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)861worker2 # [ 0.286129] ... bit width: 48862lb # [ 0.279695] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)863worker2 # [ 0.286414] ... generic counters: 6864lb # [ 0.280699] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)865worker2 # [ 0.287145] ... generic bitmap: 000000000000003f866worker1 # [ 0.282264] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)867client # [ 0.291004] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)868worker2 # [ 0.287415] ... fixed-purpose counters: 0869client # [ 0.292051] devtmpfs: initialized870worker2 # [ 0.288117] ... fixed-purpose bitmap: 0000000000000000871worker1 # [ 0.283204] Performance Events: Fam17h+ core perfctr, AMD PMU driver.872client # [ 0.292779] x86/mm: Memory block size: 128MB873worker2 # [ 0.288415] ... value mask: 0000ffffffffffff874lb # [ 0.283517] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)875worker1 # [ 0.283615] ... version: 2876worker2 # [ 0.289270] ... max period: 00007fffffffffff877worker1 # [ 0.284329] ... bit width: 48878worker1 # [ 0.284613] ... generic counters: 6879worker2 # [ 0.290087] ... global_ctrl mask: 000000000000003f880lb # [ 0.284203] Performance Events: Fam17h+ core perfctr, AMD PMU driver.881client # [ 0.294332] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)882lb # [ 0.284664] ... version: 2883worker1 # [ 0.285291] ... generic bitmap: 000000000000003f884worker2 # [ 0.290528] signal: max sigframe size: 3376885lb # [ 0.285339] ... bit width: 48886worker1 # [ 0.285623] ... fixed-purpose counters: 0887worker2 # [ 0.291336] rcu: Hierarchical SRCU implementation.888client # [ 0.294821] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).889lb # [ 0.285662] ... generic counters: 6890worker1 # [ 0.286329] ... fixed-purpose bitmap: 0000000000000000891worker2 # [ 0.292001] rcu: Max phase no-delay instances is 400.892client # [ 0.295886] pinctrl core: initialized pinctrl subsystem893lb # [ 0.286337] ... generic bitmap: 000000000000003f894worker1 # [ 0.286613] ... value mask: 0000ffffffffffff895lb # [ 0.286720] ... fixed-purpose counters: 0896client # [ 0.297002] PM: RTC time: 15:03:11, date: 2026-09-22897worker1 # [ 0.287521] ... max period: 00007fffffffffff898lb # [ 0.287438] ... fixed-purpose bitmap: 0000000000000000899worker1 # [ 0.288293] ... global_ctrl mask: 000000000000003f900lb # [ 0.287663] ... value mask: 0000ffffffffffff901worker1 # [ 0.288737] signal: max sigframe size: 3376902lb # [ 0.288544] ... max period: 00007fffffffffff903worker1 # [ 0.289503] rcu: Hierarchical SRCU implementation.904lb # [ 0.289333] ... global_ctrl mask: 000000000000003f905client # [ 0.300568] NET: Registered PF_NETLINK/PF_ROUTE protocol family906worker1 # [ 0.290192] rcu: Max phase no-delay instances is 400.907lb # [ 0.289778] signal: max sigframe size: 3376908worker2 # [ 0.297188] smp: Bringing up secondary CPUs ...909worker2 # [ 0.297428] smp: Brought up 1 node, 1 CPU910lb # [ 0.290532] rcu: Hierarchical SRCU implementation.911client # [ 0.302036] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations912lb # [ 0.291218] rcu: Max phase no-delay instances is 400.913worker2 # [ 0.298096] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)914client # [ 0.302818] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations915client # [ 0.304078] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations916worker2 # [ 0.298636] Memory: 1454580K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)917client # [ 0.304806] audit: initializing netlink subsys (disabled)918worker2 # [ 0.299736] devtmpfs: initialized919worker1 # [ 0.295243] smp: Bringing up secondary CPUs ...920worker1 # [ 0.295626] smp: Brought up 1 node, 1 CPU921worker2 # [ 0.300541] x86/mm: Memory block size: 128MB922client # [ 0.305916] thermal_sys: Registered thermal governor 'fair_share'923worker1 # [ 0.296276] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)924lb # [ 0.296262] smp: Bringing up secondary CPUs ...925client # [ 0.305918] thermal_sys: Registered thermal governor 'bang_bang'926lb # [ 0.296676] smp: Brought up 1 node, 1 CPU927client # [ 0.306797] thermal_sys: Registered thermal governor 'step_wise'928worker2 # [ 0.302335] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)929lb # [ 0.297324] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)930client # [ 0.307745] thermal_sys: Registered thermal governor 'user_space'931worker1 # [ 0.296836] Memory: 1454596K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)932worker2 # [ 0.303272] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).933worker1 # [ 0.297945] devtmpfs: initialized934client # [ 0.308512] audit: type=2000 audit(1790089392.028:1): state=initialized audit_enabled=0 res=1935worker2 # [ 0.304503] pinctrl core: initialized pinctrl subsystem936lb # [ 0.297872] Memory: 1454616K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)937worker1 # [ 0.298778] x86/mm: Memory block size: 128MB938client # [ 0.309799] thermal_sys: Registered thermal governor 'power_allocator'939worker2 # [ 0.305591] PM: RTC time: 15:03:11, date: 2026-09-22940lb # [ 0.298933] devtmpfs: initialized941client # [ 0.309813] cpuidle: using governor menu942lb # [ 0.299759] x86/mm: Memory block size: 128MB943worker1 # [ 0.300518] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)944client # [ 0.312023] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5945worker1 # [ 0.301517] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).946lb # [ 0.301522] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)947client # [ 0.313042] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]948worker2 # [ 0.309362] NET: Registered PF_NETLINK/PF_ROUTE protocol family949worker1 # [ 0.302698] pinctrl core: initialized pinctrl subsystem950client # [ 0.313799] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry951lb # [ 0.302546] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).952worker1 # [ 0.303791] PM: RTC time: 15:03:11, date: 2026-09-22953client # [ 0.314808] PCI: Using configuration type 1 for base access954worker2 # [ 0.310658] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations955lb # [ 0.303746] pinctrl core: initialized pinctrl subsystem956worker2 # [ 0.311439] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations957lb # [ 0.304859] PM: RTC time: 15:03:11, date: 2026-09-22958client # [ 0.315899] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.959worker2 # [ 0.312701] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations960worker1 # [ 0.307537] NET: Registered PF_NETLINK/PF_ROUTE protocol family961worker2 # [ 0.313426] audit: initializing netlink subsys (disabled)962worker2 # [ 0.314602] thermal_sys: Registered thermal governor 'fair_share'963worker1 # [ 0.308873] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations964lb # [ 0.308550] NET: Registered PF_NETLINK/PF_ROUTE protocol family965worker2 # [ 0.314604] thermal_sys: Registered thermal governor 'bang_bang'966worker1 # [ 0.309640] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations967lb # [ 0.309935] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations968client # [ 0.321037] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages969worker2 # [ 0.315415] thermal_sys: Registered thermal governor 'step_wise'970worker2 # [ 0.316378] thermal_sys: Registered thermal governor 'user_space'971worker1 # [ 0.310900] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations972client # [ 0.321797] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page973lb # [ 0.310684] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations974worker1 # [ 0.311623] audit: initializing netlink subsys (disabled)975worker2 # [ 0.317113] audit: type=2000 audit(1790089392.042:1): state=initialized audit_enabled=0 res=1976lb # [ 0.311961] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations977worker1 # [ 0.312753] thermal_sys: Registered thermal governor 'fair_share'978worker2 # [ 0.318417] thermal_sys: Registered thermal governor 'power_allocator'979lb # [ 0.312678] audit: initializing netlink subsys (disabled)980worker2 # [ 0.318430] cpuidle: using governor menu981worker1 # [ 0.312755] thermal_sys: Registered thermal governor 'bang_bang'982worker1 # [ 0.313614] thermal_sys: Registered thermal governor 'step_wise'983lb # [ 0.313664] audit: type=2000 audit(1790089392.050:1): state=initialized audit_enabled=0 res=1984worker2 # [ 0.320650] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5985worker1 # [ 0.314569] thermal_sys: Registered thermal governor 'user_space'986client # [ 0.326797] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages987lb # [ 0.314852] thermal_sys: Registered thermal governor 'fair_share'988client # [ 0.327795] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page989lb # [ 0.314855] thermal_sys: Registered thermal governor 'bang_bang'990worker2 # [ 0.321635] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]991worker1 # [ 0.315365] audit: type=2000 audit(1790089392.048:1): state=initialized audit_enabled=0 res=1992lb # [ 0.315663] thermal_sys: Registered thermal governor 'step_wise'993worker2 # [ 0.322418] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry994worker1 # [ 0.316616] thermal_sys: Registered thermal governor 'power_allocator'995lb # [ 0.316596] thermal_sys: Registered thermal governor 'user_space'996worker1 # [ 0.316642] cpuidle: using governor menu997worker2 # [ 0.323435] PCI: Using configuration type 1 for base access998lb # [ 0.317395] thermal_sys: Registered thermal governor 'power_allocator'999worker1 # [ 0.318869] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51000lb # [ 0.317686] cpuidle: using governor menu1001worker2 # [ 0.324529] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1002worker1 # [ 0.319891] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1003lb # [ 0.319887] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51004worker1 # [ 0.320616] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1005lb # [ 0.320918] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1006worker1 # [ 0.321625] PCI: Using configuration type 1 for base access1007lb # [ 0.321666] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1008worker2 # [ 0.329495] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1009worker1 # [ 0.322747] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1010lb # [ 0.322675] PCI: Using configuration type 1 for base access1011worker2 # [ 0.330415] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1012client # [ 0.338181] ACPI: Added _OSI(Module Device)1013lb # [ 0.323747] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1014client # [ 0.338797] ACPI: Added _OSI(Processor Device)1015client # [ 0.339534] ACPI: Added _OSI(Processor Aggregator Device)1016worker2 # [ 0.335416] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1017worker2 # [ 0.336415] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1018worker1 # [ 0.329889] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1019worker1 # [ 0.330614] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1020lb # [ 0.330688] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1021lb # [ 0.331662] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1022client # [ 0.345733] ACPI: 1 ACPI AML tables successfully acquired and loaded1023worker1 # [ 0.335614] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1024worker1 # [ 0.336613] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1025lb # [ 0.336663] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1026lb # [ 0.337663] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1027client # [ 0.352202] ACPI: Interpreter enabled1028client # [ 0.352811] ACPI: PM: (supports S0 S3 S4 S5)1029worker2 # [ 0.346767] ACPI: Added _OSI(Module Device)1030client # [ 0.353549] ACPI: Using IOAPIC for interrupt routing1031worker2 # [ 0.347416] ACPI: Added _OSI(Processor Device)1032worker2 # [ 0.348176] ACPI: Added _OSI(Processor Aggregator Device)1033client # [ 0.353871] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1034client # [ 0.354796] PCI: Using E820 reservations for host bridge windows1035worker1 # [ 0.345214] ACPI: Added _OSI(Module Device)1036client # [ 0.357947] ACPI: Enabled 2 GPEs in block 00 to 3F1037worker1 # [ 0.345614] ACPI: Added _OSI(Processor Device)1038worker1 # [ 0.346356] ACPI: Added _OSI(Processor Aggregator Device)1039lb # [ 0.348909] ACPI: Added _OSI(Module Device)1040lb # [ 0.349613] ACPI: Added _OSI(Processor Device)1041worker2 # [ 0.356553] ACPI: 1 ACPI AML tables successfully acquired and loaded1042lb # [ 0.350202] ACPI: Added _OSI(Processor Aggregator Device)1043client # [ 0.366542] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1044worker1 # [ 0.353160] ACPI: 1 ACPI AML tables successfully acquired and loaded1045worker2 # [ 0.360836] ACPI: Interpreter enabled1046worker2 # [ 0.361431] ACPI: PM: (supports S0 S3 S4 S5)1047client # [ 0.366802] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1048worker2 # [ 0.362121] ACPI: Using IOAPIC for interrupt routing1049lb # [ 0.356038] ACPI: 1 ACPI AML tables successfully acquired and loaded1050client # [ 0.368446] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1051worker1 # [ 0.356517] ACPI: Interpreter enabled1052worker1 # [ 0.357071] ACPI: PM: (supports S0 S3 S4 S5)1053client # [ 0.368919] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1054worker1 # [ 0.357614] ACPI: Using IOAPIC for interrupt routing1055worker2 # [ 0.364476] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1056client # [ 0.370140] PCI host bridge to bus 0000:001057worker2 # [ 0.365415] PCI: Using E820 reservations for host bridge windows1058client # [ 0.370800] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1059lb # [ 0.360067] ACPI: Interpreter enabled1060worker1 # [ 0.358493] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1061lb # [ 0.360678] ACPI: PM: (supports S0 S3 S4 S5)1062client # [ 0.371796] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1063lb # [ 0.361377] ACPI: Using IOAPIC for interrupt routing1064client # [ 0.372795] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1065worker2 # [ 0.368511] ACPI: Enabled 2 GPEs in block 00 to 3F1066client # [ 0.373796] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1067client # [ 0.374796] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1068lb # [ 0.363725] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1069worker1 # [ 0.363620] PCI: Using E820 reservations for host bridge windows1070lb # [ 0.364663] PCI: Using E820 reservations for host bridge windows1071client # [ 0.375796] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1072worker1 # [ 0.364815] ACPI: Enabled 2 GPEs in block 00 to 3F1073client # [ 0.376796] pci_bus 0000:00: root bus resource [bus 00-ff]1074client # [ 0.377840] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1075lb # [ 0.367723] ACPI: Enabled 2 GPEs in block 00 to 3F1076client # [ 0.379252] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1077worker2 # [ 0.376213] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1078worker2 # [ 0.377213] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1079worker2 # [ 0.378493] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1080worker2 # [ 0.379546] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1081worker1 # [ 0.373596] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1082worker2 # [ 0.380758] PCI host bridge to bus 0000:001083lb # [ 0.375224] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1084worker2 # [ 0.381419] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1085lb # [ 0.375668] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1086worker2 # [ 0.382415] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1087worker1 # [ 0.375967] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1088lb # [ 0.376740] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1089client # [ 0.381861] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1090worker2 # [ 0.383414] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1091worker1 # [ 0.376692] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1092client # [ 0.382808] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1093lb # [ 0.377784] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1094worker2 # [ 0.384415] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1095worker1 # [ 0.377735] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1096lb # [ 0.379019] PCI host bridge to bus 0000:001097client # [ 0.383738] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1098worker2 # [ 0.385421] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1099worker1 # [ 0.378996] PCI host bridge to bus 0000:001100lb # [ 0.379666] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1101worker1 # [ 0.379617] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1102worker2 # [ 0.386415] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1103client # [ 0.385080] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1104lb # [ 0.380663] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1105worker1 # [ 0.380613] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1106worker2 # [ 0.387445] pci_bus 0000:00: root bus resource [bus 00-ff]1107lb # [ 0.381662] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1108worker1 # [ 0.381613] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1109worker2 # [ 0.388435] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1110client # [ 0.386701] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1111lb # [ 0.382663] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1112worker1 # [ 0.382612] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1113lb # [ 0.383662] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1114worker2 # [ 0.389917] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1115worker1 # [ 0.383613] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1116client # [ 0.388814] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1117lb # [ 0.384662] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1118worker1 # [ 0.384612] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1119client # [ 0.389707] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1120lb # [ 0.385663] pci_bus 0000:00: root bus resource [bus 00-ff]1121worker1 # [ 0.385613] pci_bus 0000:00: root bus resource [bus 00-ff]1122client # [ 0.390506] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1123lb # [ 0.386612] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1124worker1 # [ 0.386614] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1125client # [ 0.390802] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1126lb # [ 0.388061] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1127worker1 # [ 0.388106] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1128client # [ 0.392814] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1129worker2 # [ 0.392513] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1130client # [ 0.394837] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1131worker2 # [ 0.393427] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1132client # [ 0.395715] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1133client # [ 0.396594] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1134worker2 # [ 0.394362] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1135worker2 # [ 0.395588] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1136client # [ 0.398246] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1137lb # [ 0.390761] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1138lb # [ 0.391674] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1139worker2 # [ 0.397157] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1140worker1 # [ 0.390822] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1141lb # [ 0.392638] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1142worker1 # [ 0.391667] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1143lb # [ 0.394003] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1144client # [ 0.401774] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1145worker1 # [ 0.392635] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1146worker2 # [ 0.400456] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1147client # [ 0.402437] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1148worker1 # [ 0.393937] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1149worker2 # [ 0.401387] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1150lb # [ 0.395381] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1151client # [ 0.402818] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1152worker2 # [ 0.402189] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1153worker1 # [ 0.395358] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1154client # [ 0.404342] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1155worker2 # [ 0.403319] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1156lb # [ 0.397701] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1157lb # [ 0.398556] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1158worker2 # [ 0.404971] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1159client # [ 0.405848] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1160worker1 # [ 0.398633] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1161lb # [ 0.399433] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1162client # [ 0.406687] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1163worker1 # [ 0.399465] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1164lb # [ 0.400490] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1165worker1 # [ 0.400348] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1166client # [ 0.407503] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1167worker2 # [ 0.407453] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1168client # [ 0.407802] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1169lb # [ 0.402113] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1170worker1 # [ 0.400619] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1171worker2 # [ 0.408327] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1172worker2 # [ 0.409185] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1173client # [ 0.409335] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1174worker1 # [ 0.402166] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1175lb # [ 0.403700] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1176lb # [ 0.404543] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1177worker2 # [ 0.410852] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1178client # [ 0.410808] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1179lb # [ 0.405391] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1180worker1 # [ 0.404632] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1181client # [ 0.411818] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1182worker1 # [ 0.405466] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1183lb # [ 0.407080] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1184worker1 # [ 0.406338] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1185client # [ 0.413393] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1186worker2 # [ 0.414373] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1187worker1 # [ 0.407192] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1188worker2 # [ 0.415097] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1189client # [ 0.415327] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1190worker2 # [ 0.415437] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1191lb # [ 0.409705] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1192client # [ 0.415818] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1193lb # [ 0.410694] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1194worker2 # [ 0.416987] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1195lb # [ 0.411652] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1196worker1 # [ 0.409628] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1197client # [ 0.417372] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1198worker1 # [ 0.410499] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1199lb # [ 0.413085] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1200worker2 # [ 0.419435] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1201worker1 # [ 0.411357] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1202client # [ 0.418808] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1203worker2 # [ 0.420270] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1204client # [ 0.419773] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1205worker1 # [ 0.412187] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1206lb # [ 0.414670] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1207lb # [ 0.415498] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1208worker2 # [ 0.421171] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1209client # [ 0.421235] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1210worker2 # [ 0.421422] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1211lb # [ 0.416388] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1212worker1 # [ 0.414652] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1213lb # [ 0.416669] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1214client # [ 0.423306] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1215worker2 # [ 0.422953] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1216worker1 # [ 0.415526] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1217client # [ 0.423818] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1218lb # [ 0.418236] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1219worker1 # [ 0.416362] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1220worker2 # [ 0.424427] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1221client # [ 0.425322] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1222worker1 # [ 0.417619] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1223lb # [ 0.419578] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1224worker2 # [ 0.425397] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1225lb # [ 0.420415] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1226worker1 # [ 0.419185] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1227client # [ 0.427402] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1228worker2 # [ 0.426857] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1229lb # [ 0.421264] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1230client # [ 0.427803] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1231worker1 # [ 0.420626] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1232client # [ 0.428811] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1233worker1 # [ 0.421618] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1234lb # [ 0.422705] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1235worker2 # [ 0.428429] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1236lb # [ 0.423662] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1237worker2 # [ 0.430147] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1238client # [ 0.430380] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1239worker1 # [ 0.423213] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1240lb # [ 0.425238] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1241worker2 # [ 0.430953] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1242client # [ 0.431804] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1243worker1 # [ 0.425328] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1244client # [ 0.432707] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1245lb # [ 0.426434] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1246worker2 # [ 0.432462] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1247worker1 # [ 0.425636] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1248client # [ 0.433537] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1249lb # [ 0.426685] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1250worker2 # [ 0.433437] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1251worker1 # [ 0.427252] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1252client # [ 0.434384] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1253lb # [ 0.428233] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1254worker2 # [ 0.434985] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1255client # [ 0.435521] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1256lb # [ 0.429676] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1257worker1 # [ 0.428657] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1258lb # [ 0.430631] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1259client # [ 0.435991] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1260worker1 # [ 0.429661] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1261worker2 # [ 0.436429] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1262lb # [ 0.432254] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1263worker2 # [ 0.438196] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1264worker1 # [ 0.431151] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1265client # [ 0.438250] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1266client # [ 0.439003] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1267worker2 # [ 0.439804] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1268lb # [ 0.433707] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1269lb # [ 0.434595] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1270worker1 # [ 0.433289] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1271client # [ 0.440456] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1272worker1 # [ 0.433636] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1273lb # [ 0.435397] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1274worker2 # [ 0.442269] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1275worker2 # [ 0.443092] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1276worker1 # [ 0.435161] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1277lb # [ 0.436209] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1278worker2 # [ 0.443436] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1279worker1 # [ 0.437410] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1280lb # [ 0.437705] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1281worker2 # [ 0.444975] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1282worker1 # [ 0.438251] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1283lb # [ 0.438694] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1284worker1 # [ 0.438635] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1285lb # [ 0.439607] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1286worker2 # [ 0.446433] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1287lb # [ 0.441059] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1288worker2 # [ 0.447274] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1289worker1 # [ 0.440158] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1290worker2 # [ 0.448151] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1291lb # [ 0.442227] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1292worker1 # [ 0.441620] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1293lb # [ 0.442871] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1294worker2 # [ 0.449000] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1295client # [ 0.441022] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1296worker1 # [ 0.442443] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1297lb # [ 0.444198] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1298worker2 # [ 0.450150] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1299worker1 # [ 0.443341] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1300client # [ 0.443428] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1301lb # [ 0.444864] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1302worker1 # [ 0.444184] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1303client # [ 0.444065] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1304worker2 # [ 0.450623] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1305lb # [ 0.446452] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1306worker1 # [ 0.445360] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1307client # [ 0.445203] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1308worker2 # [ 0.452260] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1309worker1 # [ 0.445809] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1310client # [ 0.446039] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1311worker2 # [ 0.453248] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1312worker2 # [ 0.455248] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1313client # [ 0.447824] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1314worker1 # [ 0.448088] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1315client # [ 0.448684] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1316worker1 # [ 0.448808] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1317client # [ 0.449871] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1318worker1 # [ 0.450373] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1319client # [ 0.451355] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1320lb # [ 0.447510] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1321client # [ 0.455439] ACPI: PCI: Interrupt link LNKA configured for IRQ 101322lb # [ 0.449175] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1323client # [ 0.455903] ACPI: PCI: Interrupt link LNKB configured for IRQ 101324lb # [ 0.449942] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1325client # [ 0.456897] ACPI: PCI: Interrupt link LNKC configured for IRQ 111326client # [ 0.457895] ACPI: PCI: Interrupt link LNKD configured for IRQ 111327lb # [ 0.450951] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1328worker2 # [ 0.456286] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1329client # [ 0.458896] ACPI: PCI: Interrupt link LNKE configured for IRQ 101330lb # [ 0.451919] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1331client # [ 0.459891] ACPI: PCI: Interrupt link LNKF configured for IRQ 101332worker2 # [ 0.458193] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1333client # [ 0.460898] ACPI: PCI: Interrupt link LNKG configured for IRQ 111334lb # [ 0.453669] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1335worker1 # [ 0.450832] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1336worker2 # [ 0.458663] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1337client # [ 0.461902] ACPI: PCI: Interrupt link LNKH configured for IRQ 111338lb # [ 0.454477] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1339client # [ 0.462833] ACPI: PCI: Interrupt link GSIA configured for IRQ 161340worker1 # [ 0.453380] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1341worker2 # [ 0.459703] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1342lb # [ 0.455669] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1343client # [ 0.463809] ACPI: PCI: Interrupt link GSIB configured for IRQ 171344worker1 # [ 0.453945] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1345client # [ 0.464806] ACPI: PCI: Interrupt link GSIC configured for IRQ 181346worker2 # [ 0.460675] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1347lb # [ 0.457168] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1348client # [ 0.465753] ACPI: PCI: Interrupt link GSID configured for IRQ 191349worker1 # [ 0.454898] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1350client # [ 0.466577] ACPI: PCI: Interrupt link GSIE configured for IRQ 201351worker2 # [ 0.462443] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1352client # [ 0.467552] ACPI: PCI: Interrupt link GSIF configured for IRQ 211353worker2 # [ 0.463302] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1354worker1 # [ 0.455866] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1355client # [ 0.467822] ACPI: PCI: Interrupt link GSIG configured for IRQ 221356lb # [ 0.460727] ACPI: PCI: Interrupt link LNKA configured for IRQ 101357worker2 # [ 0.464441] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1358client # [ 0.468782] ACPI: PCI: Interrupt link GSIH configured for IRQ 231359lb # [ 0.461771] ACPI: PCI: Interrupt link LNKB configured for IRQ 101360worker1 # [ 0.457641] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1361worker1 # [ 0.458456] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1362lb # [ 0.462761] ACPI: PCI: Interrupt link LNKC configured for IRQ 111363worker2 # [ 0.466117] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1364client # [ 0.470534] iommu: Default domain type: Translated1365lb # [ 0.463766] ACPI: PCI: Interrupt link LNKD configured for IRQ 111366client # [ 0.470803] iommu: DMA domain TLB invalidation policy: lazy mode1367worker1 # [ 0.459708] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1368lb # [ 0.464763] ACPI: PCI: Interrupt link LNKE configured for IRQ 101369client # [ 0.472037] ACPI: bus type USB registered1370lb # [ 0.465759] ACPI: PCI: Interrupt link LNKF configured for IRQ 101371worker1 # [ 0.461308] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1372client # [ 0.472803] usbcore: registered new interface driver usbfs1373lb # [ 0.466760] ACPI: PCI: Interrupt link LNKG configured for IRQ 111374client # [ 0.473659] usbcore: registered new interface driver hub1375worker2 # [ 0.470166] ACPI: PCI: Interrupt link LNKA configured for IRQ 101376client # [ 0.474454] usbcore: registered new device driver usb1377lb # [ 0.467758] ACPI: PCI: Interrupt link LNKH configured for IRQ 111378worker2 # [ 0.470521] ACPI: PCI: Interrupt link LNKB configured for IRQ 101379lb # [ 0.468698] ACPI: PCI: Interrupt link GSIA configured for IRQ 161380client # [ 0.475845] NetLabel: Initializing1381worker2 # [ 0.471516] ACPI: PCI: Interrupt link LNKC configured for IRQ 111382lb # [ 0.469641] ACPI: PCI: Interrupt link GSIB configured for IRQ 171383client # [ 0.476455] NetLabel: domain hash size = 1281384worker1 # [ 0.465154] ACPI: PCI: Interrupt link LNKA configured for IRQ 101385lb # [ 0.470377] ACPI: PCI: Interrupt link GSIC configured for IRQ 181386worker2 # [ 0.472513] ACPI: PCI: Interrupt link LNKD configured for IRQ 111387client # [ 0.476796] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1388worker1 # [ 0.465718] ACPI: PCI: Interrupt link LNKB configured for IRQ 101389lb # [ 0.470679] ACPI: PCI: Interrupt link GSID configured for IRQ 191390worker2 # [ 0.473517] ACPI: PCI: Interrupt link LNKE configured for IRQ 101391client # [ 0.477688] NetLabel: unlabeled traffic allowed by default1392client # [ 0.478440] PCI: Using ACPI for IRQ routing1393lb # [ 0.471598] ACPI: PCI: Interrupt link GSIE configured for IRQ 201394worker1 # [ 0.466714] ACPI: PCI: Interrupt link LNKC configured for IRQ 111395worker2 # [ 0.474512] ACPI: PCI: Interrupt link LNKF configured for IRQ 101396lb # [ 0.472442] ACPI: PCI: Interrupt link GSIF configured for IRQ 211397worker1 # [ 0.467711] ACPI: PCI: Interrupt link LNKD configured for IRQ 111398worker2 # [ 0.475514] ACPI: PCI: Interrupt link LNKG configured for IRQ 111399lb # [ 0.473380] ACPI: PCI: Interrupt link GSIG configured for IRQ 221400worker1 # [ 0.468715] ACPI: PCI: Interrupt link LNKE configured for IRQ 101401worker2 # [ 0.476545] ACPI: PCI: Interrupt link LNKH configured for IRQ 111402lb # [ 0.473674] ACPI: PCI: Interrupt link GSIH configured for IRQ 231403worker1 # [ 0.469707] ACPI: PCI: Interrupt link LNKF configured for IRQ 101404worker2 # [ 0.477459] ACPI: PCI: Interrupt link GSIA configured for IRQ 161405worker1 # [ 0.470710] ACPI: PCI: Interrupt link LNKG configured for IRQ 111406lb # [ 0.475638] iommu: Default domain type: Translated1407worker2 # [ 0.478432] ACPI: PCI: Interrupt link GSIB configured for IRQ 171408lb # [ 0.476238] iommu: DMA domain TLB invalidation policy: lazy mode1409worker1 # [ 0.471723] ACPI: PCI: Interrupt link LNKH configured for IRQ 111410worker2 # [ 0.479430] ACPI: PCI: Interrupt link GSIC configured for IRQ 181411lb # [ 0.476941] ACPI: bus type USB registered1412worker1 # [ 0.472655] ACPI: PCI: Interrupt link GSIA configured for IRQ 161413worker2 # [ 0.480431] ACPI: PCI: Interrupt link GSID configured for IRQ 191414lb # [ 0.477671] usbcore: registered new interface driver usbfs1415worker1 # [ 0.473635] ACPI: PCI: Interrupt link GSIB configured for IRQ 171416worker2 # [ 0.481427] ACPI: PCI: Interrupt link GSIE configured for IRQ 201417lb # [ 0.478521] usbcore: registered new interface driver hub1418worker1 # [ 0.474613] ACPI: PCI: Interrupt link GSIC configured for IRQ 181419worker2 # [ 0.482427] ACPI: PCI: Interrupt link GSIF configured for IRQ 211420lb # [ 0.479284] usbcore: registered new device driver usb1421worker1 # [ 0.475578] ACPI: PCI: Interrupt link GSID configured for IRQ 191422worker2 # [ 0.483431] ACPI: PCI: Interrupt link GSIG configured for IRQ 221423lb # [ 0.480579] NetLabel: Initializing1424worker1 # [ 0.476360] ACPI: PCI: Interrupt link GSIE configured for IRQ 201425worker2 # [ 0.484429] ACPI: PCI: Interrupt link GSIH configured for IRQ 231426lb # [ 0.481107] NetLabel: domain hash size = 1281427worker1 # [ 0.476624] ACPI: PCI: Interrupt link GSIF configured for IRQ 211428lb # [ 0.481662] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1429worker1 # [ 0.477612] ACPI: PCI: Interrupt link GSIG configured for IRQ 221430worker2 # [ 0.486539] iommu: Default domain type: Translated1431lb # [ 0.482588] NetLabel: unlabeled traffic allowed by default1432worker1 # [ 0.478595] ACPI: PCI: Interrupt link GSIH configured for IRQ 231433lb # [ 0.483316] PCI: Using ACPI for IRQ routing1434worker2 # [ 0.487355] iommu: DMA domain TLB invalidation policy: lazy mode1435worker2 # [ 0.488499] ACPI: bus type USB registered1436worker1 # [ 0.480399] iommu: Default domain type: Translated1437worker2 # [ 0.489292] usbcore: registered new interface driver usbfs1438worker1 # [ 0.481237] iommu: DMA domain TLB invalidation policy: lazy mode1439worker2 # [ 0.490181] usbcore: registered new interface driver hub1440worker1 # [ 0.481888] ACPI: bus type USB registered1441worker2 # [ 0.490551] usbcore: registered new device driver usb1442worker1 # [ 0.482664] usbcore: registered new interface driver usbfs1443worker1 # [ 0.483598] usbcore: registered new interface driver hub1444worker2 # [ 0.492351] NetLabel: Initializing1445worker1 # [ 0.484326] usbcore: registered new device driver usb1446worker2 # [ 0.492895] NetLabel: domain hash size = 1281447worker2 # [ 0.493415] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1448worker1 # [ 0.485743] NetLabel: Initializing1449worker2 # [ 0.494388] NetLabel: unlabeled traffic allowed by default1450worker1 # [ 0.486372] NetLabel: domain hash size = 1281451worker2 # [ 0.495130] PCI: Using ACPI for IRQ routing1452worker1 # [ 0.486619] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453worker1 # [ 0.487591] NetLabel: unlabeled traffic allowed by default1454worker1 # [ 0.488353] PCI: Using ACPI for IRQ routing1455client # [ 0.521718] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456client # [ 0.521792] pci 0000:00:01.0: vgaarb: bridge control possible1457client # [ 0.521792] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458client # [ 0.521801] vgaarb: loaded1459client # [ 0.522565] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460client # [ 0.522796] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461client # [ 0.526882] clocksource: Switched to clocksource kvm-clock1462client # [ 0.528574] VFS: Disk quotas dquot_6.6.01463lb # [ 0.526281] pci 0000:00:01.0: vgaarb: setting as boot VGA device1464client # [ 0.531110] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1465lb # [ 0.526658] pci 0000:00:01.0: vgaarb: bridge control possible1466client # [ 0.532675] pnp: PnP ACPI init1467lb # [ 0.526658] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1468client # [ 0.533563] ACPI: IRQ 4 override to edge(!), high(!)1469lb # [ 0.526699] vgaarb: loaded1470client # [ 0.534653] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1471lb # [ 0.527571] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01472client # [ 0.536218] pnp: PnP ACPI: found 5 devices1473lb # [ 0.528312] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1474worker2 # [ 0.539282] pci 0000:00:01.0: vgaarb: setting as boot VGA device1475lb # [ 0.532812] clocksource: Switched to clocksource kvm-clock1476worker2 # [ 0.539411] pci 0000:00:01.0: vgaarb: bridge control possible1477worker2 # [ 0.539411] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1478worker1 # [ 0.533056] pci 0000:00:01.0: vgaarb: setting as boot VGA device1479worker2 # [ 0.539419] vgaarb: loaded1480client # [ 0.543854] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1481worker1 # [ 0.533609] pci 0000:00:01.0: vgaarb: bridge control possible1482client # [ 0.545221] clocksource: Switched to clocksource acpi_pm1483lb # [ 0.536702] VFS: Disk quotas dquot_6.6.01484worker2 # [ 0.540155] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01485client # [ 0.546200] NET: Registered PF_INET protocol family1486worker2 # [ 0.540415] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1487worker1 # [ 0.533609] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1488lb # [ 0.537403] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1489worker1 # [ 0.533618] vgaarb: loaded1490lb # [ 0.538800] pnp: PnP ACPI init1491client # [ 0.547327] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1492worker1 # [ 0.534366] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01493lb # [ 0.539670] ACPI: IRQ 4 override to edge(!), high(!)1494worker1 # [ 0.534614] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1495lb # [ 0.540696] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1496lb # [ 0.542268] pnp: PnP ACPI: found 5 devices1497worker2 # [ 0.544540] clocksource: Switched to clocksource kvm-clock1498worker2 # [ 0.546328] VFS: Disk quotas dquot_6.6.01499worker1 # [ 0.539712] clocksource: Switched to clocksource kvm-clock1500worker2 # [ 0.548875] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1501worker2 # [ 0.550395] pnp: PnP ACPI init1502worker1 # [ 0.543330] VFS: Disk quotas dquot_6.6.01503worker2 # [ 0.551332] ACPI: IRQ 4 override to edge(!), high(!)1504worker1 # [ 0.544045] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1505worker1 # [ 0.545474] pnp: PnP ACPI init1506lb # [ 0.550028] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1507worker2 # [ 0.552492] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1508lb # [ 0.551499] clocksource: Switched to clocksource acpi_pm1509worker1 # [ 0.546268] ACPI: IRQ 4 override to edge(!), high(!)1510worker2 # [ 0.554323] pnp: PnP ACPI: found 5 devices1511lb # [ 0.552542] NET: Registered PF_INET protocol family1512worker1 # [ 0.547238] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1513worker1 # [ 0.548796] pnp: PnP ACPI: found 5 devices1514lb # [ 0.553722] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1515client # [ 0.564088] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1516client # [ 0.565658] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1517client # [ 0.567046] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1518client # [ 0.568470] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1519worker2 # [ 0.562273] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1520client # [ 0.569786] TCP: Hash tables configured (established 16384 bind 16384)1521worker2 # [ 0.563780] clocksource: Switched to clocksource acpi_pm1522worker1 # [ 0.556412] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1523client # [ 0.570944] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1524worker2 # [ 0.564776] NET: Registered PF_INET protocol family1525worker1 # [ 0.557819] clocksource: Switched to clocksource acpi_pm1526client # [ 0.572188] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1527worker1 # [ 0.558802] NET: Registered PF_INET protocol family1528worker2 # [ 0.566033] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1529client # [ 0.573248] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1530worker1 # [ 0.559952] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1531client # [ 0.574502] NET: Registered PF_UNIX/PF_LOCAL protocol family1532client # [ 0.575463] NET: Registered PF_XDP protocol family1533client # [ 0.576338] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1534client # [ 0.577387] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1535client # [ 0.578444] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1536client # [ 0.579628] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1537lb # [ 0.570618] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1538client # [ 0.580749] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1539lb # [ 0.572082] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1540client # [ 0.581877] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1541lb # [ 0.573417] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1542client # [ 0.583586] ACPI: \_SB_.GSIA: Enabled at IRQ 161543lb # [ 0.574817] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1544lb # [ 0.576089] TCP: Hash tables configured (established 16384 bind 16384)1545client # [ 0.585960] ACPI: \_SB_.GSIB: Enabled at IRQ 171546lb # [ 0.577248] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1547lb # [ 0.578631] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1548client # [ 0.587991] ACPI: \_SB_.GSIC: Enabled at IRQ 181549lb # [ 0.579823] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1550worker2 # [ 0.582354] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1551lb # [ 0.581073] NET: Registered PF_UNIX/PF_LOCAL protocol family1552client # [ 0.590044] ACPI: \_SB_.GSID: Enabled at IRQ 191553lb # [ 0.582053] NET: Registered PF_XDP protocol family1554worker2 # [ 0.583934] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1555worker1 # [ 0.576624] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1556client # [ 0.591758] PCI: CLS 0 bytes, default 641557lb # [ 0.582965] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1558worker2 # [ 0.585224] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1559worker1 # [ 0.578104] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1560client # [ 0.592816] Trying to unpack rootfs image as initramfs...1561lb # [ 0.583958] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1562worker2 # [ 0.586574] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1563lb # [ 0.584942] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1564worker1 # [ 0.579428] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1565worker2 # [ 0.587812] TCP: Hash tables configured (established 16384 bind 16384)1566lb # [ 0.586000] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1567worker1 # [ 0.580783] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1568lb # [ 0.587068] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1569worker2 # [ 0.589019] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1570worker1 # [ 0.582019] TCP: Hash tables configured (established 16384 bind 16384)1571lb # [ 0.588134] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1572worker2 # [ 0.590424] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1573worker1 # [ 0.583206] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1574lb # [ 0.589890] ACPI: \_SB_.GSIA: Enabled at IRQ 161575worker2 # [ 0.591622] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1576worker1 # [ 0.584560] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1577client # [ 0.598833] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1578worker2 # [ 0.592817] NET: Registered PF_UNIX/PF_LOCAL protocol family1579worker1 # [ 0.585669] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1580worker2 # [ 0.593793] NET: Registered PF_XDP protocol family1581worker1 # [ 0.586885] NET: Registered PF_UNIX/PF_LOCAL protocol family1582lb # [ 0.592906] ACPI: \_SB_.GSIB: Enabled at IRQ 171583worker2 # [ 0.594716] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1584worker1 # [ 0.587859] NET: Registered PF_XDP protocol family1585worker2 # [ 0.595730] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1586worker1 # [ 0.588700] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1587lb # [ 0.594887] ACPI: \_SB_.GSIC: Enabled at IRQ 181588worker2 # [ 0.596774] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1589worker1 # [ 0.589705] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1590worker2 # [ 0.597883] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1591worker1 # [ 0.590682] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1592lb # [ 0.596838] ACPI: \_SB_.GSID: Enabled at IRQ 191593worker2 # [ 0.599054] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1594worker1 # [ 0.591817] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1595worker2 # [ 0.600188] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1596worker1 # [ 0.592910] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1597lb # [ 0.598765] PCI: CLS 0 bytes, default 641598worker1 # [ 0.594005] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1599worker2 # [ 0.601886] ACPI: \_SB_.GSIA: Enabled at IRQ 161600lb # [ 0.599674] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1601worker1 # [ 0.595736] ACPI: \_SB_.GSIA: Enabled at IRQ 161602lb # [ 0.601354] Trying to unpack rootfs image as initramfs...1603worker2 # [ 0.604427] ACPI: \_SB_.GSIB: Enabled at IRQ 171604worker1 # [ 0.598371] ACPI: \_SB_.GSIB: Enabled at IRQ 171605worker2 # [ 0.606734] ACPI: \_SB_.GSIC: Enabled at IRQ 181606worker1 # [ 0.600423] ACPI: \_SB_.GSIC: Enabled at IRQ 181607worker2 # [ 0.608718] ACPI: \_SB_.GSID: Enabled at IRQ 191608worker1 # [ 0.602424] ACPI: \_SB_.GSID: Enabled at IRQ 191609worker2 # [ 0.610429] PCI: CLS 0 bytes, default 641610worker1 # [ 0.604157] PCI: CLS 0 bytes, default 641611worker2 # [ 0.611348] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1612worker1 # [ 0.605044] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1613worker2 # [ 0.613123] Trying to unpack rootfs image as initramfs...1614worker1 # [ 0.606781] Trying to unpack rootfs image as initramfs...1615client # [ 0.639552] Initialise system trusted keyrings1616client # [ 0.642781] workingset: timestamp_bits=40 max_order=19 bucket_order=01617lb # [ 0.643835] Initialise system trusted keyrings1618lb # [ 0.646654] workingset: timestamp_bits=40 max_order=19 bucket_order=01619client # [ 0.662079] Key type asymmetric registered1620worker1 # [ 0.652523] Initialise system trusted keyrings1621client # [ 0.666726] Asymmetric key parser 'x509' registered1622worker2 # [ 0.659939] Initialise system trusted keyrings1623client # [ 0.667554] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1624worker1 # [ 0.655495] workingset: timestamp_bits=40 max_order=19 bucket_order=01625client # [ 0.670819] io scheduler mq-deadline registered1626worker2 # [ 0.663716] workingset: timestamp_bits=40 max_order=19 bucket_order=01627client # [ 0.671596] io scheduler kyber registered1628client # [ 0.674231] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1629client # [ 0.675562] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1630lb # [ 0.669545] Key type asymmetric registered1631lb # [ 0.670240] Asymmetric key parser 'x509' registered1632lb # [ 0.671071] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1633client # [ 0.681477] Linux agpgart interface v0.1031634client # [ 0.682262] ACPI: bus type drm_connector registered1635lb # [ 0.676634] io scheduler mq-deadline registered1636client # [ 0.685171] usbcore: registered new interface driver usbserial_generic1637lb # [ 0.677394] io scheduler kyber registered1638client # [ 0.686244] usbserial: USB Serial support registered for generic1639lb # [ 0.680114] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1640worker1 # [ 0.675009] Key type asymmetric registered1641client # [ 0.689733] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1642lb # [ 0.681420] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1643client # [ 0.689733] Please enable it if your BIOS has the CPPC option.1644worker2 # [ 0.684918] Key type asymmetric registered1645client # [ 0.691976] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1646worker1 # [ 0.679488] Asymmetric key parser 'x509' registered1647worker2 # [ 0.687548] Asymmetric key parser 'x509' registered1648worker1 # [ 0.680374] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1649client # [ 0.694877] drop_monitor: Initializing network drop monitor service1650worker2 # [ 0.688419] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1651client # [ 0.696003] NET: Registered PF_INET6 protocol family1652lb # [ 0.687512] Linux agpgart interface v0.1031653lb # [ 0.688231] ACPI: bus type drm_connector registered1654worker1 # [ 0.683577] io scheduler mq-deadline registered1655worker1 # [ 0.684367] io scheduler kyber registered1656worker2 # [ 0.691646] io scheduler mq-deadline registered1657client # [ 0.699205] Segment Routing with IPv61658worker2 # [ 0.692440] io scheduler kyber registered1659worker1 # [ 0.687158] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1660lb # [ 0.692993] usbcore: registered new interface driver usbserial_generic1661worker2 # [ 0.695102] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1662client # [ 0.702749] In-situ OAM (IOAM) with IPv61663lb # [ 0.694091] usbserial: USB Serial support registered for generic1664client # [ 0.703836] IPI shorthand broadcast: enabled1665worker2 # [ 0.696382] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1666worker1 # [ 0.690482] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1667lb # [ 0.696472] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1668lb # [ 0.696472] Please enable it if your BIOS has the CPPC option.1669lb # [ 0.698722] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1670worker1 # [ 0.694103] Linux agpgart interface v0.1031671worker2 # [ 0.702334] Linux agpgart interface v0.1031672worker1 # [ 0.694916] ACPI: bus type drm_connector registered1673worker2 # [ 0.703075] ACPI: bus type drm_connector registered1674lb # [ 0.701627] drop_monitor: Initializing network drop monitor service1675lb # [ 0.702806] NET: Registered PF_INET6 protocol family1676worker1 # [ 0.697910] usbcore: registered new interface driver usbserial_generic1677client # [ 0.712047] sched_clock: Marking stable (579013907, 132693057)->(787318553, -75611589)1678worker1 # [ 0.699008] usbserial: USB Serial support registered for generic1679worker2 # [ 0.707665] usbcore: registered new interface driver usbserial_generic1680client # [ 0.715959] registered taskstats version 11681worker2 # [ 0.708815] usbserial: USB Serial support registered for generic1682lb # [ 0.708000] Segment Routing with IPv61683client # [ 0.716925] Loading compiled-in X.509 certificates1684worker1 # [ 0.703486] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1685lb # [ 0.710485] In-situ OAM (IOAM) with IPv61686worker2 # [ 0.711548] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1687worker1 # [ 0.703486] Please enable it if your BIOS has the CPPC option.1688worker2 # [ 0.711548] Please enable it if your BIOS has the CPPC option.1689lb # [ 0.711566] IPI shorthand broadcast: enabled1690worker1 # [ 0.705809] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1691worker2 # [ 0.713785] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1692worker1 # [ 0.708620] drop_monitor: Initializing network drop monitor service1693worker2 # [ 0.716690] drop_monitor: Initializing network drop monitor service1694worker1 # [ 0.709799] NET: Registered PF_INET6 protocol family1695worker2 # [ 0.717907] NET: Registered PF_INET6 protocol family1696worker1 # [ 0.714574] Segment Routing with IPv61697lb # [ 0.719879] sched_clock: Marking stable (586015274, 133437563)->(789538712, -70085875)1698worker1 # [ 0.715234] In-situ OAM (IOAM) with IPv61699worker2 # [ 0.723053] Segment Routing with IPv61700worker1 # [ 0.717855] IPI shorthand broadcast: enabled1701worker2 # [ 0.725572] In-situ OAM (IOAM) with IPv61702lb # [ 0.724721] registered taskstats version 11703worker2 # [ 0.726687] IPI shorthand broadcast: enabled1704lb # [ 0.725691] Loading compiled-in X.509 certificates1705client # [ 0.734726] Demotion targets for Node 0: null1706client # [ 0.737753] Key type .fscrypt registered1707client # [ 0.738459] Key type fscrypt-provisioning registered1708client # [ 0.739436] ima: No TPM chip found, activating TPM-bypass!1709worker1 # [ 0.726119] sched_clock: Marking stable (592014529, 133445972)->(801124054, -75663553)1710client # [ 0.741724] ima: Allocated hash algorithm: sha11711client # [ 0.742546] ima: No architecture policies found1712worker2 # [ 0.736084] sched_clock: Marking stable (602014613, 133514142)->(811622681, -76093926)1713worker1 # [ 0.729666] registered taskstats version 11714worker1 # [ 0.730678] Loading compiled-in X.509 certificates1715worker2 # [ 0.739712] registered taskstats version 11716client # [ 0.746726] PM: Magic number: 10:775:831717worker2 # [ 0.740724] Loading compiled-in X.509 certificates1718client # [ 0.748354] RAS: Correctable Errors collector initialized.1719lb # [ 0.744469] Demotion targets for Node 0: null1720lb # [ 0.745369] Key type .fscrypt registered1721client # [ 0.756393] clk: Disabling unused clocks1722lb # [ 0.748457] Key type fscrypt-provisioning registered1723lb # [ 0.749463] ima: No TPM chip found, activating TPM-bypass!1724lb # [ 0.750379] ima: Allocated hash algorithm: sha11725lb # [ 0.751212] ima: No architecture policies found1726client # [ 0.760731] PM: genpd: Disabling unused power domains1727lb # [ 0.753469] PM: Magic number: 10:775:831728worker1 # [ 0.749474] Demotion targets for Node 0: null1729lb # [ 0.755159] RAS: Correctable Errors collector initialized.1730worker1 # [ 0.750417] Key type .fscrypt registered1731worker2 # [ 0.759547] Demotion targets for Node 0: null1732worker1 # [ 0.752469] Key type fscrypt-provisioning registered1733worker1 # [ 0.753475] ima: No TPM chip found, activating TPM-bypass!1734worker1 # [ 0.754435] ima: Allocated hash algorithm: sha11735worker2 # [ 0.762575] Key type .fscrypt registered1736worker2 # [ 0.763264] Key type fscrypt-provisioning registered1737worker2 # [ 0.764271] ima: No TPM chip found, activating TPM-bypass!1738worker1 # [ 0.757493] ima: No architecture policies found1739worker2 # [ 0.766546] ima: Allocated hash algorithm: sha11740worker2 # [ 0.767380] ima: No architecture policies found1741worker1 # [ 0.760476] PM: Magic number: 10:775:831742lb # [ 0.766209] clk: Disabling unused clocks1743worker2 # [ 0.769545] PM: Magic number: 10:775:831744worker1 # [ 0.762197] RAS: Correctable Errors collector initialized.1745lb # [ 0.768475] PM: genpd: Disabling unused power domains1746worker2 # [ 0.771154] RAS: Correctable Errors collector initialized.1747worker1 # [ 0.772182] clk: Disabling unused clocks1748worker2 # [ 0.781214] clk: Disabling unused clocks1749worker1 # [ 0.774479] PM: genpd: Disabling unused power domains1750worker2 # [ 0.783547] PM: genpd: Disabling unused power domains1751client # [ 0.923040] Freeing initrd memory: 29056K1752client # [ 0.926247] Freeing unused decrypted memory: 2028K1753client # [ 0.928945] Freeing unused kernel image (initmem) memory: 3652K1754client # [ 0.930081] Write protecting the kernel read-only data: 32768k1755client # [ 0.931985] Freeing unused kernel image (text/rodata gap) memory: 1184K1756client # [ 0.933547] Freeing unused kernel image (rodata/data gap) memory: 720K1757lb # [ 0.936622] Freeing initrd memory: 29064K1758lb # [ 0.939928] Freeing unused decrypted memory: 2028K1759lb # [ 0.942809] Freeing unused kernel image (initmem) memory: 3652K1760worker1 # [ 0.937775] Freeing initrd memory: 29056K1761lb # [ 0.943938] Write protecting the kernel read-only data: 32768k1762worker2 # [ 0.947396] Freeing initrd memory: 29060K1763lb # [ 0.945887] Freeing unused kernel image (text/rodata gap) memory: 1184K1764worker1 # [ 0.941037] Freeing unused decrypted memory: 2028K1765lb # [ 0.947386] Freeing unused kernel image (rodata/data gap) memory: 720K1766worker2 # [ 0.950699] Freeing unused decrypted memory: 2028K1767worker1 # [ 0.943847] Freeing unused kernel image (initmem) memory: 3652K1768worker1 # [ 0.944982] Write protecting the kernel read-only data: 32768k1769worker2 # [ 0.953430] Freeing unused kernel image (initmem) memory: 3652K1770worker1 # [ 0.946931] Freeing unused kernel image (text/rodata gap) memory: 1184K1771worker2 # [ 0.954600] Write protecting the kernel read-only data: 32768k1772worker1 # [ 0.948522] Freeing unused kernel image (rodata/data gap) memory: 720K1773worker2 # [ 0.956481] Freeing unused kernel image (text/rodata gap) memory: 1184K1774worker2 # [ 0.958061] Freeing unused kernel image (rodata/data gap) memory: 720K1775client # [ 0.984490] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776client # [ 0.985604] Run /init as init process1777client # [ 0.996237] systemd[1]: Inserted module 'autofs4'1778lb # [ 0.998341] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779lb # [ 0.999482] Run /init as init process1780worker1 # [ 0.999609] x86/mm: Checked W+X mappings: passed, no W+X pages found.1781worker1 # [ 1.000699] Run /init as init process1782client # [ 1.016060] fuse: init (API version 7.45)1783worker2 # [ 1.008926] x86/mm: Checked W+X mappings: passed, no W+X pages found.1784worker2 # [ 1.010030] Run /init as init process1785lb # [ 1.010972] systemd[1]: Inserted module 'autofs4'1786client # [ 1.023384] ACPI: \_SB_.GSIG: Enabled at IRQ 221787client # [ 1.025792] ACPI: \_SB_.GSIH: Enabled at IRQ 231788worker1 # [ 1.011620] systemd[1]: Inserted module 'autofs4'1789worker2 # [ 1.020808] systemd[1]: Inserted module 'autofs4'1790client # [ 1.028879] ACPI: \_SB_.GSIE: Enabled at IRQ 201791client # [ 1.030826] ACPI: \_SB_.GSIF: Enabled at IRQ 211792client # [ 1.036120] virtiofs virtio5: discovered new tag: nix-store1793client # [ 1.037741] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1794lb # [ 1.031146] fuse: init (API version 7.45)1795worker1 # [ 1.031587] fuse: init (API version 7.45)1796client # [ 1.047002] virtiofs virtio6: discovered new tag: shared1797lb # [ 1.038661] ACPI: \_SB_.GSIG: Enabled at IRQ 221798worker2 # [ 1.040701] fuse: init (API version 7.45)1799client # [ 1.048710] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1800lb # [ 1.041361] ACPI: \_SB_.GSIH: Enabled at IRQ 231801client # [ 1.052613] virtiofs virtio7: discovered new tag: xchg1802worker1 # [ 1.039008] ACPI: \_SB_.GSIG: Enabled at IRQ 221803lb # [ 1.044684] ACPI: \_SB_.GSIE: Enabled at IRQ 201804client # [ 1.054161] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1805worker2 # [ 1.048060] ACPI: \_SB_.GSIG: Enabled at IRQ 221806lb # [ 1.046871] ACPI: \_SB_.GSIF: Enabled at IRQ 211807worker1 # [ 1.041821] ACPI: \_SB_.GSIH: Enabled at IRQ 231808worker2 # [ 1.050890] ACPI: \_SB_.GSIH: Enabled at IRQ 231809worker1 # [ 1.044981] ACPI: \_SB_.GSIE: Enabled at IRQ 201810worker2 # [ 1.054169] ACPI: \_SB_.GSIE: Enabled at IRQ 201811worker1 # [ 1.047184] ACPI: \_SB_.GSIF: Enabled at IRQ 211812lb # [ 1.052812] virtiofs virtio5: discovered new tag: nix-store1813lb # [ 1.054325] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1814worker2 # [ 1.056558] ACPI: \_SB_.GSIF: Enabled at IRQ 211815worker1 # [ 1.053732] virtiofs virtio5: discovered new tag: nix-store1816worker2 # [ 1.062474] virtiofs virtio5: discovered new tag: nix-store1817worker1 # [ 1.055715] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1818lb # [ 1.061485] virtiofs virtio6: discovered new tag: shared1819worker2 # [ 1.064023] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1820lb # [ 1.062980] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1821lb # [ 1.066424] virtiofs virtio7: discovered new tag: xchg1822lb # [ 1.067894] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1823worker1 # [ 1.063234] virtiofs virtio6: discovered new tag: shared1824client # [ 1.077983] systemd[1]: Successfully made /usr/ read-only.1825worker2 # [ 1.071726] virtiofs virtio6: discovered new tag: shared1826worker1 # [ 1.064748] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1827worker2 # [ 1.073194] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1828worker1 # [ 1.068231] virtiofs virtio7: discovered new tag: xchg1829worker2 # [ 1.076771] virtiofs virtio7: discovered new tag: xchg1830worker1 # [ 1.069801] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1831worker2 # [ 1.078182] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1832lb # [ 1.091585] systemd[1]: Successfully made /usr/ read-only.1833worker1 # [ 1.091313] systemd[1]: Successfully made /usr/ read-only.1834worker2 # [ 1.100334] systemd[1]: Successfully made /usr/ read-only.1835client # [ 1.413675] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836client # [ 1.420534] systemd[1]: Detected virtualization kvm.1837client # [ 1.422774] systemd[1]: Detected architecture x86-64.1838client # [ 1.424929] systemd[1]: Running in initrd.1839client # [ 1.427514] systemd[1]: Initializing machine ID from random generator.1840client # [ 1.430419] systemd[1]: Hostname set to <client>.1841worker1 # [ 1.426918] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1842worker1 # [ 1.432064] systemd[1]: Detected virtualization kvm.1843worker1 # [ 1.432964] systemd[1]: Detected architecture x86-64.1844worker1 # [ 1.433839] systemd[1]: Running in initrd.1845lb # [ 1.428301] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1846worker1 # [ 1.434932] systemd[1]: Initializing machine ID from random generator.1847worker1 # [ 1.436181] systemd[1]: Hostname set to <worker1>.1848lb # [ 1.440622] systemd[1]: Detected virtualization kvm.1849lb # [ 1.442788] systemd[1]: Detected architecture x86-64.1850lb # [ 1.444955] systemd[1]: Running in initrd.1851worker2 # [ 1.437096] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1852worker2 # [ 1.449253] systemd[1]: Detected virtualization kvm.1853lb # [ 1.447535] systemd[1]: Initializing machine ID from random generator.1854worker2 # [ 1.451569] systemd[1]: Detected architecture x86-64.1855lb # [ 1.450423] systemd[1]: Hostname set to <lb>.1856worker2 # [ 1.453747] systemd[1]: Running in initrd.1857worker2 # [ 1.456205] systemd[1]: Initializing machine ID from random generator.1858worker2 # [ 1.459144] systemd[1]: Hostname set to <worker2>.1859worker1 # [ 1.625934] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.644154] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861client # [ 1.658404] systemd[1]: bpf-restrict-fs: LSM BPF program attached1862worker1 # [ 1.664265] systemd[1]: Queued start job for default target Initrd Default Target.1863worker1 # [ 1.668844] systemd[1]: Created slice Slice /system/modprobe.1864worker1 # [ 1.670275] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1865worker1 # [ 1.671733] systemd[1]: Expecting device /dev/disk/by-label/nixos...1866worker1 # [ 1.672901] systemd[1]: Reached target Path Units.1867worker1 # [ 1.673841] systemd[1]: Reached target Slice Units.1868worker1 # [ 1.674728] systemd[1]: Reached target Swaps.1869worker1 # [ 1.675549] systemd[1]: Reached target Timer Units.1870worker2 # [ 1.681789] systemd[1]: bpf-restrict-fs: LSM BPF program attached1871worker1 # [ 1.676573] systemd[1]: Listening on D-Bus System Message Bus Socket.1872worker1 # [ 1.677780] systemd[1]: Listening on Journal Socket (/dev/log).1873lb # [ 1.683602] systemd[1]: Queued start job for default target Initrd Default Target.1874worker1 # [ 1.678963] systemd[1]: Listening on Journal Sockets.1875worker1 # [ 1.679947] systemd[1]: Listening on udev Control Socket.1876worker1 # [ 1.681005] systemd[1]: Listening on udev Kernel Socket.1877worker1 # [ 1.682001] systemd[1]: Reached target Socket Units.1878lb # [ 1.687860] systemd[1]: Created slice Slice /system/modprobe.1879worker1 # [ 1.683881] systemd[1]: Starting Create List of Static Device Nodes...1880lb # [ 1.689108] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1881lb # [ 1.690634] systemd[1]: Expecting device /dev/disk/by-label/nixos...1882lb # [ 1.691803] systemd[1]: Reached target Path Units.1883lb # [ 1.692768] systemd[1]: Reached target Slice Units.1884lb # [ 1.693703] systemd[1]: Reached target Swaps.1885worker1 # [ 1.688537] systemd[1]: Starting Load Kernel Module configfs...1886lb # [ 1.694568] systemd[1]: Reached target Timer Units.1887lb # [ 1.695604] systemd[1]: Listening on D-Bus System Message Bus Socket.1888lb # [ 1.696884] systemd[1]: Listening on Journal Socket (/dev/log).1889lb # [ 1.698115] systemd[1]: Listening on Journal Sockets.1890lb # [ 1.699179] systemd[1]: Listening on udev Control Socket.1891lb # [ 1.700256] systemd[1]: Listening on udev Kernel Socket.1892lb # [ 1.701246] systemd[1]: Reached target Socket Units.1893lb # [ 1.703187] systemd[1]: Starting Create List of Static Device Nodes...1894client # [ 1.713600] systemd[1]: Queued start job for default target Initrd Default Target.1895lb # [ 1.707524] systemd[1]: Starting Load Kernel Module configfs...1896client # [ 1.718166] systemd[1]: Created slice Slice /system/modprobe.1897client # [ 1.719338] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1898client # [ 1.720770] systemd[1]: Expecting device /dev/disk/by-label/nixos...1899client # [ 1.721902] systemd[1]: Reached target Path Units.1900worker1 # [ 1.696648] systemd[1]: Starting Journal Service...1901client # [ 1.722823] systemd[1]: Reached target Slice Units.1902client # [ 1.723730] systemd[1]: Reached target Swaps.1903client # [ 1.724585] systemd[1]: Reached target Timer Units.1904client # [ 1.725573] systemd[1]: Listening on D-Bus System Message Bus Socket.1905client # [ 1.726908] systemd[1]: Listening on Journal Socket (/dev/log).1906client # [ 1.728095] systemd[1]: Listening on Journal Sockets.1907client # [ 1.729111] systemd[1]: Listening on udev Control Socket.1908client # [ 1.730144] systemd[1]: Listening on udev Kernel Socket.1909client # [ 1.731109] systemd[1]: Reached target Socket Units.1910client # [ 1.733044] systemd[1]: Starting Create List of Static Device Nodes...1911worker1 # [ 1.722637] systemd[1]: Starting Load Kernel Modules...1912client # [ 1.737848] systemd[1]: Starting Load Kernel Module configfs...1913lb # [ 1.719139] systemd[1]: Starting Journal Service...1914worker1 # [ 1.726556] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1915worker2 # [ 1.734901] systemd[1]: Queued start job for default target Initrd Default Target.1916worker2 # [ 1.738817] systemd[1]: Created slice Slice /system/modprobe.1917worker2 # [ 1.740054] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1918worker1 # [ 1.733649] systemd[1]: Starting Coldplug All udev Devices...1919worker2 # [ 1.741483] systemd[1]: Expecting device /dev/disk/by-label/nixos...1920worker2 # [ 1.742651] systemd[1]: Reached target Path Units.1921worker2 # [ 1.743587] systemd[1]: Reached target Slice Units.1922worker2 # [ 1.744501] systemd[1]: Reached target Swaps.1923worker2 # [ 1.745343] systemd[1]: Reached target Timer Units.1924lb # [ 1.743655] systemd[1]: Starting Load Kernel Modules...1925worker2 # [ 1.746300] systemd[1]: Listening on D-Bus System Message Bus Socket.1926worker2 # [ 1.747587] systemd[1]: Listening on Journal Socket (/dev/log).1927worker2 # [ 1.748796] systemd[1]: Listening on Journal Sockets.1928worker2 # [ 1.749886] systemd[1]: Listening on udev Control Socket.1929lb # [ 1.747577] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1930worker2 # [ 1.750914] systemd[1]: Listening on udev Kernel Socket.1931worker2 # [ 1.751903] systemd[1]: Reached target Socket Units.1932worker1 # [ 1.745015] systemd-journald[66]: Collecting audit messages is disabled.1933client # [ 1.748936] systemd[1]: Starting Journal Service...1934worker2 # [ 1.754605] systemd[1]: Starting Create List of Static Device Nodes...1935worker1 # [ 1.747904] systemd[1]: Finished Create List of Static Device Nodes.1936worker2 # [ 1.757595] systemd[1]: Starting Load Kernel Module configfs...1937lb # [ 1.756542] systemd[1]: Starting Coldplug All udev Devices...1938worker1 # [ 1.753181] systemd[1]: modprobe@configfs.service: Deactivated successfully.1939worker1 # [ 1.759864] systemd[1]: Finished Load Kernel Module configfs.1940client # [ 1.774916] systemd[1]: Starting Load Kernel Modules...1941lb # [ 1.766051] systemd-journald[66]: Collecting audit messages is disabled.1942client # [ 1.778828] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1943worker1 # [ 1.765870] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1944lb # [ 1.775733] systemd[1]: Finished Create List of Static Device Nodes.1945worker2 # [ 1.766595] systemd[1]: Starting Journal Service...1946client # [ 1.788954] systemd[1]: Starting Coldplug All udev Devices...1947lb # [ 1.780118] systemd[1]: modprobe@configfs.service: Deactivated successfully.1948worker1 # [ 1.778531] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1949lb # [ 1.787848] systemd[1]: Finished Load Kernel Module configfs.1950client # [ 1.799410] systemd-journald[66]: Collecting audit messages is disabled.1951worker2 # [ 1.792709] systemd[1]: Starting Load Kernel Modules...1952worker1 # [ 1.786518] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1953client # [ 1.802037] systemd[1]: Finished Create List of Static Device Nodes.1954lb # [ 1.793891] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1955worker2 # [ 1.796622] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1956client # [ 1.809047] systemd[1]: modprobe@configfs.service: Deactivated successfully.1957worker2 # [ 1.804759] systemd[1]: Starting Coldplug All udev Devices...1958worker1 # [ 1.797639] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1959lb # [ 1.805524] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1960client # [ 1.815181] systemd[1]: Finished Load Kernel Module configfs.1961lb # [ 1.808017] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1962client # [ 1.821667] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1963worker2 # [ 1.816911] systemd-journald[66]: Collecting audit messages is disabled.1964worker2 # [ 1.818182] systemd[1]: Finished Create List of Static Device Nodes.1965lb # [ 1.819500] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1966worker2 # [ 1.826052] systemd[1]: modprobe@configfs.service: Deactivated successfully.1967worker1 # [ 1.819045] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1968client # [ 1.837004] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1969worker2 # [ 1.832894] systemd[1]: Finished Load Kernel Module configfs.1970worker1 # [ 1.827664] systemd[1]: Starting Create Static Device Nodes in /dev...1971client # [ 1.845549] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1972worker2 # [ 1.838922] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1973worker1 # [ 1.840018] systemd[1]: Finished Load Kernel Modules.1974lb # [ 1.847076] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1975worker2 # [ 1.849610] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1976client # [ 1.859122] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1977worker1 # [ 1.848661] systemd[1]: Starting Apply Kernel Variables...1978lb # [ 1.857011] systemd[1]: Starting Create Static Device Nodes in /dev...1979worker2 # [ 1.858707] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1980worker2 # [ 1.866602] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1981lb # [ 1.867056] systemd[1]: Finished Load Kernel Modules.1982worker1 # [ 1.733459] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.868212] systemd[1]: Started Journal Service.1983client # [ 1.882343] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1984lb # [ 1.875662] systemd[1]: Starting Apply Kernel Variables...1985worker1 # 1986worker1 # [ 1.738322] systemd-modules-load[68]: Inserted module 'virtio_balloon'1987worker1 # [ 1.739344] systemd-modules-load[68]: Inserted module 'virtio_gpu'1988worker1 # [ 1.745042] systemd[1]: Finished Create Static Device Nodes in /dev.1989client # [ 1.893928] systemd[1]: Starting Create Static Device Nodes in /dev...1990worker2 # [ 1.890082] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1991lb # [ 1.891324] systemd[1]: Started Journal Service.1992worker1 # [ 1.752531] systemd[1]: Reached target Preparation for Local File Systems.1993worker1 # [ 1.753581] systemd[1]: Reached target Local File Systems.1994lb # [ 1.756739] systemd-modules-load[68]: Inserted module 'dm_mod'1995lb # [ 1.760934] systemd-modules-load[68]: Inserted module 'virtio_balloon'1996worker1 # [ 1.755696] systemd[1]: Starting Create System Files and Directories...1997lb # [ 1.761939] systemd-modules-load[68]: Inserted module 'virtio_gpu'1998client # [ 1.904794] systemd[1]: Finished Load Kernel Modules.1999worker2 # [ 1.898720] systemd[1]: Starting Create Static Device Nodes in /dev...2000lb # [ 1.768109] systemd[1]: Finished Create Static Device Nodes in /dev.
loading 2,709 hidden lines…
4710lb # [ 41.460478] niks3-server[990]: 2026/09/22 15:03:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4711worker1 # [ 41.460282] nix-grpc-daemon[880]: time=2026-09-22T15:03:52.801Z level=INFO msg="Uploading 1 narinfos"4712lb # [ 41.465137] niks3-server[990]: 2026/09/22 15:03:52 INFO Signed narinfos id=4 count=14713lb # [ 41.470069] niks3-server[990]: 2026/09/22 15:03:52 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4714lb # [ 41.474216] niks3-server[990]: 2026/09/22 15:03:52 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4715lb # [ 41.478950] niks3-server[990]: 2026/09/22 15:03:52 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4716lb # [ 41.483994] niks3-server[990]: 2026/09/22 15:03:52 INFO Completed upload id=44717worker1 # [ 41.479859] nix-grpc-daemon[880]: time=2026-09-22T15:03:52.821Z level=INFO msg="Upload complete. (158ms)"4718lb # [ 41.485830] niks3-server[990]: 2026/09/22 15:03:52 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4719worker1 # [ 41.485654] nix-grpc-daemon[783]: ts=2026-09-22T15:03:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14720worker1 # [ 41.491559] nix-grpc-daemon[783]: ts=2026-09-22T15:03:52Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04721worker1 # [ 41.493486] nix-grpc-daemon[783]: ts=2026-09-22T15:03:52Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14722worker1 # [ 41.501405] nix-daemon[865]: reaped child process 941, status = succeeded4723worker1 # [ 41.534650] nix-grpc-daemon[783]: ts=2026-09-22T15:03:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608984724worker1 # [ 41.543300] nix-grpc-daemon[783]: ts=2026-09-22T15:03:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608984725worker1 # [ 41.552766] nix-grpc-daemon[880]: time=2026-09-22T15:03:52.894Z level=INFO msg="All 2 paths already cached"4726client # copying 1 paths to 'lb:50051'...4727client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4728worker1 # [ 41.558569] nix-grpc-daemon[783]: ts=2026-09-22T15:03:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608984729worker1 # [ 41.563370] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4730worker1 # [ 41.654609] nix-daemon[865]: accepted connection from pid 969, user nix-grpc-daemon (trusted)4731worker1 # [ 41.669174] nix-daemon[865]: reaped child process 972, status = succeeded4732worker1 # [ 41.756349] nix-daemon[865]: accepted connection from pid 974, user nix-grpc-daemon (trusted)4733worker1 # [ 41.772978] nix-daemon[865]: reaped child process 977, status = succeeded4734lb # [ 41.781922] niks3-server[990]: 2026/09/22 15:03:53 INFO Received uploads request method=POST path=/api/pending_closures4735worker1 # [ 41.782102] nix-grpc-daemon[880]: time=2026-09-22T15:03:53.123Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4736worker1 # [ 41.783867] nix-grpc-daemon[880]: time=2026-09-22T15:03:53.123Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4737lb # [ 41.797593] niks3-server[990]: 2026/09/22 15:03:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4738lb # [ 41.800341] niks3-server[990]: 2026/09/22 15:03:53 INFO Signed narinfos id=5 count=14739worker1 # [ 41.795319] nix-grpc-daemon[880]: time=2026-09-22T15:03:53.136Z level=INFO msg="Uploading 1 narinfos"4740lb # [ 41.802880] niks3-server[990]: 2026/09/22 15:03:53 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4741lb # [ 41.806552] niks3-server[990]: 2026/09/22 15:03:53 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4742lb # [ 41.810772] niks3-server[990]: 2026/09/22 15:03:53 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4743lb # [ 41.818597] niks3-server[990]: 2026/09/22 15:03:53 INFO Completed upload id=54744worker1 # [ 41.813229] nix-grpc-daemon[880]: time=2026-09-22T15:03:53.154Z level=INFO msg="Upload complete. (243ms)"4745lb # [ 41.821252] niks3-server[990]: 2026/09/22 15:03:53 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4746worker1 # [ 41.815962] nix-grpc-daemon[783]: ts=2026-09-22T15:03:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_in=7524747worker1 # [ 41.819871] nix-grpc-daemon[783]: ts=2026-09-22T15:03:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608984748worker1 # [ 41.827722] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4749worker1 # [ 41.830907] nix-daemon[865]: reaped child process 967, status = succeeded4750worker1 # [ 41.837907] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4751client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4752client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4753client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4754client # farm-top> LOG-farm-top-t14755worker1 # [ 43.111638] nix-daemon[865]: reaped child process 981, status = succeeded4756worker1 # [ 43.154306] nix-daemon[865]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4757worker1 # [ 43.169770] nix-daemon[865]: reaped child process 1000, status = succeeded4758lb # [ 43.179331] niks3-server[990]: 2026/09/22 15:03:54 INFO Received uploads request method=POST path=/api/pending_closures4759worker1 # [ 43.178674] nix-grpc-daemon[880]: time=2026-09-22T15:03:54.519Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4760worker1 # [ 43.180454] nix-grpc-daemon[880]: time=2026-09-22T15:03:54.521Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4761lb # [ 43.203815] niks3-server[990]: 2026/09/22 15:03:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4762lb # [ 43.208924] niks3-server[990]: 2026/09/22 15:03:54 INFO Signed narinfos id=6 count=14763worker1 # [ 43.203590] nix-grpc-daemon[880]: time=2026-09-22T15:03:54.544Z level=INFO msg="Uploading 1 narinfos"4764lb # [ 43.210393] niks3-server[990]: 2026/09/22 15:03:54 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4765lb # [ 43.214151] niks3-server[990]: 2026/09/22 15:03:54 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4766lb # [ 43.217387] niks3-server[990]: 2026/09/22 15:03:54 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4767lb # [ 43.222576] niks3-server[990]: 2026/09/22 15:03:54 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4768lb # [ 43.226726] niks3-server[990]: 2026/09/22 15:03:54 INFO Completed upload id=64769worker1 # [ 43.221292] nix-grpc-daemon[880]: time=2026-09-22T15:03:54.562Z level=INFO msg="Upload complete. (174ms)"4770lb # [ 43.228673] niks3-server[990]: 2026/09/22 15:03:54 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4771worker1 # [ 43.224283] nix-grpc-daemon[783]: ts=2026-09-22T15:03:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14772worker1 # [ 43.233262] nix-daemon[865]: reaped child process 979, status = succeeded4773client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 5.80 seconds)4774client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4775client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds)4776client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4777client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4778client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4779client # copying 3 paths...4780client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4781client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4782client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4783client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.14 seconds)4784(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.10 seconds)4785subtest: repeat is answered Cached by the scheduler without building4786worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4787worker1 # finding garbage collector roots...4788worker1 # deleting specified paths...4789worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4790worker1 # deleting unused links...4791worker1 # note: hard linking is currently saving 0.0 KiB4792worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4793worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4794worker2 # finding garbage collector roots...4795worker2 # deleting specified paths...4796worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4797worker2 # deleting unused links...4798worker2 # note: hard linking is currently saving 0.0 KiB4799worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.15 seconds)4800client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24801worker1 # [ 44.005700] nix-grpc-daemon[783]: ts=2026-09-22T15:03:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608984802client # this path will be fetched (0.0 KiB download, 0.2 KiB unpacked):4803client # /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4804worker1 # [ 44.017029] nix-grpc-daemon[783]: ts=2026-09-22T15:03:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608984805worker1 # [ 44.023259] nix-grpc-daemon[880]: time=2026-09-22T15:03:55.364Z level=INFO msg="All 2 paths already cached"4806worker1 # [ 44.033524] nix-grpc-daemon[783]: ts=2026-09-22T15:03:55Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14807client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.19 seconds)4808client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4809client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.09 seconds)4810lb: must succeed: curl -sf localhost:9901/clusters4811lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4812lb: must succeed: curl -sf localhost:9901/clusters4813lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4814worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4815worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4816(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.72 seconds)4817subtest: inputs already in the cache are not sent to the scheduler4818lb: must succeed: curl -sf localhost:9901/clusters4819lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4820lb: must succeed: curl -sf localhost:9901/clusters4821lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4822worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4823worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.02 seconds)4824client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24825worker2 # [ 44.501334] nix-grpc-daemon[782]: ts=2026-09-22T15:03:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395484826client # don't know how to build these paths:4827client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4828worker2 # [ 44.507983] nix-grpc-daemon[782]: ts=2026-09-22T15:03:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395484829worker1 # [ 44.515995] nix-grpc-daemon[783]: ts=2026-09-22T15:03:55Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04830worker1 # [ 44.520306] nix-grpc-daemon[783]: ts=2026-09-22T15:03:55Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14831worker1 # [ 44.523835] nix-grpc-daemon[783]: ts=2026-09-22T15:03:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608984832worker1 # [ 44.532310] nix-grpc-daemon[783]: ts=2026-09-22T15:03:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608984833worker1 # [ 44.540941] nix-grpc-daemon[880]: time=2026-09-22T15:03:55.882Z level=INFO msg="All 2 paths already cached"4834client # copying 1 paths to 'lb:50051'...4835client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4836worker1 # [ 44.545601] nix-grpc-daemon[783]: ts=2026-09-22T15:03:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608984837worker1 # [ 44.551301] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4838worker1 # [ 44.644070] nix-daemon[865]: accepted connection from pid 1029, user nix-grpc-daemon (trusted)4839worker1 # [ 44.659623] nix-daemon[865]: reaped child process 1032, status = succeeded4840worker1 # [ 44.745771] nix-daemon[865]: accepted connection from pid 1034, user nix-grpc-daemon (trusted)4841worker1 # [ 44.760742] nix-daemon[865]: reaped child process 1037, status = succeeded4842lb # [ 44.771395] niks3-server[990]: 2026/09/22 15:03:56 INFO Received uploads request method=POST path=/api/pending_closures4843worker1 # [ 44.776226] nix-grpc-daemon[880]: time=2026-09-22T15:03:56.117Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4844worker1 # [ 44.777604] nix-grpc-daemon[880]: time=2026-09-22T15:03:56.117Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4845lb # [ 44.798137] niks3-server[990]: 2026/09/22 15:03:56 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4846lb # [ 44.808611] niks3-server[990]: 2026/09/22 15:03:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4847worker1 # [ 44.810964] nix-grpc-daemon[880]: time=2026-09-22T15:03:56.152Z level=INFO msg="Uploading 1 narinfos"4848lb # [ 44.816770] niks3-server[990]: 2026/09/22 15:03:56 INFO Signed narinfos id=7 count=14849lb # [ 44.821961] niks3-server[990]: 2026/09/22 15:03:56 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4850lb # [ 44.833920] niks3-server[990]: 2026/09/22 15:03:56 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4851lb # [ 44.841449] niks3-server[990]: 2026/09/22 15:03:56 INFO Completed upload id=74852worker1 # [ 44.838558] nix-grpc-daemon[880]: time=2026-09-22T15:03:56.179Z level=INFO msg="Upload complete. (282ms)"4853lb # [ 44.845221] niks3-server[990]: 2026/09/22 15:03:56 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4854worker1 # [ 44.843086] nix-grpc-daemon[783]: ts=2026-09-22T15:03:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_in=7524855worker1 # [ 44.847553] nix-grpc-daemon[783]: ts=2026-09-22T15:03:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608984856worker1 # [ 44.857277] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4857worker1 # [ 44.864841] nix-daemon[865]: reaped child process 1027, status = succeeded4858client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4859client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4860worker1 # [ 44.868717] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)4861client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4862client # farm-top> LOG-farm-top-t1b4863worker1 # [ 45.683967] nix-daemon[865]: reaped child process 1041, status = succeeded4864worker1 # [ 45.765796] nix-daemon[865]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)4865worker1 # [ 45.779584] nix-daemon[865]: reaped child process 1060, status = succeeded4866lb # [ 45.791991] niks3-server[990]: 2026/09/22 15:03:57 INFO Received uploads request method=POST path=/api/pending_closures4867worker1 # [ 45.793032] nix-grpc-daemon[880]: time=2026-09-22T15:03:57.134Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4868worker1 # [ 45.794570] nix-grpc-daemon[880]: time=2026-09-22T15:03:57.136Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4869lb # [ 45.816070] niks3-server[990]: 2026/09/22 15:03:57 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4870lb # [ 45.820076] niks3-server[990]: 2026/09/22 15:03:57 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4871lb # [ 45.824188] niks3-server[990]: 2026/09/22 15:03:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4872lb # [ 45.826963] niks3-server[990]: 2026/09/22 15:03:57 INFO Signed narinfos id=8 count=14873worker1 # [ 45.821887] nix-grpc-daemon[880]: time=2026-09-22T15:03:57.163Z level=INFO msg="Uploading 1 narinfos"4874lb # [ 45.828751] niks3-server[990]: 2026/09/22 15:03:57 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4875lb # [ 45.835053] niks3-server[990]: 2026/09/22 15:03:57 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4876lb # [ 45.837851] niks3-server[990]: 2026/09/22 15:03:57 INFO Completed upload id=84877worker1 # [ 45.832364] nix-grpc-daemon[880]: time=2026-09-22T15:03:57.173Z level=INFO msg="Upload complete. (158ms)"4878lb # [ 45.839590] niks3-server[990]: 2026/09/22 15:03:57 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4879worker1 # [ 45.835083] nix-grpc-daemon[783]: ts=2026-09-22T15:03:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14880worker1 # [ 45.843632] nix-daemon[865]: reaped child process 1039, status = succeeded4881client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.51 seconds)4882client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4883client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.10 seconds)4884lb: must succeed: curl -sf localhost:9901/clusters4885lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4886lb: must succeed: curl -sf localhost:9901/clusters4887lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4888worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4889worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.05 seconds)4890(finished: subtest: inputs already in the cache are not sent to the scheduler, in 1.78 seconds)4891subtest: outputs read back through any worker carry the cache signature4892client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24893worker1 # [ 46.180881] nix-grpc-daemon[783]: ts=2026-09-22T15:03:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608984894client # don't know how to build these paths:4895client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4896worker2 # [ 46.197415] nix-grpc-daemon[782]: ts=2026-09-22T15:03:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395484897worker1 # [ 46.202616] nix-grpc-daemon[783]: ts=2026-09-22T15:03:57Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04898worker2 # [ 46.215832] nix-grpc-daemon[782]: ts=2026-09-22T15:03:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395484899worker1 # [ 46.207220] nix-grpc-daemon[783]: ts=2026-09-22T15:03:57Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.4:50051 clients=14900worker2 # [ 46.221764] nix-grpc-daemon[782]: ts=2026-09-22T15:03:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395484901client # copying 1 paths to 'lb:50051'...4902client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4903worker2 # [ 46.227610] nix-grpc-daemon[782]: ts=2026-09-22T15:03:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395484904worker2 # [ 46.233573] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4905worker2 # [ 46.245956] nix-grpc-daemon[782]: ts=2026-09-22T15:03:57Z level=info event=niks3_push_started pid=8784906worker2 # [ 46.372959] nix-grpc-daemon[878]: time=2026-09-22T15:03:57.596Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4907worker2 # [ 46.470460] nix-daemon[850]: accepted connection from pid 891, user nix-grpc-daemon (trusted)4908worker2 # [ 46.487616] nix-daemon[850]: reaped child process 894, status = succeeded4909lb # [ 46.523148] envoy[916]: conn peer=[2001:db8:1::1]:59984 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=54910lb # [ 46.524618] envoy[916]: conn peer=[2001:db8:1::1]:60000 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=54911lb # [ 46.526055] envoy[916]: conn peer=[2001:db8:1::1]:60012 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9477 tx=4364 ms=56724912lb # [ 46.527528] envoy[916]: conn peer=[2001:db8:1::1]:52170 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=478 ms=394913lb # [ 46.528913] envoy[916]: conn peer=[2001:db8:1::1]:52176 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4122 tx=1860 ms=13544914worker2 # [ 46.583062] nix-daemon[850]: accepted connection from pid 896, user nix-grpc-daemon (trusted)4915worker2 # [ 46.597054] nix-daemon[850]: reaped child process 899, status = succeeded4916lb # [ 46.600586] niks3-server[990]: 2026/09/22 15:03:57 INFO Received uploads request method=POST path=/api/pending_closures4917worker2 # [ 46.606938] nix-grpc-daemon[878]: time=2026-09-22T15:03:57.830Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4918worker2 # [ 46.610666] nix-grpc-daemon[878]: time=2026-09-22T15:03:57.832Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4919lb # [ 46.621904] niks3-server[990]: 2026/09/22 15:03:57 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4920lb # [ 46.624205] niks3-server[990]: 2026/09/22 15:03:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4921lb # [ 46.627613] niks3-server[990]: 2026/09/22 15:03:57 INFO Signed narinfos id=9 count=14922worker2 # [ 46.629912] nix-grpc-daemon[878]: time=2026-09-22T15:03:57.853Z level=INFO msg="Uploading 1 narinfos"4923lb # [ 46.630466] niks3-server[990]: 2026/09/22 15:03:57 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4924lb # [ 46.635513] niks3-server[990]: 2026/09/22 15:03:57 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4925lb # [ 46.638890] niks3-server[990]: 2026/09/22 15:03:57 INFO Completed upload id=94926worker2 # [ 46.640908] nix-grpc-daemon[878]: time=2026-09-22T15:03:57.864Z level=INFO msg="Upload complete. (263ms)"4927worker2 # [ 46.642812] nix-grpc-daemon[782]: ts=2026-09-22T15:03:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_in=5044928lb # [ 46.643261] niks3-server[990]: 2026/09/22 15:03:57 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4929worker2 # [ 46.650339] nix-grpc-daemon[782]: ts=2026-09-22T15:03:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395484930worker2 # [ 46.652357] nix-daemon[850]: reaped child process 876, status = succeeded4931worker2 # [ 46.654826] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4932worker2 # [ 46.661142] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4933client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4934client # farm-a> worker2: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4935client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4936client # farm-a> LOG-farm-a-sig4937worker2 # [ 47.885239] nix-daemon[850]: reaped child process 903, status = succeeded4938worker2 # [ 47.958751] nix-daemon[850]: accepted connection from pid 919, user nix-grpc-daemon (trusted)4939worker2 # [ 47.971374] nix-daemon[850]: reaped child process 922, status = succeeded4940lb # [ 47.976807] niks3-server[990]: 2026/09/22 15:03:59 INFO Received uploads request method=POST path=/api/pending_closures4941worker2 # [ 47.983321] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.206Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4942worker2 # [ 47.985107] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.208Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4943lb # [ 48.000278] niks3-server[990]: 2026/09/22 15:03:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4944lb # [ 48.001992] niks3-server[990]: 2026/09/22 15:03:59 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4945worker2 # [ 48.009876] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.233Z level=INFO msg="Uploading 1 narinfos"4946lb # [ 48.007782] niks3-server[990]: 2026/09/22 15:03:59 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4947lb # [ 48.010825] niks3-server[990]: 2026/09/22 15:03:59 INFO Signed narinfos id=10 count=14948lb # [ 48.015760] niks3-server[990]: 2026/09/22 15:03:59 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4949lb # [ 48.021172] niks3-server[990]: 2026/09/22 15:03:59 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4950lb # [ 48.024286] niks3-server[990]: 2026/09/22 15:03:59 INFO Completed upload id=104951worker2 # [ 48.026261] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.250Z level=INFO msg="Upload complete. (156ms)"4952lb # [ 48.026647] niks3-server[990]: 2026/09/22 15:03:59 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4953worker2 # [ 48.029033] nix-grpc-daemon[782]: ts=2026-09-22T15:03:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=1 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14954worker2 # [ 48.035725] nix-daemon[850]: reaped child process 901, status = succeeded4955worker1 # [ 48.030955] nix-grpc-daemon[783]: ts=2026-09-22T15:03:59Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04956worker1 # [ 48.035404] nix-grpc-daemon[783]: ts=2026-09-22T15:03:59Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.4:50051 clients=14957worker2 # [ 48.081781] nix-grpc-daemon[782]: ts=2026-09-22T15:03:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395484958worker2 # [ 48.093795] nix-grpc-daemon[782]: ts=2026-09-22T15:03:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395484959worker2 # [ 48.105789] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.329Z level=INFO msg="All 1 paths already cached"4960client # copying 1 paths to 'lb:50051'...4961client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4962worker2 # [ 48.113942] nix-grpc-daemon[782]: ts=2026-09-22T15:03:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395484963worker2 # [ 48.123164] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4964worker2 # [ 48.220163] nix-daemon[850]: accepted connection from pid 926, user nix-grpc-daemon (trusted)4965worker2 # [ 48.235102] nix-daemon[850]: reaped child process 929, status = succeeded4966worker2 # [ 48.316627] nix-daemon[850]: accepted connection from pid 931, user nix-grpc-daemon (trusted)4967worker2 # [ 48.329867] nix-daemon[850]: reaped child process 934, status = succeeded4968lb # [ 48.333981] niks3-server[990]: 2026/09/22 15:03:59 INFO Received uploads request method=POST path=/api/pending_closures4969worker2 # [ 48.343647] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.567Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4970worker2 # [ 48.345184] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.568Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4971lb # [ 48.365784] niks3-server[990]: 2026/09/22 15:03:59 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4972lb # [ 48.375277] niks3-server[990]: 2026/09/22 15:03:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign4973worker2 # [ 48.382120] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.605Z level=INFO msg="Uploading 1 narinfos"4974lb # [ 48.380498] niks3-server[990]: 2026/09/22 15:03:59 INFO Signed narinfos id=11 count=14975lb # [ 48.386198] niks3-server[990]: 2026/09/22 15:03:59 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4976lb # [ 48.399737] niks3-server[990]: 2026/09/22 15:03:59 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete4977lb # [ 48.406260] niks3-server[990]: 2026/09/22 15:03:59 INFO Completed upload id=114978worker2 # [ 48.408064] nix-grpc-daemon[878]: time=2026-09-22T15:03:59.631Z level=INFO msg="Upload complete. (274ms)"4979worker2 # [ 48.412101] nix-grpc-daemon[782]: ts=2026-09-22T15:03:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_in=6244980lb # [ 48.411105] niks3-server[990]: 2026/09/22 15:03:59 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4981worker2 # [ 48.417566] nix-grpc-daemon[782]: ts=2026-09-22T15:03:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395484982worker2 # [ 48.427122] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4983worker2 # [ 48.432322] nix-daemon[850]: reaped child process 924, status = succeeded4984client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4985worker2 # [ 48.437353] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4986client # farm-b> worker2: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4987client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4988client # farm-b> LOG-farm-b-sig4989worker2 # [ 49.474215] nix-daemon[850]: reaped child process 938, status = succeeded4990worker2 # [ 49.557941] nix-daemon[850]: accepted connection from pid 954, user nix-grpc-daemon (trusted)4991worker2 # [ 49.571483] nix-daemon[850]: reaped child process 957, status = succeeded4992lb # [ 49.578681] niks3-server[990]: 2026/09/22 15:04:00 INFO Received uploads request method=POST path=/api/pending_closures4993worker2 # [ 49.587050] nix-grpc-daemon[878]: time=2026-09-22T15:04:00.810Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4994worker2 # [ 49.588617] nix-grpc-daemon[878]: time=2026-09-22T15:04:00.812Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4995lb # [ 49.620737] niks3-server[990]: 2026/09/22 15:04:00 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4996lb # [ 49.625968] niks3-server[990]: 2026/09/22 15:04:00 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4997lb # [ 49.633312] niks3-server[990]: 2026/09/22 15:04:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign4998lb # [ 49.639444] niks3-server[990]: 2026/09/22 15:04:00 INFO Signed narinfos id=12 count=14999worker2 # [ 49.642197] nix-grpc-daemon[878]: time=2026-09-22T15:04:00.865Z level=INFO msg="Uploading 1 narinfos"5000lb # [ 49.648738] niks3-server[990]: 2026/09/22 15:04:00 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls5001lb # [ 49.656092] niks3-server[990]: 2026/09/22 15:04:00 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5002lb # [ 49.663111] niks3-server[990]: 2026/09/22 15:04:00 INFO Completed upload id=125003lb # [ 49.665068] niks3-server[990]: 2026/09/22 15:04:00 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo5004worker2 # [ 49.665403] nix-grpc-daemon[878]: time=2026-09-22T15:04:00.889Z level=INFO msg="Upload complete. (220ms)"5005worker2 # [ 49.671959] nix-grpc-daemon[782]: ts=2026-09-22T15:04:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=15006worker1 # [ 49.670870] nix-grpc-daemon[783]: ts=2026-09-22T15:04:01Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=05007worker1 # [ 49.675675] nix-grpc-daemon[783]: ts=2026-09-22T15:04:01Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.4:50051 clients=15008worker2 # [ 49.688152] nix-daemon[850]: reaped child process 936, status = succeeded5009worker2 # [ 49.721273] nix-grpc-daemon[782]: ts=2026-09-22T15:04:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485010worker2 # [ 49.733092] nix-grpc-daemon[782]: ts=2026-09-22T15:04:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395485011worker2 # [ 49.745575] nix-grpc-daemon[878]: time=2026-09-22T15:04:00.969Z level=INFO msg="All 2 paths already cached"5012client # copying 1 paths to 'lb:50051'...5013worker2 # [ 49.754876] nix-grpc-daemon[782]: ts=2026-09-22T15:04:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395485014client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...5015worker2 # [ 49.777708] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5016worker2 # [ 49.881443] nix-daemon[850]: accepted connection from pid 961, user nix-grpc-daemon (trusted)5017worker2 # [ 49.895737] nix-daemon[850]: reaped child process 964, status = succeeded5018worker2 # [ 49.977894] nix-daemon[850]: accepted connection from pid 966, user nix-grpc-daemon (trusted)5019worker2 # [ 49.990553] nix-daemon[850]: reaped child process 969, status = succeeded5020lb # [ 49.996437] niks3-server[990]: 2026/09/22 15:04:01 INFO Received uploads request method=POST path=/api/pending_closures5021worker2 # [ 50.007071] nix-grpc-daemon[878]: time=2026-09-22T15:04:01.230Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5022worker2 # [ 50.008915] nix-grpc-daemon[878]: time=2026-09-22T15:04:01.232Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"5023lb # [ 50.029833] niks3-server[990]: 2026/09/22 15:04:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5024lb # [ 50.035102] niks3-server[990]: 2026/09/22 15:04:01 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst5025worker2 # [ 50.044837] nix-grpc-daemon[878]: time=2026-09-22T15:04:01.268Z level=INFO msg="Uploading 1 narinfos"5026lb # [ 50.042697] niks3-server[990]: 2026/09/22 15:04:01 INFO Signed narinfos id=13 count=15027lb # [ 50.050101] niks3-server[990]: 2026/09/22 15:04:01 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls5028lb # [ 50.063554] niks3-server[990]: 2026/09/22 15:04:01 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5029lb # [ 50.071077] niks3-server[990]: 2026/09/22 15:04:01 INFO Completed upload id=135030lb # [ 50.074231] niks3-server[990]: 2026/09/22 15:04:01 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo5031worker2 # [ 50.075898] nix-grpc-daemon[878]: time=2026-09-22T15:04:01.299Z level=INFO msg="Upload complete. (284ms)"5032worker2 # [ 50.080575] nix-grpc-daemon[782]: ts=2026-09-22T15:04:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_in=7525033worker2 # [ 50.086728] nix-grpc-daemon[782]: ts=2026-09-22T15:04:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485034worker2 # [ 50.096515] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5035worker2 # [ 50.106199] nix-daemon[850]: reaped child process 959, status = succeeded5036client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...5037worker2 # [ 50.111120] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5038client # farm-top> worker2: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5039client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5040client # farm-top> LOG-farm-top-sig5041worker2 # [ 51.350063] nix-daemon[850]: reaped child process 973, status = succeeded5042worker2 # [ 51.425783] nix-daemon[850]: accepted connection from pid 989, user nix-grpc-daemon (trusted)5043worker2 # [ 51.438329] nix-daemon[850]: reaped child process 992, status = succeeded5044lb # [ 51.443342] niks3-server[990]: 2026/09/22 15:04:02 INFO Received uploads request method=POST path=/api/pending_closures5045worker2 # [ 51.453825] nix-grpc-daemon[878]: time=2026-09-22T15:04:02.677Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5046worker2 # [ 51.455437] nix-grpc-daemon[878]: time=2026-09-22T15:04:02.679Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"5047lb # [ 51.486154] niks3-server[990]: 2026/09/22 15:04:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5048lb # [ 51.491549] niks3-server[990]: 2026/09/22 15:04:02 INFO Signed narinfos id=14 count=15049worker2 # [ 51.497534] nix-grpc-daemon[878]: time=2026-09-22T15:04:02.721Z level=INFO msg="Uploading 1 narinfos"5050lb # [ 51.496693] niks3-server[990]: 2026/09/22 15:04:02 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst5051lb # [ 51.503967] niks3-server[990]: 2026/09/22 15:04:02 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv5052lb # [ 51.512123] niks3-server[990]: 2026/09/22 15:04:02 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls5053lb # [ 51.519595] niks3-server[990]: 2026/09/22 15:04:02 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5054lb # [ 51.526143] niks3-server[990]: 2026/09/22 15:04:02 INFO Completed upload id=145055lb # [ 51.527364] niks3-server[990]: 2026/09/22 15:04:02 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo5056worker2 # [ 51.528420] nix-grpc-daemon[878]: time=2026-09-22T15:04:02.752Z level=INFO msg="Upload complete. (209ms)"5057worker2 # [ 51.534934] nix-grpc-daemon[782]: ts=2026-09-22T15:04:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=15058worker2 # [ 51.549393] nix-daemon[850]: reaped child process 971, status = succeeded5059client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 5.52 seconds)5060client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath5061client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.11 seconds)5062client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5063worker1 # [ 51.775696] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:553765064client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):5065client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a5066client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b5067client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5068worker1 # [ 51.819891] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:553765069worker1 # [ 51.823244] nix-grpc-daemon[783]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...5070worker1 # [ 51.833216] nix-grpc-daemon[783]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...5071worker1 # [ 51.840810] nix-grpc-daemon[783]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...5072client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.19 seconds)5073client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5074worker2 # [ 51.983175] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:514685075worker2 # [ 51.989480] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:514685076client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.18 seconds)5077client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top5078client # finding garbage collector roots...5079client # removing stale temporary roots file "/nix/var/nix/temproots/868"5080client # removing stale temporary roots file "/nix/var/nix/temproots/848"5081client # removing stale temporary roots file "/nix/var/nix/temproots/860"5082client # removing stale temporary roots file "/nix/var/nix/temproots/904"5083client # removing stale temporary roots file "/nix/var/nix/temproots/827"5084client # removing stale temporary roots file "/nix/var/nix/temproots/889"5085client # removing stale temporary roots file "/nix/var/nix/temproots/881"5086client # removing stale temporary roots file "/nix/var/nix/temproots/811"5087client # deleting specified paths...5088client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'5089client # deleting unused links...5090client # note: hard linking is currently saving 0.0 KiB5091worker2 # [ 52.284360] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395485092worker1 # [ 52.280382] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985093worker1 # [ 52.289960] nix-grpc-daemon[880]: time=2026-09-22T15:04:03.630Z level=INFO msg="All 1 paths already cached"5094worker2 # [ 52.302877] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395485095worker2 # [ 52.310095] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395485096client # copying 3 paths...5097worker2 # [ 52.320115] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395485098client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...5099worker2 # [ 52.336371] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395485100worker1 # [ 52.335978] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608985101worker2 # [ 52.341823] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_out=2325102worker1 # [ 52.340799] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_out=1765103worker2 # [ 52.350166] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395485104worker2 # [ 52.354930] nix-grpc-daemon[782]: ts=2026-09-22T15:04:03Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_out=1285105client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...5106client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...5107client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.38 seconds)5108(finished: subtest: outputs read back through any worker carry the cache signature, in 6.38 seconds)5109subtest: two clients wanting the same drv build it once5110client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5111client # [ 52.464483] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5112client # Running as unit: dedup1.service; invocation ID: ac21f476d45a4d59b011210af565c1fd5113client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)5114client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup5115worker1 # [ 52.575375] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608985116client # [ 52.595560] nix[957]: don't know how to build these paths:5117client # [ 52.597133] nix[957]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5118worker1 # [ 52.588723] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05119worker1 # [ 52.592902] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.3:50051 clients=15120worker1 # [ 52.598271] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985121worker1 # [ 52.607969] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985122client # [ 52.631770] nix[957]: copying 1 paths to 'lb:50051'...5123client # [ 52.632707] nix[957]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...5124worker1 # [ 52.617790] nix-grpc-daemon[783]: ts=2026-09-22T15:04:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608985125worker1 # [ 52.626798] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5126worker1 # [ 52.726343] nix-daemon[865]: accepted connection from pid 1080, user nix-grpc-daemon (trusted)5127worker1 # [ 52.741527] nix-daemon[865]: reaped child process 1083, status = succeeded5128worker1 # [ 52.829447] nix-daemon[865]: accepted connection from pid 1085, user nix-grpc-daemon (trusted)5129worker1 # [ 52.843454] nix-daemon[865]: reaped child process 1088, status = succeeded5130lb # [ 52.857818] niks3-server[990]: 2026/09/22 15:04:04 INFO Received uploads request method=POST path=/api/pending_closures5131worker1 # [ 52.858522] nix-grpc-daemon[880]: time=2026-09-22T15:04:04.200Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5132worker1 # [ 52.860084] nix-grpc-daemon[880]: time=2026-09-22T15:04:04.201Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"5133lb # [ 52.889727] niks3-server[990]: 2026/09/22 15:04:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5134lb # [ 52.895067] niks3-server[990]: 2026/09/22 15:04:04 INFO Signed narinfos id=15 count=15135worker1 # [ 52.893160] nix-grpc-daemon[880]: time=2026-09-22T15:04:04.234Z level=INFO msg="Uploading 1 narinfos"5136lb # [ 52.904884] niks3-server[990]: 2026/09/22 15:04:04 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst5137lb # [ 52.911115] niks3-server[990]: 2026/09/22 15:04:04 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls5138lb # [ 52.929646] niks3-server[990]: 2026/09/22 15:04:04 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5139lb # [ 52.937557] niks3-server[990]: 2026/09/22 15:04:04 INFO Completed upload id=155140lb # [ 52.940228] niks3-server[990]: 2026/09/22 15:04:04 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo5141worker1 # [ 52.932392] nix-grpc-daemon[880]: time=2026-09-22T15:04:04.273Z level=INFO msg="Upload complete. (296ms)"5142worker1 # [ 52.937880] nix-grpc-daemon[783]: ts=2026-09-22T15:04:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_in=4325143worker1 # [ 52.944905] nix-grpc-daemon[783]: ts=2026-09-22T15:04:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985144worker1 # [ 52.955566] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5145worker1 # [ 52.966188] nix-daemon[865]: reaped child process 1078, status = succeeded5146worker1 # [ 52.971940] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5147client # [ 52.985653] nix[957]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5148client # [ 53.516661] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.5149client # Running as unit: dedup2.service; invocation ID: 8002bc736539413c8032aacf5163fa915150client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.06 seconds)5151worker2 # [ 53.673269] nix-grpc-daemon[782]: ts=2026-09-22T15:04:04Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395485152client # [ 53.687155] nix[973]: don't know how to build these paths:5153client # [ 53.688128] nix[973]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5154worker1 # [ 53.677527] nix-grpc-daemon[783]: ts=2026-09-22T15:04:05Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=05155worker1 # [ 53.681913] nix-grpc-daemon[783]: ts=2026-09-22T15:04:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985156client # [ 53.699376] nix[973]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...5157client: must succeed: sleep 25158lb # [ 56.523490] envoy[916]: conn peer=[2001:db8:1::1]:52190 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9468 tx=4360 ms=53615159lb # [ 56.527420] envoy[916]: conn peer=[2001:db8:1::1]:52200 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1051 tx=2887 ms=1385160lb # [ 56.531266] envoy[916]: conn peer=[2001:db8:1::1]:52218 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=330 ms=705161lb # [ 56.535488] envoy[916]: conn peer=[2001:db8:1::1]:52206 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=410 tx=410 ms=845162lb # [ 56.539076] envoy[916]: conn peer=[2001:db8:1::1]:52210 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=795163client: (finished: must succeed: sleep 2, in 2.04 seconds)5164worker1 # [ 56.791132] nix-daemon[865]: reaped child process 1092, status = succeeded5165client: waiting for success: ! systemctl is-active dedup1 dedup25166worker1 # [ 56.872466] nix-daemon[865]: accepted connection from pid 1124, user nix-grpc-daemon (trusted)5167worker1 # [ 56.887387] nix-daemon[865]: reaped child process 1127, status = succeeded5168lb # [ 56.900255] niks3-server[990]: 2026/09/22 15:04:08 INFO Received uploads request method=POST path=/api/pending_closures5169worker1 # [ 56.904724] nix-grpc-daemon[880]: time=2026-09-22T15:04:08.245Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5170worker1 # [ 56.907069] nix-grpc-daemon[880]: time=2026-09-22T15:04:08.247Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"5171lb # [ 56.937083] niks3-server[990]: 2026/09/22 15:04:08 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv5172lb # [ 56.951504] niks3-server[990]: 2026/09/22 15:04:08 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst5173lb # [ 56.962986] niks3-server[990]: 2026/09/22 15:04:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5174lb # [ 56.969056] niks3-server[990]: 2026/09/22 15:04:08 INFO Signed narinfos id=16 count=15175worker1 # [ 56.966560] nix-grpc-daemon[880]: time=2026-09-22T15:04:08.307Z level=INFO msg="Uploading 1 narinfos"5176lb # [ 56.974531] niks3-server[990]: 2026/09/22 15:04:08 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls5177lb # [ 56.984679] niks3-server[990]: 2026/09/22 15:04:08 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5178lb # [ 56.987363] niks3-server[990]: 2026/09/22 15:04:08 INFO Completed upload id=165179worker1 # [ 56.983536] nix-grpc-daemon[880]: time=2026-09-22T15:04:08.324Z level=INFO msg="Upload complete. (201ms)"5180lb # [ 56.992419] niks3-server[990]: 2026/09/22 15:04:08 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo5181worker1 # [ 56.991332] nix-grpc-daemon[783]: ts=2026-09-22T15:04:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15182worker1 # [ 57.000308] nix-grpc-daemon[783]: ts=2026-09-22T15:04:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=15183worker1 # [ 57.012661] nix-daemon[865]: reaped child process 1090, status = succeeded5184client # [ 57.057341] systemd[1]: dedup2.service: Deactivated successfully.5185client # [ 57.060378] systemd[1]: dedup2.service: Consumed 138ms CPU time over 3.540s wall clock time, 35.4M memory peak, 6.1K incoming IP traffic, 5.7K outgoing IP traffic.5186client # [ 57.066309] systemd[1]: dedup1.service: Deactivated successfully.5187client # [ 57.069543] systemd[1]: dedup1.service: Consumed 114ms CPU time over 4.604s wall clock time, 43.5M memory peak, 7.3K incoming IP traffic, 8.5K outgoing IP traffic.5188client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)5189client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5190client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.03 seconds)5191worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5192worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.06 seconds)5193worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5194worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5195worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5196worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5197worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5198worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5199(finished: subtest: two clients wanting the same drv build it once, in 5.68 seconds)5200subtest: --timeout and --max-silent-time reach the builder5201worker2 # [ 58.256613] nix-grpc-daemon[782]: ts=2026-09-22T15:04:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395485202worker1 # [ 58.259188] nix-grpc-daemon[783]: ts=2026-09-22T15:04:09Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=05203worker1 # [ 58.260948] nix-grpc-daemon[783]: ts=2026-09-22T15:04:09Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.4:50051 clients=15204worker2 # [ 58.267719] nix-grpc-daemon[782]: ts=2026-09-22T15:04:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485205worker2 # [ 58.276795] nix-grpc-daemon[782]: ts=2026-09-22T15:04:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395485206worker2 # [ 58.285529] nix-grpc-daemon[782]: ts=2026-09-22T15:04:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395485207worker2 # [ 58.296347] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5208worker2 # [ 58.398179] nix-daemon[850]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)5209worker2 # [ 58.412688] nix-daemon[850]: reaped child process 1020, status = succeeded5210worker2 # [ 58.493197] nix-daemon[850]: accepted connection from pid 1022, user nix-grpc-daemon (trusted)5211worker2 # [ 58.506210] nix-daemon[850]: reaped child process 1025, status = succeeded5212lb # [ 58.510902] niks3-server[990]: 2026/09/22 15:04:09 INFO Received uploads request method=POST path=/api/pending_closures5213worker2 # [ 58.517257] nix-grpc-daemon[878]: time=2026-09-22T15:04:09.740Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5214worker2 # [ 58.518595] nix-grpc-daemon[878]: time=2026-09-22T15:04:09.740Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"5215lb # [ 58.523565] niks3-server[990]: 2026/09/22 15:04:09 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst5216lb # [ 58.526262] niks3-server[990]: 2026/09/22 15:04:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5217lb # [ 58.528315] niks3-server[990]: 2026/09/22 15:04:09 INFO Signed narinfos id=17 count=15218worker2 # [ 58.531607] nix-grpc-daemon[878]: time=2026-09-22T15:04:09.755Z level=INFO msg="Uploading 1 narinfos"5219lb # [ 58.532565] niks3-server[990]: 2026/09/22 15:04:09 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls5220lb # [ 58.537062] niks3-server[990]: 2026/09/22 15:04:09 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5221lb # [ 58.540948] niks3-server[990]: 2026/09/22 15:04:09 INFO Completed upload id=175222worker2 # [ 58.542692] nix-grpc-daemon[878]: time=2026-09-22T15:04:09.766Z level=INFO msg="Upload complete. (235ms)"5223worker2 # [ 58.544684] nix-grpc-daemon[782]: ts=2026-09-22T15:04:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_in=4245224lb # [ 58.543584] niks3-server[990]: 2026/09/22 15:04:09 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo5225worker2 # [ 58.547906] nix-grpc-daemon[782]: ts=2026-09-22T15:04:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485226worker2 # [ 58.554190] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5227worker2 # [ 58.557697] nix-daemon[850]: reaped child process 1015, status = succeeded5228worker2 # [ 58.562091] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5229worker2 # [ 64.974880] nix-grpc-daemon[782]: ts=2026-09-22T15:04:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=05230worker2 # [ 64.986282] nix-daemon[850]: reaped child process 1029, status = succeeded5231worker2 # [ 64.987263] nix-daemon[850]: reaped child process 1027, status = succeeded5232worker1 # [ 65.099630] nix-grpc-daemon[783]: ts=2026-09-22T15:04:16Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608985233worker1 # [ 65.105791] nix-grpc-daemon[783]: ts=2026-09-22T15:04:16Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=05234worker1 # [ 65.107912] nix-grpc-daemon[783]: ts=2026-09-22T15:04:16Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.3:50051 clients=15235worker1 # [ 65.110428] nix-grpc-daemon[783]: ts=2026-09-22T15:04:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985236worker1 # [ 65.118193] nix-grpc-daemon[783]: ts=2026-09-22T15:04:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985237worker1 # [ 65.125411] nix-grpc-daemon[783]: ts=2026-09-22T15:04:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608985238worker1 # [ 65.130784] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5239worker1 # [ 65.224852] nix-daemon[865]: accepted connection from pid 1144, user nix-grpc-daemon (trusted)5240worker1 # [ 65.241435] nix-daemon[865]: reaped child process 1147, status = succeeded5241worker1 # [ 65.326767] nix-daemon[865]: accepted connection from pid 1149, user nix-grpc-daemon (trusted)5242worker1 # [ 65.341201] nix-daemon[865]: reaped child process 1152, status = succeeded5243lb # [ 65.353211] niks3-server[990]: 2026/09/22 15:04:16 INFO Received uploads request method=POST path=/api/pending_closures5244worker1 # [ 65.356393] nix-grpc-daemon[880]: time=2026-09-22T15:04:16.697Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5245worker1 # [ 65.357793] nix-grpc-daemon[880]: time=2026-09-22T15:04:16.697Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"5246lb # [ 65.379925] niks3-server[990]: 2026/09/22 15:04:16 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst5247lb # [ 65.384860] niks3-server[990]: 2026/09/22 15:04:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5248lb # [ 65.386839] niks3-server[990]: 2026/09/22 15:04:16 INFO Signed narinfos id=18 count=15249worker1 # [ 65.383556] nix-grpc-daemon[880]: time=2026-09-22T15:04:16.724Z level=INFO msg="Uploading 1 narinfos"5250lb # [ 65.392320] niks3-server[990]: 2026/09/22 15:04:16 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls5251lb # [ 65.398063] niks3-server[990]: 2026/09/22 15:04:16 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5252lb # [ 65.401634] niks3-server[990]: 2026/09/22 15:04:16 INFO Completed upload id=185253worker1 # [ 65.396392] nix-grpc-daemon[880]: time=2026-09-22T15:04:16.737Z level=INFO msg="Upload complete. (260ms)"5254worker1 # [ 65.398962] nix-grpc-daemon[783]: ts=2026-09-22T15:04:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_in=4245255lb # [ 65.405212] niks3-server[990]: 2026/09/22 15:04:16 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo5256worker1 # [ 65.402613] nix-grpc-daemon[783]: ts=2026-09-22T15:04:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985257worker1 # [ 65.410668] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5258worker1 # [ 65.415121] nix-daemon[865]: reaped child process 1142, status = succeeded5259worker1 # [ 65.420531] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5260lb # [ 66.523061] envoy[916]: conn peer=[2001:db8:1::1]:52230 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2998 tx=1537 ms=44335261lb # [ 66.525118] envoy[916]: conn peer=[2001:db8:1::1]:55146 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1197 ms=33465262lb # [ 66.526914] envoy[916]: conn peer=[2001:db8:1::1]:55152 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2953 tx=935 ms=67285263worker1 # [ 71.433559] nix-grpc-daemon[783]: ts=2026-09-22T15:04:22Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=05264worker1 # [ 71.445926] nix-daemon[865]: reaped child process 1156, status = succeeded5265worker1 # [ 71.447198] nix-daemon[865]: reaped child process 1154, status = succeeded5266(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.48 seconds)5267subtest: the first client leaving does not fail the second5268client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5269client # [ 71.648533] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5270client # Running as unit: share1.service; invocation ID: 834d4cdbc4464e7e87b03cb4626803d55271client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5272client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5273worker2 # [ 71.779747] nix-grpc-daemon[782]: ts=2026-09-22T15:04:23Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:395485274client # [ 71.793883] nix[1047]: don't know how to build these paths:5275client # [ 71.794817] nix[1047]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5276worker1 # [ 71.786461] nix-grpc-daemon[783]: ts=2026-09-22T15:04:23Z level=debug event=want client=9 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05277worker1 # [ 71.791434] nix-grpc-daemon[783]: ts=2026-09-22T15:04:23Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15278worker2 # [ 71.801253] nix-grpc-daemon[782]: ts=2026-09-22T15:04:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485279worker2 # [ 71.813445] nix-grpc-daemon[782]: ts=2026-09-22T15:04:23Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395485280client # [ 71.831074] nix[1047]: copying 1 paths to 'lb:50051'...5281client # [ 71.831807] nix[1047]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5282worker2 # [ 71.824459] nix-grpc-daemon[782]: ts=2026-09-22T15:04:23Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395485283worker2 # [ 71.835147] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5284worker2 # [ 71.938860] nix-daemon[850]: accepted connection from pid 1052, user nix-grpc-daemon (trusted)5285worker2 # [ 71.951881] nix-daemon[850]: reaped child process 1055, status = succeeded5286worker2 # [ 72.036064] nix-daemon[850]: accepted connection from pid 1057, user nix-grpc-daemon (trusted)5287worker2 # [ 72.049154] nix-daemon[850]: reaped child process 1060, status = succeeded5288lb # [ 72.054901] niks3-server[990]: 2026/09/22 15:04:23 INFO Received uploads request method=POST path=/api/pending_closures5289worker2 # [ 72.066149] nix-grpc-daemon[878]: time=2026-09-22T15:04:23.289Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5290worker2 # [ 72.067546] nix-grpc-daemon[878]: time=2026-09-22T15:04:23.289Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5291lb # [ 72.085116] niks3-server[990]: 2026/09/22 15:04:23 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5292lb # [ 72.092270] niks3-server[990]: 2026/09/22 15:04:23 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5293lb # [ 72.100121] niks3-server[990]: 2026/09/22 15:04:23 INFO Signed narinfos id=19 count=15294worker2 # [ 72.105252] nix-grpc-daemon[878]: time=2026-09-22T15:04:23.328Z level=INFO msg="Uploading 1 narinfos"5295lb # [ 72.106956] niks3-server[990]: 2026/09/22 15:04:23 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5296lb # [ 72.121444] niks3-server[990]: 2026/09/22 15:04:23 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5297lb # [ 72.127110] niks3-server[990]: 2026/09/22 15:04:23 INFO Completed upload id=195298worker2 # [ 72.128981] nix-grpc-daemon[878]: time=2026-09-22T15:04:23.352Z level=INFO msg="Upload complete. (282ms)"5299lb # [ 72.130123] niks3-server[990]: 2026/09/22 15:04:23 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5300worker2 # [ 72.133280] nix-grpc-daemon[782]: ts=2026-09-22T15:04:23Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_in=4325301worker2 # [ 72.140120] nix-grpc-daemon[782]: ts=2026-09-22T15:04:23Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485302worker2 # [ 72.148712] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5303worker2 # [ 72.156885] nix-daemon[850]: reaped child process 1050, status = succeeded5304worker2 # [ 72.161669] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5305client # [ 72.168647] nix[1047]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5306client # [ 72.713942] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5307client # Running as unit: share2.service; invocation ID: e446a0eb30924db6ba3a8fb2d17af1805308client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5309client # [ 72.855954] nix[1063]: don't know how to build these paths:5310client # [ 72.856859] nix[1063]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5311worker1 # [ 72.839978] nix-grpc-daemon[783]: ts=2026-09-22T15:04:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608985312worker1 # [ 72.850421] nix-grpc-daemon[783]: ts=2026-09-22T15:04:24Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05313worker2 # [ 72.860885] nix-grpc-daemon[782]: ts=2026-09-22T15:04:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485314client # [ 72.873820] nix[1063]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5315client: must succeed: sleep 2; systemctl kill -s INT share15316client # [ 75.963481] systemd[1]: share1.service: Sent signal SIGINT to main process 1047 (nix) on client request.5317client # [ 75.971511] nix[1047]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5318client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.05 seconds)5319client: waiting for success: ! systemctl is-active share15320client # [ 75.978241] nix[1047]: error: interrupted by the user5321client # [ 76.018694] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5322client # [ 76.020212] systemd[1]: share1.service: Failed with result 'exit-code'.5323client # [ 76.022156] systemd[1]: share1.service: Consumed 115ms CPU time over 4.373s wall clock time, 41.5M memory peak, 6.1K incoming IP traffic, 8.2K outgoing IP traffic.5324lb # [ 76.524596] envoy[916]: conn peer=[2001:db8:1::1]:43050 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=933 ms=63905325lb # [ 76.528528] envoy[916]: conn peer=[2001:db8:1::1]:43056 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2982 tx=814 ms=42005326client: (finished: waiting for success: ! systemctl is-active share1, in 1.06 seconds)5327client: waiting for success: ! systemctl is-active share25328worker2 # [ 77.398393] nix-daemon[850]: reaped child process 1064, status = succeeded5329worker2 # [ 77.482561] nix-daemon[850]: accepted connection from pid 1098, user nix-grpc-daemon (trusted)5330worker2 # [ 77.496218] nix-daemon[850]: reaped child process 1101, status = succeeded5331lb # [ 77.500429] niks3-server[990]: 2026/09/22 15:04:28 INFO Received uploads request method=POST path=/api/pending_closures5332worker2 # [ 77.513585] nix-grpc-daemon[878]: time=2026-09-22T15:04:28.736Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5333worker2 # [ 77.516489] nix-grpc-daemon[878]: time=2026-09-22T15:04:28.738Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5334lb # [ 77.555616] niks3-server[990]: 2026/09/22 15:04:28 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5335lb # [ 77.560830] niks3-server[990]: 2026/09/22 15:04:28 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5336lb # [ 77.573229] niks3-server[990]: 2026/09/22 15:04:28 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5337worker2 # [ 77.581157] nix-grpc-daemon[878]: time=2026-09-22T15:04:28.804Z level=INFO msg="Uploading 1 narinfos"5338lb # [ 77.578885] niks3-server[990]: 2026/09/22 15:04:28 INFO Signed narinfos id=20 count=15339lb # [ 77.584690] niks3-server[990]: 2026/09/22 15:04:28 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5340lb # [ 77.590062] niks3-server[990]: 2026/09/22 15:04:28 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5341lb # [ 77.594619] niks3-server[990]: 2026/09/22 15:04:28 INFO Completed upload id=205342worker2 # [ 77.596625] nix-grpc-daemon[878]: time=2026-09-22T15:04:28.820Z level=INFO msg="Upload complete. (223ms)"5343lb # [ 77.596161] niks3-server[990]: 2026/09/22 15:04:28 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5344worker2 # [ 77.599460] nix-grpc-daemon[782]: ts=2026-09-22T15:04:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15345worker2 # [ 77.603405] nix-grpc-daemon[782]: ts=2026-09-22T15:04:28Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=11 outputs=15346worker2 # [ 77.616330] nix-daemon[850]: reaped child process 1062, status = succeeded5347client # [ 77.680470] systemd[1]: share2.service: Deactivated successfully.5348client # [ 77.683973] systemd[1]: share2.service: Consumed 119ms CPU time over 4.969s wall clock time, 37.5M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5349client: (finished: waiting for success: ! systemctl is-active share2, in 1.06 seconds)5350client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5351client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.02 seconds)5352(finished: subtest: the first client leaving does not fail the second, in 6.67 seconds)5353subtest: build hook: nix-daemon with builders = grpc://lb5354client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5355client # [ 78.367835] systemd[1]: Started Nix Daemon.5356client # [ 78.456496] nix-daemon[1114]: accepted connection from pid 1111, user root (trusted)5357worker1 # [ 78.562839] nix-grpc-daemon[783]: ts=2026-09-22T15:04:29Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:608985358worker1 # [ 78.569306] nix-grpc-daemon[783]: ts=2026-09-22T15:04:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985359worker1 # [ 78.585242] nix-grpc-daemon[783]: ts=2026-09-22T15:04:29Z level=debug event=want client=11 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05360worker1 # [ 78.590891] nix-grpc-daemon[783]: ts=2026-09-22T15:04:29Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15361worker1 # [ 78.596843] nix-grpc-daemon[783]: ts=2026-09-22T15:04:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985362worker1 # [ 78.608155] nix-grpc-daemon[783]: ts=2026-09-22T15:04:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985363worker1 # [ 78.619916] nix-grpc-daemon[783]: ts=2026-09-22T15:04:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608985364worker1 # [ 78.628824] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5365worker1 # [ 78.733831] nix-daemon[865]: accepted connection from pid 1201, user nix-grpc-daemon (trusted)5366worker1 # [ 78.748824] nix-daemon[865]: reaped child process 1204, status = succeeded5367worker1 # [ 78.838149] nix-daemon[865]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5368worker1 # [ 78.853178] nix-daemon[865]: reaped child process 1209, status = succeeded5369lb # [ 78.864799] niks3-server[990]: 2026/09/22 15:04:30 INFO Received uploads request method=POST path=/api/pending_closures5370worker1 # [ 78.869260] nix-grpc-daemon[880]: time=2026-09-22T15:04:30.210Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5371worker1 # [ 78.870800] nix-grpc-daemon[880]: time=2026-09-22T15:04:30.212Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5372lb # [ 78.896685] niks3-server[990]: 2026/09/22 15:04:30 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5373lb # [ 78.905529] niks3-server[990]: 2026/09/22 15:04:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5374lb # [ 78.910726] niks3-server[990]: 2026/09/22 15:04:30 INFO Signed narinfos id=21 count=15375worker1 # [ 78.908452] nix-grpc-daemon[880]: time=2026-09-22T15:04:30.249Z level=INFO msg="Uploading 1 narinfos"5376lb # [ 78.921319] niks3-server[990]: 2026/09/22 15:04:30 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5377lb # [ 78.929462] niks3-server[990]: 2026/09/22 15:04:30 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5378worker1 # [ 78.930450] nix-grpc-daemon[880]: time=2026-09-22T15:04:30.271Z level=INFO msg="Upload complete. (288ms)"5379lb # [ 78.936166] niks3-server[990]: 2026/09/22 15:04:30 INFO Completed upload id=215380worker1 # [ 78.932893] nix-grpc-daemon[783]: ts=2026-09-22T15:04:30Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_in=5125381lb # [ 78.939863] niks3-server[990]: 2026/09/22 15:04:30 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5382worker1 # [ 78.939175] nix-grpc-daemon[783]: ts=2026-09-22T15:04:30Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985383worker1 # [ 78.950069] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5384worker1 # [ 78.952741] nix-daemon[865]: reaped child process 1199, status = succeeded5385worker1 # [ 78.957702] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5386worker1 # [ 79.796457] nix-daemon[865]: reaped child process 1213, status = succeeded5387worker1 # [ 79.869070] nix-daemon[865]: accepted connection from pid 1230, user nix-grpc-daemon (trusted)5388worker1 # [ 79.883994] nix-daemon[865]: reaped child process 1233, status = succeeded5389lb # [ 79.893864] niks3-server[990]: 2026/09/22 15:04:31 INFO Received uploads request method=POST path=/api/pending_closures5390worker1 # [ 79.892985] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.234Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5391worker1 # [ 79.895278] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.236Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5392lb # [ 79.913516] niks3-server[990]: 2026/09/22 15:04:31 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5393lb # [ 79.915547] niks3-server[990]: 2026/09/22 15:04:31 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5394worker1 # [ 79.917313] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.258Z level=INFO msg="Uploading 1 narinfos"5395lb # [ 79.922398] niks3-server[990]: 2026/09/22 15:04:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5396lb # [ 79.924058] niks3-server[990]: 2026/09/22 15:04:31 INFO Signed narinfos id=22 count=15397lb # [ 79.930196] niks3-server[990]: 2026/09/22 15:04:31 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5398lb # [ 79.932165] niks3-server[990]: 2026/09/22 15:04:31 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5399lb # [ 79.937972] niks3-server[990]: 2026/09/22 15:04:31 INFO Completed upload id=225400worker1 # [ 79.932723] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.274Z level=INFO msg="Upload complete. (168ms)"5401lb # [ 79.939231] niks3-server[990]: 2026/09/22 15:04:31 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5402worker1 # [ 79.936644] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=12 outputs=15403worker1 # [ 79.944799] nix-daemon[865]: reaped child process 1211, status = succeeded5404worker2 # [ 79.950879] nix-grpc-daemon[782]: ts=2026-09-22T15:04:31Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395485405worker2 # [ 79.960882] nix-grpc-daemon[782]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5406worker2 # [ 79.983519] nix-grpc-daemon[782]: ts=2026-09-22T15:04:31Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395485407worker2 # [ 79.988687] nix-grpc-daemon[782]: ts=2026-09-22T15:04:31Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_out=1285408worker1 # [ 80.183913] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:608985409worker1 # [ 80.187589] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985410worker1 # [ 80.193527] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.535Z level=INFO msg="All 1 paths already cached"5411worker1 # [ 80.203475] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=debug event=want client=12 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05412worker1 # [ 80.205720] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15413worker1 # [ 80.208474] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985414worker1 # [ 80.215870] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985415worker1 # [ 80.223850] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.565Z level=INFO msg="All 1 paths already cached"5416worker1 # [ 80.228111] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608985417worker1 # [ 80.233072] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5418worker1 # [ 80.321286] nix-daemon[865]: accepted connection from pid 1239, user nix-grpc-daemon (trusted)5419worker1 # [ 80.337290] nix-daemon[865]: reaped child process 1242, status = succeeded5420worker1 # [ 80.421218] nix-daemon[865]: accepted connection from pid 1244, user nix-grpc-daemon (trusted)5421worker1 # [ 80.436535] nix-daemon[865]: reaped child process 1247, status = succeeded5422lb # [ 80.447099] niks3-server[990]: 2026/09/22 15:04:31 INFO Received uploads request method=POST path=/api/pending_closures5423worker1 # [ 80.450828] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.792Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5424worker1 # [ 80.452278] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.792Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5425lb # [ 80.477234] niks3-server[990]: 2026/09/22 15:04:31 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5426lb # [ 80.486410] niks3-server[990]: 2026/09/22 15:04:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5427lb # [ 80.492844] niks3-server[990]: 2026/09/22 15:04:31 INFO Signed narinfos id=23 count=15428worker1 # [ 80.490740] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.832Z level=INFO msg="Uploading 1 narinfos"5429lb # [ 80.497856] niks3-server[990]: 2026/09/22 15:04:31 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5430lb # [ 80.510410] niks3-server[990]: 2026/09/22 15:04:31 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5431worker1 # [ 80.512227] nix-grpc-daemon[880]: time=2026-09-22T15:04:31.853Z level=INFO msg="Upload complete. (273ms)"5432lb # [ 80.517612] niks3-server[990]: 2026/09/22 15:04:31 INFO Completed upload id=235433lb # [ 80.520378] niks3-server[990]: 2026/09/22 15:04:31 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5434worker1 # [ 80.515457] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_in=6245435worker1 # [ 80.521201] nix-grpc-daemon[783]: ts=2026-09-22T15:04:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985436worker1 # [ 80.530063] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5437worker1 # [ 80.535077] nix-daemon[865]: reaped child process 1237, status = succeeded5438worker1 # [ 80.539847] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5439worker1 # [ 81.575949] nix-daemon[865]: reaped child process 1251, status = succeeded5440worker1 # [ 81.661102] nix-daemon[865]: accepted connection from pid 1267, user nix-grpc-daemon (trusted)5441worker1 # [ 81.674349] nix-daemon[865]: reaped child process 1270, status = succeeded5442lb # [ 81.687819] niks3-server[990]: 2026/09/22 15:04:33 INFO Received uploads request method=POST path=/api/pending_closures5443worker1 # [ 81.692605] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.033Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5444worker1 # [ 81.694254] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.035Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5445lb # [ 81.725848] niks3-server[990]: 2026/09/22 15:04:33 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5446lb # [ 81.740698] niks3-server[990]: 2026/09/22 15:04:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5447lb # [ 81.750239] niks3-server[990]: 2026/09/22 15:04:33 INFO Signed narinfos id=24 count=15448worker1 # [ 81.748338] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.089Z level=INFO msg="Uploading 1 narinfos"5449lb # [ 81.753774] niks3-server[990]: 2026/09/22 15:04:33 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5450lb # [ 81.759397] niks3-server[990]: 2026/09/22 15:04:33 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5451lb # [ 81.770029] niks3-server[990]: 2026/09/22 15:04:33 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5452lb # [ 81.773616] niks3-server[990]: 2026/09/22 15:04:33 INFO Completed upload id=245453lb # [ 81.775955] niks3-server[990]: 2026/09/22 15:04:33 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5454worker1 # [ 81.769898] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.111Z level=INFO msg="Upload complete. (220ms)"5455worker1 # [ 81.776590] nix-grpc-daemon[783]: ts=2026-09-22T15:04:33Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=13 outputs=15456worker2 # [ 81.794491] nix-grpc-daemon[782]: ts=2026-09-22T15:04:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395485457worker1 # [ 81.792441] nix-daemon[865]: reaped child process 1249, status = succeeded5458worker2 # [ 81.803057] nix-grpc-daemon[782]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5459worker2 # [ 81.825471] nix-grpc-daemon[782]: ts=2026-09-22T15:04:33Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395485460worker2 # [ 81.830132] nix-grpc-daemon[782]: ts=2026-09-22T15:04:33Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_out=1765461worker2 # [ 82.033184] nix-grpc-daemon[782]: ts=2026-09-22T15:04:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:395485462worker2 # [ 82.040098] nix-grpc-daemon[782]: ts=2026-09-22T15:04:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395485463worker2 # [ 82.045817] nix-grpc-daemon[878]: time=2026-09-22T15:04:33.267Z level=INFO msg="All 2 paths already cached"5464worker1 # [ 82.044627] nix-grpc-daemon[783]: ts=2026-09-22T15:04:33Z level=debug event=want client=13 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05465worker1 # [ 82.047082] nix-grpc-daemon[783]: ts=2026-09-22T15:04:33Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15466worker1 # [ 82.049644] nix-grpc-daemon[783]: ts=2026-09-22T15:04:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985467worker1 # [ 82.056792] nix-grpc-daemon[783]: ts=2026-09-22T15:04:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985468worker1 # [ 82.065111] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.406Z level=INFO msg="All 2 paths already cached"5469worker1 # [ 82.069760] nix-grpc-daemon[783]: ts=2026-09-22T15:04:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608985470worker1 # [ 82.073156] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5471worker1 # [ 82.168364] nix-daemon[865]: accepted connection from pid 1276, user nix-grpc-daemon (trusted)5472worker1 # [ 82.185768] nix-daemon[865]: reaped child process 1279, status = succeeded5473worker1 # [ 82.265663] nix-daemon[865]: accepted connection from pid 1281, user nix-grpc-daemon (trusted)5474worker1 # [ 82.283141] nix-daemon[865]: reaped child process 1284, status = succeeded5475lb # [ 82.286675] niks3-server[990]: 2026/09/22 15:04:33 INFO Received uploads request method=POST path=/api/pending_closures5476worker1 # [ 82.289929] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.631Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5477worker1 # [ 82.291753] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.632Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5478lb # [ 82.315535] niks3-server[990]: 2026/09/22 15:04:33 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5479lb # [ 82.321546] niks3-server[990]: 2026/09/22 15:04:33 INFO Signed narinfos id=25 count=15480worker1 # [ 82.319488] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.660Z level=INFO msg="Uploading 1 narinfos"5481lb # [ 82.330114] niks3-server[990]: 2026/09/22 15:04:33 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5482lb # [ 82.335420] niks3-server[990]: 2026/09/22 15:04:33 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5483lb # [ 82.346260] niks3-server[990]: 2026/09/22 15:04:33 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5484lb # [ 82.354484] niks3-server[990]: 2026/09/22 15:04:33 INFO Completed upload id=255485worker1 # [ 82.349080] nix-grpc-daemon[880]: time=2026-09-22T15:04:33.690Z level=INFO msg="Upload complete. (268ms)"5486lb # [ 82.357095] niks3-server[990]: 2026/09/22 15:04:33 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5487worker1 # [ 82.352573] nix-grpc-daemon[783]: ts=2026-09-22T15:04:33Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_in=7525488worker1 # [ 82.358604] nix-grpc-daemon[783]: ts=2026-09-22T15:04:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985489worker1 # [ 82.367566] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5490worker1 # [ 82.370595] nix-daemon[865]: reaped child process 1274, status = succeeded5491worker1 # [ 82.376131] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5492worker1 # [ 83.200805] nix-daemon[865]: reaped child process 1288, status = succeeded5493worker1 # [ 83.277182] nix-daemon[865]: accepted connection from pid 1305, user nix-grpc-daemon (trusted)5494worker1 # [ 83.291651] nix-daemon[865]: reaped child process 1308, status = succeeded5495lb # [ 83.304209] niks3-server[990]: 2026/09/22 15:04:34 INFO Received uploads request method=POST path=/api/pending_closures5496worker1 # [ 83.311859] nix-grpc-daemon[880]: time=2026-09-22T15:04:34.653Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5497worker1 # [ 83.313372] nix-grpc-daemon[880]: time=2026-09-22T15:04:34.654Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5498lb # [ 83.346113] niks3-server[990]: 2026/09/22 15:04:34 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5499lb # [ 83.350345] niks3-server[990]: 2026/09/22 15:04:34 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5500lb # [ 83.361512] niks3-server[990]: 2026/09/22 15:04:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5501lb # [ 83.366174] niks3-server[990]: 2026/09/22 15:04:34 INFO Signed narinfos id=26 count=15502worker1 # [ 83.363211] nix-grpc-daemon[880]: time=2026-09-22T15:04:34.704Z level=INFO msg="Uploading 1 narinfos"5503lb # [ 83.372490] niks3-server[990]: 2026/09/22 15:04:34 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5504lb # [ 83.376560] niks3-server[990]: 2026/09/22 15:04:34 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5505lb # [ 83.379412] niks3-server[990]: 2026/09/22 15:04:34 INFO Completed upload id=265506lb # [ 83.381904] niks3-server[990]: 2026/09/22 15:04:34 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5507worker1 # [ 83.375461] nix-grpc-daemon[880]: time=2026-09-22T15:04:34.716Z level=INFO msg="Upload complete. (192ms)"5508worker1 # [ 83.383568] nix-grpc-daemon[783]: ts=2026-09-22T15:04:34Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=14 outputs=15509worker2 # [ 83.394814] nix-grpc-daemon[782]: ts=2026-09-22T15:04:34Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395485510worker1 # [ 83.396427] nix-daemon[865]: reaped child process 1286, status = succeeded5511worker2 # [ 83.403637] nix-grpc-daemon[782]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5512worker2 # [ 83.425695] nix-grpc-daemon[782]: ts=2026-09-22T15:04:34Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395485513worker2 # [ 83.430731] nix-grpc-daemon[782]: ts=2026-09-22T15:04:34Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_out=2325514client # [ 83.550410] nix-daemon[1114]: reaped child process 1116, status = succeeded5515client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.28 seconds)5516client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5517client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5518client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5519client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5520client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5521client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5522(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.34 seconds)5523subtest: an input only one worker has still reaches the builder5524client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5525client # this derivation will be built:5526client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5527client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5528client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)5529client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5530client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5531worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5532worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.10 seconds)5533worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5534worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5535client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25536client # [ 84.185559] nix-daemon[1114]: accepted connection from pid 1197, user root (trusted)5537client # this derivation will be built:5538client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5539worker2 # [ 84.290794] nix-grpc-daemon[782]: ts=2026-09-22T15:04:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:395485540client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5541worker1 # [ 84.287343] nix-grpc-daemon[783]: ts=2026-09-22T15:04:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985542worker1 # [ 84.375628] nix-daemon[865]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5543worker1 # [ 84.389151] nix-daemon[865]: reaped child process 1318, status = succeeded5544lb # [ 84.401773] niks3-server[990]: 2026/09/22 15:04:35 INFO Received uploads request method=POST path=/api/pending_closures5545worker1 # [ 84.402826] nix-grpc-daemon[880]: time=2026-09-22T15:04:35.744Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5546worker1 # [ 84.404602] nix-grpc-daemon[880]: time=2026-09-22T15:04:35.745Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5547lb # [ 84.426413] niks3-server[990]: 2026/09/22 15:04:35 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5548lb # [ 84.441081] niks3-server[990]: 2026/09/22 15:04:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5549lb # [ 84.447058] niks3-server[990]: 2026/09/22 15:04:35 INFO Signed narinfos id=27 count=15550worker1 # [ 84.444755] nix-grpc-daemon[880]: time=2026-09-22T15:04:35.786Z level=INFO msg="Uploading 1 narinfos"5551lb # [ 84.453684] niks3-server[990]: 2026/09/22 15:04:35 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5552lb # [ 84.465874] niks3-server[990]: 2026/09/22 15:04:35 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5553lb # [ 84.471700] niks3-server[990]: 2026/09/22 15:04:35 INFO Completed upload id=275554worker1 # [ 84.465941] nix-grpc-daemon[880]: time=2026-09-22T15:04:35.807Z level=INFO msg="Upload complete. (176ms)"5555lb # [ 84.474536] niks3-server[990]: 2026/09/22 15:04:35 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5556worker1 # [ 84.484348] nix-grpc-daemon[783]: ts=2026-09-22T15:04:35Z level=debug event=want client=14 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05557worker1 # [ 84.488551] nix-grpc-daemon[783]: ts=2026-09-22T15:04:35Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15558worker2 # [ 84.535167] nix-grpc-daemon[782]: ts=2026-09-22T15:04:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485559worker2 # [ 84.544035] nix-grpc-daemon[782]: ts=2026-09-22T15:04:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395485560client # copying 2 paths to 'lb:50051'...5561worker2 # [ 84.554333] nix-grpc-daemon[782]: ts=2026-09-22T15:04:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395485562client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5563client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5564worker2 # [ 84.569318] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5565worker2 # [ 84.684277] nix-daemon[850]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5566worker2 # [ 84.700717] nix-daemon[850]: reaped child process 1114, status = succeeded5567worker2 # [ 84.788862] nix-daemon[850]: accepted connection from pid 1116, user nix-grpc-daemon (trusted)5568worker2 # [ 84.805513] nix-daemon[850]: reaped child process 1119, status = succeeded5569lb # [ 84.808070] niks3-server[990]: 2026/09/22 15:04:36 INFO Received uploads request method=POST path=/api/pending_closures5570lb # [ 84.813936] niks3-server[990]: 2026/09/22 15:04:36 INFO Received uploads request method=POST path=/api/pending_closures5571worker2 # [ 84.821489] nix-grpc-daemon[878]: time=2026-09-22T15:04:36.044Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5572worker2 # [ 84.822881] nix-grpc-daemon[878]: time=2026-09-22T15:04:36.044Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5573worker2 # [ 84.824477] nix-grpc-daemon[878]: time=2026-09-22T15:04:36.044Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5574lb # [ 84.832669] niks3-server[990]: 2026/09/22 15:04:36 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5575lb # [ 84.839376] niks3-server[990]: 2026/09/22 15:04:36 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5576lb # [ 84.842621] niks3-server[990]: 2026/09/22 15:04:36 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5577lb # [ 84.847304] niks3-server[990]: 2026/09/22 15:04:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5578lb # [ 84.850732] niks3-server[990]: 2026/09/22 15:04:36 INFO Signed narinfos id=29 count=25579lb # [ 84.853467] niks3-server[990]: 2026/09/22 15:04:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5580lb # [ 84.855174] niks3-server[990]: 2026/09/22 15:04:36 INFO Signed narinfos id=28 count=15581worker2 # [ 84.858782] nix-grpc-daemon[878]: time=2026-09-22T15:04:36.082Z level=INFO msg="Uploading 3 narinfos"5582lb # [ 84.858657] niks3-server[990]: 2026/09/22 15:04:36 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5583lb # [ 84.871877] niks3-server[990]: 2026/09/22 15:04:36 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5584lb # [ 84.875296] niks3-server[990]: 2026/09/22 15:04:36 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5585lb # [ 84.878118] niks3-server[990]: 2026/09/22 15:04:36 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5586lb # [ 84.880308] niks3-server[990]: 2026/09/22 15:04:36 INFO Completed upload id=285587lb # [ 84.881682] niks3-server[990]: 2026/09/22 15:04:36 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5588lb # [ 84.884208] niks3-server[990]: 2026/09/22 15:04:36 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5589lb # [ 84.886248] niks3-server[990]: 2026/09/22 15:04:36 INFO Completed upload id=295590worker2 # [ 84.888259] nix-grpc-daemon[878]: time=2026-09-22T15:04:36.111Z level=INFO msg="Upload complete. (305ms)"5591worker2 # [ 84.891707] nix-grpc-daemon[782]: ts=2026-09-22T15:04:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=2 nar_bytes_in=9765592worker2 # [ 84.894426] nix-grpc-daemon[782]: ts=2026-09-22T15:04:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485593worker2 # [ 84.902057] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5594worker2 # [ 84.906180] nix-daemon[850]: reaped child process 1109, status = succeeded5595worker2 # [ 84.913309] nix-grpc-daemon[782]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5596worker2 # [ 84.920132] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5597client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5598client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5599client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5600worker2 # [ 85.757564] nix-daemon[850]: reaped child process 1124, status = succeeded5601worker2 # [ 85.841588] nix-daemon[850]: accepted connection from pid 1140, user nix-grpc-daemon (trusted)5602worker2 # [ 85.856882] nix-daemon[850]: reaped child process 1143, status = succeeded5603lb # [ 85.862673] niks3-server[990]: 2026/09/22 15:04:37 INFO Received uploads request method=POST path=/api/pending_closures5604worker2 # [ 85.874777] nix-grpc-daemon[878]: time=2026-09-22T15:04:37.098Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5605lb # [ 85.896591] niks3-server[990]: 2026/09/22 15:04:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5606worker2 # [ 85.902971] nix-grpc-daemon[878]: time=2026-09-22T15:04:37.126Z level=INFO msg="Uploading 1 narinfos"5607lb # [ 85.900934] niks3-server[990]: 2026/09/22 15:04:37 INFO Signed narinfos id=30 count=15608lb # [ 85.908572] niks3-server[990]: 2026/09/22 15:04:37 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5609lb # [ 85.920112] niks3-server[990]: 2026/09/22 15:04:37 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5610lb # [ 85.924863] niks3-server[990]: 2026/09/22 15:04:37 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5611worker2 # [ 85.933431] nix-grpc-daemon[878]: time=2026-09-22T15:04:37.157Z level=INFO msg="Upload complete. (204ms)"5612lb # [ 85.931487] niks3-server[990]: 2026/09/22 15:04:37 INFO Completed upload id=305613worker2 # [ 85.936740] nix-grpc-daemon[782]: ts=2026-09-22T15:04:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=15 outputs=15614client # copying 1 paths...5615lb # [ 85.938783] niks3-server[990]: 2026/09/22 15:04:37 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5616worker2 # [ 85.946782] nix-daemon[850]: reaped child process 1121, status = succeeded5617worker1 # [ 85.938825] nix-grpc-daemon[783]: ts=2026-09-22T15:04:37Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:608985618worker1 # [ 85.947351] nix-grpc-daemon[783]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5619client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5620worker2 # [ 85.979969] nix-grpc-daemon[782]: ts=2026-09-22T15:04:37Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:395485621worker2 # [ 85.982117] nix-grpc-daemon[782]: ts=2026-09-22T15:04:37Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_out=1205622client # [ 86.058979] nix-daemon[1114]: reaped child process 1200, status = succeeded5623client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.99 seconds)5624client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25625client # [ 86.151096] nix-daemon[1114]: accepted connection from pid 1219, user root (trusted)5626client # this derivation will be built:5627client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5628worker2 # [ 86.252842] nix-grpc-daemon[782]: ts=2026-09-22T15:04:37Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:395485629client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5630worker2 # [ 86.258817] nix-grpc-daemon[782]: ts=2026-09-22T15:04:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395485631worker2 # [ 86.263743] nix-grpc-daemon[878]: time=2026-09-22T15:04:37.486Z level=INFO msg="All 1 paths already cached"5632worker1 # [ 86.266891] nix-grpc-daemon[783]: ts=2026-09-22T15:04:37Z level=debug event=want client=15 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05633worker1 # [ 86.273336] nix-grpc-daemon[783]: ts=2026-09-22T15:04:37Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15634worker1 # [ 86.279082] nix-grpc-daemon[783]: ts=2026-09-22T15:04:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985635worker1 # [ 86.290962] nix-grpc-daemon[783]: ts=2026-09-22T15:04:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:608985636client # copying 1 paths to 'lb:50051'...5637client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5638worker1 # [ 86.309202] nix-grpc-daemon[783]: ts=2026-09-22T15:04:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:608985639worker1 # [ 86.320379] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5640worker1 # [ 86.336476] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 10 ms (attempt 1/5)5641worker1 # [ 86.352942] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 15 ms (attempt 2/5)5642worker1 # [ 86.369455] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 335 ms (attempt 3/5)5643lb # [ 86.525243] envoy[916]: conn peer=[2001:db8:1::1]:57134 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1453 tx=1197 ms=48065644lb # [ 86.528909] envoy[916]: conn peer=[2001:db8:1::1]:57150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3429 tx=1940 ms=14715645lb # [ 86.532470] envoy[916]: conn peer=[2001:db8:1::1]:57156 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=575646lb # [ 86.535905] envoy[916]: conn peer=[2001:db8:1::1]:57164 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3863 tx=2138 ms=16905647lb # [ 86.539414] envoy[916]: conn peer=[2001:db8:1::1]:57172 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=378 ms=575648lb # [ 86.543677] envoy[916]: conn peer=[2001:db8:1::1]:57180 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4299 tx=2395 ms=14485649lb # [ 86.547286] envoy[916]: conn peer=[2001:db8:1::1]:52096 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=565650lb # [ 86.550810] envoy[916]: conn peer=[2001:db8:1::1]:52110 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3864 tx=2042 ms=17025651lb # [ 86.554291] envoy[916]: conn peer=[2001:db8:1::1]:52126 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=145652worker1 # [ 86.711899] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 630 ms (attempt 4/5)5653worker1 # [ 87.345790] nix-grpc-daemon[783]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5654worker1 # [ 87.349405] nix-grpc-daemon[783]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5655worker1 # [ 87.354648] nix-grpc-daemon[783]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5656worker1 # [ 87.474820] nix-daemon[865]: accepted connection from pid 1332, user nix-grpc-daemon (trusted)5657worker1 # [ 87.489768] nix-daemon[865]: reaped child process 1335, status = succeeded5658worker1 # [ 87.574580] nix-daemon[865]: accepted connection from pid 1337, user nix-grpc-daemon (trusted)5659worker1 # [ 87.592102] nix-daemon[865]: reaped child process 1340, status = succeeded5660lb # [ 87.603737] niks3-server[990]: 2026/09/22 15:04:38 INFO Received uploads request method=POST path=/api/pending_closures5661worker1 # [ 87.606863] nix-grpc-daemon[880]: time=2026-09-22T15:04:38.947Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5662worker1 # [ 87.608322] nix-grpc-daemon[880]: time=2026-09-22T15:04:38.947Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5663lb # [ 87.628258] niks3-server[990]: 2026/09/22 15:04:38 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5664lb # [ 87.646148] niks3-server[990]: 2026/09/22 15:04:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5665lb # [ 87.651155] niks3-server[990]: 2026/09/22 15:04:38 INFO Signed narinfos id=31 count=15666worker1 # [ 87.648853] nix-grpc-daemon[880]: time=2026-09-22T15:04:38.990Z level=INFO msg="Uploading 1 narinfos"5667lb # [ 87.658393] niks3-server[990]: 2026/09/22 15:04:38 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5668lb # [ 87.670055] niks3-server[990]: 2026/09/22 15:04:38 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5669lb # [ 87.677118] niks3-server[990]: 2026/09/22 15:04:38 INFO Completed upload id=315670worker1 # [ 87.671929] nix-grpc-daemon[880]: time=2026-09-22T15:04:39.013Z level=INFO msg="Upload complete. (298ms)"5671lb # [ 87.680309] niks3-server[990]: 2026/09/22 15:04:39 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5672worker1 # [ 87.676394] nix-grpc-daemon[783]: ts=2026-09-22T15:04:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=1 paths=1 nar_bytes_in=5605673worker1 # [ 87.681535] nix-grpc-daemon[783]: ts=2026-09-22T15:04:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:608985674worker1 # [ 87.691434] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5675worker1 # [ 87.700099] nix-daemon[865]: reaped child process 1325, status = succeeded5676client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5677worker1 # [ 87.703080] nix-daemon[865]: accepted connection from pid 783, user nix-grpc-daemon (trusted)5678client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5679client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5680worker1 # [ 88.718981] nix-daemon[865]: reaped child process 1344, status = succeeded5681worker1 # [ 88.801859] nix-daemon[865]: accepted connection from pid 1360, user nix-grpc-daemon (trusted)5682worker1 # [ 88.817333] nix-daemon[865]: reaped child process 1363, status = succeeded5683lb # [ 88.828812] niks3-server[990]: 2026/09/22 15:04:40 INFO Received uploads request method=POST path=/api/pending_closures5684worker1 # [ 88.827843] nix-grpc-daemon[880]: time=2026-09-22T15:04:40.169Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5685lb # [ 88.844063] niks3-server[990]: 2026/09/22 15:04:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5686lb # [ 88.847061] niks3-server[990]: 2026/09/22 15:04:40 INFO Signed narinfos id=32 count=15687worker1 # [ 88.843146] nix-grpc-daemon[880]: time=2026-09-22T15:04:40.184Z level=INFO msg="Uploading 1 narinfos"5688lb # [ 88.849217] niks3-server[990]: 2026/09/22 15:04:40 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5689lb # [ 88.853141] niks3-server[990]: 2026/09/22 15:04:40 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5690lb # [ 88.858373] niks3-server[990]: 2026/09/22 15:04:40 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5691lb # [ 88.861348] niks3-server[990]: 2026/09/22 15:04:40 INFO Completed upload id=325692worker1 # [ 88.855938] nix-grpc-daemon[880]: time=2026-09-22T15:04:40.197Z level=INFO msg="Upload complete. (145ms)"5693lb # [ 88.863388] niks3-server[990]: 2026/09/22 15:04:40 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5694worker1 # [ 88.858634] nix-grpc-daemon[783]: ts=2026-09-22T15:04:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=16 outputs=15695client # copying 1 paths...5696worker1 # [ 88.866936] nix-daemon[865]: reaped child process 1342, status = succeeded5697worker2 # [ 88.874365] nix-grpc-daemon[782]: ts=2026-09-22T15:04:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395485698worker2 # [ 88.880388] nix-grpc-daemon[782]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5699client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5700worker1 # [ 88.893344] nix-grpc-daemon[783]: ts=2026-09-22T15:04:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608985701worker1 # [ 88.895592] nix-grpc-daemon[783]: ts=2026-09-22T15:04:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_out=1205702client # [ 89.019650] nix-daemon[1114]: reaped child process 1222, status = succeeded5703client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.96 seconds)5704client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25705client # [ 89.118528] nix-daemon[1114]: accepted connection from pid 1241, user root (trusted)5706client # this derivation will be built:5707client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5708worker1 # [ 89.215197] nix-grpc-daemon[783]: ts=2026-09-22T15:04:40Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:608985709client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5710worker2 # [ 89.225467] nix-grpc-daemon[782]: ts=2026-09-22T15:04:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395485711worker2 # [ 89.231362] nix-grpc-daemon[878]: time=2026-09-22T15:04:40.454Z level=INFO msg="All 1 paths already cached"5712worker1 # [ 89.235282] nix-grpc-daemon[783]: ts=2026-09-22T15:04:40Z level=debug event=want client=16 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05713worker2 # [ 89.249166] nix-grpc-daemon[782]: ts=2026-09-22T15:04:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485714worker1 # [ 89.240094] nix-grpc-daemon[783]: ts=2026-09-22T15:04:40Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15715worker2 # [ 89.257393] nix-grpc-daemon[782]: ts=2026-09-22T15:04:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:395485716worker2 # [ 89.265487] nix-grpc-daemon[878]: time=2026-09-22T15:04:40.489Z level=INFO msg="All 1 paths already cached"5717client # copying 1 paths to 'lb:50051'...5718client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5719worker2 # [ 89.270213] nix-grpc-daemon[782]: ts=2026-09-22T15:04:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:395485720worker2 # [ 89.276807] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5721worker2 # [ 89.373836] nix-daemon[850]: accepted connection from pid 1148, user nix-grpc-daemon (trusted)5722worker2 # [ 89.388979] nix-daemon[850]: reaped child process 1151, status = succeeded5723worker2 # [ 89.479955] nix-daemon[850]: accepted connection from pid 1153, user nix-grpc-daemon (trusted)5724worker2 # [ 89.494602] nix-daemon[850]: reaped child process 1156, status = succeeded5725lb # [ 89.500865] niks3-server[990]: 2026/09/22 15:04:40 INFO Received uploads request method=POST path=/api/pending_closures5726worker2 # [ 89.511891] nix-grpc-daemon[878]: time=2026-09-22T15:04:40.735Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5727worker2 # [ 89.513287] nix-grpc-daemon[878]: time=2026-09-22T15:04:40.735Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5728lb # [ 89.529530] niks3-server[990]: 2026/09/22 15:04:40 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5729lb # [ 89.539332] niks3-server[990]: 2026/09/22 15:04:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5730lb # [ 89.546567] niks3-server[990]: 2026/09/22 15:04:40 INFO Signed narinfos id=33 count=15731worker2 # [ 89.551775] nix-grpc-daemon[878]: time=2026-09-22T15:04:40.775Z level=INFO msg="Uploading 1 narinfos"5732lb # [ 89.551930] niks3-server[990]: 2026/09/22 15:04:40 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5733lb # [ 89.564777] niks3-server[990]: 2026/09/22 15:04:40 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5734lb # [ 89.571380] niks3-server[990]: 2026/09/22 15:04:40 INFO Completed upload id=335735worker2 # [ 89.575324] nix-grpc-daemon[878]: time=2026-09-22T15:04:40.799Z level=INFO msg="Upload complete. (292ms)"5736lb # [ 89.573576] niks3-server[990]: 2026/09/22 15:04:40 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5737worker2 # [ 89.577836] nix-grpc-daemon[782]: ts=2026-09-22T15:04:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=0 paths=1 nar_bytes_in=5605738worker2 # [ 89.581269] nix-grpc-daemon[782]: ts=2026-09-22T15:04:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:395485739worker2 # [ 89.587146] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5740worker2 # [ 89.595332] nix-daemon[850]: reaped child process 1146, status = succeeded5741worker2 # [ 89.597966] nix-daemon[850]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5742client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5743client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5744client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5745worker2 # [ 90.633499] nix-daemon[850]: reaped child process 1160, status = succeeded5746worker2 # [ 90.722887] nix-daemon[850]: accepted connection from pid 1176, user nix-grpc-daemon (trusted)5747worker2 # [ 90.738804] nix-daemon[850]: reaped child process 1179, status = succeeded5748lb # [ 90.742844] niks3-server[990]: 2026/09/22 15:04:42 INFO Received uploads request method=POST path=/api/pending_closures5749worker2 # [ 90.755259] nix-grpc-daemon[878]: time=2026-09-22T15:04:41.978Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5750lb # [ 90.776191] niks3-server[990]: 2026/09/22 15:04:42 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5751lb # [ 90.784751] niks3-server[990]: 2026/09/22 15:04:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5752lb # [ 90.791645] niks3-server[990]: 2026/09/22 15:04:42 INFO Signed narinfos id=34 count=15753worker2 # [ 90.797033] nix-grpc-daemon[878]: time=2026-09-22T15:04:42.020Z level=INFO msg="Uploading 1 narinfos"5754lb # [ 90.798245] niks3-server[990]: 2026/09/22 15:04:42 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5755lb # [ 90.815972] niks3-server[990]: 2026/09/22 15:04:42 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5756lb # [ 90.822202] niks3-server[990]: 2026/09/22 15:04:42 INFO Completed upload id=345757worker2 # [ 90.824054] nix-grpc-daemon[878]: time=2026-09-22T15:04:42.047Z level=INFO msg="Upload complete. (217ms)"5758lb # [ 90.823946] niks3-server[990]: 2026/09/22 15:04:42 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5759worker2 # [ 90.829776] nix-grpc-daemon[782]: ts=2026-09-22T15:04:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:39548 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=17 outputs=15760worker2 # [ 90.844981] nix-daemon[850]: reaped child process 1158, status = succeeded5761client # copying 1 paths...5762worker2 # [ 90.877389] nix-grpc-daemon[782]: ts=2026-09-22T15:04:42Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:395485763client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5764worker1 # [ 90.892534] nix-grpc-daemon[783]: ts=2026-09-22T15:04:42Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:608985765worker1 # [ 90.903927] nix-grpc-daemon[783]: copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'http://lb:5751'...5766worker1 # [ 90.917777] nix-grpc-daemon[783]: ts=2026-09-22T15:04:42Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:60898 duration_s=0 paths=1 nar_bytes_out=1205767client # [ 90.998548] nix-daemon[1114]: reaped child process 1244, status = succeeded5768client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.98 seconds)5769(finished: subtest: an input only one worker has still reaches the builder, in 7.39 seconds)5770subtest: upload whose reference the worker lacks is completed from the cache5771client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5772client # this derivation will be built:5773client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5774client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5775client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.23 seconds)5776client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5777client # this derivation will be built:5778client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5779client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5780client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.22 seconds)5781client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5782client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5783worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5784worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.12 seconds)5785worker2 # [ 91.711310] systemd[1]: Stopping Nix worker-protocol over gRPC...5786worker2 # [ 91.777149] nix-grpc-daemon[782]: ts=2026-09-22T15:04:43Z level=info event=signal_exit5787worker1 # [ 92.946419] nix-grpc-daemon[783]: ts=2026-09-22T15:04:44Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:608945788worker2 # [ 92.948925] nix-grpc-daemon[782]: ts=2026-09-22T15:04:44Z level=info event=scheduler_restarting5789worker1 # [ 92.947999] nix-grpc-daemon[783]: ts=2026-09-22T15:04:44Z level=info event=worker_gone addr=192.168.1.4:500515790worker2 # [ 92.966496] nix-grpc-daemon[782]: ts=2026-09-22T15:04:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5791worker2 # [ 92.985981] nix-daemon[850]: reaped child process 852, status = succeeded5792worker2 # [ 93.907840] nix-grpc-daemon[782]: E0000 00:00:1790089485.131286 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true5793worker2 # [ 93.913065] nix-grpc-daemon[782]: E0000 00:00:1790089485.131493 827 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5794worker2 # [ 94.171871] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5795worker2 # [ 94.174634] systemd[1]: Stopped Nix worker-protocol over gRPC.5796worker2 # [ 94.178146] systemd[1]: nix-grpc-daemon.service: Consumed 2.319s CPU time over 1min 17.561s wall clock time, 60.2M memory peak, 79.9K incoming IP traffic, 195.2K outgoing IP traffic.5797worker2 # [ 94.190883] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5798worker2 # [ 94.195239] systemd[1]: Closed Nix worker-protocol over gRPC.5799worker2 # [ 94.198387] systemd[1]: nix-grpc-daemon.socket: Consumed 312us CPU time over 1min 26.695s wall clock time, 256K memory peak, 90.5K incoming IP traffic, 81.5K outgoing IP traffic.5800lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5801lb: must succeed: curl -sf localhost:9901/clusters5802lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5803lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5804client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25805worker1 # [ 94.357917] nix-grpc-daemon[783]: ts=2026-09-22T15:04:45Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:608985806worker1 # [ 94.364388] nix-grpc-daemon[783]: ts=2026-09-22T15:04:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:608985807client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5808client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.14 seconds)5809worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5810worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5811worker2 # [ 94.454117] systemd[1]: Listening on Nix worker-protocol over gRPC.5812worker1 # [ 94.491941] systemd[1]: Stopping Nix worker-protocol over gRPC...5813worker1 # [ 94.721615] nix-grpc-daemon[783]: ts=2026-09-22T15:04:46Z level=info event=signal_exit5814lb # [ 94.949936] niks3-server[990]: 2026/09/22 15:04:46 INFO lead: released remote=192.168.1.3:484945815worker2 # [ 95.165527] systemd[1]: Starting scheduler-token.service...5816worker2 # [ 95.221775] systemd[1]: scheduler-token.service: Deactivated successfully.5817worker2 # [ 95.224970] systemd[1]: Finished scheduler-token.service.5818worker2 # [ 95.227675] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 58ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5819worker2 # [ 95.235921] systemd[1]: Starting Nix worker-protocol over gRPC...5820worker2 # [ 95.313140] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5821worker2 # [ 95.315109] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5822worker2 # [ 95.318327] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15823worker2 # [ 95.322620] nix-grpc-daemon[1203]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5824worker2 # [ 95.323995] nix-grpc-daemon[1203]: E0000 00:00:1790089486.546189 1203 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5825worker2 # [ 95.330062] nix-grpc-daemon[1203]: E0000 00:00:1790089486.551563 1203 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5826worker2 # [ 95.332195] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5827worker2 # [ 95.333779] systemd[1]: Started Nix worker-protocol over gRPC.5828worker2 # [ 95.340061] nix-grpc-daemon[1203]: E0000 00:00:1790089486.563694 1244 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5829worker2 # [ 95.343880] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=scheduler_take_over5830lb # [ 95.340321] niks3-server[990]: 2026/09/22 15:04:46 INFO lead: acquired remote=192.168.1.4:367645831worker2 # [ 95.355853] nix-grpc-daemon[1203]: E0000 00:00:1790089486.579585 1244 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5832worker2 # [ 95.360055] nix-grpc-daemon[1203]: E0000 00:00:1790089486.581896 1244 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5833worker2 # [ 95.369768] nix-grpc-daemon[1203]: E0000 00:00:1790089486.593488 1244 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5834worker2 # [ 95.378547] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=scheduler_connected addr=lb:500515835worker2 # [ 95.379970] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5836worker2 # [ 95.581351] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=scheduler_connected addr=lb:500515837worker2 # [ 95.585215] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:46Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5838worker2 # [ 95.786885] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=scheduler_connected addr=lb:500515839worker2 # [ 95.790555] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5840worker1 # [ 95.942734] nix-grpc-daemon[783]: ts=2026-09-22T15:04:47Z level=info event=scheduler_restarting5841worker1 # [ 95.947251] nix-grpc-daemon[783]: ts=2026-09-22T15:04:47Z level=info event=scheduler_restarting addr=lb:500515842worker1 # [ 95.948950] nix-grpc-daemon[783]: ts=2026-09-22T15:04:47Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:608945843worker1 # [ 95.950751] nix-grpc-daemon[783]: ts=2026-09-22T15:04:47Z level=info event=worker_gone addr=192.168.1.3:500515844worker1 # [ 95.966635] nix-daemon[865]: reaped child process 867, status = succeeded5845worker2 # [ 95.991100] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=scheduler_connected addr=lb:500515846worker2 # [ 95.992904] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5847worker1 # [ 96.160718] nix-grpc-daemon[783]: ts=2026-09-22T15:04:47Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5848worker2 # [ 96.193715] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=scheduler_connected addr=lb:500515849worker2 # [ 96.195542] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5850worker2 # [ 96.396434] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=scheduler_connected addr=lb:500515851worker2 # [ 96.402883] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=oidc_jwks provider=mock keys=15852worker2 # [ 96.404321] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:331505853worker2 # [ 96.405927] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:47Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05854lb # [ 96.525752] envoy[916]: conn peer=[2001:db8:1::1]:52132 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3728 tx=2121 ms=27025855lb # [ 96.529444] envoy[916]: conn peer=[2001:db8:1::1]:52148 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=535856lb # [ 96.532833] envoy[916]: conn peer=[2001:db8:1::1]:52150 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3742 tx=2100 ms=17145857lb # [ 96.536311] envoy[916]: conn peer=[2001:db8:1::1]:52158 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=329 ms=395858lb # [ 96.539640] envoy[916]: conn peer=[2001:db8:1::4]:48558 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=63529 tx=3226 ms=761645859lb # [ 96.543259] envoy[916]: conn peer=[2001:db8:1::1]:48316 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=195860lb # [ 96.547445] envoy[916]: conn peer=[2001:db8:1::3]:47162 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=17179 tx=3499 ms=794085861worker1 # [ 96.860463] nix-grpc-daemon[783]: E0000 00:00:1790089488.200910 828 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 19, started true, shutdown true5862worker1 # [ 96.865330] nix-grpc-daemon[783]: E0000 00:00:1790089488.201897 828 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true5863worker1 # [ 98.123810] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5864worker1 # [ 98.126380] systemd[1]: Stopped Nix worker-protocol over gRPC.5865worker1 # [ 98.127377] systemd[1]: nix-grpc-daemon.service: Consumed 3.249s CPU time over 1min 21.606s wall clock time, 70.6M memory peak, 114K incoming IP traffic, 248.3K outgoing IP traffic.5866worker1 # [ 98.132631] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5867worker1 # [ 98.134563] systemd[1]: Closed Nix worker-protocol over gRPC.5868worker1 # [ 98.136120] systemd[1]: nix-grpc-daemon.socket: Consumed 323us CPU time over 1min 29.702s wall clock time, 256K memory peak, 122.2K incoming IP traffic, 103.5K outgoing IP traffic.5869lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5870lb: must succeed: curl -sf localhost:9901/clusters5871lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)5872lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5873client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25874worker2 # [ 98.337358] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331585875worker2 # [ 98.348310] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5876worker2 # [ 98.367131] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 66 ms (attempt 1/5)5877worker2 # [ 98.440585] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 2/5)5878worker2 # [ 98.472989] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 331 ms (attempt 3/5)5879worker2 # [ 98.811778] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 364 ms (attempt 4/5)5880worker2 # [ 99.184341] nix-grpc-daemon[1203]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5881worker2 # [ 99.187280] nix-grpc-daemon[1203]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5882client # copying 2 paths...5883client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5884worker2 # [ 99.207780] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331585885client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5886worker2 # [ 99.217691] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5887worker2 # [ 99.240551] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:50Z level=info event=niks3_push_started pid=12635888worker2 # [ 99.343026] nix-grpc-daemon[1263]: time=2026-09-22T15:04:50.566Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5889worker2 # [ 99.431191] nix-daemon[850]: accepted connection from pid 1275, user nix-grpc-daemon (trusted)5890worker2 # [ 99.443725] nix-daemon[850]: reaped child process 1278, status = succeeded5891lb # [ 99.448507] niks3-server[990]: 2026/09/22 15:04:50 INFO Received uploads request method=POST path=/api/pending_closures5892lb # [ 99.460662] niks3-server[990]: 2026/09/22 15:04:50 INFO Received uploads request method=POST path=/api/pending_closures5893worker2 # [ 99.470369] nix-grpc-daemon[1263]: time=2026-09-22T15:04:50.694Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5894worker2 # [ 99.471986] nix-grpc-daemon[1263]: time=2026-09-22T15:04:50.695Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5895worker2 # [ 99.474602] nix-grpc-daemon[1263]: time=2026-09-22T15:04:50.695Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5896lb # [ 99.502927] niks3-server[990]: 2026/09/22 15:04:50 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5897lb # [ 99.521107] niks3-server[990]: 2026/09/22 15:04:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5898lb # [ 99.530045] niks3-server[990]: 2026/09/22 15:04:50 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5899lb # [ 99.536062] niks3-server[990]: 2026/09/22 15:04:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5900lb # [ 99.540047] niks3-server[990]: 2026/09/22 15:04:50 INFO Signed narinfos id=35 count=15901lb # [ 99.541229] niks3-server[990]: 2026/09/22 15:04:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5902lb # [ 99.543529] niks3-server[990]: 2026/09/22 15:04:50 INFO Signed narinfos id=36 count=25903worker2 # [ 99.546129] nix-grpc-daemon[1263]: time=2026-09-22T15:04:50.769Z level=INFO msg="Uploading 3 narinfos"5904lb # [ 99.551257] niks3-server[990]: 2026/09/22 15:04:50 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5905lb # [ 99.562493] niks3-server[990]: 2026/09/22 15:04:50 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5906lb # [ 99.565061] niks3-server[990]: 2026/09/22 15:04:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5907lb # [ 99.569091] niks3-server[990]: 2026/09/22 15:04:50 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5908lb # [ 99.575656] niks3-server[990]: 2026/09/22 15:04:50 INFO Completed upload id=355909lb # [ 99.577163] niks3-server[990]: 2026/09/22 15:04:50 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5910lb # [ 99.579730] niks3-server[990]: 2026/09/22 15:04:50 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5911lb # [ 99.582710] niks3-server[990]: 2026/09/22 15:04:50 INFO Completed upload id=365912worker2 # [ 99.585104] nix-grpc-daemon[1263]: time=2026-09-22T15:04:50.808Z level=INFO msg="Upload complete. (238ms)"5913worker2 # [ 99.590825] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33158 duration_s=0 paths=2 nar_bytes_in=3125914worker2 # [ 99.604338] nix-daemon[850]: reaped child process 1261, status = succeeded5915client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.42 seconds)5916worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5917worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5918worker1 # [ 99.678716] systemd[1]: Listening on Nix worker-protocol over gRPC.5919worker1 # [ 99.685914] systemd[1]: Starting Nix worker-protocol over gRPC...5920worker1 # [ 99.780269] nix-grpc-daemon[1389]: ts=2026-09-22T15:04:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5921worker1 # [ 99.785055] nix-grpc-daemon[1389]: ts=2026-09-22T15:04:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5922worker1 # [ 99.789189] nix-grpc-daemon[1389]: ts=2026-09-22T15:04:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15923worker1 # [ 99.793258] nix-grpc-daemon[1389]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5924worker1 # [ 99.795553] nix-grpc-daemon[1389]: E0000 00:00:1790089491.129180 1389 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5925worker1 # [ 99.800099] nix-grpc-daemon[1389]: E0000 00:00:1790089491.133286 1389 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5926worker1 # [ 99.804520] nix-grpc-daemon[1389]: ts=2026-09-22T15:04:51Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5927worker1 # [ 99.807053] systemd[1]: Started Nix worker-protocol over gRPC.5928worker1 # [ 99.809102] nix-grpc-daemon[1389]: E0000 00:00:1790089491.149808 1431 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5929worker1 # [ 99.821836] nix-grpc-daemon[1389]: E0000 00:00:1790089491.163281 1431 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5930worker1 # [ 99.825174] nix-grpc-daemon[1389]: E0000 00:00:1790089491.165563 1431 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5931lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5932lb: must succeed: curl -sf localhost:9901/clusters5933worker1 # [ 99.837232] nix-grpc-daemon[1389]: E0000 00:00:1790089491.178699 1431 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5934worker1 # [ 99.851277] nix-grpc-daemon[1389]: ts=2026-09-22T15:04:51Z level=info event=scheduler_connected addr=lb:500515935worker2 # [ 99.859932] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:51Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:331505936worker2 # [ 99.864702] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:51Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05937lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5938lb: must succeed: curl -sf localhost:9901/clusters5939lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5940lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.08 seconds)5941(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 9.91 seconds)5942subtest: a niks3 restart does not cost the leader its role5943lb: must succeed: curl -sf localhost:9901/clusters5944lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5945lb: must succeed: curl -sf localhost:9901/clusters5946lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5947worker2 # [ 100.968220] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:52Z level=info event=scheduler_lock_lost status=2005948lb # [ 100.965672] niks3-server[990]: 2026/09/22 15:04:52 INFO Shutdown signal received, draining in-flight requests timeout=10s5949lb # [ 100.968326] niks3-server[990]: 2026/09/22 15:04:52 INFO lead: released remote=192.168.1.4:367645950worker1 # [ 100.961623] nix-grpc-daemon[1389]: ts=2026-09-22T15:04:52Z level=info event=scheduler_lock_lost status=2005951lb # [ 100.972096] systemd[1]: Stopping niks3 server...5952lb # [ 100.981934] systemd[1]: niks3.service: Deactivated successfully.5953lb # [ 100.983519] systemd[1]: Stopped niks3 server.5954lb # [ 100.985070] systemd[1]: niks3.service: Consumed 637ms CPU time over 1min 20.267s wall clock time, 28.7M memory peak, 99.5K incoming IP traffic, 141.4K outgoing IP traffic.5955lb # [ 100.991750] systemd[1]: Starting niks3 server...5956lb # [ 101.039813] niks3-server[1129]: 2026/09/22 15:04:52 goose: no migrations to run. current version: 202609200000005957lb # [ 101.045732] niks3-server[1129]: 2026/09/22 15:04:52 OK 1_commit_pending_closure.sql (4.1ms)5958lb # [ 101.049188] niks3-server[1129]: 2026/09/22 15:04:52 OK 2_object_stats_trigger.sql (1.9ms)5959lb # [ 101.050230] niks3-server[1129]: 2026/09/22 15:04:52 goose: up to current file version: 25960lb # [ 101.051421] niks3-server[1129]: 2026/09/22 15:04:52 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5961lb # [ 101.061753] niks3-server[1129]: 2026/09/22 15:04:52 INFO Read proxy enabled — serving cache objects from S35962lb # [ 101.063171] niks3-server[1129]: 2026/09/22 15:04:52 INFO Using socket-activated listener address=0.0.0.0:57515963lb # [ 101.065313] systemd[1]: Started niks3 server.5964lb # [ 101.067326] niks3-server[1129]: 2026/09/22 15:04:52 INFO systemd watchdog enabled interval=15s5965lb # [ 101.068732] niks3-server[1129]: 2026/09/22 15:04:52 INFO Starting HTTP server address=0.0.0.0:57515966lb: waiting for unit niks3.service5967lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5968lb: must succeed: curl -sf localhost:9901/clusters5969lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5970worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5971worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5972lb: must succeed: curl -sf localhost:9901/clusters5973lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5974lb: must succeed: curl -sf localhost:9901/clusters5975lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5976worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5977worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5978client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25979worker1 # [ 101.371181] nix-grpc-daemon[1389]: ts=2026-09-22T15:04:52Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:432405980worker1 # [ 101.383695] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)5981client # don't know how to build these paths:5982client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5983worker1 # [ 101.398511] nix-grpc-daemon[1389]: ts=2026-09-22T15:04:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432405984worker1 # [ 101.409123] nix-grpc-daemon[1389]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 7 ms (attempt 1/5)5985worker1 # [ 101.423144] nix-grpc-daemon[1389]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 65 ms (attempt 2/5)5986worker1 # [ 101.489650] nix-grpc-daemon[1389]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 379 ms (attempt 3/5)5987worker1 # [ 101.870982] nix-grpc-daemon[1389]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 270 ms (attempt 4/5)5988lb # [ 101.972513] niks3-server[1129]: 2026/09/22 15:04:53 INFO lead: acquired remote=192.168.1.4:368025989worker1 # [ 102.143931] nix-grpc-daemon[1389]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5990worker1 # [ 102.148084] nix-grpc-daemon[1389]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5991worker2 # [ 102.195362] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:53Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05992worker2 # [ 102.199573] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:53Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15993worker2 # [ 102.238599] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331585994worker2 # [ 102.252942] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331585995client # copying 1 paths to 'lb:50051'...5996client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5997worker2 # [ 102.265551] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331585998worker2 # [ 102.277625] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)5999worker2 # [ 102.376293] nix-daemon[850]: accepted connection from pid 1297, user nix-grpc-daemon (trusted)6000worker2 # [ 102.392757] nix-daemon[850]: reaped child process 1300, status = succeeded6001worker2 # [ 102.480381] nix-daemon[850]: accepted connection from pid 1302, user nix-grpc-daemon (trusted)6002worker2 # [ 102.496559] nix-daemon[850]: reaped child process 1305, status = succeeded6003lb # [ 102.499842] niks3-server[1129]: 2026/09/22 15:04:53 INFO Received uploads request method=POST path=/api/pending_closures6004worker2 # [ 102.514890] nix-grpc-daemon[1263]: time=2026-09-22T15:04:53.738Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6005worker2 # [ 102.516386] nix-grpc-daemon[1263]: time=2026-09-22T15:04:53.738Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"6006lb # [ 102.537199] niks3-server[1129]: 2026/09/22 15:04:53 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst6007lb # [ 102.545221] niks3-server[1129]: 2026/09/22 15:04:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6008lb # [ 102.565380] niks3-server[1129]: 2026/09/22 15:04:53 INFO Signed narinfos id=37 count=16009worker2 # [ 102.569345] nix-grpc-daemon[1263]: time=2026-09-22T15:04:53.792Z level=INFO msg="Uploading 1 narinfos"6010lb # [ 102.576636] niks3-server[1129]: 2026/09/22 15:04:53 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls6011lb # [ 102.578274] niks3-server[1129]: 2026/09/22 15:04:53 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6012lb # [ 102.584882] niks3-server[1129]: 2026/09/22 15:04:53 INFO Completed upload id=376013worker2 # [ 102.586929] nix-grpc-daemon[1263]: time=2026-09-22T15:04:53.810Z level=INFO msg="Upload complete. (303ms)"6014lb # [ 102.587270] niks3-server[1129]: 2026/09/22 15:04:53 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo6015worker2 # [ 102.589418] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33158 duration_s=0 paths=1 nar_bytes_in=5366016worker2 # [ 102.593811] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331586017worker2 # [ 102.601277] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6018worker2 # [ 102.603892] nix-daemon[850]: reaped child process 1295, status = succeeded6019worker2 # [ 102.609387] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6020client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...6021client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6022client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6023client # farm-a> LOG-farm-a-after-niks3-restart6024worker2 # [ 103.251905] nix-daemon[850]: reaped child process 1309, status = succeeded6025worker2 # [ 103.333644] nix-daemon[850]: accepted connection from pid 1325, user nix-grpc-daemon (trusted)6026worker2 # [ 103.349848] nix-daemon[850]: reaped child process 1328, status = succeeded6027lb # [ 103.352641] niks3-server[1129]: 2026/09/22 15:04:54 INFO Received uploads request method=POST path=/api/pending_closures6028worker2 # [ 103.364911] nix-grpc-daemon[1263]: time=2026-09-22T15:04:54.588Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6029worker2 # [ 103.368206] nix-grpc-daemon[1263]: time=2026-09-22T15:04:54.590Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"6030lb # [ 103.399253] niks3-server[1129]: 2026/09/22 15:04:54 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst6031lb # [ 103.405949] niks3-server[1129]: 2026/09/22 15:04:54 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv6032lb # [ 103.414582] niks3-server[1129]: 2026/09/22 15:04:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6033lb # [ 103.423358] niks3-server[1129]: 2026/09/22 15:04:54 INFO Signed narinfos id=38 count=16034worker2 # [ 103.426275] nix-grpc-daemon[1263]: time=2026-09-22T15:04:54.649Z level=INFO msg="Uploading 1 narinfos"6035lb # [ 103.432267] niks3-server[1129]: 2026/09/22 15:04:54 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls6036lb # [ 103.440541] niks3-server[1129]: 2026/09/22 15:04:54 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6037lb # [ 103.447987] niks3-server[1129]: 2026/09/22 15:04:54 INFO Completed upload id=386038worker2 # [ 103.450363] nix-grpc-daemon[1263]: time=2026-09-22T15:04:54.674Z level=INFO msg="Upload complete. (233ms)"6039lb # [ 103.450042] niks3-server[1129]: 2026/09/22 15:04:54 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo6040worker2 # [ 103.457719] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:54Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33158 duration_s=0 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=16041worker2 # [ 103.474122] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:54Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=06042worker2 # [ 103.477302] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:54Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=16043worker2 # [ 103.487808] nix-daemon[850]: reaped child process 1307, status = succeeded6044worker2 # [ 103.516769] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331586045worker2 # [ 103.529093] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331586046worker2 # [ 103.543460] nix-grpc-daemon[1263]: time=2026-09-22T15:04:54.766Z level=INFO msg="All 1 paths already cached"6047client # copying 1 paths to 'lb:50051'...6048client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...6049worker2 # [ 103.552145] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331586050worker2 # [ 103.562132] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6051worker2 # [ 103.662858] nix-daemon[850]: accepted connection from pid 1334, user nix-grpc-daemon (trusted)6052worker2 # [ 103.678651] nix-daemon[850]: reaped child process 1337, status = succeeded6053worker2 # [ 103.764524] nix-daemon[850]: accepted connection from pid 1339, user nix-grpc-daemon (trusted)6054worker2 # [ 103.779074] nix-daemon[850]: reaped child process 1342, status = succeeded6055lb # [ 103.784135] niks3-server[1129]: 2026/09/22 15:04:55 INFO Received uploads request method=POST path=/api/pending_closures6056worker2 # [ 103.791690] nix-grpc-daemon[1263]: time=2026-09-22T15:04:55.015Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6057worker2 # [ 103.793711] nix-grpc-daemon[1263]: time=2026-09-22T15:04:55.017Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"6058lb # [ 103.801423] niks3-server[1129]: 2026/09/22 15:04:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6059lb # [ 103.806828] niks3-server[1129]: 2026/09/22 15:04:55 INFO Signed narinfos id=39 count=16060worker2 # [ 103.808997] nix-grpc-daemon[1263]: time=2026-09-22T15:04:55.032Z level=INFO msg="Uploading 1 narinfos"6061lb # [ 103.809041] niks3-server[1129]: 2026/09/22 15:04:55 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst6062lb # [ 103.821201] niks3-server[1129]: 2026/09/22 15:04:55 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6063lb # [ 103.824952] niks3-server[1129]: 2026/09/22 15:04:55 INFO Completed upload id=396064worker2 # [ 103.828319] nix-grpc-daemon[1263]: time=2026-09-22T15:04:55.052Z level=INFO msg="Upload complete. (256ms)"6065lb # [ 103.827941] niks3-server[1129]: 2026/09/22 15:04:55 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo6066worker2 # [ 103.830636] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33158 duration_s=0 paths=1 nar_bytes_in=6566067lb # [ 103.831334] niks3-server[1129]: 2026/09/22 15:04:55 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls6068worker2 # [ 103.837575] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331586069worker2 # [ 103.842739] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6070worker2 # [ 103.845314] nix-daemon[850]: reaped child process 1332, status = succeeded6071worker2 # [ 103.850865] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6072client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...6073client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6074client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6075client # farm-b> LOG-farm-b-after-niks3-restart6076worker2 # [ 104.894329] nix-daemon[850]: reaped child process 1346, status = succeeded6077worker2 # [ 104.964888] nix-daemon[850]: accepted connection from pid 1363, user nix-grpc-daemon (trusted)6078worker2 # [ 104.980490] nix-daemon[850]: reaped child process 1366, status = succeeded6079lb # [ 104.985427] niks3-server[1129]: 2026/09/22 15:04:56 INFO Received uploads request method=POST path=/api/pending_closures6080worker2 # [ 104.997038] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.220Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6081worker2 # [ 104.998707] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.222Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"6082lb # [ 105.029906] niks3-server[1129]: 2026/09/22 15:04:56 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst6083lb # [ 105.037306] niks3-server[1129]: 2026/09/22 15:04:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6084worker2 # [ 105.045865] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.269Z level=INFO msg="Uploading 1 narinfos"6085lb # [ 105.043540] niks3-server[1129]: 2026/09/22 15:04:56 INFO Signed narinfos id=40 count=16086lb # [ 105.055964] niks3-server[1129]: 2026/09/22 15:04:56 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv6087lb # [ 105.058379] niks3-server[1129]: 2026/09/22 15:04:56 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls6088lb # [ 105.062069] niks3-server[1129]: 2026/09/22 15:04:56 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6089lb # [ 105.065655] niks3-server[1129]: 2026/09/22 15:04:56 INFO Completed upload id=406090worker2 # [ 105.067823] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.291Z level=INFO msg="Upload complete. (208ms)"6091lb # [ 105.068652] niks3-server[1129]: 2026/09/22 15:04:56 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo6092worker2 # [ 105.077420] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33158 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=16093worker2 # [ 105.084779] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:56Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=06094worker2 # [ 105.088939] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:56Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=16095worker2 # [ 105.099273] nix-daemon[850]: reaped child process 1344, status = succeeded6096worker2 # [ 105.128761] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331586097worker2 # [ 105.140967] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:331586098worker2 # [ 105.156152] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.379Z level=INFO msg="All 2 paths already cached"6099client # copying 1 paths to 'lb:50051'...6100client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...6101worker2 # [ 105.165300] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:331586102worker2 # [ 105.173571] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6103worker2 # [ 105.275984] nix-daemon[850]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)6104worker2 # [ 105.291694] nix-daemon[850]: reaped child process 1375, status = succeeded6105worker2 # [ 105.381242] nix-daemon[850]: accepted connection from pid 1377, user nix-grpc-daemon (trusted)6106worker2 # [ 105.396779] nix-daemon[850]: reaped child process 1380, status = succeeded6107lb # [ 105.400239] niks3-server[1129]: 2026/09/22 15:04:56 INFO Received uploads request method=POST path=/api/pending_closures6108worker2 # [ 105.413862] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.637Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6109worker2 # [ 105.415811] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.639Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"6110lb # [ 105.430794] niks3-server[1129]: 2026/09/22 15:04:56 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst6111lb # [ 105.441536] niks3-server[1129]: 2026/09/22 15:04:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6112worker2 # [ 105.449586] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.673Z level=INFO msg="Uploading 1 narinfos"6113lb # [ 105.447687] niks3-server[1129]: 2026/09/22 15:04:56 INFO Signed narinfos id=41 count=16114lb # [ 105.455164] niks3-server[1129]: 2026/09/22 15:04:56 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls6115lb # [ 105.464561] niks3-server[1129]: 2026/09/22 15:04:56 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6116lb # [ 105.469120] niks3-server[1129]: 2026/09/22 15:04:56 INFO Completed upload id=416117worker2 # [ 105.470787] nix-grpc-daemon[1263]: time=2026-09-22T15:04:56.694Z level=INFO msg="Upload complete. (284ms)"6118lb # [ 105.470779] niks3-server[1129]: 2026/09/22 15:04:56 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo6119worker2 # [ 105.475155] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:33158 duration_s=0 paths=1 nar_bytes_in=7846120worker2 # [ 105.480440] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:331586121worker2 # [ 105.488250] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6122worker2 # [ 105.492107] nix-daemon[850]: reaped child process 1370, status = succeeded6123worker2 # [ 105.495982] nix-daemon[850]: accepted connection from pid 1203, user nix-grpc-daemon (trusted)6124client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...6125client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6126client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6127lb # [ 106.526821] envoy[916]: conn peer=[2001:db8:1::1]:48322 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=12666128client # farm-top> LOG-farm-top-after-niks3-restart6129worker2 # [ 106.710802] nix-daemon[850]: reaped child process 1384, status = succeeded6130worker2 # [ 106.785379] nix-daemon[850]: accepted connection from pid 1400, user nix-grpc-daemon (trusted)6131worker2 # [ 106.799191] nix-daemon[850]: reaped child process 1403, status = succeeded6132lb # [ 106.803608] niks3-server[1129]: 2026/09/22 15:04:58 INFO Received uploads request method=POST path=/api/pending_closures6133worker2 # [ 106.817968] nix-grpc-daemon[1263]: time=2026-09-22T15:04:58.041Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6134worker2 # [ 106.819460] nix-grpc-daemon[1263]: time=2026-09-22T15:04:58.043Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"6135lb # [ 106.845111] niks3-server[1129]: 2026/09/22 15:04:58 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv6136lb # [ 106.854648] niks3-server[1129]: 2026/09/22 15:04:58 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst6137lb # [ 106.866709] niks3-server[1129]: 2026/09/22 15:04:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6138lb # [ 106.872235] niks3-server[1129]: 2026/09/22 15:04:58 INFO Signed narinfos id=42 count=16139worker2 # [ 106.877507] nix-grpc-daemon[1263]: time=2026-09-22T15:04:58.100Z level=INFO msg="Uploading 1 narinfos"6140lb # [ 106.882791] niks3-server[1129]: 2026/09/22 15:04:58 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6141lb # [ 106.889775] niks3-server[1129]: 2026/09/22 15:04:58 INFO Completed upload id=426142worker2 # [ 106.891893] nix-grpc-daemon[1263]: time=2026-09-22T15:04:58.115Z level=INFO msg="Upload complete. (189ms)"6143lb # [ 106.891403] niks3-server[1129]: 2026/09/22 15:04:58 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls6144lb # [ 106.894506] niks3-server[1129]: 2026/09/22 15:04:58 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo6145worker2 # [ 106.898275] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:33158 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=16146worker2 # [ 106.913917] nix-daemon[850]: reaped child process 1382, status = succeeded6147client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 5.70 seconds)6148client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath6149client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)6150(finished: subtest: a niks3 restart does not cost the leader its role, in 6.14 seconds)6151subtest: scheduler down: the other node takes the lock and keeps it6152lb: must succeed: curl -sf localhost:9901/clusters6153lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6154lb: must succeed: curl -sf localhost:9901/clusters6155lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6156lb: must succeed: curl -sf localhost:9901/clusters6157lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6158lb: must succeed: curl -sf localhost:9901/clusters6159lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6160worker2 # [ 107.200563] systemd[1]: Stopping Nix worker-protocol over gRPC...6161worker2 # [ 107.333388] nix-grpc-daemon[1203]: ts=2026-09-22T15:04:58Z level=info event=signal_exit6162lb # [ 107.977067] niks3-server[1129]: 2026/09/22 15:04:59 INFO lead: released remote=192.168.1.4:368026163worker1 # [ 108.973180] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:00Z level=info event=scheduler_restarting addr=lb:500516164worker2 # [ 108.983448] nix-grpc-daemon[1203]: ts=2026-09-22T15:05:00Z level=info event=scheduler_restarting6165worker2 # [ 108.987324] nix-grpc-daemon[1203]: ts=2026-09-22T15:05:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:331506166worker2 # [ 108.991475] nix-grpc-daemon[1203]: ts=2026-09-22T15:05:00Z level=info event=worker_gone addr=192.168.1.3:500516167worker2 # [ 108.994938] nix-grpc-daemon[1203]: ts=2026-09-22T15:05:00Z level=info event=scheduler_restarting addr=lb:500516168worker2 # [ 108.998295] nix-grpc-daemon[1203]: ts=2026-09-22T15:05:00Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:331506169worker2 # [ 109.001909] nix-grpc-daemon[1203]: ts=2026-09-22T15:05:00Z level=info event=worker_gone addr=192.168.1.4:500516170worker2 # [ 109.022887] nix-daemon[850]: reaped child process 1255, status = succeeded6171worker1 # [ 109.172905] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:00Z level=info event=scheduler_connected addr=lb:500516172worker1 # [ 109.177136] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6173worker2 # [ 109.193681] nix-grpc-daemon[1203]: ts=2026-09-22T15:05:00Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6174worker2 # [ 109.376410] nix-grpc-daemon[1203]: E0000 00:00:1790089500.600049 1248 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true6175worker2 # [ 109.378623] nix-grpc-daemon[1203]: E0000 00:00:1790089500.600142 1248 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6176worker1 # [ 109.379148] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:00Z level=info event=scheduler_connected addr=lb:500516177worker1 # [ 109.382359] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6178worker1 # [ 109.580187] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:00Z level=info event=scheduler_connected addr=lb:500516179worker1 # [ 109.584352] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:00Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6180worker1 # [ 109.784888] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_connected addr=lb:500516181worker1 # [ 109.787208] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6182worker1 # [ 109.988221] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_connected addr=lb:500516183worker1 # [ 109.990042] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6184worker1 # [ 110.190478] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_connected addr=lb:500516185worker1 # [ 110.192294] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6186worker1 # [ 110.393187] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_connected addr=lb:500516187worker1 # [ 110.395366] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6188worker1 # [ 110.596138] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_connected addr=lb:500516189worker1 # [ 110.597946] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:01Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6190worker1 # [ 110.798878] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_connected addr=lb:500516191worker1 # [ 110.802988] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6192worker1 # [ 111.005379] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_connected addr=lb:500516193worker1 # [ 111.008658] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6194worker2 # [ 111.196283] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6195worker2 # [ 111.200132] systemd[1]: Stopped Nix worker-protocol over gRPC.6196worker2 # [ 111.204276] systemd[1]: nix-grpc-daemon.service: Consumed 1.124s CPU time over 15.963s wall clock time, 42.4M memory peak, 45.1K incoming IP traffic, 82.8K outgoing IP traffic.6197worker1 # [ 111.206367] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_connected addr=lb:500516198worker2 # [ 111.215422] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.6199worker2 # [ 111.219453] systemd[1]: Closed Nix worker-protocol over gRPC.6200worker1 # [ 111.210850] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6201worker2 # [ 111.223360] systemd[1]: nix-grpc-daemon.socket: Consumed 308us CPU time over 16.767s wall clock time, 256K memory peak, 36.9K incoming IP traffic, 27.1K outgoing IP traffic.6202worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over6203worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.05 seconds)6204worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6205worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)6206worker1 # [ 111.411786] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_connected addr=lb:500516207worker1 # [ 111.415810] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6208worker1 # [ 111.617188] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_connected addr=lb:500516209worker1 # [ 111.621287] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:02Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6210worker1 # [ 111.822239] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=scheduler_connected addr=lb:500516211worker1 # [ 111.826354] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6212lb # [ 111.973398] niks3-server[1129]: 2026/09/22 15:05:03 INFO lead: acquired remote=192.168.1.3:481586213worker1 # [ 111.967831] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=scheduler_take_over6214worker1 # [ 112.028562] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=scheduler_connected addr=lb:500516215worker1 # [ 112.031627] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6216worker1 # [ 112.229752] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=scheduler_connected addr=lb:500516217worker1 # [ 112.237175] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:419846218worker1 # [ 112.240846] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06219worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6220worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6221client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&26222worker1 # [ 112.482728] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:432406223client # don't know how to build these paths:6224client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6225worker1 # [ 112.491791] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432406226worker1 # [ 112.514556] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=06227worker1 # [ 112.519619] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=16228worker1 # [ 112.557851] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432406229worker1 # [ 112.568435] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432406230client # copying 1 paths to 'lb:50051'...6231client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...6232worker1 # [ 112.578656] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432406233worker1 # [ 112.587907] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6234worker1 # [ 112.610138] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:03Z level=info event=niks3_push_started pid=14656235worker1 # [ 112.716867] nix-grpc-daemon[1465]: time=2026-09-22T15:05:04.057Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6236worker1 # [ 112.808887] nix-daemon[865]: accepted connection from pid 1478, user nix-grpc-daemon (trusted)6237worker1 # [ 112.823344] nix-daemon[865]: reaped child process 1481, status = succeeded6238worker1 # [ 112.914130] nix-daemon[865]: accepted connection from pid 1483, user nix-grpc-daemon (trusted)6239worker1 # [ 112.928770] nix-daemon[865]: reaped child process 1486, status = succeeded6240lb # [ 112.941120] niks3-server[1129]: 2026/09/22 15:05:04 INFO Received uploads request method=POST path=/api/pending_closures6241worker1 # [ 112.944493] nix-grpc-daemon[1465]: time=2026-09-22T15:05:04.286Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6242worker1 # [ 112.946026] nix-grpc-daemon[1465]: time=2026-09-22T15:05:04.287Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"6243lb # [ 112.981526] niks3-server[1129]: 2026/09/22 15:05:04 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6244lb # [ 112.992260] niks3-server[1129]: 2026/09/22 15:05:04 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst6245lb # [ 113.002908] niks3-server[1129]: 2026/09/22 15:05:04 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls6246lb # [ 113.012943] niks3-server[1129]: 2026/09/22 15:05:04 INFO Signed narinfos id=43 count=16247worker1 # [ 113.008191] nix-grpc-daemon[1465]: time=2026-09-22T15:05:04.349Z level=INFO msg="Uploading 1 narinfos"6248lb # [ 113.022604] niks3-server[1129]: 2026/09/22 15:05:04 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6249lb # [ 113.031255] niks3-server[1129]: 2026/09/22 15:05:04 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo6250lb # [ 113.034791] niks3-server[1129]: 2026/09/22 15:05:04 INFO Completed upload id=436251worker1 # [ 113.029830] nix-grpc-daemon[1465]: time=2026-09-22T15:05:04.371Z level=INFO msg="Upload complete. (313ms)"6252worker1 # [ 113.035772] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43240 duration_s=0 paths=1 nar_bytes_in=5206253worker1 # [ 113.042431] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432406254worker1 # [ 113.053281] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6255worker1 # [ 113.065103] nix-daemon[865]: reaped child process 1463, status = succeeded6256client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...6257client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6258worker1 # [ 113.067432] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6259client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6260client # farm-a> LOG-farm-a-on-standby6261worker1 # [ 114.283731] nix-daemon[865]: reaped child process 1490, status = succeeded6262worker1 # [ 114.364323] nix-daemon[865]: accepted connection from pid 1507, user nix-grpc-daemon (trusted)6263worker1 # [ 114.376391] nix-daemon[865]: reaped child process 1510, status = succeeded6264lb # [ 114.390705] niks3-server[1129]: 2026/09/22 15:05:05 INFO Received uploads request method=POST path=/api/pending_closures6265worker1 # [ 114.395788] nix-grpc-daemon[1465]: time=2026-09-22T15:05:05.737Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6266worker1 # [ 114.399057] nix-grpc-daemon[1465]: time=2026-09-22T15:05:05.738Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6267lb # [ 114.426226] niks3-server[1129]: 2026/09/22 15:05:05 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6268lb # [ 114.442672] niks3-server[1129]: 2026/09/22 15:05:05 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6269lb # [ 114.455825] niks3-server[1129]: 2026/09/22 15:05:05 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6270worker1 # [ 114.455226] nix-grpc-daemon[1465]: time=2026-09-22T15:05:05.796Z level=INFO msg="Uploading 1 narinfos"6271lb # [ 114.460483] niks3-server[1129]: 2026/09/22 15:05:05 INFO Signed narinfos id=44 count=16272lb # [ 114.465495] niks3-server[1129]: 2026/09/22 15:05:05 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6273lb # [ 114.474228] niks3-server[1129]: 2026/09/22 15:05:05 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6274lb # [ 114.476969] niks3-server[1129]: 2026/09/22 15:05:05 INFO Completed upload id=446275worker1 # [ 114.473154] nix-grpc-daemon[1465]: time=2026-09-22T15:05:05.814Z level=INFO msg="Upload complete. (198ms)"6276lb # [ 114.480460] niks3-server[1129]: 2026/09/22 15:05:05 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6277worker1 # [ 114.476083] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43240 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16278worker1 # [ 114.484157] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:05Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06279worker1 # [ 114.486238] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:05Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=16280worker1 # [ 114.490970] nix-daemon[865]: reaped child process 1488, status = succeeded6281worker1 # [ 114.527444] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432406282worker1 # [ 114.536148] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432406283worker1 # [ 114.545790] nix-grpc-daemon[1465]: time=2026-09-22T15:05:05.887Z level=INFO msg="All 1 paths already cached"6284client # copying 1 paths to 'lb:50051'...6285worker1 # [ 114.551483] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432406286client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6287worker1 # [ 114.556207] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6288worker1 # [ 114.645221] nix-daemon[865]: accepted connection from pid 1516, user nix-grpc-daemon (trusted)6289worker1 # [ 114.658806] nix-daemon[865]: reaped child process 1519, status = succeeded6290worker1 # [ 114.747257] nix-daemon[865]: accepted connection from pid 1521, user nix-grpc-daemon (trusted)6291worker1 # [ 114.761310] nix-daemon[865]: reaped child process 1524, status = succeeded6292lb # [ 114.773757] niks3-server[1129]: 2026/09/22 15:05:06 INFO Received uploads request method=POST path=/api/pending_closures6293worker1 # [ 114.776817] nix-grpc-daemon[1465]: time=2026-09-22T15:05:06.117Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6294worker1 # [ 114.778155] nix-grpc-daemon[1465]: time=2026-09-22T15:05:06.117Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6295lb # [ 114.800334] niks3-server[1129]: 2026/09/22 15:05:06 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6296lb # [ 114.809562] niks3-server[1129]: 2026/09/22 15:05:06 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6297lb # [ 114.814367] niks3-server[1129]: 2026/09/22 15:05:06 INFO Signed narinfos id=45 count=16298worker1 # [ 114.812425] nix-grpc-daemon[1465]: time=2026-09-22T15:05:06.153Z level=INFO msg="Uploading 1 narinfos"6299lb # [ 114.829152] niks3-server[1129]: 2026/09/22 15:05:06 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6300lb # [ 114.835900] niks3-server[1129]: 2026/09/22 15:05:06 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6301worker1 # [ 114.834666] nix-grpc-daemon[1465]: time=2026-09-22T15:05:06.176Z level=INFO msg="Upload complete. (272ms)"6302lb # [ 114.840122] niks3-server[1129]: 2026/09/22 15:05:06 INFO Completed upload id=456303worker1 # [ 114.836912] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43240 duration_s=0 paths=1 nar_bytes_in=6406304lb # [ 114.845717] niks3-server[1129]: 2026/09/22 15:05:06 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6305worker1 # [ 114.844120] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432406306worker1 # [ 114.845875] nix-daemon[865]: reaped child process 1514, status = succeeded6307worker1 # [ 114.849798] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6308worker1 # [ 114.855399] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6309client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6310client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6311client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6312client # farm-b> LOG-farm-b-on-standby6313worker1 # [ 116.095745] nix-daemon[865]: reaped child process 1528, status = succeeded6314worker1 # [ 116.177806] nix-daemon[865]: accepted connection from pid 1544, user nix-grpc-daemon (trusted)6315worker1 # [ 116.190734] nix-daemon[865]: reaped child process 1547, status = succeeded6316lb # [ 116.203591] niks3-server[1129]: 2026/09/22 15:05:07 INFO Received uploads request method=POST path=/api/pending_closures6317worker1 # [ 116.207484] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.548Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6318worker1 # [ 116.208983] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.550Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6319lb # [ 116.240740] niks3-server[1129]: 2026/09/22 15:05:07 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6320lb # [ 116.251667] niks3-server[1129]: 2026/09/22 15:05:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6321lb # [ 116.258701] niks3-server[1129]: 2026/09/22 15:05:07 INFO Signed narinfos id=46 count=16322worker1 # [ 116.256448] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.597Z level=INFO msg="Uploading 1 narinfos"6323lb # [ 116.262097] niks3-server[1129]: 2026/09/22 15:05:07 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6324lb # [ 116.268649] niks3-server[1129]: 2026/09/22 15:05:07 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6325lb # [ 116.274499] niks3-server[1129]: 2026/09/22 15:05:07 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6326lb # [ 116.277498] niks3-server[1129]: 2026/09/22 15:05:07 INFO Completed upload id=466327worker1 # [ 116.272237] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.613Z level=INFO msg="Upload complete. (206ms)"6328lb # [ 116.279948] niks3-server[1129]: 2026/09/22 15:05:07 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6329worker1 # [ 116.281654] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43240 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16330worker1 # [ 116.288915] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:07Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06331worker1 # [ 116.293229] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:07Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16332worker1 # [ 116.303074] nix-daemon[865]: reaped child process 1526, status = succeeded6333worker1 # [ 116.332980] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432406334worker1 # [ 116.344721] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432406335worker1 # [ 116.358277] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.699Z level=INFO msg="All 2 paths already cached"6336client # copying 1 paths to 'lb:50051'...6337client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6338worker1 # [ 116.367147] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432406339worker1 # [ 116.377221] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6340worker1 # [ 116.471548] nix-daemon[865]: accepted connection from pid 1553, user nix-grpc-daemon (trusted)6341worker1 # [ 116.485465] nix-daemon[865]: reaped child process 1556, status = succeeded6342lb # [ 116.526994] envoy[916]: conn peer=[2001:db8:1::1]:48328 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9722 tx=4423 ms=55436343lb # [ 116.530847] envoy[916]: conn peer=[2001:db8:1::4]:53198 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=5880 tx=648 ms=138116344worker1 # [ 116.570467] nix-daemon[865]: accepted connection from pid 1558, user nix-grpc-daemon (trusted)6345worker1 # [ 116.584043] nix-daemon[865]: reaped child process 1561, status = succeeded6346lb # [ 116.596531] niks3-server[1129]: 2026/09/22 15:05:07 INFO Received uploads request method=POST path=/api/pending_closures6347worker1 # [ 116.600108] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.941Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6348worker1 # [ 116.601717] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.943Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6349lb # [ 116.631675] niks3-server[1129]: 2026/09/22 15:05:07 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6350lb # [ 116.636518] niks3-server[1129]: 2026/09/22 15:05:07 INFO Signed narinfos id=47 count=16351worker1 # [ 116.633785] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.975Z level=INFO msg="Uploading 1 narinfos"6352lb # [ 116.640191] niks3-server[1129]: 2026/09/22 15:05:07 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6353lb # [ 116.650197] niks3-server[1129]: 2026/09/22 15:05:07 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6354lb # [ 116.658739] niks3-server[1129]: 2026/09/22 15:05:07 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6355lb # [ 116.661512] niks3-server[1129]: 2026/09/22 15:05:07 INFO Completed upload id=476356worker1 # [ 116.656260] nix-grpc-daemon[1465]: time=2026-09-22T15:05:07.997Z level=INFO msg="Upload complete. (269ms)"6357lb # [ 116.664141] niks3-server[1129]: 2026/09/22 15:05:07 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6358worker1 # [ 116.658837] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43240 duration_s=0 paths=1 nar_bytes_in=7686359worker1 # [ 116.663800] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432406360worker1 # [ 116.674186] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6361worker1 # [ 116.676150] nix-daemon[865]: reaped child process 1551, status = succeeded6362worker1 # [ 116.681441] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6363client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6364client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6365client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6366client # farm-top> LOG-farm-top-on-standby6367worker1 # [ 117.912134] nix-daemon[865]: reaped child process 1565, status = succeeded6368worker1 # [ 117.982737] nix-daemon[865]: accepted connection from pid 1581, user nix-grpc-daemon (trusted)6369worker1 # [ 117.996341] nix-daemon[865]: reaped child process 1584, status = succeeded6370lb # [ 118.008340] niks3-server[1129]: 2026/09/22 15:05:09 INFO Received uploads request method=POST path=/api/pending_closures6371worker1 # [ 118.011721] nix-grpc-daemon[1465]: time=2026-09-22T15:05:09.353Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6372worker1 # [ 118.014067] nix-grpc-daemon[1465]: time=2026-09-22T15:05:09.354Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6373lb # [ 118.051090] niks3-server[1129]: 2026/09/22 15:05:09 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6374lb # [ 118.056716] niks3-server[1129]: 2026/09/22 15:05:09 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6375lb # [ 118.061715] niks3-server[1129]: 2026/09/22 15:05:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6376worker1 # [ 118.062548] nix-grpc-daemon[1465]: time=2026-09-22T15:05:09.403Z level=INFO msg="Uploading 1 narinfos"6377lb # [ 118.067459] niks3-server[1129]: 2026/09/22 15:05:09 INFO Signed narinfos id=48 count=16378lb # [ 118.074556] niks3-server[1129]: 2026/09/22 15:05:09 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6379lb # [ 118.082968] niks3-server[1129]: 2026/09/22 15:05:09 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6380lb # [ 118.086434] niks3-server[1129]: 2026/09/22 15:05:09 INFO Completed upload id=486381lb # [ 118.087805] niks3-server[1129]: 2026/09/22 15:05:09 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6382worker1 # [ 118.081336] nix-grpc-daemon[1465]: time=2026-09-22T15:05:09.422Z level=INFO msg="Upload complete. (192ms)"6383worker1 # [ 118.089320] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43240 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16384worker1 # [ 118.103290] nix-daemon[865]: reaped child process 1563, status = succeeded6385client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.76 seconds)6386client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6387client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.09 seconds)6388worker2 # [ 118.256699] systemd[1]: Listening on Nix worker-protocol over gRPC.6389worker2 # [ 118.264152] systemd[1]: Starting scheduler-token.service...6390worker2 # [ 118.307972] systemd[1]: scheduler-token.service: Deactivated successfully.6391worker2 # [ 118.309681] systemd[1]: Finished scheduler-token.service.6392worker2 # [ 118.312217] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 46ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6393worker2 # [ 118.315980] systemd[1]: Starting Nix worker-protocol over gRPC...6394worker2 # [ 118.386495] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6395worker2 # [ 118.393113] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6396worker2 # [ 118.399937] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16397worker2 # [ 118.404043] nix-grpc-daemon[1424]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6398worker2 # [ 118.406097] nix-grpc-daemon[1424]: E0000 00:00:1790089509.615995 1424 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6399worker2 # [ 118.411205] nix-grpc-daemon[1424]: E0000 00:00:1790089509.619133 1424 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6400worker2 # [ 118.414116] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6401worker2 # [ 118.418102] nix-grpc-daemon[1424]: E0000 00:00:1790089509.627791 1465 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6402worker2 # [ 118.421120] systemd[1]: Started Nix worker-protocol over gRPC.6403worker2 # [ 118.426045] nix-grpc-daemon[1424]: E0000 00:00:1790089509.649551 1465 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6404worker2 # [ 118.428140] nix-grpc-daemon[1424]: E0000 00:00:1790089509.649808 1465 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6405lb: must succeed: curl -sf localhost:9901/clusters6406worker2 # [ 118.440606] nix-grpc-daemon[1424]: E0000 00:00:1790089509.664378 1465 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6407worker2 # [ 118.452807] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:09Z level=info event=scheduler_connected addr=lb:500516408worker1 # [ 118.455172] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:09Z level=info event=oidc_jwks provider=mock keys=16409worker1 # [ 118.458934] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:419846410worker1 # [ 118.462983] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06411lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6412worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6413worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6414lb: must succeed: curl -sf localhost:9901/clusters6415lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6416lb: must succeed: curl -sf localhost:9901/clusters6417lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6418worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6419worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6420(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 11.57 seconds)6421subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6422client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6423client # [ 118.647492] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6424client # Running as unit: rs.service; invocation ID: 0d72554b15ca4cbf9194387c358536cf6425client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.03 seconds)6426worker1 # [ 118.741468] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:10Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:432406427client # [ 118.762893] nix[1368]: don't know how to build these paths:6428client # [ 118.763954] nix[1368]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6429worker1 # [ 118.754246] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:10Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06430worker1 # [ 118.759421] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:10Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16431worker1 # [ 118.797870] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432406432worker1 # [ 118.810308] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:432406433client # [ 118.834929] nix[1368]: copying 1 paths to 'lb:50051'...6434client # [ 118.836021] nix[1368]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6435worker1 # [ 118.821139] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:432406436worker1 # [ 118.830336] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6437worker1 # [ 118.930652] nix-daemon[865]: accepted connection from pid 1597, user nix-grpc-daemon (trusted)6438worker1 # [ 118.945560] nix-daemon[865]: reaped child process 1600, status = succeeded6439worker1 # [ 119.024422] nix-daemon[865]: accepted connection from pid 1602, user nix-grpc-daemon (trusted)6440worker1 # [ 119.037504] nix-daemon[865]: reaped child process 1605, status = succeeded6441lb # [ 119.050376] niks3-server[1129]: 2026/09/22 15:05:10 INFO Received uploads request method=POST path=/api/pending_closures6442worker1 # [ 119.053132] nix-grpc-daemon[1465]: time=2026-09-22T15:05:10.394Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6443worker1 # [ 119.054731] nix-grpc-daemon[1465]: time=2026-09-22T15:05:10.396Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6444lb # [ 119.077498] niks3-server[1129]: 2026/09/22 15:05:10 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6445lb # [ 119.088551] niks3-server[1129]: 2026/09/22 15:05:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6446worker1 # [ 119.090895] nix-grpc-daemon[1465]: time=2026-09-22T15:05:10.432Z level=INFO msg="Uploading 1 narinfos"6447lb # [ 119.096734] niks3-server[1129]: 2026/09/22 15:05:10 INFO Signed narinfos id=49 count=16448lb # [ 119.103312] niks3-server[1129]: 2026/09/22 15:05:10 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6449lb # [ 119.116977] niks3-server[1129]: 2026/09/22 15:05:10 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6450worker1 # [ 119.117131] nix-grpc-daemon[1465]: time=2026-09-22T15:05:10.458Z level=INFO msg="Upload complete. (274ms)"6451lb # [ 119.122829] niks3-server[1129]: 2026/09/22 15:05:10 INFO Completed upload id=496452lb # [ 119.125894] niks3-server[1129]: 2026/09/22 15:05:10 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6453worker1 # [ 119.119969] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:43240 duration_s=0 paths=1 nar_bytes_in=4166454worker1 # [ 119.125853] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:432406455worker1 # [ 119.136925] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6456worker1 # [ 119.140898] nix-daemon[865]: reaped child process 1595, status = succeeded6457client # [ 119.160148] nix[1368]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6458worker1 # [ 119.145669] nix-daemon[865]: accepted connection from pid 1389, user nix-grpc-daemon (trusted)6459lb: must succeed: curl -sf localhost:9901/clusters6460lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6461lb: must succeed: curl -sf localhost:9901/clusters6462lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6463lb: must succeed: curl -sf localhost:9901/clusters6464lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6465lb: must succeed: curl -sf localhost:9901/clusters6466lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6467worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6468worker2 # [ 121.186314] systemd-journald[283]: Received client request to rotate journal, rotating.6469worker2 # [ 121.196696] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/179d1bf382724742a5c58e5b48c89102.6470worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.03 seconds)6471worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6472worker1 # [ 121.089995] systemd[1]: Stopping Nix worker-protocol over gRPC...6473worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6474client: must succeed: sleep 5.06475worker1 # [ 121.803377] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:13Z level=info event=draining6476client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6477worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6478worker1 # [ 126.370271] nix-daemon[865]: reaped child process 1609, status = succeeded6479worker1 # [ 126.442971] nix-daemon[865]: accepted connection from pid 1650, user nix-grpc-daemon (trusted)6480worker1 # [ 126.456672] nix-daemon[865]: reaped child process 1653, status = succeeded6481lb # [ 126.468101] niks3-server[1129]: 2026/09/22 15:05:17 INFO Received uploads request method=POST path=/api/pending_closures6482worker1 # [ 126.467203] nix-grpc-daemon[1465]: time=2026-09-22T15:05:17.808Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6483worker1 # [ 126.468779] nix-grpc-daemon[1465]: time=2026-09-22T15:05:17.810Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6484lb # [ 126.490522] niks3-server[1129]: 2026/09/22 15:05:17 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6485lb # [ 126.495502] niks3-server[1129]: 2026/09/22 15:05:17 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6486lb # [ 126.497797] niks3-server[1129]: 2026/09/22 15:05:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6487lb # [ 126.499685] niks3-server[1129]: 2026/09/22 15:05:17 INFO Signed narinfos id=50 count=16488worker1 # [ 126.495843] nix-grpc-daemon[1465]: time=2026-09-22T15:05:17.837Z level=INFO msg="Uploading 1 narinfos"6489lb # [ 126.503451] niks3-server[1129]: 2026/09/22 15:05:17 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6490lb # [ 126.509053] niks3-server[1129]: 2026/09/22 15:05:17 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6491lb # [ 126.512717] niks3-server[1129]: 2026/09/22 15:05:17 INFO Completed upload id=506492lb # [ 126.514289] niks3-server[1129]: 2026/09/22 15:05:17 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6493worker1 # [ 126.509235] nix-grpc-daemon[1465]: time=2026-09-22T15:05:17.850Z level=INFO msg="Upload complete. (149ms)"6494worker1 # [ 126.513079] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:43240 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16495worker1 # [ 126.519910] nix-daemon[865]: reaped child process 1607, status = succeeded6496lb # [ 126.526742] envoy[916]: conn peer=[2001:db8:1::1]:48222 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9547 tx=4344 ms=56156497lb # [ 126.528314] envoy[916]: conn peer=[2001:db8:1::1]:57572 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=1539 ms=77826498client # [ 126.546292] systemd[1]: rs.service: Deactivated successfully.6499client # [ 126.548065] systemd[1]: rs.service: Consumed 105ms CPU time over 7.900s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6500worker1 # [ 126.805276] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:18Z level=info event=signal_exit6501lb # [ 126.975244] niks3-server[1129]: 2026/09/22 15:05:18 INFO lead: released remote=192.168.1.3:481586502worker1 # [ 127.969504] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:19Z level=info event=scheduler_restarting6503worker1 # [ 127.973614] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:19Z level=info event=scheduler_restarting addr=lb:500516504worker2 # [ 127.979788] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:19Z level=info event=scheduler_restarting addr=lb:500516505worker1 # [ 127.975171] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:419846506worker1 # [ 127.976732] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:19Z level=info event=worker_gone addr=192.168.1.4:500516507worker1 # [ 127.978908] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:19Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:419846508worker1 # [ 127.982674] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:19Z level=info event=worker_gone addr=192.168.1.3:500516509worker1 # [ 127.999399] nix-daemon[865]: reaped child process 1441, status = succeeded6510worker2 # [ 128.180344] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:19Z level=info event=scheduler_connected addr=lb:500516511worker2 # [ 128.184644] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6512worker1 # [ 128.182344] nix-grpc-daemon[1389]: E0000 00:00:1790089519.523739 1434 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 18, started true, shutdown true6513worker1 # [ 128.190956] nix-grpc-daemon[1389]: E0000 00:00:1790089519.529113 1434 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true6514worker1 # [ 128.196299] nix-grpc-daemon[1389]: ts=2026-09-22T15:05:19Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6515worker2 # [ 128.385987] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:19Z level=info event=scheduler_connected addr=lb:500516516worker2 # [ 128.390317] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6517lb # [ 128.402681] niks3-server[1129]: 2026/09/22 15:05:19 INFO lead: acquired remote=192.168.1.4:359246518worker2 # [ 128.404457] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:19Z level=info event=scheduler_take_over6519worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 2.10 seconds)6520client: waiting for success: ! systemctl is-active rs6521client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6522client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6523client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6524client: must fail: journalctl -u rs | grep -q 'reconnecting'6525client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6526lb: must succeed: curl -sf localhost:9901/clusters6527lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6528worker1 # [ 128.562661] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6529worker1 # [ 128.565570] systemd[1]: Stopped Nix worker-protocol over gRPC.6530worker1 # [ 128.568365] systemd[1]: nix-grpc-daemon.service: Consumed 1.311s CPU time over 28.877s wall clock time, 39M memory peak, 47.8K incoming IP traffic, 100.5K outgoing IP traffic.6531worker2 # [ 128.588371] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:19Z level=info event=scheduler_connected addr=lb:500516532worker1 # [ 128.583842] systemd[1]: Starting Nix worker-protocol over gRPC...6533worker2 # [ 128.592306] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:19Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6534worker1 # [ 128.672719] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6535worker1 # [ 128.674740] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6536worker1 # [ 128.677932] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16537worker1 # [ 128.683058] nix-grpc-daemon[1660]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6538worker1 # [ 128.684412] nix-grpc-daemon[1660]: E0000 00:00:1790089520.024092 1660 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6539worker1 # [ 128.690323] nix-grpc-daemon[1660]: E0000 00:00:1790089520.029032 1660 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6540worker1 # [ 128.693544] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6541worker1 # [ 128.696208] systemd[1]: Started Nix worker-protocol over gRPC.6542worker1 # [ 128.698064] nix-grpc-daemon[1660]: E0000 00:00:1790089520.039130 1701 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6543worker1 # [ 128.709816] nix-grpc-daemon[1660]: E0000 00:00:1790089520.051252 1701 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6544worker1 # [ 128.714142] nix-grpc-daemon[1660]: E0000 00:00:1790089520.053568 1701 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6545worker1 # [ 128.724118] nix-grpc-daemon[1660]: E0000 00:00:1790089520.065608 1701 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6546worker1 # [ 128.733389] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=scheduler_connected addr=lb:500516547worker1 # [ 128.734975] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6548worker2 # [ 128.793389] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=scheduler_connected addr=lb:500516549worker2 # [ 128.797370] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6550worker1 # [ 128.935959] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=scheduler_connected addr=lb:500516551worker1 # [ 128.939930] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6552worker2 # [ 129.003548] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=scheduler_connected addr=lb:500516553worker2 # [ 129.006797] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6554worker1 # [ 129.140703] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=scheduler_connected addr=lb:500516555worker1 # [ 129.144598] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6556worker2 # [ 129.204775] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=scheduler_connected addr=lb:500516557worker2 # [ 129.208657] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6558worker1 # [ 129.345618] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:20Z level=info event=scheduler_connected addr=lb:500516559worker2 # [ 129.358196] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:542986560worker2 # [ 129.362733] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06561worker2 # [ 129.409535] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=scheduler_connected addr=lb:500516562worker2 # [ 129.419139] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=oidc_jwks provider=mock keys=16563worker2 # [ 129.422744] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:542986564worker2 # [ 129.426719] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:20Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06565lb: must succeed: curl -sf localhost:9901/clusters6566lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6567worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6568worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6569(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.00 seconds)6570subtest: scheduler restart mid-build: build finishes, no double build6571client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6572client # [ 129.669902] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6573client # Running as unit: mid.service; invocation ID: e92f5200ab374597b510a6c20456fc076574client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6575worker2 # [ 129.793671] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:543146576worker2 # [ 129.805781] nix-daemon[850]: accepted connection from pid 1424, user nix-grpc-daemon (trusted)6577client # [ 129.825611] nix[1404]: don't know how to build these paths:6578client # [ 129.826523] nix[1404]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6579worker2 # [ 129.824505] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:21Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06580worker2 # [ 129.829311] nix-grpc-daemon[1424]: ts=2026-09-22T15:05:21Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16581worker1 # [ 129.871469] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:21Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454986582worker1 # [ 129.876448] nix-daemon[865]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6583worker1 # [ 129.883907] nix-grpc-daemon[1660]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 92 ms (attempt 1/5)6584worker1 # [ 129.980390] nix-grpc-daemon[1660]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 114 ms (attempt 2/5)6585worker1 # [ 130.096596] nix-grpc-daemon[1660]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 151 ms (attempt 3/5)6586worker1 # [ 130.249758] nix-grpc-daemon[1660]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 339 ms (attempt 4/5)6587worker1 # [ 130.590881] nix-grpc-daemon[1660]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6588worker1 # [ 130.594395] nix-grpc-daemon[1660]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6589worker1 # [ 130.611362] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454986590client # [ 130.635982] nix[1404]: copying 1 paths to 'lb:50051'...6591client # [ 130.639915] nix[1404]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6592worker1 # [ 130.626061] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:21Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454986593worker1 # [ 130.634352] nix-daemon[865]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6594worker1 # [ 130.656193] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:21Z level=info event=niks3_push_started pid=17246595worker1 # [ 130.761913] nix-grpc-daemon[1724]: time=2026-09-22T15:05:22.102Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6596worker1 # [ 130.854877] nix-daemon[865]: accepted connection from pid 1740, user nix-grpc-daemon (trusted)6597worker1 # [ 130.868994] nix-daemon[865]: reaped child process 1743, status = succeeded6598worker1 # [ 130.951244] nix-daemon[865]: accepted connection from pid 1745, user nix-grpc-daemon (trusted)6599worker1 # [ 130.965112] nix-daemon[865]: reaped child process 1748, status = succeeded6600lb # [ 130.976667] niks3-server[1129]: 2026/09/22 15:05:22 INFO Received uploads request method=POST path=/api/pending_closures6601worker1 # [ 130.980065] nix-grpc-daemon[1724]: time=2026-09-22T15:05:22.320Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6602worker1 # [ 130.981505] nix-grpc-daemon[1724]: time=2026-09-22T15:05:22.321Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6603lb # [ 131.010893] niks3-server[1129]: 2026/09/22 15:05:22 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6604lb # [ 131.017904] niks3-server[1129]: 2026/09/22 15:05:22 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6605worker1 # [ 131.020159] nix-grpc-daemon[1724]: time=2026-09-22T15:05:22.361Z level=INFO msg="Uploading 1 narinfos"6606lb # [ 131.025367] niks3-server[1129]: 2026/09/22 15:05:22 INFO Signed narinfos id=51 count=16607lb # [ 131.036494] niks3-server[1129]: 2026/09/22 15:05:22 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6608lb # [ 131.046218] niks3-server[1129]: 2026/09/22 15:05:22 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6609lb # [ 131.053347] niks3-server[1129]: 2026/09/22 15:05:22 INFO Completed upload id=516610worker1 # [ 131.047672] nix-grpc-daemon[1724]: time=2026-09-22T15:05:22.389Z level=INFO msg="Upload complete. (282ms)"6611lb # [ 131.056430] niks3-server[1129]: 2026/09/22 15:05:22 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6612worker1 # [ 131.052485] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:22Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45498 duration_s=0 paths=1 nar_bytes_in=4246613worker1 # [ 131.058354] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:22Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454986614worker1 # [ 131.068264] nix-daemon[865]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6615worker1 # [ 131.073106] nix-daemon[865]: reaped child process 1722, status = succeeded6616client # [ 131.091143] nix[1404]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6617worker1 # [ 131.076352] nix-daemon[865]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6618lb: must succeed: curl -sf localhost:9901/clusters6619lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6620lb: must succeed: curl -sf localhost:9901/clusters6621lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6622worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6623worker2 # [ 133.225966] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1424 (nix-grpc-daemon) on client request.6624worker2 # [ 133.239813] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6625lb # [ 133.246880] niks3-server[1129]: 2026/09/22 15:05:24 INFO lead: released remote=192.168.1.4:359246626worker1 # [ 133.246325] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="Stream removed (RST_STREAM (Received RST_STREAM with error code 2))"6627client # [ 133.261988] nix[1404]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6628worker2 # [ 133.256598] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6629worker2 # [ 133.263499] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6630worker2 # [ 133.268336] systemd[1]: nix-grpc-daemon.service: Consumed 202ms CPU time over 14.940s wall clock time, 13.4M memory peak, 10.6K incoming IP traffic, 22K outgoing IP traffic.6631worker2 # [ 133.285156] systemd[1]: Starting scheduler-token.service...6632worker2 # [ 133.289695] nix-daemon[850]: reaped child process 1533, status = succeeded6633worker2 # [ 133.319709] systemd[1]: scheduler-token.service: Deactivated successfully.6634worker2 # [ 133.322326] systemd[1]: Finished scheduler-token.service.6635worker2 # [ 133.323276] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 36ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6636worker2 # [ 133.325762] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6637worker2 # [ 133.329522] systemd[1]: Starting scheduler-token.service...6638worker2 # [ 133.360838] systemd[1]: scheduler-token.service: Deactivated successfully.6639worker2 # [ 133.362678] systemd[1]: Finished scheduler-token.service.6640worker2 # [ 133.364139] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 33ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6641worker2 # [ 133.369161] systemd[1]: Starting Nix worker-protocol over gRPC...6642worker2 # [ 133.439215] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6643worker2 # [ 133.441430] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6644worker2 # [ 133.444885] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16645worker2 # [ 133.450051] nix-grpc-daemon[1567]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6646worker2 # [ 133.451565] nix-grpc-daemon[1567]: E0000 00:00:1790089524.673325 1567 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6647worker1 # [ 133.446605] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:24Z level=info event=scheduler_connected addr=lb:500516648worker2 # [ 133.459567] nix-grpc-daemon[1567]: E0000 00:00:1790089524.679040 1567 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6649worker2 # [ 133.463936] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6650worker2 # [ 133.469972] systemd[1]: Started Nix worker-protocol over gRPC.6651lb # [ 133.468683] niks3-server[1129]: 2026/09/22 15:05:24 INFO lead: acquired remote=192.168.1.4:529766652worker2 # [ 133.472173] nix-grpc-daemon[1567]: E0000 00:00:1790089524.692603 1608 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6653worker1 # [ 133.467153] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:24Z level=info event=scheduler_disconnected addr=lb:50051 error="scheduler passive, another node serves"6654worker2 # [ 133.478175] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=scheduler_take_over6655worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.29 seconds)6656client: must succeed: sleep 5.06657worker2 # [ 133.487280] nix-grpc-daemon[1567]: E0000 00:00:1790089524.711052 1608 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6658worker2 # [ 133.491125] nix-grpc-daemon[1567]: E0000 00:00:1790089524.713612 1608 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6659worker2 # [ 133.502365] nix-grpc-daemon[1567]: E0000 00:00:1790089524.726097 1608 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6660worker2 # [ 133.511891] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=scheduler_connected addr=lb:500516661worker2 # [ 133.517392] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=oidc_jwks provider=mock keys=16662worker2 # [ 133.518943] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:375466663worker2 # [ 133.520777] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06664worker1 # [ 133.667701] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:25Z level=info event=scheduler_connected addr=lb:500516665worker2 # [ 133.677465] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:375466666worker2 # [ 133.682138] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:24Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=16667lb # [ 136.527910] envoy[916]: conn peer=[2001:db8:1::3]:51500 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4581 tx=1106 ms=283346668lb # [ 136.532367] envoy[916]: conn peer=[2001:db8:1::4]:54846 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6953 tx=546 ms=148076669client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6670client: waiting for success: ! systemctl is-active mid6671worker1 # [ 138.700396] nix-daemon[865]: reaped child process 1752, status = succeeded6672worker1 # [ 138.772389] nix-daemon[865]: accepted connection from pid 1789, user nix-grpc-daemon (trusted)6673worker1 # [ 138.785757] nix-daemon[865]: reaped child process 1792, status = succeeded6674lb # [ 138.798506] niks3-server[1129]: 2026/09/22 15:05:30 INFO Received uploads request method=POST path=/api/pending_closures6675worker1 # [ 138.801040] nix-grpc-daemon[1724]: time=2026-09-22T15:05:30.142Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6676worker1 # [ 138.802977] nix-grpc-daemon[1724]: time=2026-09-22T15:05:30.144Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6677lb # [ 138.860807] niks3-server[1129]: 2026/09/22 15:05:30 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6678lb # [ 138.868928] niks3-server[1129]: 2026/09/22 15:05:30 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6679lb # [ 138.874974] niks3-server[1129]: 2026/09/22 15:05:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6680lb # [ 138.880878] niks3-server[1129]: 2026/09/22 15:05:30 INFO Signed narinfos id=52 count=16681worker1 # [ 138.875592] nix-grpc-daemon[1724]: time=2026-09-22T15:05:30.217Z level=INFO msg="Uploading 1 narinfos"6682lb # [ 138.884960] niks3-server[1129]: 2026/09/22 15:05:30 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6683lb # [ 138.890776] niks3-server[1129]: 2026/09/22 15:05:30 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6684lb # [ 138.897326] niks3-server[1129]: 2026/09/22 15:05:30 INFO Completed upload id=526685worker1 # [ 138.892304] nix-grpc-daemon[1724]: time=2026-09-22T15:05:30.233Z level=INFO msg="Upload complete. (203ms)"6686lb # [ 138.900163] niks3-server[1129]: 2026/09/22 15:05:30 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6687worker1 # [ 138.898985] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45498 duration_s=7 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16688worker1 # [ 138.913473] nix-daemon[865]: reaped child process 1750, status = succeeded6689client # [ 138.945971] systemd[1]: mid.service: Deactivated successfully.6690client # [ 138.949868] systemd[1]: mid.service: Consumed 127ms CPU time over 9.276s wall clock time, 43.4M memory peak, 7.8K incoming IP traffic, 8.8K outgoing IP traffic.6691client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6692client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6693client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6694(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 10.18 seconds)6695subtest: interrupting the client stops the build on the worker6696client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6697client # [ 139.832337] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6698client # Running as unit: intr.service; invocation ID: 5875aa67c10444e89cab18a49ee188b26699client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6700worker2 # [ 139.950439] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375646701worker2 # [ 139.961237] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6702client # [ 139.980887] nix[1440]: don't know how to build these paths:6703client # [ 139.981837] nix[1440]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6704worker2 # [ 139.979332] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:31Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06705worker2 # [ 139.982879] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:31Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16706worker2 # [ 140.030835] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375646707worker2 # [ 140.041991] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 96 ms (attempt 1/5)6708worker2 # [ 140.144935] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 126 ms (attempt 2/5)6709worker2 # [ 140.272094] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 191 ms (attempt 3/5)6710worker2 # [ 140.464247] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 146 ms (attempt 4/5)6711worker2 # [ 140.613872] nix-grpc-daemon[1567]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6712worker2 # [ 140.617435] nix-grpc-daemon[1567]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6713worker2 # [ 140.632057] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375646714client # [ 140.648587] nix[1440]: copying 1 paths to 'lb:50051'...6715client # [ 140.652127] nix[1440]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6716worker2 # [ 140.644379] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375646717worker2 # [ 140.654156] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6718worker2 # [ 140.674158] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:31Z level=info event=niks3_push_started pid=16416719worker2 # [ 140.776583] nix-grpc-daemon[1641]: time=2026-09-22T15:05:32.000Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6720worker2 # [ 140.862391] nix-daemon[850]: accepted connection from pid 1654, user nix-grpc-daemon (trusted)6721worker2 # [ 140.875933] nix-daemon[850]: reaped child process 1657, status = succeeded6722worker2 # [ 140.955840] nix-daemon[850]: accepted connection from pid 1659, user nix-grpc-daemon (trusted)6723worker2 # [ 140.969123] nix-daemon[850]: reaped child process 1662, status = succeeded6724lb # [ 140.976073] niks3-server[1129]: 2026/09/22 15:05:32 INFO Received uploads request method=POST path=/api/pending_closures6725worker2 # [ 140.985734] nix-grpc-daemon[1641]: time=2026-09-22T15:05:32.208Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6726worker2 # [ 140.987233] nix-grpc-daemon[1641]: time=2026-09-22T15:05:32.208Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6727lb # [ 141.005858] niks3-server[1129]: 2026/09/22 15:05:32 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6728lb # [ 141.016374] niks3-server[1129]: 2026/09/22 15:05:32 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6729lb # [ 141.022294] niks3-server[1129]: 2026/09/22 15:05:32 INFO Signed narinfos id=53 count=16730worker2 # [ 141.027404] nix-grpc-daemon[1641]: time=2026-09-22T15:05:32.250Z level=INFO msg="Uploading 1 narinfos"6731lb # [ 141.034539] niks3-server[1129]: 2026/09/22 15:05:32 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6732lb # [ 141.044209] niks3-server[1129]: 2026/09/22 15:05:32 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6733lb # [ 141.052224] niks3-server[1129]: 2026/09/22 15:05:32 INFO Completed upload id=536734worker2 # [ 141.054299] nix-grpc-daemon[1641]: time=2026-09-22T15:05:32.278Z level=INFO msg="Upload complete. (274ms)"6735lb # [ 141.054458] niks3-server[1129]: 2026/09/22 15:05:32 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6736worker2 # [ 141.057069] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:32Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 paths=1 nar_bytes_in=4246737worker2 # [ 141.060678] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:32Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375646738worker2 # [ 141.067309] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6739worker2 # [ 141.069595] nix-daemon[850]: reaped child process 1639, status = succeeded6740worker2 # [ 141.074556] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6741client # [ 141.082601] nix[1440]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6742client: must succeed: systemctl kill -s INT intr6743client # [ 143.185902] systemd[1]: intr.service: Sent signal SIGINT to main process 1440 (nix) on client request.6744client # [ 143.187851] nix[1440]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6745client # [ 143.190561] nix[1440]: error: interrupted by the user6746worker2 # [ 143.182221] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:34Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6747client: (finished: must succeed: systemctl kill -s INT intr, in 0.02 seconds)6748client # [ 143.207149] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6749client # [ 143.208739] systemd[1]: intr.service: Failed with result 'exit-code'.6750client # [ 143.210545] systemd[1]: intr.service: Consumed 113ms CPU time over 3.376s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6751worker2 # [ 143.293553] nix-daemon[1670]: unexpected Nix daemon error: error: interrupted by the user6752worker2 # [ 143.296322] nix-daemon[850]: reaped child process 1668, status = succeeded6753worker2 # [ 143.299680] nix-daemon[850]: reaped child process 1670, status = failed with exit code 16754(finished: subtest: interrupting the client stops the build on the worker, in 4.56 seconds)6755subtest: a deploy mid-build drains: the build finishes, the new generation takes over6756client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6757client # [ 144.399615] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6758client # Running as unit: sw.service; invocation ID: 28f221e03d314d9eb5f3ef4852a890286759client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.04 seconds)6760worker2 # [ 144.522405] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375646761client # [ 144.531503] nix[1461]: don't know how to build these paths:6762client # [ 144.532367] nix[1461]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6763worker2 # [ 144.528801] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:35Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06764worker2 # [ 144.530509] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:35Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16765worker1 # [ 144.567245] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454986766worker1 # [ 144.580430] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:454986767client # [ 144.606128] nix[1461]: copying 1 paths to 'lb:50051'...6768client # [ 144.607193] nix[1461]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6769worker1 # [ 144.592127] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:454986770worker1 # [ 144.604248] nix-daemon[865]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6771worker1 # [ 144.705185] nix-daemon[865]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6772worker1 # [ 144.718987] nix-daemon[865]: reaped child process 1827, status = succeeded6773worker1 # [ 144.799826] nix-daemon[865]: accepted connection from pid 1829, user nix-grpc-daemon (trusted)6774worker1 # [ 144.813532] nix-daemon[865]: reaped child process 1832, status = succeeded6775lb # [ 144.825233] niks3-server[1129]: 2026/09/22 15:05:36 INFO Received uploads request method=POST path=/api/pending_closures6776worker1 # [ 144.828347] nix-grpc-daemon[1724]: time=2026-09-22T15:05:36.169Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6777worker1 # [ 144.829774] nix-grpc-daemon[1724]: time=2026-09-22T15:05:36.169Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6778lb # [ 144.851752] niks3-server[1129]: 2026/09/22 15:05:36 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6779lb # [ 144.862955] niks3-server[1129]: 2026/09/22 15:05:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6780worker1 # [ 144.864433] nix-grpc-daemon[1724]: time=2026-09-22T15:05:36.205Z level=INFO msg="Uploading 1 narinfos"6781lb # [ 144.869564] niks3-server[1129]: 2026/09/22 15:05:36 INFO Signed narinfos id=54 count=16782lb # [ 144.874217] niks3-server[1129]: 2026/09/22 15:05:36 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6783lb # [ 144.888441] niks3-server[1129]: 2026/09/22 15:05:36 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6784worker1 # [ 144.890023] nix-grpc-daemon[1724]: time=2026-09-22T15:05:36.231Z level=INFO msg="Upload complete. (274ms)"6785lb # [ 144.895343] niks3-server[1129]: 2026/09/22 15:05:36 INFO Completed upload id=546786worker1 # [ 144.892127] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:45498 duration_s=0 paths=1 nar_bytes_in=4166787lb # [ 144.899668] niks3-server[1129]: 2026/09/22 15:05:36 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6788worker1 # [ 144.896871] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:454986789worker1 # [ 144.906927] nix-daemon[865]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6790worker1 # [ 144.909268] nix-daemon[865]: reaped child process 1822, status = succeeded6791client # [ 144.928761] nix[1461]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6792worker1 # [ 144.914484] nix-daemon[865]: accepted connection from pid 1660, user nix-grpc-daemon (trusted)6793worker1: must succeed: systemctl reload nix-grpc-daemon.service6794worker1 # [ 145.647234] systemd[1]: Reloading Nix worker-protocol over gRPC...6795worker1 # [ 145.673607] systemd[1]: Reloaded Nix worker-protocol over gRPC.6796worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6797worker1: must succeed: pgrep -f 'read -t [9]0 x'6798worker1 # [ 145.695323] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:37Z level=info event=draining6799worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6800client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26801client # don't know how to build these paths:6802client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6803worker2 # [ 145.842903] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375646804worker2 # [ 145.850154] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375646805worker2 # [ 145.870254] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06806worker2 # [ 145.875919] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16807worker2 # [ 145.919908] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375646808worker2 # [ 145.933137] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375646809client # copying 1 paths to 'lb:50051'...6810worker2 # [ 145.948596] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375646811client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6812worker2 # [ 145.976496] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6813worker2 # [ 146.081807] nix-daemon[850]: accepted connection from pid 1722, user nix-grpc-daemon (trusted)6814worker2 # [ 146.095177] nix-daemon[850]: reaped child process 1725, status = succeeded6815worker2 # [ 146.178230] nix-daemon[850]: accepted connection from pid 1727, user nix-grpc-daemon (trusted)6816worker2 # [ 146.191566] nix-daemon[850]: reaped child process 1730, status = succeeded6817lb # [ 146.197106] niks3-server[1129]: 2026/09/22 15:05:37 INFO Received uploads request method=POST path=/api/pending_closures6818worker2 # [ 146.206161] nix-grpc-daemon[1641]: time=2026-09-22T15:05:37.429Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6819worker2 # [ 146.207723] nix-grpc-daemon[1641]: time=2026-09-22T15:05:37.429Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6820lb # [ 146.221411] niks3-server[1129]: 2026/09/22 15:05:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6821worker2 # [ 146.228422] nix-grpc-daemon[1641]: time=2026-09-22T15:05:37.452Z level=INFO msg="Uploading 1 narinfos"6822lb # [ 146.226313] niks3-server[1129]: 2026/09/22 15:05:37 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6823lb # [ 146.228167] niks3-server[1129]: 2026/09/22 15:05:37 INFO Signed narinfos id=55 count=16824lb # [ 146.244086] niks3-server[1129]: 2026/09/22 15:05:37 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6825lb # [ 146.247516] niks3-server[1129]: 2026/09/22 15:05:37 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6826lb # [ 146.250091] niks3-server[1129]: 2026/09/22 15:05:37 INFO Completed upload id=556827worker2 # [ 146.251996] nix-grpc-daemon[1641]: time=2026-09-22T15:05:37.475Z level=INFO msg="Upload complete. (263ms)"6828worker2 # [ 146.254088] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 paths=1 nar_bytes_in=5286829lb # [ 146.253073] niks3-server[1129]: 2026/09/22 15:05:37 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6830worker2 # [ 146.257705] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375646831worker2 # [ 146.264707] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6832worker2 # [ 146.267160] nix-daemon[850]: reaped child process 1720, status = succeeded6833worker2 # [ 146.272488] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6834client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6835client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6836lb # [ 146.527753] envoy[916]: conn peer=[2001:db8:1::1]:44280 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3087 tx=1606 ms=91326837lb # [ 146.529392] envoy[916]: conn peer=[2001:db8:1::1]:41392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3006 tx=829 ms=32486838client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6839client # farm-a> LOG-farm-a-during-drain6840worker2 # [ 147.722722] nix-daemon[850]: reaped child process 1734, status = succeeded6841worker2 # [ 147.785088] nix-daemon[850]: accepted connection from pid 1750, user nix-grpc-daemon (trusted)6842worker2 # [ 147.797529] nix-daemon[850]: reaped child process 1753, status = succeeded6843lb # [ 147.800510] niks3-server[1129]: 2026/09/22 15:05:39 INFO Received uploads request method=POST path=/api/pending_closures6844worker2 # [ 147.807987] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.031Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6845worker2 # [ 147.812440] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.033Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6846lb # [ 147.819813] niks3-server[1129]: 2026/09/22 15:05:39 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6847lb # [ 147.830510] niks3-server[1129]: 2026/09/22 15:05:39 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6848lb # [ 147.841652] niks3-server[1129]: 2026/09/22 15:05:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6849lb # [ 147.845106] niks3-server[1129]: 2026/09/22 15:05:39 INFO Signed narinfos id=56 count=16850worker2 # [ 147.848770] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.072Z level=INFO msg="Uploading 1 narinfos"6851lb # [ 147.847509] niks3-server[1129]: 2026/09/22 15:05:39 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6852lb # [ 147.853959] niks3-server[1129]: 2026/09/22 15:05:39 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6853lb # [ 147.859700] niks3-server[1129]: 2026/09/22 15:05:39 INFO Completed upload id=566854worker2 # [ 147.861671] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.085Z level=INFO msg="Upload complete. (181ms)"6855lb # [ 147.861498] niks3-server[1129]: 2026/09/22 15:05:39 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6856worker2 # [ 147.864521] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:39Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=3 outputs=16857worker2 # [ 147.871170] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:39Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06858worker2 # [ 147.873563] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:39Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16859worker2 # [ 147.878436] nix-daemon[850]: reaped child process 1732, status = succeeded6860worker2 # [ 147.915274] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375646861worker2 # [ 147.926419] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375646862worker2 # [ 147.938791] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.162Z level=INFO msg="All 1 paths already cached"6863client # copying 1 paths to 'lb:50051'...6864worker2 # [ 147.946883] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375646865client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6866worker2 # [ 147.965132] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6867worker2 # [ 148.062464] nix-daemon[850]: accepted connection from pid 1759, user nix-grpc-daemon (trusted)6868worker2 # [ 148.075605] nix-daemon[850]: reaped child process 1762, status = succeeded6869worker2 # [ 148.159748] nix-daemon[850]: accepted connection from pid 1764, user nix-grpc-daemon (trusted)6870worker2 # [ 148.173738] nix-daemon[850]: reaped child process 1767, status = succeeded6871lb # [ 148.175651] niks3-server[1129]: 2026/09/22 15:05:39 INFO Received uploads request method=POST path=/api/pending_closures6872worker2 # [ 148.190085] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.413Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6873worker2 # [ 148.191518] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.413Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6874lb # [ 148.212845] niks3-server[1129]: 2026/09/22 15:05:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6875lb # [ 148.219110] niks3-server[1129]: 2026/09/22 15:05:39 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6876worker2 # [ 148.225345] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.448Z level=INFO msg="Uploading 1 narinfos"6877lb # [ 148.223703] niks3-server[1129]: 2026/09/22 15:05:39 INFO Signed narinfos id=57 count=16878lb # [ 148.230135] niks3-server[1129]: 2026/09/22 15:05:39 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6879lb # [ 148.241906] niks3-server[1129]: 2026/09/22 15:05:39 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6880lb # [ 148.248090] niks3-server[1129]: 2026/09/22 15:05:39 INFO Completed upload id=576881worker2 # [ 148.250038] nix-grpc-daemon[1641]: time=2026-09-22T15:05:39.473Z level=INFO msg="Upload complete. (274ms)"6882lb # [ 148.251253] niks3-server[1129]: 2026/09/22 15:05:39 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6883worker2 # [ 148.254121] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 paths=1 nar_bytes_in=6406884worker2 # [ 148.258810] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375646885worker2 # [ 148.269943] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6886worker2 # [ 148.276122] nix-daemon[850]: reaped child process 1757, status = succeeded6887worker2 # [ 148.279178] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6888client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6889client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6890client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6891client # farm-b> LOG-farm-b-during-drain6892worker2 # [ 149.095943] nix-daemon[850]: reaped child process 1771, status = succeeded6893worker2 # [ 149.170851] nix-daemon[850]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6894worker2 # [ 149.184072] nix-daemon[850]: reaped child process 1790, status = succeeded6895lb # [ 149.189258] niks3-server[1129]: 2026/09/22 15:05:40 INFO Received uploads request method=POST path=/api/pending_closures6896worker2 # [ 149.195487] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.419Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6897worker2 # [ 149.197633] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.420Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6898lb # [ 149.207412] niks3-server[1129]: 2026/09/22 15:05:40 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6899lb # [ 149.212238] niks3-server[1129]: 2026/09/22 15:05:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6900lb # [ 149.218253] niks3-server[1129]: 2026/09/22 15:05:40 INFO Signed narinfos id=58 count=16901worker2 # [ 149.223186] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.446Z level=INFO msg="Uploading 1 narinfos"6902lb # [ 149.224872] niks3-server[1129]: 2026/09/22 15:05:40 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6903lb # [ 149.230667] niks3-server[1129]: 2026/09/22 15:05:40 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6904lb # [ 149.234685] niks3-server[1129]: 2026/09/22 15:05:40 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6905lb # [ 149.237319] niks3-server[1129]: 2026/09/22 15:05:40 INFO Completed upload id=586906worker2 # [ 149.239315] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.463Z level=INFO msg="Upload complete. (153ms)"6907lb # [ 149.239325] niks3-server[1129]: 2026/09/22 15:05:40 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6908worker2 # [ 149.242192] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=4 outputs=16909worker2 # [ 149.249472] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:40Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06910worker2 # [ 149.251651] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:40Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16911worker2 # [ 149.255922] nix-daemon[850]: reaped child process 1769, status = succeeded6912worker2 # [ 149.292995] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375646913worker2 # [ 149.304270] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:40Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375646914worker2 # [ 149.315348] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.539Z level=INFO msg="All 2 paths already cached"6915client # copying 1 paths to 'lb:50051'...6916worker2 # [ 149.323302] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:40Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375646917client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6918worker2 # [ 149.342117] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6919worker2 # [ 149.437063] nix-daemon[850]: accepted connection from pid 1796, user nix-grpc-daemon (trusted)6920worker2 # [ 149.450460] nix-daemon[850]: reaped child process 1799, status = succeeded6921worker2 # [ 149.535985] nix-daemon[850]: accepted connection from pid 1801, user nix-grpc-daemon (trusted)6922worker2 # [ 149.549491] nix-daemon[850]: reaped child process 1804, status = succeeded6923lb # [ 149.554524] niks3-server[1129]: 2026/09/22 15:05:40 INFO Received uploads request method=POST path=/api/pending_closures6924worker2 # [ 149.563727] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.787Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6925worker2 # [ 149.565514] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.789Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6926lb # [ 149.585683] niks3-server[1129]: 2026/09/22 15:05:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6927worker2 # [ 149.595283] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.818Z level=INFO msg="Uploading 1 narinfos"6928lb # [ 149.593145] niks3-server[1129]: 2026/09/22 15:05:40 INFO Signed narinfos id=59 count=16929lb # [ 149.599753] niks3-server[1129]: 2026/09/22 15:05:40 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6930lb # [ 149.614123] niks3-server[1129]: 2026/09/22 15:05:40 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6931lb # [ 149.619534] niks3-server[1129]: 2026/09/22 15:05:40 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6932lb # [ 149.625739] niks3-server[1129]: 2026/09/22 15:05:40 INFO Completed upload id=596933worker2 # [ 149.627880] nix-grpc-daemon[1641]: time=2026-09-22T15:05:40.851Z level=INFO msg="Upload complete. (275ms)"6934lb # [ 149.629182] niks3-server[1129]: 2026/09/22 15:05:40 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6935worker2 # [ 149.632158] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:40Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 paths=1 nar_bytes_in=7686936worker2 # [ 149.637404] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:40Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375646937worker2 # [ 149.646692] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6938worker2 # [ 149.654850] nix-daemon[850]: reaped child process 1794, status = succeeded6939worker2 # [ 149.659808] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)6940client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6941client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6942client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6943client # farm-top> LOG-farm-top-during-drain6944worker2 # [ 150.073509] nix-daemon[850]: reaped child process 1808, status = succeeded6945worker2 # [ 150.142546] nix-daemon[850]: accepted connection from pid 1824, user nix-grpc-daemon (trusted)6946worker2 # [ 150.156609] nix-daemon[850]: reaped child process 1827, status = succeeded6947lb # [ 150.158666] niks3-server[1129]: 2026/09/22 15:05:41 INFO Received uploads request method=POST path=/api/pending_closures6948worker2 # [ 150.166498] nix-grpc-daemon[1641]: time=2026-09-22T15:05:41.390Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6949worker2 # [ 150.168044] nix-grpc-daemon[1641]: time=2026-09-22T15:05:41.391Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6950lb # [ 150.190983] niks3-server[1129]: 2026/09/22 15:05:41 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6951lb # [ 150.205301] niks3-server[1129]: 2026/09/22 15:05:41 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6952lb # [ 150.211296] niks3-server[1129]: 2026/09/22 15:05:41 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6953worker2 # [ 150.221988] nix-grpc-daemon[1641]: time=2026-09-22T15:05:41.445Z level=INFO msg="Uploading 1 narinfos"6954lb # [ 150.219723] niks3-server[1129]: 2026/09/22 15:05:41 INFO Signed narinfos id=60 count=16955lb # [ 150.225437] niks3-server[1129]: 2026/09/22 15:05:41 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6956lb # [ 150.235889] niks3-server[1129]: 2026/09/22 15:05:41 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6957lb # [ 150.238745] niks3-server[1129]: 2026/09/22 15:05:41 INFO Completed upload id=606958worker2 # [ 150.240776] nix-grpc-daemon[1641]: time=2026-09-22T15:05:41.464Z level=INFO msg="Upload complete. (175ms)"6959lb # [ 150.240784] niks3-server[1129]: 2026/09/22 15:05:41 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6960worker2 # [ 150.243498] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:41Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=5 outputs=16961worker2 # [ 150.250993] nix-daemon[850]: reaped child process 1806, status = succeeded6962client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.60 seconds)6963client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6964client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.12 seconds)6965worker1: must succeed: pkill -f 'read -t [9]0 x'6966worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6967client: waiting for success: ! systemctl is-active sw6968worker1 # [ 150.536428] nix-daemon[865]: reaped child process 1836, status = succeeded6969worker1 # [ 150.612457] nix-daemon[865]: accepted connection from pid 1873, user nix-grpc-daemon (trusted)6970worker1 # [ 150.626048] nix-daemon[865]: reaped child process 1876, status = succeeded6971lb # [ 150.638521] niks3-server[1129]: 2026/09/22 15:05:41 INFO Received uploads request method=POST path=/api/pending_closures6972worker1 # [ 150.640985] nix-grpc-daemon[1724]: time=2026-09-22T15:05:41.982Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6973worker1 # [ 150.642531] nix-grpc-daemon[1724]: time=2026-09-22T15:05:41.984Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6974lb # [ 150.675482] niks3-server[1129]: 2026/09/22 15:05:41 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6975lb # [ 150.685109] niks3-server[1129]: 2026/09/22 15:05:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6976lb # [ 150.690715] niks3-server[1129]: 2026/09/22 15:05:42 INFO Signed narinfos id=61 count=16977worker1 # [ 150.688147] nix-grpc-daemon[1724]: time=2026-09-22T15:05:42.029Z level=INFO msg="Uploading 1 narinfos"6978lb # [ 150.698769] niks3-server[1129]: 2026/09/22 15:05:42 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6979lb # [ 150.710895] niks3-server[1129]: 2026/09/22 15:05:42 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6980lb # [ 150.716303] niks3-server[1129]: 2026/09/22 15:05:42 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6981lb # [ 150.723675] niks3-server[1129]: 2026/09/22 15:05:42 INFO Completed upload id=616982worker1 # [ 150.718535] nix-grpc-daemon[1724]: time=2026-09-22T15:05:42.059Z level=INFO msg="Upload complete. (193ms)"6983lb # [ 150.725516] niks3-server[1129]: 2026/09/22 15:05:42 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6984worker1 # [ 150.725713] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:42Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:45498 duration_s=5 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=2 outputs=16985worker1 # [ 150.739780] nix-daemon[865]: reaped child process 1834, status = succeeded6986client # [ 150.769708] systemd[1]: sw.service: Deactivated successfully.6987client # [ 150.774181] systemd[1]: sw.service: Consumed 119ms CPU time over 6.372s wall clock time, 43.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6988client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6989client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6990client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6991worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6992worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 0.04 seconds)6993lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6994lb: must succeed: curl -sf localhost:9901/clusters6995lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6996worker1 # [ 151.697183] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:43Z level=info event=signal_exit6997lb: must succeed: curl -sf localhost:9901/clusters6998lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6999worker2 # [ 152.714548] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:43Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:375467000worker2 # [ 152.716291] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:43Z level=info event=worker_gone addr=192.168.1.3:500517001worker1 # [ 152.707978] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:44Z level=info event=scheduler_restarting7002worker1 # [ 152.722535] nix-grpc-daemon[1660]: ts=2026-09-22T15:05:44Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"7003worker1 # [ 152.744193] nix-daemon[865]: reaped child process 1716, status = succeeded7004worker1 # [ 153.613742] nix-grpc-daemon[1660]: E0000 00:00:1790089544.955000 1705 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true7005worker1 # [ 153.619072] nix-grpc-daemon[1660]: E0000 00:00:1790089544.955523 1705 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true7006lb: must succeed: curl -sf localhost:9901/clusters7007lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)7008worker1 # [ 154.723528] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.7009worker1 # [ 154.727370] systemd[1]: nix-grpc-daemon.service: Consumed 762ms CPU time over 26.139s wall clock time, 37M memory peak, 24.7K incoming IP traffic, 34.7K outgoing IP traffic.7010lb: must succeed: curl -sf localhost:9901/clusters7011lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7012worker1 # [ 155.719427] systemd[1]: Starting Nix worker-protocol over gRPC...7013lb: must succeed: curl -sf localhost:9901/clusters7014worker1 # [ 155.823354] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:47Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"7015worker1 # [ 155.825473] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:47Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc7016worker1 # [ 155.829111] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:47Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=17017lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7018worker1 # [ 155.833082] nix-grpc-daemon[1888]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR7019worker1 # [ 155.834851] nix-grpc-daemon[1888]: E0000 00:00:1790089547.174367 1888 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7020worker1 # [ 155.841069] nix-grpc-daemon[1888]: E0000 00:00:1790089547.180007 1888 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7021worker1 # [ 155.843188] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:47Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket7022worker1 # [ 155.847283] systemd[1]: Started Nix worker-protocol over gRPC.7023worker1 # [ 155.852065] nix-grpc-daemon[1888]: E0000 00:00:1790089547.192639 1927 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7024worker1 # [ 155.865863] nix-grpc-daemon[1888]: E0000 00:00:1790089547.207337 1927 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7025worker1 # [ 155.871054] nix-grpc-daemon[1888]: E0000 00:00:1790089547.209739 1927 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.7026worker1 # [ 155.880231] nix-grpc-daemon[1888]: E0000 00:00:1790089547.221716 1927 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.7027worker1 # [ 155.889303] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:47Z level=info event=scheduler_connected addr=lb:500517028worker2 # [ 155.898853] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:47Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:375467029worker2 # [ 155.903369] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:47Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=07030lb # [ 156.528384] envoy[916]: conn peer=[2001:db8:1::1]:51744 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9568 tx=4410 ms=44597031lb # [ 156.529874] envoy[916]: conn peer=[2001:db8:1::1]:51736 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2988 tx=1516 ms=62197032lb # [ 156.531280] envoy[916]: conn peer=[2001:db8:1::3]:44296 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2012 tx=583 ms=239827033lb: must succeed: curl -sf localhost:9901/clusters7034lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7035lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 5.26 seconds)7036lb: must succeed: curl -sf localhost:9901/clusters7037lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7038worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7039worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7040(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 12.56 seconds)7041subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does7042client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&17043worker2 # [ 157.066780] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:48Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375647044worker2 # [ 157.074207] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:48Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375647045worker2 # [ 157.116822] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:48Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=07046worker2 # [ 157.120982] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:48Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=17047worker2 # [ 157.125450] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:48Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=07048worker2 # [ 157.129388] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:48Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=07049worker1 # [ 157.157601] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547050worker1 # [ 157.169211] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7051worker1 # [ 157.185596] nix-grpc-daemon[1888]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 23 ms (attempt 1/5)7052worker1 # [ 157.216184] nix-grpc-daemon[1888]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 121 ms (attempt 2/5)7053worker1 # [ 157.339239] nix-grpc-daemon[1888]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 130 ms (attempt 3/5)7054worker1 # [ 157.471484] nix-grpc-daemon[1888]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 394 ms (attempt 4/5)7055worker1 # [ 157.868237] nix-grpc-daemon[1888]: disabling binary cache 'https://cache.nixos.org' for 60 seconds7056worker1 # [ 157.871212] nix-grpc-daemon[1888]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org7057worker1 # [ 157.885476] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547058worker1 # [ 157.898304] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547059worker1 # [ 157.906700] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7060worker1 # [ 157.928147] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:49Z level=info event=niks3_push_started pid=19487061worker1 # [ 158.026439] nix-grpc-daemon[1948]: time=2026-09-22T15:05:49.367Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""7062worker1 # [ 158.110673] nix-daemon[865]: accepted connection from pid 1961, user nix-grpc-daemon (trusted)7063worker1 # [ 158.123706] nix-daemon[865]: reaped child process 1964, status = succeeded7064worker1 # [ 158.203326] nix-daemon[865]: accepted connection from pid 1966, user nix-grpc-daemon (trusted)7065worker1 # [ 158.216710] nix-daemon[865]: reaped child process 1969, status = succeeded7066lb # [ 158.229315] niks3-server[1129]: 2026/09/22 15:05:49 INFO Received uploads request method=POST path=/api/pending_closures7067worker1 # [ 158.234067] nix-grpc-daemon[1948]: time=2026-09-22T15:05:49.574Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7068worker1 # [ 158.235495] nix-grpc-daemon[1948]: time=2026-09-22T15:05:49.575Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"7069lb # [ 158.264626] niks3-server[1129]: 2026/09/22 15:05:49 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst7070lb # [ 158.279098] niks3-server[1129]: 2026/09/22 15:05:49 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign7071worker1 # [ 158.279633] nix-grpc-daemon[1948]: time=2026-09-22T15:05:49.620Z level=INFO msg="Uploading 1 narinfos"7072lb # [ 158.284666] niks3-server[1129]: 2026/09/22 15:05:49 INFO Signed narinfos id=62 count=17073lb # [ 158.301065] niks3-server[1129]: 2026/09/22 15:05:49 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls7074lb # [ 158.305479] niks3-server[1129]: 2026/09/22 15:05:49 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete7075lb # [ 158.310166] niks3-server[1129]: 2026/09/22 15:05:49 INFO Completed upload id=627076worker1 # [ 158.305200] nix-grpc-daemon[1948]: time=2026-09-22T15:05:49.646Z level=INFO msg="Upload complete. (275ms)"7077lb # [ 158.312503] niks3-server[1129]: 2026/09/22 15:05:49 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo7078worker1 # [ 158.309949] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=5047079worker1 # [ 158.316321] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547080worker1 # [ 158.326101] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7081worker1 # [ 158.332133] nix-daemon[865]: reaped child process 1946, status = succeeded7082worker1 # [ 158.335370] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7083worker1 # [ 158.948691] nix-daemon[865]: reaped child process 1973, status = succeeded7084worker1 # [ 159.028479] nix-daemon[865]: accepted connection from pid 1989, user nix-grpc-daemon (trusted)7085worker1 # [ 159.040218] nix-daemon[865]: reaped child process 1992, status = succeeded7086lb # [ 159.054391] niks3-server[1129]: 2026/09/22 15:05:50 INFO Received uploads request method=POST path=/api/pending_closures7087worker1 # [ 159.060797] nix-grpc-daemon[1948]: time=2026-09-22T15:05:50.402Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7088worker1 # [ 159.062816] nix-grpc-daemon[1948]: time=2026-09-22T15:05:50.403Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7089lb # [ 159.098162] niks3-server[1129]: 2026/09/22 15:05:50 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7090lb # [ 159.102678] niks3-server[1129]: 2026/09/22 15:05:50 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7091lb # [ 159.110620] niks3-server[1129]: 2026/09/22 15:05:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign7092lb # [ 159.114886] niks3-server[1129]: 2026/09/22 15:05:50 INFO Signed narinfos id=63 count=17093worker1 # [ 159.111799] nix-grpc-daemon[1948]: time=2026-09-22T15:05:50.453Z level=INFO msg="Uploading 1 narinfos"7094lb # [ 159.122611] niks3-server[1129]: 2026/09/22 15:05:50 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7095lb # [ 159.127059] niks3-server[1129]: 2026/09/22 15:05:50 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete7096lb # [ 159.130613] niks3-server[1129]: 2026/09/22 15:05:50 INFO Completed upload id=637097worker1 # [ 159.126595] nix-grpc-daemon[1948]: time=2026-09-22T15:05:50.468Z level=INFO msg="Upload complete. (185ms)"7098lb # [ 159.133535] niks3-server[1129]: 2026/09/22 15:05:50 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7099worker1 # [ 159.131350] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=6 outputs=17100worker2 # [ 159.140435] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:50Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17101worker1 # [ 159.144241] nix-daemon[865]: reaped child process 1971, status = succeeded7102worker2 # [ 159.150418] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:50Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07103worker1 # [ 159.184842] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547104worker1 # [ 159.196097] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:50Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547105worker1 # [ 159.208333] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:50Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547106worker1 # [ 159.218524] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7107worker1 # [ 159.315982] nix-daemon[865]: accepted connection from pid 1996, user nix-grpc-daemon (trusted)7108worker1 # [ 159.329300] nix-daemon[865]: reaped child process 1999, status = succeeded7109worker1 # [ 159.414939] nix-daemon[865]: accepted connection from pid 2001, user nix-grpc-daemon (trusted)7110worker1 # [ 159.428827] nix-daemon[865]: reaped child process 2004, status = succeeded7111lb # [ 159.441590] niks3-server[1129]: 2026/09/22 15:05:50 INFO Received uploads request method=POST path=/api/pending_closures7112worker1 # [ 159.445135] nix-grpc-daemon[1948]: time=2026-09-22T15:05:50.786Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7113worker1 # [ 159.446711] nix-grpc-daemon[1948]: time=2026-09-22T15:05:50.788Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7114lb # [ 159.473963] niks3-server[1129]: 2026/09/22 15:05:50 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7115lb # [ 159.484085] niks3-server[1129]: 2026/09/22 15:05:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7116worker1 # [ 159.484770] nix-grpc-daemon[1948]: time=2026-09-22T15:05:50.826Z level=INFO msg="Uploading 1 narinfos"7117lb # [ 159.489910] niks3-server[1129]: 2026/09/22 15:05:50 INFO Signed narinfos id=64 count=17118lb # [ 159.497972] niks3-server[1129]: 2026/09/22 15:05:50 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7119lb # [ 159.508726] niks3-server[1129]: 2026/09/22 15:05:50 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7120lb # [ 159.513062] niks3-server[1129]: 2026/09/22 15:05:50 INFO Completed upload id=647121lb # [ 159.515058] niks3-server[1129]: 2026/09/22 15:05:50 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7122worker1 # [ 159.508087] nix-grpc-daemon[1948]: time=2026-09-22T15:05:50.849Z level=INFO msg="Upload complete. (278ms)"7123worker1 # [ 159.512533] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=5047124worker1 # [ 159.517627] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547125worker1 # [ 159.526904] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7126worker1 # [ 159.533875] nix-daemon[865]: reaped child process 1994, status = succeeded7127worker1 # [ 159.537546] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7128worker1 # [ 160.579211] nix-daemon[865]: reaped child process 2008, status = succeeded7129worker1 # [ 160.656959] nix-daemon[865]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)7130worker1 # [ 160.670477] nix-daemon[865]: reaped child process 2028, status = succeeded7131lb # [ 160.683896] niks3-server[1129]: 2026/09/22 15:05:52 INFO Received uploads request method=POST path=/api/pending_closures7132worker1 # [ 160.687165] nix-grpc-daemon[1948]: time=2026-09-22T15:05:52.028Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7133worker1 # [ 160.688773] nix-grpc-daemon[1948]: time=2026-09-22T15:05:52.030Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7134lb # [ 160.721205] niks3-server[1129]: 2026/09/22 15:05:52 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7135lb # [ 160.731249] niks3-server[1129]: 2026/09/22 15:05:52 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7136lb # [ 160.743202] niks3-server[1129]: 2026/09/22 15:05:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7137lb # [ 160.749700] niks3-server[1129]: 2026/09/22 15:05:52 INFO Signed narinfos id=65 count=17138worker1 # [ 160.747067] nix-grpc-daemon[1948]: time=2026-09-22T15:05:52.088Z level=INFO msg="Uploading 1 narinfos"7139lb # [ 160.755471] niks3-server[1129]: 2026/09/22 15:05:52 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7140lb # [ 160.765455] niks3-server[1129]: 2026/09/22 15:05:52 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7141lb # [ 160.768559] niks3-server[1129]: 2026/09/22 15:05:52 INFO Completed upload id=657142worker1 # [ 160.763335] nix-grpc-daemon[1948]: time=2026-09-22T15:05:52.104Z level=INFO msg="Upload complete. (219ms)"7143lb # [ 160.770939] niks3-server[1129]: 2026/09/22 15:05:52 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7144worker2 # [ 160.777687] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:52Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17145worker1 # [ 160.769570] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:52Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=7 outputs=17146worker2 # [ 160.782474] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:52Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07147worker1 # [ 160.778314] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547148worker1 # [ 160.789479] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547149worker1 # [ 160.797620] nix-daemon[865]: reaped child process 2006, status = succeeded7150worker1 # [ 160.800535] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547151worker1 # [ 160.805843] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7152worker1 # [ 160.891652] nix-daemon[865]: accepted connection from pid 2032, user nix-grpc-daemon (trusted)7153worker1 # [ 160.904782] nix-daemon[865]: reaped child process 2035, status = succeeded7154worker1 # [ 160.982673] nix-daemon[865]: accepted connection from pid 2037, user nix-grpc-daemon (trusted)7155worker1 # [ 160.996088] nix-daemon[865]: reaped child process 2040, status = succeeded7156lb # [ 161.007649] niks3-server[1129]: 2026/09/22 15:05:52 INFO Received uploads request method=POST path=/api/pending_closures7157worker1 # [ 161.010359] nix-grpc-daemon[1948]: time=2026-09-22T15:05:52.351Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7158worker1 # [ 161.012245] nix-grpc-daemon[1948]: time=2026-09-22T15:05:52.353Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7159lb # [ 161.038753] niks3-server[1129]: 2026/09/22 15:05:52 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7160lb # [ 161.047574] niks3-server[1129]: 2026/09/22 15:05:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7161worker1 # [ 161.047893] nix-grpc-daemon[1948]: time=2026-09-22T15:05:52.389Z level=INFO msg="Uploading 1 narinfos"7162lb # [ 161.053256] niks3-server[1129]: 2026/09/22 15:05:52 INFO Signed narinfos id=66 count=17163lb # [ 161.059613] niks3-server[1129]: 2026/09/22 15:05:52 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7164lb # [ 161.072524] niks3-server[1129]: 2026/09/22 15:05:52 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7165lb # [ 161.079262] niks3-server[1129]: 2026/09/22 15:05:52 INFO Completed upload id=667166worker1 # [ 161.073981] nix-grpc-daemon[1948]: time=2026-09-22T15:05:52.415Z level=INFO msg="Upload complete. (262ms)"7167lb # [ 161.082537] niks3-server[1129]: 2026/09/22 15:05:52 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7168worker1 # [ 161.077961] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=5047169worker1 # [ 161.082944] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547170worker1 # [ 161.092105] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7171worker1 # [ 161.099185] nix-daemon[865]: reaped child process 2030, status = succeeded7172worker1 # [ 161.104140] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7173worker1 # [ 161.946416] nix-daemon[865]: reaped child process 2044, status = succeeded7174worker1 # [ 162.021769] nix-daemon[865]: accepted connection from pid 2060, user nix-grpc-daemon (trusted)7175worker1 # [ 162.035593] nix-daemon[865]: reaped child process 2063, status = succeeded7176lb # [ 162.048403] niks3-server[1129]: 2026/09/22 15:05:53 INFO Received uploads request method=POST path=/api/pending_closures7177worker1 # [ 162.050777] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.392Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7178worker1 # [ 162.053055] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.393Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7179lb # [ 162.082242] niks3-server[1129]: 2026/09/22 15:05:53 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7180lb # [ 162.097347] niks3-server[1129]: 2026/09/22 15:05:53 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7181lb # [ 162.102335] niks3-server[1129]: 2026/09/22 15:05:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7182lb # [ 162.106762] niks3-server[1129]: 2026/09/22 15:05:53 INFO Signed narinfos id=67 count=17183worker1 # [ 162.107672] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.448Z level=INFO msg="Uploading 1 narinfos"7184lb # [ 162.118151] niks3-server[1129]: 2026/09/22 15:05:53 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7185lb # [ 162.125852] niks3-server[1129]: 2026/09/22 15:05:53 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7186lb # [ 162.130056] niks3-server[1129]: 2026/09/22 15:05:53 INFO Completed upload id=677187worker1 # [ 162.124835] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.466Z level=INFO msg="Upload complete. (213ms)"7188lb # [ 162.132769] niks3-server[1129]: 2026/09/22 15:05:53 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7189worker1 # [ 162.130883] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=8 outputs=17190worker2 # [ 162.139482] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:53Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17191worker1 # [ 162.139470] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547192worker2 # [ 162.148986] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:53Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07193worker1 # [ 162.149586] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547194worker1 # [ 162.158475] nix-daemon[865]: reaped child process 2042, status = succeeded7195worker1 # [ 162.160250] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.501Z level=INFO msg="All 1 paths already cached"7196worker1 # [ 162.164435] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547197worker1 # [ 162.168936] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7198worker1 # [ 162.256024] nix-daemon[865]: accepted connection from pid 2067, user nix-grpc-daemon (trusted)7199worker1 # [ 162.269504] nix-daemon[865]: reaped child process 2070, status = succeeded7200worker1 # [ 162.355179] nix-daemon[865]: accepted connection from pid 2072, user nix-grpc-daemon (trusted)7201worker1 # [ 162.368721] nix-daemon[865]: reaped child process 2075, status = succeeded7202lb # [ 162.380650] niks3-server[1129]: 2026/09/22 15:05:53 INFO Received uploads request method=POST path=/api/pending_closures7203worker1 # [ 162.380030] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.721Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7204worker1 # [ 162.381644] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.723Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7205lb # [ 162.408609] niks3-server[1129]: 2026/09/22 15:05:53 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7206lb # [ 162.417360] niks3-server[1129]: 2026/09/22 15:05:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7207worker1 # [ 162.420157] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.761Z level=INFO msg="Uploading 1 narinfos"7208lb # [ 162.425297] niks3-server[1129]: 2026/09/22 15:05:53 INFO Signed narinfos id=68 count=17209lb # [ 162.433374] niks3-server[1129]: 2026/09/22 15:05:53 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7210lb # [ 162.444852] niks3-server[1129]: 2026/09/22 15:05:53 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7211lb # [ 162.450793] niks3-server[1129]: 2026/09/22 15:05:53 INFO Completed upload id=687212worker1 # [ 162.445515] nix-grpc-daemon[1948]: time=2026-09-22T15:05:53.786Z level=INFO msg="Upload complete. (271ms)"7213lb # [ 162.453894] niks3-server[1129]: 2026/09/22 15:05:53 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7214worker1 # [ 162.449534] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:53Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=6247215worker1 # [ 162.454408] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547216worker1 # [ 162.464539] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7217worker1 # [ 162.468747] nix-daemon[865]: reaped child process 2065, status = succeeded7218worker1 # [ 162.473298] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7219worker1 # [ 163.712579] nix-daemon[865]: reaped child process 2079, status = succeeded7220worker1 # [ 163.796742] nix-daemon[865]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)7221worker1 # [ 163.809938] nix-daemon[865]: reaped child process 2098, status = succeeded7222lb # [ 163.824451] niks3-server[1129]: 2026/09/22 15:05:55 INFO Received uploads request method=POST path=/api/pending_closures7223worker1 # [ 163.827761] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.169Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7224worker1 # [ 163.829883] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.170Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7225lb # [ 163.851163] niks3-server[1129]: 2026/09/22 15:05:55 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7226lb # [ 163.855787] niks3-server[1129]: 2026/09/22 15:05:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7227lb # [ 163.857505] niks3-server[1129]: 2026/09/22 15:05:55 INFO Signed narinfos id=69 count=17228worker1 # [ 163.854571] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.195Z level=INFO msg="Uploading 1 narinfos"7229lb # [ 163.861719] niks3-server[1129]: 2026/09/22 15:05:55 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7230lb # [ 163.865543] niks3-server[1129]: 2026/09/22 15:05:55 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7231lb # [ 163.869844] niks3-server[1129]: 2026/09/22 15:05:55 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7232lb # [ 163.873494] niks3-server[1129]: 2026/09/22 15:05:55 INFO Completed upload id=697233worker1 # [ 163.868033] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.209Z level=INFO msg="Upload complete. (184ms)"7234lb # [ 163.875662] niks3-server[1129]: 2026/09/22 15:05:55 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7235worker1 # [ 163.870693] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=9 outputs=17236worker2 # [ 163.880789] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:55Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17237worker1 # [ 163.878363] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547238worker2 # [ 163.886796] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:55Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07239worker1 # [ 163.882375] nix-daemon[865]: reaped child process 2077, status = succeeded7240worker1 # [ 163.885757] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547241worker1 # [ 163.891417] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.232Z level=INFO msg="All 1 paths already cached"7242worker1 # [ 163.895339] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547243worker1 # [ 163.898990] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7244worker1 # [ 163.986636] nix-daemon[865]: accepted connection from pid 2102, user nix-grpc-daemon (trusted)7245worker1 # [ 164.000122] nix-daemon[865]: reaped child process 2105, status = succeeded7246worker1 # [ 164.085826] nix-daemon[865]: accepted connection from pid 2107, user nix-grpc-daemon (trusted)7247worker1 # [ 164.099190] nix-daemon[865]: reaped child process 2110, status = succeeded7248lb # [ 164.111548] niks3-server[1129]: 2026/09/22 15:05:55 INFO Received uploads request method=POST path=/api/pending_closures7249worker1 # [ 164.111177] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.452Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7250worker1 # [ 164.112638] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.452Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7251lb # [ 164.125946] niks3-server[1129]: 2026/09/22 15:05:55 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7252lb # [ 164.130510] niks3-server[1129]: 2026/09/22 15:05:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7253lb # [ 164.133176] niks3-server[1129]: 2026/09/22 15:05:55 INFO Signed narinfos id=70 count=17254worker1 # [ 164.127891] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.469Z level=INFO msg="Uploading 1 narinfos"7255lb # [ 164.136595] niks3-server[1129]: 2026/09/22 15:05:55 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7256lb # [ 164.141918] niks3-server[1129]: 2026/09/22 15:05:55 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7257lb # [ 164.145981] niks3-server[1129]: 2026/09/22 15:05:55 INFO Completed upload id=707258worker1 # [ 164.140760] nix-grpc-daemon[1948]: time=2026-09-22T15:05:55.482Z level=INFO msg="Upload complete. (235ms)"7259worker1 # [ 164.142781] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=6247260lb # [ 164.149080] niks3-server[1129]: 2026/09/22 15:05:55 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7261worker1 # [ 164.146764] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547262worker1 # [ 164.152767] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7263worker1 # [ 164.155076] nix-daemon[865]: reaped child process 2100, status = succeeded7264worker1 # [ 164.160197] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7265worker1 # [ 164.776361] nix-daemon[865]: reaped child process 2114, status = succeeded7266worker1 # [ 164.853834] nix-daemon[865]: accepted connection from pid 2130, user nix-grpc-daemon (trusted)7267worker1 # [ 164.867341] nix-daemon[865]: reaped child process 2133, status = succeeded7268lb # [ 164.878269] niks3-server[1129]: 2026/09/22 15:05:56 INFO Received uploads request method=POST path=/api/pending_closures7269worker1 # [ 164.878358] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.219Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7270worker1 # [ 164.879896] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.221Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7271lb # [ 164.896872] niks3-server[1129]: 2026/09/22 15:05:56 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7272lb # [ 164.901679] niks3-server[1129]: 2026/09/22 15:05:56 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7273lb # [ 164.906692] niks3-server[1129]: 2026/09/22 15:05:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7274lb # [ 164.908966] niks3-server[1129]: 2026/09/22 15:05:56 INFO Signed narinfos id=71 count=17275worker1 # [ 164.903863] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.245Z level=INFO msg="Uploading 1 narinfos"7276lb # [ 164.913768] niks3-server[1129]: 2026/09/22 15:05:56 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7277lb # [ 164.918802] niks3-server[1129]: 2026/09/22 15:05:56 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7278lb # [ 164.922239] niks3-server[1129]: 2026/09/22 15:05:56 INFO Completed upload id=717279worker1 # [ 164.916791] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.258Z level=INFO msg="Upload complete. (150ms)"7280lb # [ 164.925533] niks3-server[1129]: 2026/09/22 15:05:56 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7281worker1 # [ 164.919506] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=10 outputs=17282worker2 # [ 164.927859] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:56Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17283worker1 # [ 164.925390] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547284worker1 # [ 164.930107] nix-daemon[865]: reaped child process 2112, status = succeeded7285worker2 # [ 164.935519] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:56Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07286worker1 # [ 164.933335] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547287worker1 # [ 164.938987] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.280Z level=INFO msg="All 1 paths already cached"7288worker1 # [ 164.941571] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547289worker1 # [ 164.945321] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7290worker1 # [ 165.027524] nix-daemon[865]: accepted connection from pid 2137, user nix-grpc-daemon (trusted)7291worker1 # [ 165.041576] nix-daemon[865]: reaped child process 2140, status = succeeded7292worker1 # [ 165.120479] nix-daemon[865]: accepted connection from pid 2142, user nix-grpc-daemon (trusted)7293worker1 # [ 165.134187] nix-daemon[865]: reaped child process 2145, status = succeeded7294lb # [ 165.145501] niks3-server[1129]: 2026/09/22 15:05:56 INFO Received uploads request method=POST path=/api/pending_closures7295worker1 # [ 165.149220] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.490Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7296worker1 # [ 165.150800] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.492Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7297lb # [ 165.179822] niks3-server[1129]: 2026/09/22 15:05:56 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7298lb # [ 165.187864] niks3-server[1129]: 2026/09/22 15:05:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7299lb # [ 165.192632] niks3-server[1129]: 2026/09/22 15:05:56 INFO Signed narinfos id=72 count=17300worker1 # [ 165.189784] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.531Z level=INFO msg="Uploading 1 narinfos"7301lb # [ 165.204257] niks3-server[1129]: 2026/09/22 15:05:56 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7302lb # [ 165.208033] niks3-server[1129]: 2026/09/22 15:05:56 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7303lb # [ 165.211427] niks3-server[1129]: 2026/09/22 15:05:56 INFO Completed upload id=727304worker1 # [ 165.206250] nix-grpc-daemon[1948]: time=2026-09-22T15:05:56.547Z level=INFO msg="Upload complete. (255ms)"7305lb # [ 165.214284] niks3-server[1129]: 2026/09/22 15:05:56 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7306worker1 # [ 165.209429] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=6247307worker1 # [ 165.215121] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547308worker1 # [ 165.223270] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7309worker1 # [ 165.228059] nix-daemon[865]: reaped child process 2135, status = succeeded7310worker1 # [ 165.233717] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7311worker1 # [ 166.477130] nix-daemon[865]: reaped child process 2149, status = succeeded7312worker1 # [ 166.555765] nix-daemon[865]: accepted connection from pid 2165, user nix-grpc-daemon (trusted)7313worker1 # [ 166.569964] nix-daemon[865]: reaped child process 2168, status = succeeded7314lb # [ 166.581894] niks3-server[1129]: 2026/09/22 15:05:57 INFO Received uploads request method=POST path=/api/pending_closures7315worker1 # [ 166.585264] nix-grpc-daemon[1948]: time=2026-09-22T15:05:57.926Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7316worker1 # [ 166.588143] nix-grpc-daemon[1948]: time=2026-09-22T15:05:57.928Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7317lb # [ 166.622754] niks3-server[1129]: 2026/09/22 15:05:57 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7318lb # [ 166.630279] niks3-server[1129]: 2026/09/22 15:05:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7319worker1 # [ 166.629079] nix-grpc-daemon[1948]: time=2026-09-22T15:05:57.970Z level=INFO msg="Uploading 1 narinfos"7320lb # [ 166.634246] niks3-server[1129]: 2026/09/22 15:05:57 INFO Signed narinfos id=73 count=17321lb # [ 166.641481] niks3-server[1129]: 2026/09/22 15:05:57 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7322lb # [ 166.646137] niks3-server[1129]: 2026/09/22 15:05:57 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7323lb # [ 166.655246] niks3-server[1129]: 2026/09/22 15:05:57 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7324lb # [ 166.659542] niks3-server[1129]: 2026/09/22 15:05:57 INFO Completed upload id=737325worker1 # [ 166.654433] nix-grpc-daemon[1948]: time=2026-09-22T15:05:57.995Z level=INFO msg="Upload complete. (213ms)"7326lb # [ 166.661103] niks3-server[1129]: 2026/09/22 15:05:57 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7327worker2 # [ 166.668163] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:57Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17328worker2 # [ 166.673406] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:57Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07329worker1 # [ 166.662128] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=11 outputs=17330worker1 # [ 166.670097] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547331worker1 # [ 166.682529] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547332worker1 # [ 166.689438] nix-daemon[865]: reaped child process 2147, status = succeeded7333worker1 # [ 166.691853] nix-grpc-daemon[1948]: time=2026-09-22T15:05:58.033Z level=INFO msg="All 2 paths already cached"7334worker1 # [ 166.695807] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547335worker1 # [ 166.699582] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7336worker1 # [ 166.784359] nix-daemon[865]: accepted connection from pid 2172, user nix-grpc-daemon (trusted)7337worker1 # [ 166.797781] nix-daemon[865]: reaped child process 2175, status = succeeded7338worker1 # [ 166.882836] nix-daemon[865]: accepted connection from pid 2177, user nix-grpc-daemon (trusted)7339worker1 # [ 166.895611] nix-daemon[865]: reaped child process 2180, status = succeeded7340lb # [ 166.906582] niks3-server[1129]: 2026/09/22 15:05:58 INFO Received uploads request method=POST path=/api/pending_closures7341worker1 # [ 166.909565] nix-grpc-daemon[1948]: time=2026-09-22T15:05:58.251Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7342worker1 # [ 166.911173] nix-grpc-daemon[1948]: time=2026-09-22T15:05:58.252Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7343lb # [ 166.939630] niks3-server[1129]: 2026/09/22 15:05:58 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7344lb # [ 166.947741] niks3-server[1129]: 2026/09/22 15:05:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7345lb # [ 166.952824] niks3-server[1129]: 2026/09/22 15:05:58 INFO Signed narinfos id=74 count=17346worker1 # [ 166.950821] nix-grpc-daemon[1948]: time=2026-09-22T15:05:58.292Z level=INFO msg="Uploading 1 narinfos"7347lb # [ 166.964223] niks3-server[1129]: 2026/09/22 15:05:58 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7348lb # [ 166.972900] niks3-server[1129]: 2026/09/22 15:05:58 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7349lb # [ 166.978239] niks3-server[1129]: 2026/09/22 15:05:58 INFO Completed upload id=747350worker1 # [ 166.973115] nix-grpc-daemon[1948]: time=2026-09-22T15:05:58.314Z level=INFO msg="Upload complete. (267ms)"7351worker1 # [ 166.977327] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=7527352worker1 # [ 166.982315] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547353worker1 # [ 166.991354] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7354worker1 # [ 166.997554] nix-daemon[865]: reaped child process 2170, status = succeeded7355worker1 # [ 167.001971] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7356worker1 # [ 168.241466] nix-daemon[865]: reaped child process 2184, status = succeeded7357worker1 # [ 168.324604] nix-daemon[865]: accepted connection from pid 2200, user nix-grpc-daemon (trusted)7358worker1 # [ 168.337384] nix-daemon[865]: reaped child process 2203, status = succeeded7359lb # [ 168.349750] niks3-server[1129]: 2026/09/22 15:05:59 INFO Received uploads request method=POST path=/api/pending_closures7360worker1 # [ 168.354902] nix-grpc-daemon[1948]: time=2026-09-22T15:05:59.696Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7361worker1 # [ 168.357952] nix-grpc-daemon[1948]: time=2026-09-22T15:05:59.697Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7362lb # [ 168.385691] niks3-server[1129]: 2026/09/22 15:05:59 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7363lb # [ 168.401253] niks3-server[1129]: 2026/09/22 15:05:59 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7364lb # [ 168.409730] niks3-server[1129]: 2026/09/22 15:05:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7365lb # [ 168.413822] niks3-server[1129]: 2026/09/22 15:05:59 INFO Signed narinfos id=75 count=17366worker1 # [ 168.411909] nix-grpc-daemon[1948]: time=2026-09-22T15:05:59.753Z level=INFO msg="Uploading 1 narinfos"7367lb # [ 168.419494] niks3-server[1129]: 2026/09/22 15:05:59 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7368lb # [ 168.428472] niks3-server[1129]: 2026/09/22 15:05:59 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7369lb # [ 168.432229] niks3-server[1129]: 2026/09/22 15:05:59 INFO Completed upload id=757370worker1 # [ 168.427053] nix-grpc-daemon[1948]: time=2026-09-22T15:05:59.768Z level=INFO msg="Upload complete. (216ms)"7371lb # [ 168.433824] niks3-server[1129]: 2026/09/22 15:05:59 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7372worker1 # [ 168.430264] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=12 outputs=17373worker2 # [ 168.440351] nix-grpc-daemon[1567]: ts=2026-09-22T15:05:59Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17374worker1 # [ 168.439484] nix-daemon[865]: reaped child process 2182, status = succeeded7375worker1 # [ 168.475469] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547376worker1 # [ 168.482791] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547377worker1 # [ 168.491581] nix-grpc-daemon[1948]: time=2026-09-22T15:05:59.833Z level=INFO msg="All 2 paths already cached"7378worker1 # [ 168.495900] nix-grpc-daemon[1888]: ts=2026-09-22T15:05:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547379worker1 # [ 168.500134] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7380worker1 # [ 168.587477] nix-daemon[865]: accepted connection from pid 2207, user nix-grpc-daemon (trusted)7381worker1 # [ 168.601880] nix-daemon[865]: reaped child process 2210, status = succeeded7382worker1 # [ 168.682109] nix-daemon[865]: accepted connection from pid 2212, user nix-grpc-daemon (trusted)7383worker1 # [ 168.695293] nix-daemon[865]: reaped child process 2215, status = succeeded7384lb # [ 168.707316] niks3-server[1129]: 2026/09/22 15:06:00 INFO Received uploads request method=POST path=/api/pending_closures7385worker1 # [ 168.709630] nix-grpc-daemon[1948]: time=2026-09-22T15:06:00.051Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7386worker1 # [ 168.711539] nix-grpc-daemon[1948]: time=2026-09-22T15:06:00.052Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7387lb # [ 168.738257] niks3-server[1129]: 2026/09/22 15:06:00 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7388lb # [ 168.746447] niks3-server[1129]: 2026/09/22 15:06:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/76/sign7389worker1 # [ 168.746978] nix-grpc-daemon[1948]: time=2026-09-22T15:06:00.088Z level=INFO msg="Uploading 1 narinfos"7390lb # [ 168.752371] niks3-server[1129]: 2026/09/22 15:06:00 INFO Signed narinfos id=76 count=17391lb # [ 168.758401] niks3-server[1129]: 2026/09/22 15:06:00 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7392lb # [ 168.771864] niks3-server[1129]: 2026/09/22 15:06:00 INFO Received complete upload request method=POST path=/api/pending_closures/76/complete7393worker1 # [ 168.772024] nix-grpc-daemon[1948]: time=2026-09-22T15:06:00.113Z level=INFO msg="Upload complete. (264ms)"7394lb # [ 168.777628] niks3-server[1129]: 2026/09/22 15:06:00 INFO Completed upload id=767395worker1 # [ 168.774214] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=7527396lb # [ 168.782519] niks3-server[1129]: 2026/09/22 15:06:00 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7397worker1 # [ 168.781505] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547398worker1 # [ 168.783201] nix-daemon[865]: reaped child process 2205, status = succeeded7399worker1 # [ 168.786210] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7400worker1 # [ 168.792889] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7401worker1 # [ 170.010409] nix-daemon[865]: reaped child process 2219, status = succeeded7402worker1 # [ 170.082955] nix-daemon[865]: accepted connection from pid 2235, user nix-grpc-daemon (trusted)7403worker1 # [ 170.096546] nix-daemon[865]: reaped child process 2238, status = succeeded7404lb # [ 170.107147] niks3-server[1129]: 2026/09/22 15:06:01 INFO Received uploads request method=POST path=/api/pending_closures7405worker1 # [ 170.106518] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.447Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7406worker1 # [ 170.108548] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.449Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7407lb # [ 170.126190] niks3-server[1129]: 2026/09/22 15:06:01 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7408lb # [ 170.129421] niks3-server[1129]: 2026/09/22 15:06:01 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7409lb # [ 170.135094] niks3-server[1129]: 2026/09/22 15:06:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/77/sign7410lb # [ 170.140626] niks3-server[1129]: 2026/09/22 15:06:01 INFO Signed narinfos id=77 count=17411worker1 # [ 170.138192] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.479Z level=INFO msg="Uploading 1 narinfos"7412lb # [ 170.145602] niks3-server[1129]: 2026/09/22 15:06:01 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7413lb # [ 170.151782] niks3-server[1129]: 2026/09/22 15:06:01 INFO Received complete upload request method=POST path=/api/pending_closures/77/complete7414lb # [ 170.155762] niks3-server[1129]: 2026/09/22 15:06:01 INFO Completed upload id=777415worker1 # [ 170.150424] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.491Z level=INFO msg="Upload complete. (149ms)"7416lb # [ 170.157909] niks3-server[1129]: 2026/09/22 15:06:01 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7417worker1 # [ 170.153132] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=13 outputs=17418worker2 # [ 170.163732] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:01Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17419worker1 # [ 170.161164] nix-daemon[865]: reaped child process 2217, status = succeeded7420worker1 # [ 170.200742] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547421worker1 # [ 170.212270] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547422worker1 # [ 170.225501] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.566Z level=INFO msg="All 2 paths already cached"7423worker1 # [ 170.233708] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354547424worker1 # [ 170.243277] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7425worker1 # [ 170.331317] nix-daemon[865]: accepted connection from pid 2242, user nix-grpc-daemon (trusted)7426worker1 # [ 170.351507] nix-daemon[865]: reaped child process 2245, status = succeeded7427worker1 # [ 170.424026] nix-daemon[865]: accepted connection from pid 2247, user nix-grpc-daemon (trusted)7428worker1 # [ 170.437191] nix-daemon[865]: reaped child process 2250, status = succeeded7429lb # [ 170.449340] niks3-server[1129]: 2026/09/22 15:06:01 INFO Received uploads request method=POST path=/api/pending_closures7430worker1 # [ 170.451919] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.793Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7431worker1 # [ 170.453285] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.793Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7432lb # [ 170.477628] niks3-server[1129]: 2026/09/22 15:06:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/78/sign7433lb # [ 170.483729] niks3-server[1129]: 2026/09/22 15:06:01 INFO Signed narinfos id=78 count=17434worker1 # [ 170.481363] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.822Z level=INFO msg="Uploading 1 narinfos"7435lb # [ 170.491565] niks3-server[1129]: 2026/09/22 15:06:01 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7436lb # [ 170.505769] niks3-server[1129]: 2026/09/22 15:06:01 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7437lb # [ 170.512448] niks3-server[1129]: 2026/09/22 15:06:01 INFO Received complete upload request method=POST path=/api/pending_closures/78/complete7438lb # [ 170.518152] niks3-server[1129]: 2026/09/22 15:06:01 INFO Completed upload id=787439worker1 # [ 170.515161] nix-grpc-daemon[1948]: time=2026-09-22T15:06:01.856Z level=INFO msg="Upload complete. (261ms)"7440lb # [ 170.521472] niks3-server[1129]: 2026/09/22 15:06:01 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7441worker1 # [ 170.519540] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=0 paths=1 nar_bytes_in=7527442worker1 # [ 170.524599] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354547443worker1 # [ 170.534966] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7444worker1 # [ 170.542095] nix-daemon[865]: reaped child process 2240, status = succeeded7445worker1 # [ 170.544947] nix-daemon[865]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7446worker1 # [ 171.371728] nix-daemon[865]: reaped child process 2254, status = succeeded7447worker1 # [ 171.442708] nix-daemon[865]: accepted connection from pid 2270, user nix-grpc-daemon (trusted)7448worker1 # [ 171.455744] nix-daemon[865]: reaped child process 2273, status = succeeded7449lb # [ 171.469872] niks3-server[1129]: 2026/09/22 15:06:02 INFO Received uploads request method=POST path=/api/pending_closures7450worker1 # [ 171.470901] nix-grpc-daemon[1948]: time=2026-09-22T15:06:02.812Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7451worker1 # [ 171.473056] nix-grpc-daemon[1948]: time=2026-09-22T15:06:02.814Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7452lb # [ 171.508220] niks3-server[1129]: 2026/09/22 15:06:02 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7453lb # [ 171.516400] niks3-server[1129]: 2026/09/22 15:06:02 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7454lb # [ 171.530546] niks3-server[1129]: 2026/09/22 15:06:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/79/sign7455lb # [ 171.534260] niks3-server[1129]: 2026/09/22 15:06:02 INFO Signed narinfos id=79 count=17456worker1 # [ 171.531626] nix-grpc-daemon[1948]: time=2026-09-22T15:06:02.872Z level=INFO msg="Uploading 1 narinfos"7457lb # [ 171.540834] niks3-server[1129]: 2026/09/22 15:06:02 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7458lb # [ 171.549160] niks3-server[1129]: 2026/09/22 15:06:02 INFO Received complete upload request method=POST path=/api/pending_closures/79/complete7459lb # [ 171.552505] niks3-server[1129]: 2026/09/22 15:06:02 INFO Completed upload id=797460worker1 # [ 171.547275] nix-grpc-daemon[1948]: time=2026-09-22T15:06:02.888Z level=INFO msg="Upload complete. (195ms)"7461lb # [ 171.554457] niks3-server[1129]: 2026/09/22 15:06:02 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7462worker1 # [ 171.553456] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35454 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=14 outputs=17463worker1 # [ 171.568072] nix-daemon[865]: reaped child process 2252, status = succeeded7464client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.72 seconds)7465worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7466worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7467client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7468client # [ 171.747982] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7469client # Running as unit: gpu.service; invocation ID: 37bd0bad5a644c6bb6d8f0f07a99a37b7470client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7471lb: must succeed: curl -sf localhost:9901/clusters7472lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7473lb: must succeed: curl -sf localhost:9901/clusters7474lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7475worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7476worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7477worker2 # [ 171.874133] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:03Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375647478client # [ 171.882866] nix[1541]: don't know how to build these paths:7479client # [ 171.883708] nix[1541]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7480worker2 # [ 171.878046] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375647481worker2 # [ 171.889706] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:03Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07482client # [ 171.898223] nix[1541]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7483lb: must succeed: curl -sf localhost:9901/clusters7484lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7485lb: must succeed: curl -sf localhost:9901/clusters7486lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7487worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7488worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)7489client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7490lb # [ 176.530588] envoy[916]: conn peer=[2001:db8:1::1]:41526 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27038 tx=12297 ms=145497491client # [ 201.898940] nix[1541]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7492client # [ 201.904789] nix[1541]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7493client # [ 201.919273] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7494client # [ 201.921123] systemd[1]: gpu.service: Failed with result 'exit-code'.7495client # [ 201.922759] systemd[1]: gpu.service: Consumed 121ms CPU time over 30.173s wall clock time, 37.6M memory peak, 5.2K incoming IP traffic, 5K outgoing IP traffic.7496client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.06 seconds)7497client: must succeed: journalctl -u gpu -o cat7498client: (finished: must succeed: journalctl -u gpu -o cat, in 0.01 seconds)7499lb: must succeed: curl -sf localhost:9901/clusters7500lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7501lb: must succeed: curl -sf localhost:9901/clusters7502lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7503worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7504worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7505(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.17 seconds)7506subtest: low disk drains a worker and builds go to the other7507worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7508worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.91 seconds)7509worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7510worker1 # [ 203.856670] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:35Z level=info event=unhealthy reason=min_free7511worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds)7512client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27513client # don't know how to build these paths:7514client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7515worker2 # [ 204.245075] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:35Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:375647516worker1 # [ 204.243481] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354547517worker2 # [ 204.273886] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:35Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07518worker2 # [ 204.279215] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:35Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17519worker2 # [ 204.317735] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375647520worker2 # [ 204.329864] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375647521client # copying 1 paths to 'lb:50051'...7522client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7523worker2 # [ 204.342401] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375647524worker2 # [ 204.353494] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7525worker2 # [ 204.453723] nix-daemon[850]: accepted connection from pid 1861, user nix-grpc-daemon (trusted)7526worker2 # [ 204.468548] nix-daemon[850]: reaped child process 1864, status = succeeded7527worker2 # [ 204.549188] nix-daemon[850]: accepted connection from pid 1866, user nix-grpc-daemon (trusted)7528worker2 # [ 204.563053] nix-daemon[850]: reaped child process 1869, status = succeeded7529lb # [ 204.566510] niks3-server[1129]: 2026/09/22 15:06:35 INFO Received uploads request method=POST path=/api/pending_closures7530worker2 # [ 204.577064] nix-grpc-daemon[1641]: time=2026-09-22T15:06:35.800Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7531worker2 # [ 204.578897] nix-grpc-daemon[1641]: time=2026-09-22T15:06:35.802Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7532lb # [ 204.601600] niks3-server[1129]: 2026/09/22 15:06:35 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7533lb # [ 204.612230] niks3-server[1129]: 2026/09/22 15:06:35 INFO Received sign narinfos request method=POST path=/api/pending_closures/80/sign7534worker2 # [ 204.619086] nix-grpc-daemon[1641]: time=2026-09-22T15:06:35.842Z level=INFO msg="Uploading 1 narinfos"7535lb # [ 204.617621] niks3-server[1129]: 2026/09/22 15:06:35 INFO Signed narinfos id=80 count=17536lb # [ 204.638154] niks3-server[1129]: 2026/09/22 15:06:35 INFO Received complete upload request method=POST path=/api/pending_closures/80/complete7537lb # [ 204.641483] niks3-server[1129]: 2026/09/22 15:06:35 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7538lb # [ 204.644643] niks3-server[1129]: 2026/09/22 15:06:35 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7539lb # [ 204.646759] niks3-server[1129]: 2026/09/22 15:06:35 INFO Completed upload id=807540worker2 # [ 204.648782] nix-grpc-daemon[1641]: time=2026-09-22T15:06:35.872Z level=INFO msg="Upload complete. (284ms)"7541worker2 # [ 204.650885] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 paths=1 nar_bytes_in=5127542worker2 # [ 204.655083] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375647543worker2 # [ 204.661567] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7544worker2 # [ 204.664116] nix-daemon[850]: reaped child process 1859, status = succeeded7545worker2 # [ 204.668868] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7546client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7547client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7548client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7549client # farm-a> LOG-farm-a-drain7550worker2 # [ 206.087447] nix-daemon[850]: reaped child process 1873, status = succeeded7551worker2 # [ 206.166579] nix-daemon[850]: accepted connection from pid 1889, user nix-grpc-daemon (trusted)7552worker2 # [ 206.181082] nix-daemon[850]: reaped child process 1892, status = succeeded7553lb # [ 206.183809] niks3-server[1129]: 2026/09/22 15:06:37 INFO Received uploads request method=POST path=/api/pending_closures7554worker2 # [ 206.194316] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.417Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7555worker2 # [ 206.196348] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.419Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7556lb # [ 206.222052] niks3-server[1129]: 2026/09/22 15:06:37 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7557lb # [ 206.232848] niks3-server[1129]: 2026/09/22 15:06:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/81/sign7558lb # [ 206.238937] niks3-server[1129]: 2026/09/22 15:06:37 INFO Signed narinfos id=81 count=17559worker2 # [ 206.243665] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.467Z level=INFO msg="Uploading 1 narinfos"7560lb # [ 206.244172] niks3-server[1129]: 2026/09/22 15:06:37 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7561lb # [ 206.247395] niks3-server[1129]: 2026/09/22 15:06:37 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7562lb # [ 206.254647] niks3-server[1129]: 2026/09/22 15:06:37 INFO Received complete upload request method=POST path=/api/pending_closures/81/complete7563lb # [ 206.259171] niks3-server[1129]: 2026/09/22 15:06:37 INFO Completed upload id=817564worker2 # [ 206.261311] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.485Z level=INFO msg="Upload complete. (184ms)"7565lb # [ 206.261690] niks3-server[1129]: 2026/09/22 15:06:37 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7566worker2 # [ 206.266969] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:37Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=15 outputs=17567worker2 # [ 206.277509] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:37Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07568worker2 # [ 206.281628] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:37Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17569worker2 # [ 206.289304] nix-daemon[850]: reaped child process 1871, status = succeeded7570worker2 # [ 206.320848] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375647571worker2 # [ 206.330612] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:37Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375647572worker2 # [ 206.341060] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.564Z level=INFO msg="All 1 paths already cached"7573client # copying 1 paths to 'lb:50051'...7574client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7575worker2 # [ 206.348840] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375647576worker2 # [ 206.357283] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7577worker2 # [ 206.456212] nix-daemon[850]: accepted connection from pid 1898, user nix-grpc-daemon (trusted)7578worker2 # [ 206.473384] nix-daemon[850]: reaped child process 1901, status = succeeded7579lb # [ 206.532353] envoy[916]: conn peer=[2001:db8:1::1]:50766 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=561 ms=300337580worker2 # [ 206.557901] nix-daemon[850]: accepted connection from pid 1903, user nix-grpc-daemon (trusted)7581worker2 # [ 206.572995] nix-daemon[850]: reaped child process 1906, status = succeeded7582lb # [ 206.576811] niks3-server[1129]: 2026/09/22 15:06:37 INFO Received uploads request method=POST path=/api/pending_closures7583worker2 # [ 206.587091] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.810Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7584worker2 # [ 206.588585] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.810Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7585lb # [ 206.603992] niks3-server[1129]: 2026/09/22 15:06:37 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7586lb # [ 206.613322] niks3-server[1129]: 2026/09/22 15:06:37 INFO Received sign narinfos request method=POST path=/api/pending_closures/82/sign7587worker2 # [ 206.622581] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.846Z level=INFO msg="Uploading 1 narinfos"7588lb # [ 206.620282] niks3-server[1129]: 2026/09/22 15:06:37 INFO Signed narinfos id=82 count=17589lb # [ 206.627871] niks3-server[1129]: 2026/09/22 15:06:37 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7590lb # [ 206.641078] niks3-server[1129]: 2026/09/22 15:06:37 INFO Received complete upload request method=POST path=/api/pending_closures/82/complete7591lb # [ 206.649181] niks3-server[1129]: 2026/09/22 15:06:37 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7592lb # [ 206.652058] niks3-server[1129]: 2026/09/22 15:06:37 INFO Completed upload id=827593worker2 # [ 206.654114] nix-grpc-daemon[1641]: time=2026-09-22T15:06:37.877Z level=INFO msg="Upload complete. (286ms)"7594worker2 # [ 206.657575] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 paths=1 nar_bytes_in=6247595worker2 # [ 206.662426] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375647596worker2 # [ 206.671874] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7597worker2 # [ 206.681195] nix-daemon[850]: reaped child process 1896, status = succeeded7598worker2 # [ 206.686584] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7599client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7600client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7601client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7602client # farm-b> LOG-farm-b-drain7603worker2 # [ 207.528443] nix-daemon[850]: reaped child process 1910, status = succeeded7604worker2 # [ 207.597741] nix-daemon[850]: accepted connection from pid 1926, user nix-grpc-daemon (trusted)7605worker2 # [ 207.611952] nix-daemon[850]: reaped child process 1929, status = succeeded7606lb # [ 207.614843] niks3-server[1129]: 2026/09/22 15:06:38 INFO Received uploads request method=POST path=/api/pending_closures7607worker2 # [ 207.625440] nix-grpc-daemon[1641]: time=2026-09-22T15:06:38.848Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7608worker2 # [ 207.626959] nix-grpc-daemon[1641]: time=2026-09-22T15:06:38.850Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7609lb # [ 207.656948] niks3-server[1129]: 2026/09/22 15:06:38 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7610lb # [ 207.668184] niks3-server[1129]: 2026/09/22 15:06:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/83/sign7611worker2 # [ 207.678363] nix-grpc-daemon[1641]: time=2026-09-22T15:06:38.901Z level=INFO msg="Uploading 1 narinfos"7612lb # [ 207.676497] niks3-server[1129]: 2026/09/22 15:06:38 INFO Signed narinfos id=83 count=17613lb # [ 207.682110] niks3-server[1129]: 2026/09/22 15:06:39 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7614lb # [ 207.687640] niks3-server[1129]: 2026/09/22 15:06:39 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7615lb # [ 207.699146] niks3-server[1129]: 2026/09/22 15:06:39 INFO Received complete upload request method=POST path=/api/pending_closures/83/complete7616lb # [ 207.702620] niks3-server[1129]: 2026/09/22 15:06:39 INFO Completed upload id=837617worker2 # [ 207.704772] nix-grpc-daemon[1641]: time=2026-09-22T15:06:38.928Z level=INFO msg="Upload complete. (210ms)"7618lb # [ 207.704528] niks3-server[1129]: 2026/09/22 15:06:39 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7619worker2 # [ 207.715234] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=16 outputs=17620worker2 # [ 207.724384] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:38Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07621worker2 # [ 207.728289] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:38Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17622worker2 # [ 207.731746] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375647623worker2 # [ 207.743285] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:375647624worker2 # [ 207.745522] nix-daemon[850]: reaped child process 1908, status = succeeded7625worker2 # [ 207.750059] nix-grpc-daemon[1641]: time=2026-09-22T15:06:38.973Z level=INFO msg="All 2 paths already cached"7626client # copying 1 paths to 'lb:50051'...7627client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7628worker2 # [ 207.754480] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:375647629worker2 # [ 207.758490] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7630worker2 # [ 207.855658] nix-daemon[850]: accepted connection from pid 1935, user nix-grpc-daemon (trusted)7631worker2 # [ 207.872172] nix-daemon[850]: reaped child process 1938, status = succeeded7632worker2 # [ 207.957691] nix-daemon[850]: accepted connection from pid 1940, user nix-grpc-daemon (trusted)7633worker2 # [ 207.973407] nix-daemon[850]: reaped child process 1943, status = succeeded7634lb # [ 207.975637] niks3-server[1129]: 2026/09/22 15:06:39 INFO Received uploads request method=POST path=/api/pending_closures7635worker2 # [ 207.982058] nix-grpc-daemon[1641]: time=2026-09-22T15:06:39.205Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7636worker2 # [ 207.983397] nix-grpc-daemon[1641]: time=2026-09-22T15:06:39.205Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7637lb # [ 207.989490] niks3-server[1129]: 2026/09/22 15:06:39 INFO Received sign narinfos request method=POST path=/api/pending_closures/84/sign7638lb # [ 207.992368] niks3-server[1129]: 2026/09/22 15:06:39 INFO Signed narinfos id=84 count=17639worker2 # [ 207.994657] nix-grpc-daemon[1641]: time=2026-09-22T15:06:39.218Z level=INFO msg="Uploading 1 narinfos"7640lb # [ 207.994771] niks3-server[1129]: 2026/09/22 15:06:39 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7641lb # [ 207.997651] niks3-server[1129]: 2026/09/22 15:06:39 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7642lb # [ 208.002878] niks3-server[1129]: 2026/09/22 15:06:39 INFO Received complete upload request method=POST path=/api/pending_closures/84/complete7643lb # [ 208.005780] niks3-server[1129]: 2026/09/22 15:06:39 INFO Completed upload id=847644worker2 # [ 208.007851] nix-grpc-daemon[1641]: time=2026-09-22T15:06:39.231Z level=INFO msg="Upload complete. (242ms)"7645worker2 # [ 208.010080] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=0 paths=1 nar_bytes_in=7527646lb # [ 208.009061] niks3-server[1129]: 2026/09/22 15:06:39 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7647worker2 # [ 208.013238] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:375647648worker2 # [ 208.018321] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7649worker2 # [ 208.024506] nix-daemon[850]: reaped child process 1933, status = succeeded7650client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7651client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7652worker2 # [ 208.028928] nix-daemon[850]: accepted connection from pid 1567, user nix-grpc-daemon (trusted)7653client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7654client # farm-top> LOG-farm-top-drain7655worker2 # [ 209.047225] nix-daemon[850]: reaped child process 1947, status = succeeded7656worker2 # [ 209.124482] nix-daemon[850]: accepted connection from pid 1963, user nix-grpc-daemon (trusted)7657worker2 # [ 209.140874] nix-daemon[850]: reaped child process 1966, status = succeeded7658lb # [ 209.142391] niks3-server[1129]: 2026/09/22 15:06:40 INFO Received uploads request method=POST path=/api/pending_closures7659worker2 # [ 209.148682] nix-grpc-daemon[1641]: time=2026-09-22T15:06:40.372Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7660worker2 # [ 209.150312] nix-grpc-daemon[1641]: time=2026-09-22T15:06:40.374Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7661lb # [ 209.161654] niks3-server[1129]: 2026/09/22 15:06:40 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7662lb # [ 209.166387] niks3-server[1129]: 2026/09/22 15:06:40 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7663lb # [ 209.168470] niks3-server[1129]: 2026/09/22 15:06:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/85/sign7664lb # [ 209.170595] niks3-server[1129]: 2026/09/22 15:06:40 INFO Signed narinfos id=85 count=17665worker2 # [ 209.172667] nix-grpc-daemon[1641]: time=2026-09-22T15:06:40.396Z level=INFO msg="Uploading 1 narinfos"7666lb # [ 209.174396] niks3-server[1129]: 2026/09/22 15:06:40 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7667lb # [ 209.178700] niks3-server[1129]: 2026/09/22 15:06:40 INFO Received complete upload request method=POST path=/api/pending_closures/85/complete7668lb # [ 209.181785] niks3-server[1129]: 2026/09/22 15:06:40 INFO Completed upload id=857669worker2 # [ 209.183862] nix-grpc-daemon[1641]: time=2026-09-22T15:06:40.407Z level=INFO msg="Upload complete. (147ms)"7670lb # [ 209.184314] niks3-server[1129]: 2026/09/22 15:06:40 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7671worker2 # [ 209.186683] nix-grpc-daemon[1567]: ts=2026-09-22T15:06:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:37564 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=17 outputs=17672worker2 # [ 209.195076] nix-daemon[850]: reaped child process 1945, status = succeeded7673client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.12 seconds)7674client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7675client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7676worker1: must succeed: rm /nix/.rw-store/fill7677worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.09 seconds)7678worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7679worker1 # [ 209.860616] nix-grpc-daemon[1888]: ts=2026-09-22T15:06:41Z level=info event=healthy reason=min_free7680worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.10 seconds)7681(finished: subtest: low disk drains a worker and builds go to the other, in 8.47 seconds)7682(finished: run the VM test script, in 211.50 seconds)7683test script finished in 211.67s7684cleanup7685kill QemuMachine (pid 49)7686client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7687client # [2026-09-22T15:06:42Z INFO virtiofsd] Client disconnected, shutting down7688client # [2026-09-22T15:06:42Z INFO virtiofsd] Client disconnected, shutting down7689client # [2026-09-22T15:06:42Z INFO virtiofsd] Client disconnected, shutting down7690kill QemuMachine (pid 50)7691lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7692lb # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7693lb # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7694lb # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7695kill QemuMachine (pid 52)7696worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7697worker1 # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7698worker1 # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7699worker1 # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7700kill QemuMachine (pid 51)7701worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7702worker2 # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7703worker2 # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7704worker2 # [2026-09-22T15:06:43Z INFO virtiofsd] Client disconnected, shutting down7705(finished: cleanup, in 1.13 seconds)7706additionally exposed symbols:7707 client, lb, worker1, worker2,7708 vlan1,7709 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh