vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #479
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker1 # Disk image does not exist, creating the virtualisation disk image...13worker1 # Formatting '/build/vm-state-worker1/tmp.5hurYdPoLc', fmt=raw size=107374182414worker1: QEMU running (pid 51)15worker1 # mke2fs 1.47.4 (6-Mar-2025)16worker1 # Discarding device blocks: 0/262144 done17client # Disk image does not exist, creating the virtualisation disk image...18client: QEMU running (pid 49)19client # Formatting '/build/vm-state-client/tmp.Ee7gBFYovT', fmt=raw size=107374182420worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes21client # mke2fs 1.47.4 (6-Mar-2025)22worker1 # Filesystem UUID: bba8e287-ba5f-4e23-8aca-8e249ddc43e423client # Discarding device blocks: 0/262144 done24lb # Disk image does not exist, creating the virtualisation disk image...25client # Creating filesystem with 262144 4k blocks and 65536 inodes26worker1 # Superblock backups stored on blocks:27worker2 # Disk image does not exist, creating the virtualisation disk image...28worker2: QEMU running (pid 52)29lb # Formatting '/build/vm-state-lb/tmp.zukaCZbZgL', fmt=raw size=107374182430lb: QEMU running (pid 50)31client # Filesystem UUID: 8b055e39-b99b-41a2-aedd-b933cd0e48e232lb # mke2fs 1.47.4 (6-Mar-2025)33worker1 # 32768, 98304, 163840, 22937634worker2 # Formatting '/build/vm-state-worker2/tmp.PXMVD9ukWs', fmt=raw size=107374182435client # Superblock backups stored on blocks:36lb # Discarding device blocks: 0/262144 done37worker1 # 38worker2 # mke2fs 1.47.4 (6-Mar-2025)39lb # Creating filesystem with 262144 4k blocks and 65536 inodes40worker1 # Allocating group tables: 0/8 done41worker2 # Discarding device blocks: 0/262144 done42lb # Filesystem UUID: bf614ca3-9c9e-42d3-9c60-d1f66add456443worker1 # Writing inode tables: 0/8 done44worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes45lb # Superblock backups stored on blocks:46worker1 # Creating journal (8192 blocks): done47worker2 # Filesystem UUID: 1770fd8a-c118-4945-975a-ee7a58457db148lb # 32768, 98304, 163840, 22937649worker1 # Writing superblocks and filesystem accounting information: 0/8 done50worker2 # Superblock backups stored on blocks:51lb # 52worker1 # 53worker2 # 32768, 98304, 163840, 22937654lb # Allocating group tables: 0/8 done55worker1 # Virtualisation disk image created.56worker2 # 57lb # Writing inode tables: 0/8 done58worker1 # Starting virtiofs daemons...59worker2 # Allocating group tables: 0/8 done60lb # Creating journal (8192 blocks): done61worker1 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)62worker2 # Writing inode tables: 0/8 done63lb # Writing superblocks and filesystem accounting information: 0/8 done64worker1 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether65worker2 # Creating journal (8192 blocks): done66lb # 67worker1 # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...68worker2 # Writing superblocks and filesystem accounting information: 0/8 done69lb # Virtualisation disk image created.70worker1 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)71worker2 # 72lb # Starting virtiofs daemons...73worker1 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether74worker2 # Virtualisation disk image created.75lb # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)76worker1 # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...77worker2 # Starting virtiofs daemons...78lb # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether79worker1 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)80worker2 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)81lb # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...82worker1 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether83worker2 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether84lb # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)85worker1 # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...86worker2 # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...87lb # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether88worker1 # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests89worker2 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)90lb # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...91worker1 # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests92worker2 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether93lb # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94worker1 # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests95worker2 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)96lb # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether97client # 32768, 98304, 163840, 22937698(finished: start all VMs, in 0.24 seconds)99worker2 # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether100lb # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...101client # 102client: waiting for unit network-addresses-eth1.service103worker2 # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...104lb # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests105client # Allocating group tables: 0/8 done106client: waiting for the VM to finish booting107worker2 # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...108lb # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests109worker2 # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests110lb # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests111worker2 # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests112client # Writing inode tables: 0/8 done113worker2 # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests114client # Creating journal (8192 blocks): done115client # Writing superblocks and filesystem accounting information: 0/8 done116client # 117client # Virtualisation disk image created.118client # Starting virtiofs daemons...119client # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)120client # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...122client # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123client # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124client # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...125client # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126client # [2026-09-21T14:28:56Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127client # [2026-09-21T14:28:56Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-21T14:28:56Z INFO virtiofsd] Client connected, servicing requests131worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker2 # 136worker2 # 137worker1 # 138worker1 # 139worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00140worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00141worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...142worker2 # 143worker2 # 144worker2 # 145worker2 # 146worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...147worker1 # 148worker1 # 149worker1 # 150worker1 # 151worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00152worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00153lb # 154lb # 155lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00156worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...157worker2 # 158worker2 # 159worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...160worker1 # 161worker1 # 162lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...163lb # 164lb # 165lb # 166lb # 167lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00168lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169lb # 170lb # 171client # 172client # 173client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00174client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...175client # 176client # 177client # 178client # 179client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...181client # 182client # 183worker2 # Booting from ROM...184worker1 # Booting from ROM...185lb # Booting from ROM...186client # Booting from ROM...187worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yfs0c81qbblhjyw6lfs25cwbnx2791iq-nixos-system-worker2-test/init regInfo=/nix/store/l7xw28r8q3bra83p81s8rvp2dxz8nzjr-closure-info/registration console=ttyS0,115200n8 console=tty0189worker2 # [ 0.000000] BIOS-provided physical RAM map:190worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200worker2 # [ 0.000000] NX (Execute Disable) protection: active201worker2 # [ 0.000000] APIC: Static calls initialized202worker2 # [ 0.000000] SMBIOS 2.8 present.203worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014204worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026205worker2 # [ 0.000000] DMI: Memory slots populated: 1/1206worker2 # [ 0.000000] Hypervisor detected: KVM207worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000208worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00209worker2 # [ 0.000000] kvm-clock: using sched offset of 469005695 cycles210worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qggyx523n6nbpvgymzwfzhq707m01zph-nixos-system-worker1-test/init regInfo=/nix/store/xzabmsah3l5jc5ynkbhvd8b35120bd6n-closure-info/registration console=ttyS0,115200n8 console=tty0211worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns212worker1 # [ 0.000000] BIOS-provided physical RAM map:213worker2 # [ 0.000005] tsc: Detected 2400.012 MHz processor214worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable215worker2 # [ 0.000809] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000216worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved217worker2 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs218worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved219worker2 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT220worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable221worker2 # [ 0.002783] found SMP MP-table at [mem 0x000f5450-0x000f545f]222worker2 # [ 0.002794] Using GB pages for direct mapping223lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026224worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved225worker2 # [ 0.002885] RAMDISK: [mem 0x5e36f000-0x5ffcffff]226worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved227worker2 # [ 0.002893] ACPI: Early table checksum verification disabled228worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved229worker2 # [ 0.002896] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )230worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved231worker2 # [ 0.002899] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)232worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved233lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ywx2cfw2bd45f1sp7w8ib468r0dscq27-nixos-system-lb-test/init regInfo=/nix/store/lhzhb7lk7gq3c4r6bxsdb6n35xs7cgxx-closure-info/registration console=ttyS0,115200n8 console=tty0234worker2 # [ 0.002903] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)235lb # [ 0.000000] BIOS-provided physical RAM map:236worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved237client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026238worker2 # [ 0.002910] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)239worker1 # [ 0.000000] NX (Execute Disable) protection: active240lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable241worker1 # [ 0.000000] APIC: Static calls initialized242worker2 # [ 0.002912] ACPI: FACS 0x000000005FFE0000 000040243lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved244worker1 # [ 0.000000] SMBIOS 2.8 present.245worker2 # [ 0.002914] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)246lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved247worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014248lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable249worker1 # [ 0.000000] DMI: Memory slots populated: 1/1250worker2 # [ 0.002915] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)251worker1 # [ 0.000000] Hypervisor detected: KVM252lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved253client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test/init regInfo=/nix/store/6yakikkdias5blvs43n5xx753sdmgk3b-closure-info/registration console=ttyS0,115200n8 console=tty0254worker2 # [ 0.002917] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)255worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000256client # [ 0.000000] BIOS-provided physical RAM map:257lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved258worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00259worker2 # [ 0.002919] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)260client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable261lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved262worker1 # [ 0.000001] kvm-clock: using sched offset of 491830642 cycles263worker2 # [ 0.002920] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]264client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved265lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved266worker2 # [ 0.002921] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]267worker1 # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns268client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved269lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved270worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor271client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable272worker2 # [ 0.002921] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]273lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved274worker1 # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000275client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved276lb # [ 0.000000] NX (Execute Disable) protection: active277worker2 # [ 0.002922] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]278lb # [ 0.000000] APIC: Static calls initialized279worker1 # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs280client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved281worker2 # [ 0.002922] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]282lb # [ 0.000000] SMBIOS 2.8 present.283worker1 # [ 0.000841] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT284client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved285worker2 # [ 0.002923] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]286worker1 # [ 0.002744] found SMP MP-table at [mem 0x000f5450-0x000f545f]287lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014288client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved289worker2 # [ 0.002923] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]290worker1 # [ 0.002755] Using GB pages for direct mapping291lb # [ 0.000000] DMI: Memory slots populated: 1/1292worker2 # [ 0.003155] No NUMA configuration found293lb # [ 0.000000] Hypervisor detected: KVM294worker1 # [ 0.002822] RAMDISK: [mem 0x5e370000-0x5ffcffff]295client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved296lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000297worker2 # [ 0.003156] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]298worker1 # [ 0.002833] ACPI: Early table checksum verification disabled299client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved300worker2 # [ 0.003159] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]301lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00302worker1 # [ 0.002836] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )303worker2 # [ 0.005459] Zone ranges:304client # [ 0.000000] NX (Execute Disable) protection: active305lb # [ 0.000000] kvm-clock: using sched offset of 490704667 cycles306client # [ 0.000000] APIC: Static calls initialized307worker2 # [ 0.005460] DMA [mem 0x0000000000001000-0x0000000000ffffff]308client # [ 0.000000] SMBIOS 2.8 present.309worker1 # [ 0.002840] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)310worker2 # [ 0.005461] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]311lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns312worker2 # [ 0.005463] Normal empty313worker1 # [ 0.002844] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)314client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014315worker2 # [ 0.005463] Device empty316lb # [ 0.000005] tsc: Detected 2400.012 MHz processor317client # [ 0.000000] DMI: Memory slots populated: 1/1318worker2 # [ 0.005464] Movable zone start for each node319lb # [ 0.000812] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000320worker1 # [ 0.002852] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)321client # [ 0.000000] Hypervisor detected: KVM322worker2 # [ 0.005465] Early memory node ranges323worker1 # [ 0.002854] ACPI: FACS 0x000000005FFE0000 000040324client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000325lb # [ 0.000839] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs326worker2 # [ 0.005465] node 0: [mem 0x0000000000001000-0x000000000009efff]327client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00328worker1 # [ 0.002856] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)329worker2 # [ 0.005466] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]330lb # [ 0.000842] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT331client # [ 0.000001] kvm-clock: using sched offset of 499180575 cycles332lb # [ 0.002733] found SMP MP-table at [mem 0x000f5450-0x000f545f]333worker1 # [ 0.002857] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)334worker2 # [ 0.005467] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]335lb # [ 0.002744] Using GB pages for direct mapping336worker2 # [ 0.005487] On node 0, zone DMA: 1 pages in unavailable ranges337client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns338lb # [ 0.002888] RAMDISK: [mem 0x5e36e000-0x5ffcffff]339worker1 # [ 0.002859] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)340client # [ 0.000005] tsc: Detected 2400.012 MHz processor341worker2 # [ 0.005752] On node 0, zone DMA: 97 pages in unavailable ranges342lb # [ 0.002899] ACPI: Early table checksum verification disabled343client # [ 0.000805] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000344worker1 # [ 0.002860] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)345worker2 # [ 0.032740] On node 0, zone DMA32: 40 pages in unavailable ranges346lb # [ 0.002902] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )347worker2 # [ 0.033201] ACPI: PM-Timer IO Port: 0x608348worker1 # [ 0.002862] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]349client # [ 0.000832] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs350worker2 # [ 0.033211] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])351lb # [ 0.002906] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)352client # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT353worker1 # [ 0.002863] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]354worker2 # [ 0.033237] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23355lb # [ 0.002910] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)356client # [ 0.002721] found SMP MP-table at [mem 0x000f5450-0x000f545f]357worker1 # [ 0.002863] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]358worker2 # [ 0.033239] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)359client # [ 0.002732] Using GB pages for direct mapping360lb # [ 0.002917] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)361worker1 # [ 0.002864] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]362client # [ 0.002868] RAMDISK: [mem 0x5e370000-0x5ffcffff]363worker2 # [ 0.033241] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)364lb # [ 0.002919] ACPI: FACS 0x000000005FFE0000 000040365client # [ 0.002876] ACPI: Early table checksum verification disabled366worker1 # [ 0.002864] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]367worker2 # [ 0.033242] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)368client # [ 0.002878] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )369lb # [ 0.002920] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)370worker1 # [ 0.002865] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]371worker2 # [ 0.033243] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)372client # [ 0.002882] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)373lb # [ 0.002922] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)374worker2 # [ 0.033244] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)375worker1 # [ 0.002865] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]376worker1 # [ 0.003099] No NUMA configuration found377worker2 # [ 0.033246] ACPI: Using ACPI (MADT) for SMP configuration information378client # [ 0.002885] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)379lb # [ 0.002923] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)380worker1 # [ 0.003100] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]381worker2 # [ 0.033247] ACPI: HPET id: 0x8086a201 base: 0xfed00000382worker2 # [ 0.033250] TSC deadline timer available383client # [ 0.002892] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)384lb # [ 0.002925] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)385worker1 # [ 0.003102] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]386worker1 # [ 0.005904] Zone ranges:387client # [ 0.002894] ACPI: FACS 0x000000005FFE0000 000040388worker2 # [ 0.033254] CPU topo: Max. logical packages: 1389lb # [ 0.002926] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]390worker1 # [ 0.005906] DMA [mem 0x0000000000001000-0x0000000000ffffff]391worker2 # [ 0.033255] CPU topo: Max. logical dies: 1392client # [ 0.002896] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)393worker2 # [ 0.033255] CPU topo: Max. dies per package: 1394lb # [ 0.002927] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]395worker1 # [ 0.005907] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]396worker2 # [ 0.033258] CPU topo: Max. threads per core: 1397worker1 # [ 0.005909] Normal empty398worker1 # [ 0.005909] Device empty399lb # [ 0.002928] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]400client # [ 0.002897] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)401worker2 # [ 0.033259] CPU topo: Num. cores per package: 1402worker1 # [ 0.005910] Movable zone start for each node403worker2 # [ 0.033259] CPU topo: Num. threads per package: 1404lb # [ 0.002928] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]405worker1 # [ 0.005911] Early memory node ranges406client # [ 0.002899] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)407worker2 # [ 0.033259] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs408lb # [ 0.002928] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]409worker1 # [ 0.005911] node 0: [mem 0x0000000000001000-0x000000000009efff]410client # [ 0.002901] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)411worker2 # [ 0.033277] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()412worker1 # [ 0.005912] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]413lb # [ 0.002929] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]414client # [ 0.002902] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]415worker2 # [ 0.033306] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]416worker1 # [ 0.005913] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]417lb # [ 0.002929] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]418client # [ 0.002903] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]419lb # [ 0.003180] No NUMA configuration found420worker1 # [ 0.005938] On node 0, zone DMA: 1 pages in unavailable ranges421worker2 # [ 0.033308] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]422client # [ 0.002903] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]423lb # [ 0.003181] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]424worker1 # [ 0.006211] On node 0, zone DMA: 97 pages in unavailable ranges425worker2 # [ 0.033309] [mem 0x60000000-0xafffffff] available for PCI devices426lb # [ 0.003184] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]427client # [ 0.002904] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]428worker1 # [ 0.034471] On node 0, zone DMA32: 40 pages in unavailable ranges429lb # [ 0.005594] Zone ranges:430worker2 # [ 0.033311] Booting paravirtualized kernel on KVM431worker1 # [ 0.034941] ACPI: PM-Timer IO Port: 0x608432client # [ 0.002904] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]433lb # [ 0.005595] DMA [mem 0x0000000000001000-0x0000000000ffffff]434worker1 # [ 0.034952] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])435worker2 # [ 0.033314] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns436client # [ 0.002905] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]437lb # [ 0.005597] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]438lb # [ 0.005598] Normal empty439worker1 # [ 0.034981] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23440worker2 # [ 0.037765] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1441lb # [ 0.005599] Device empty442client # [ 0.002905] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]443worker1 # [ 0.034984] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)444lb # [ 0.005599] Movable zone start for each node445worker2 # [ 0.039957] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152446client # [ 0.003136] No NUMA configuration found447lb # [ 0.005600] Early memory node ranges448worker2 # [ 0.040001] kvm-guest: PV spinlocks disabled, single CPU449worker1 # [ 0.034986] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)450client # [ 0.003137] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]451lb # [ 0.005601] node 0: [mem 0x0000000000001000-0x000000000009efff]452client # [ 0.003139] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]453worker1 # [ 0.034987] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)454client # [ 0.005424] Zone ranges:455lb # [ 0.005602] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]456worker1 # [ 0.034988] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)457client # [ 0.005425] DMA [mem 0x0000000000001000-0x0000000000ffffff]458lb # [ 0.005603] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]459worker1 # [ 0.034989] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)460client # [ 0.005427] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]461lb # [ 0.005625] On node 0, zone DMA: 1 pages in unavailable ranges462client # [ 0.005428] Normal empty463worker1 # [ 0.034991] ACPI: Using ACPI (MADT) for SMP configuration information464client # [ 0.005429] Device empty465lb # [ 0.005899] On node 0, zone DMA: 97 pages in unavailable ranges466worker2 # [ 0.040003] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/yfs0c81qbblhjyw6lfs25cwbnx2791iq-nixos-system-worker2-test/init regInfo=/nix/store/l7xw28r8q3bra83p81s8rvp2dxz8nzjr-closure-info/registration console=ttyS0,115200n8 console=tty0467client # [ 0.005430] Movable zone start for each node468worker1 # [ 0.034993] ACPI: HPET id: 0x8086a201 base: 0xfed00000469lb # [ 0.034402] On node 0, zone DMA32: 40 pages in unavailable ranges470client # [ 0.005430] Early memory node ranges471worker1 # [ 0.034996] TSC deadline timer available472lb # [ 0.034866] ACPI: PM-Timer IO Port: 0x608473worker1 # [ 0.035000] CPU topo: Max. logical packages: 1474client # [ 0.005431] node 0: [mem 0x0000000000001000-0x000000000009efff]475lb # [ 0.034877] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])476worker2 # [ 0.040094] Unknown kernel command line parameters "regInfo=/nix/store/l7xw28r8q3bra83p81s8rvp2dxz8nzjr-closure-info/registration", will be passed to user space.477worker1 # [ 0.035001] CPU topo: Max. logical dies: 1478client # [ 0.005432] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]479worker2 # [ 0.040106] random: crng init done480worker1 # [ 0.035002] CPU topo: Max. dies per package: 1481lb # [ 0.034906] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23482worker1 # [ 0.035005] CPU topo: Max. threads per core: 1483client # [ 0.005433] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]484worker2 # [ 0.040107] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes485lb # [ 0.034909] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)486worker1 # [ 0.035006] CPU topo: Num. cores per package: 1487client # [ 0.005454] On node 0, zone DMA: 1 pages in unavailable ranges488worker2 # [ 0.042271] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)489lb # [ 0.034910] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)490worker1 # [ 0.035006] CPU topo: Num. threads per package: 1491client # [ 0.005715] On node 0, zone DMA: 97 pages in unavailable ranges492worker2 # [ 0.043613] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)493worker1 # [ 0.035006] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs494lb # [ 0.034911] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)495client # [ 0.032658] On node 0, zone DMA32: 40 pages in unavailable ranges496worker2 # [ 0.043645] Fallback order for Node 0: 0497lb # [ 0.034912] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)498worker1 # [ 0.035025] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()499client # [ 0.033118] ACPI: PM-Timer IO Port: 0x608500worker2 # [ 0.043648] Built 1 zonelists, mobility grouping on. Total pages: 393078501client # [ 0.033128] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])502lb # [ 0.034913] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)503worker2 # [ 0.043649] Policy zone: DMA32504worker1 # [ 0.035056] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]505client # [ 0.033154] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23506lb # [ 0.034915] ACPI: Using ACPI (MADT) for SMP configuration information507worker2 # [ 0.046174] mem auto-init: stack:all(zero), heap alloc:on, heap free:off508worker1 # [ 0.035058] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]509lb # [ 0.034916] ACPI: HPET id: 0x8086a201 base: 0xfed00000510client # [ 0.033156] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)511worker2 # [ 0.049216] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1512worker1 # [ 0.035059] [mem 0x60000000-0xafffffff] available for PCI devices513lb # [ 0.034920] TSC deadline timer available514worker2 # [ 0.052545] allocated 3145728 bytes of page_ext515worker1 # [ 0.035061] Booting paravirtualized kernel on KVM516client # [ 0.033158] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)517lb # [ 0.034923] CPU topo: Max. logical packages: 1518worker2 # [ 0.062197] ftrace: allocating 48787 entries in 192 pages519client # [ 0.033159] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)520lb # [ 0.034924] CPU topo: Max. logical dies: 1521worker2 # [ 0.062198] ftrace: allocated 192 pages with 2 groups522lb # [ 0.034925] CPU topo: Max. dies per package: 1523worker1 # [ 0.035064] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns524client # [ 0.033160] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)525worker2 # [ 0.063053] Dynamic Preempt: lazy526lb # [ 0.034928] CPU topo: Max. threads per core: 1527worker2 # [ 0.063244] rcu: Preemptible hierarchical RCU implementation.528worker1 # [ 0.039554] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1529client # [ 0.033161] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)530lb # [ 0.034928] CPU topo: Num. cores per package: 1531worker2 # [ 0.063244] rcu: RCU event tracing is enabled.532lb # [ 0.034929] CPU topo: Num. threads per package: 1533client # [ 0.033163] ACPI: Using ACPI (MADT) for SMP configuration information534worker1 # [ 0.041895] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152535worker2 # [ 0.063245] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.536worker1 # [ 0.041942] kvm-guest: PV spinlocks disabled, single CPU537lb # [ 0.034929] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs538client # [ 0.033164] ACPI: HPET id: 0x8086a201 base: 0xfed00000539worker2 # [ 0.063246] Trampoline variant of Tasks RCU enabled.540client # [ 0.033168] TSC deadline timer available541lb # [ 0.034947] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()542worker2 # [ 0.063247] Rude variant of Tasks RCU enabled.543client # [ 0.033171] CPU topo: Max. logical packages: 1544worker2 # [ 0.063247] Tracing variant of Tasks RCU enabled.545lb # [ 0.034978] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]546client # [ 0.033172] CPU topo: Max. logical dies: 1547client # [ 0.033172] CPU topo: Max. dies per package: 1548worker2 # [ 0.063248] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.549lb # [ 0.034979] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]550client # [ 0.033176] CPU topo: Max. threads per core: 1551worker2 # [ 0.063248] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1552lb # [ 0.034981] [mem 0x60000000-0xafffffff] available for PCI devices553client # [ 0.033176] CPU topo: Num. cores per package: 1554worker1 # [ 0.041944] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/qggyx523n6nbpvgymzwfzhq707m01zph-nixos-system-worker1-test/init regInfo=/nix/store/xzabmsah3l5jc5ynkbhvd8b35120bd6n-closure-info/registration console=ttyS0,115200n8 console=tty0555lb # [ 0.034982] Booting paravirtualized kernel on KVM556client # [ 0.033177] CPU topo: Num. threads per package: 1557worker2 # [ 0.063270] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.558client # [ 0.033177] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs559lb # [ 0.034985] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns560worker2 # [ 0.063272] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.561worker1 # [ 0.042040] Unknown kernel command line parameters "regInfo=/nix/store/xzabmsah3l5jc5ynkbhvd8b35120bd6n-closure-info/registration", will be passed to user space.562client # [ 0.033193] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()563worker1 # [ 0.042053] random: crng init done564lb # [ 0.039484] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1565client # [ 0.033224] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]566worker2 # [ 0.063272] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.567worker1 # [ 0.042054] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes568lb # [ 0.041759] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152569worker2 # [ 0.067647] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16570lb # [ 0.041807] kvm-guest: PV spinlocks disabled, single CPU571client # [ 0.033225] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]572worker1 # [ 0.044301] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)573worker2 # [ 0.068217] rcu: srcu_init: Setting srcu_struct sizes based on contention.574client # [ 0.033227] [mem 0x60000000-0xafffffff] available for PCI devices575worker1 # [ 0.045413] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)576client # [ 0.033228] Booting paravirtualized kernel on KVM577worker2 # [ 0.068224] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns578worker1 # [ 0.045445] Fallback order for Node 0: 0579worker1 # [ 0.045449] Built 1 zonelists, mobility grouping on. Total pages: 393078580client # [ 0.033231] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns581worker2 # [ 0.068322] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)582worker1 # [ 0.045450] Policy zone: DMA32583worker2 # [ 0.071844] Console: colour VGA+ 80x25584client # [ 0.037722] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1585lb # [ 0.041808] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/ywx2cfw2bd45f1sp7w8ib468r0dscq27-nixos-system-lb-test/init regInfo=/nix/store/lhzhb7lk7gq3c4r6bxsdb6n35xs7cgxx-closure-info/registration console=ttyS0,115200n8 console=tty0586worker1 # [ 0.048373] mem auto-init: stack:all(zero), heap alloc:on, heap free:off587worker2 # [ 0.071847] printk: legacy console [tty0] enabled588client # [ 0.039910] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152589worker2 # [ 0.101276] printk: legacy console [ttyS0] enabled590worker1 # [ 0.051722] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1591client # [ 0.039958] kvm-guest: PV spinlocks disabled, single CPU592worker1 # [ 0.055254] allocated 3145728 bytes of page_ext593worker2 # [ 0.198566] ACPI: Core revision 20250807594lb # [ 0.041904] Unknown kernel command line parameters "regInfo=/nix/store/lhzhb7lk7gq3c4r6bxsdb6n35xs7cgxx-closure-info/registration", will be passed to user space.595worker1 # [ 0.065428] ftrace: allocating 48787 entries in 192 pages596lb # [ 0.041916] random: crng init done597worker1 # [ 0.065430] ftrace: allocated 192 pages with 2 groups598lb # [ 0.041917] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes599worker1 # [ 0.066316] Dynamic Preempt: lazy600worker2 # [ 0.199463] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns601worker1 # [ 0.066487] rcu: Preemptible hierarchical RCU implementation.602lb # [ 0.044140] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)603worker2 # [ 0.200939] APIC: Switch to symmetric I/O mode setup604worker1 # [ 0.066488] rcu: RCU event tracing is enabled.605worker2 # [ 0.201900] x2apic enabled606client # [ 0.039959] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test/init regInfo=/nix/store/6yakikkdias5blvs43n5xx753sdmgk3b-closure-info/registration console=ttyS0,115200n8 console=tty0607lb # [ 0.045526] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)608worker1 # [ 0.066488] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.609lb # [ 0.045557] Fallback order for Node 0: 0610worker1 # [ 0.066490] Trampoline variant of Tasks RCU enabled.611worker2 # [ 0.202632] APIC: Switched APIC routing to: physical x2apic612lb # [ 0.045560] Built 1 zonelists, mobility grouping on. Total pages: 393078613worker1 # [ 0.066491] Rude variant of Tasks RCU enabled.614client # [ 0.040050] Unknown kernel command line parameters "regInfo=/nix/store/6yakikkdias5blvs43n5xx753sdmgk3b-closure-info/registration", will be passed to user space.615lb # [ 0.045561] Policy zone: DMA32616worker1 # [ 0.066491] Tracing variant of Tasks RCU enabled.617client # [ 0.040062] random: crng init done618lb # [ 0.048176] mem auto-init: stack:all(zero), heap alloc:on, heap free:off619worker1 # [ 0.066491] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.620client # [ 0.040063] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes621worker2 # [ 0.204441] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1622lb # [ 0.051474] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1623worker1 # [ 0.066492] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1624lb # [ 0.054924] allocated 3145728 bytes of page_ext625client # [ 0.042213] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)626lb # [ 0.064940] ftrace: allocating 48787 entries in 192 pages627worker2 # [ 0.205397] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns628worker1 # [ 0.066514] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.629client # [ 0.043608] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)630lb # [ 0.064942] ftrace: allocated 192 pages with 2 groups631client # [ 0.043640] Fallback order for Node 0: 0632lb # [ 0.065819] Dynamic Preempt: lazy633worker1 # [ 0.066516] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.634worker2 # [ 0.207043] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)635client # [ 0.043642] Built 1 zonelists, mobility grouping on. Total pages: 393078636lb # [ 0.065983] rcu: Preemptible hierarchical RCU implementation.637client # [ 0.043643] Policy zone: DMA32638lb # [ 0.065984] rcu: RCU event tracing is enabled.639worker1 # [ 0.066517] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.640worker2 # [ 0.208127] x86/cpu: User Mode Instruction Prevention (UMIP) activated641client # [ 0.046223] mem auto-init: stack:all(zero), heap alloc:on, heap free:off642lb # [ 0.065985] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.643worker1 # [ 0.070964] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16644client # [ 0.049274] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1645lb # [ 0.065986] Trampoline variant of Tasks RCU enabled.646worker2 # [ 0.209152] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127647worker1 # [ 0.071251] rcu: srcu_init: Setting srcu_struct sizes based on contention.648lb # [ 0.065987] Rude variant of Tasks RCU enabled.649client # [ 0.052614] allocated 3145728 bytes of page_ext650lb # [ 0.065987] Tracing variant of Tasks RCU enabled.651worker2 # [ 0.210039] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0652client # [ 0.062436] ftrace: allocating 48787 entries in 192 pages653worker1 # [ 0.071258] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns654client # [ 0.062438] ftrace: allocated 192 pages with 2 groups655lb # [ 0.065988] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.656client # [ 0.063279] Dynamic Preempt: lazy657worker1 # [ 0.071365] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)658lb # [ 0.065988] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1659worker2 # [ 0.211043] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto660client # [ 0.063451] rcu: Preemptible hierarchical RCU implementation.661worker1 # [ 0.074913] Console: colour VGA+ 80x25662client # [ 0.063451] rcu: RCU event tracing is enabled.663worker1 # [ 0.074916] printk: legacy console [tty0] enabled664lb # [ 0.066054] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.665client # [ 0.063452] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.666worker1 # [ 0.104364] printk: legacy console [ttyS0] enabled667worker2 # [ 0.212039] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl668client # [ 0.063454] Trampoline variant of Tasks RCU enabled.669lb # [ 0.066056] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.670worker1 # [ 0.201243] ACPI: Core revision 20250807671client # [ 0.063454] Rude variant of Tasks RCU enabled.672worker2 # [ 0.213039] Transient Scheduler Attacks: Vulnerable: No microcode673client # [ 0.063454] Tracing variant of Tasks RCU enabled.674lb # [ 0.066057] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.675worker2 # [ 0.214039] Spectre V2 : Mitigation: Enhanced / Automatic IBRS676lb # [ 0.070811] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16677client # [ 0.063455] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.678worker1 # [ 0.202197] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns679worker2 # [ 0.215039] Speculative Return Stack Overflow: Mitigation: Safe RET680lb # [ 0.071100] rcu: srcu_init: Setting srcu_struct sizes based on contention.681client # [ 0.063456] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1682worker1 # [ 0.203717] APIC: Switch to symmetric I/O mode setup683worker1 # [ 0.204722] x2apic enabled684client # [ 0.063477] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.685lb # [ 0.071108] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns686client # [ 0.063478] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.687worker1 # [ 0.205491] APIC: Switched APIC routing to: physical x2apic688lb # [ 0.071222] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)689lb # [ 0.074766] Console: colour VGA+ 80x25690client # [ 0.063479] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.691lb # [ 0.074769] printk: legacy console [tty0] enabled692worker2 # [ 0.216039] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization693client # [ 0.068156] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16694lb # [ 0.104232] printk: legacy console [ttyS0] enabled695worker1 # [ 0.207373] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1696worker2 # [ 0.218046] Spectre V2 : Enabling IBPB for BPF697client # [ 0.068432] rcu: srcu_init: Setting srcu_struct sizes based on contention.698lb # [ 0.199796] ACPI: Core revision 20250807699worker1 # [ 0.208358] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns700client # [ 0.068439] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns701worker2 # [ 0.219040] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier702lb # [ 0.200696] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns703worker2 # [ 0.220040] active return thunk: srso_alias_return_thunk704client # [ 0.068544] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)705worker1 # [ 0.210046] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)706lb # [ 0.202198] APIC: Switch to symmetric I/O mode setup707client # [ 0.072066] Console: colour VGA+ 80x25708lb # [ 0.203125] x2apic enabled709worker2 # [ 0.221059] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'710client # [ 0.072068] printk: legacy console [tty0] enabled711worker1 # [ 0.211132] x86/cpu: User Mode Instruction Prevention (UMIP) activated712client # [ 0.101573] printk: legacy console [ttyS0] enabled713worker2 # [ 0.222039] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'714lb # [ 0.203860] APIC: Switched APIC routing to: physical x2apic715client # [ 0.197138] ACPI: Core revision 20250807716worker1 # [ 0.212183] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127717worker2 # [ 0.223039] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'718worker1 # [ 0.213711] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0719client # [ 0.198014] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns720lb # [ 0.205676] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1721worker2 # [ 0.224039] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'722client # [ 0.199490] APIC: Switch to symmetric I/O mode setup723worker2 # [ 0.225039] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'724worker1 # [ 0.214820] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto725client # [ 0.200460] x2apic enabled726lb # [ 0.206658] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns727worker2 # [ 0.226039] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'728worker1 # [ 0.216043] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl729client # [ 0.201215] APIC: Switched APIC routing to: physical x2apic730lb # [ 0.208336] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)731worker2 # [ 0.227039] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'732worker1 # [ 0.217044] Transient Scheduler Attacks: Vulnerable: No microcode733worker2 # [ 0.228936] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256734lb # [ 0.209422] x86/cpu: User Mode Instruction Prevention (UMIP) activated735worker1 # [ 0.218760] Spectre V2 : Mitigation: Enhanced / Automatic IBRS736client # [ 0.203025] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1737worker2 # [ 0.229733] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64738lb # [ 0.210460] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127739worker1 # [ 0.219717] Speculative Return Stack Overflow: Mitigation: Safe RET740worker2 # [ 0.230748] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512741client # [ 0.203987] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns742lb # [ 0.211333] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0743worker2 # [ 0.231762] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024744client # [ 0.205619] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)745lb # [ 0.212337] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto746worker2 # [ 0.232772] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8747client # [ 0.206720] x86/cpu: User Mode Instruction Prevention (UMIP) activated748worker1 # [ 0.220043] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization749lb # [ 0.214332] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl750worker2 # [ 0.233039] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.751client # [ 0.207759] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127752worker1 # [ 0.222051] Spectre V2 : Enabling IBPB for BPF753lb # [ 0.215333] Transient Scheduler Attacks: Vulnerable: No microcode754client # [ 0.208617] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0755worker1 # [ 0.223044] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier756lb # [ 0.217066] Spectre V2 : Mitigation: Enhanced / Automatic IBRS757worker1 # [ 0.224043] active return thunk: srso_alias_return_thunk758lb # [ 0.218028] Speculative Return Stack Overflow: Mitigation: Safe RET759client # [ 0.209620] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto760worker1 # [ 0.225689] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'761client # [ 0.211616] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl762worker1 # [ 0.227043] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'763client # [ 0.212617] Transient Scheduler Attacks: Vulnerable: No microcode764worker1 # [ 0.228035] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'765client # [ 0.213616] Spectre V2 : Mitigation: Enhanced / Automatic IBRS766lb # [ 0.218333] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization767worker1 # [ 0.229043] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'768client # [ 0.214616] Speculative Return Stack Overflow: Mitigation: Safe RET769lb # [ 0.220340] Spectre V2 : Enabling IBPB for BPF770worker1 # [ 0.230043] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'771lb # [ 0.221333] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier772worker1 # [ 0.231038] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'773lb # [ 0.222334] active return thunk: srso_alias_return_thunk774worker1 # [ 0.232043] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'775lb # [ 0.224003] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'776client # [ 0.215616] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization777worker1 # [ 0.233044] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256778client # [ 0.217623] Spectre V2 : Enabling IBPB for BPF779lb # [ 0.225204] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'780worker1 # [ 0.234043] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64781lb # [ 0.226097] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'782client # [ 0.218617] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier783worker1 # [ 0.235043] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512784lb # [ 0.227075] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'785client # [ 0.219617] active return thunk: srso_alias_return_thunk786worker1 # [ 0.236043] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024787lb # [ 0.228092] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'788client # [ 0.220642] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'789worker1 # [ 0.237043] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8790lb # [ 0.229081] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'791client # [ 0.221616] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'792worker1 # [ 0.238043] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.793client # [ 0.222616] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'794lb # [ 0.230113] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'795client # [ 0.223616] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'796lb # [ 0.231333] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256797client # [ 0.224616] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'798lb # [ 0.232332] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64799lb # [ 0.233332] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512800client # [ 0.225616] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'801lb # [ 0.234332] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024802client # [ 0.226616] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'803lb # [ 0.235332] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8804client # [ 0.228534] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256805client # [ 0.229348] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64806lb # [ 0.236332] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.807client # [ 0.230342] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512808client # [ 0.231339] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024809client # [ 0.232338] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8810client # [ 0.233336] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.811worker2 # [ 0.267855] Freeing SMP alternatives memory: 44K812worker2 # [ 0.268591] pid_max: default: 32768 minimum: 301813worker2 # [ 0.269124] LSM: initializing lsm=capability,landlock,yama,bpf,ima814worker2 # [ 0.270141] landlock: Up and running.815worker2 # [ 0.271040] Yama: becoming mindful.816worker2 # [ 0.272253] LSM support for eBPF active817worker2 # [ 0.273102] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)818worker2 # [ 0.274078] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)819worker2 # [ 0.276743] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)820worker2 # [ 0.277558] Performance Events: Fam17h+ core perfctr, AMD PMU driver.821worker2 # [ 0.278044] ... version: 2822worker2 # [ 0.278781] ... bit width: 48823worker2 # [ 0.279042] ... generic counters: 6824worker2 # [ 0.279799] ... generic bitmap: 000000000000003f825worker2 # [ 0.280093] ... fixed-purpose counters: 0826worker2 # [ 0.280833] ... fixed-purpose bitmap: 0000000000000000827worker2 # [ 0.281043] ... value mask: 0000ffffffffffff828worker1 # [ 0.273587] Freeing SMP alternatives memory: 44K829worker2 # [ 0.281970] ... max period: 00007fffffffffff830worker1 # [ 0.274042] pid_max: default: 32768 minimum: 301831worker2 # [ 0.282733] ... global_ctrl mask: 000000000000003f832worker2 # [ 0.283151] signal: max sigframe size: 3376833worker1 # [ 0.275150] LSM: initializing lsm=capability,landlock,yama,bpf,ima834worker1 # [ 0.276131] landlock: Up and running.835worker2 # [ 0.284010] rcu: Hierarchical SRCU implementation.836worker1 # [ 0.277044] Yama: becoming mindful.837worker2 # [ 0.284699] rcu: Max phase no-delay instances is 400.838worker1 # [ 0.278070] LSM support for eBPF active839worker1 # [ 0.278879] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)840lb # [ 0.271758] Freeing SMP alternatives memory: 44K841worker1 # [ 0.280084] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)842lb # [ 0.272335] pid_max: default: 32768 minimum: 301843lb # [ 0.273422] LSM: initializing lsm=capability,landlock,yama,bpf,ima844worker2 # [ 0.289708] smp: Bringing up secondary CPUs ...845lb # [ 0.274440] landlock: Up and running.846client # [ 0.267399] Freeing SMP alternatives memory: 44K847worker2 # [ 0.290055] smp: Brought up 1 node, 1 CPU848worker1 # [ 0.283042] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)849lb # [ 0.275035] Yama: becoming mindful.850client # [ 0.268158] pid_max: default: 32768 minimum: 301851worker2 # [ 0.290733] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)852lb # [ 0.275948] LSM support for eBPF active853client # [ 0.268717] LSM: initializing lsm=capability,landlock,yama,bpf,ima854worker1 # [ 0.283630] Performance Events: Fam17h+ core perfctr, AMD PMU driver.855worker1 # [ 0.284048] ... version: 2856client # [ 0.269722] landlock: Up and running.857lb # [ 0.276479] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)858client # [ 0.270617] Yama: becoming mindful.859worker1 # [ 0.284718] ... bit width: 48860worker2 # [ 0.291209] Memory: 1454584K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)861lb # [ 0.277374] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)862client # [ 0.271402] LSM support for eBPF active863worker1 # [ 0.285045] ... generic counters: 6864worker2 # [ 0.292301] devtmpfs: initialized865worker1 # [ 0.285733] ... generic bitmap: 000000000000003f866worker2 # [ 0.293102] x86/mm: Memory block size: 128MB867client # [ 0.272182] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)868worker1 # [ 0.286057] ... fixed-purpose counters: 0869worker1 # [ 0.286746] ... fixed-purpose bitmap: 0000000000000000870lb # [ 0.280452] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)871client # [ 0.273633] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)872worker1 # [ 0.287046] ... value mask: 0000ffffffffffff873worker2 # [ 0.294937] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)874worker1 # [ 0.287930] ... max period: 00007fffffffffff875lb # [ 0.281967] Performance Events: Fam17h+ core perfctr, AMD PMU driver.876lb # [ 0.282342] ... version: 2877worker2 # [ 0.295986] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).878worker1 # [ 0.288709] ... global_ctrl mask: 000000000000003f879client # [ 0.275863] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)880lb # [ 0.283054] ... bit width: 48881worker1 # [ 0.289174] signal: max sigframe size: 3376882worker2 # [ 0.297139] pinctrl core: initialized pinctrl subsystem883lb # [ 0.283336] ... generic counters: 6884worker1 # [ 0.289933] rcu: Hierarchical SRCU implementation.885client # [ 0.277218] Performance Events: Fam17h+ core perfctr, AMD PMU driver.886worker2 # [ 0.298234] PM: RTC time: 14:28:56, date: 2026-09-21887lb # [ 0.284067] ... generic bitmap: 000000000000003f888worker1 # [ 0.290597] rcu: Max phase no-delay instances is 400.889client # [ 0.277626] ... version: 2890lb # [ 0.284336] ... fixed-purpose counters: 0891client # [ 0.278357] ... bit width: 48892lb # [ 0.285074] ... fixed-purpose bitmap: 0000000000000000893client # [ 0.278619] ... generic counters: 6894lb # [ 0.285335] ... value mask: 0000ffffffffffff895client # [ 0.279324] ... generic bitmap: 000000000000003f896client # [ 0.279619] ... fixed-purpose counters: 0897lb # [ 0.286201] ... max period: 00007fffffffffff898worker2 # [ 0.301779] NET: Registered PF_NETLINK/PF_ROUTE protocol family899client # [ 0.280311] ... fixed-purpose bitmap: 0000000000000000900lb # [ 0.286983] ... global_ctrl mask: 000000000000003f901worker2 # [ 0.302548] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations902client # [ 0.280618] ... value mask: 0000ffffffffffff903lb # [ 0.287446] signal: max sigframe size: 3376904worker1 # [ 0.295680] smp: Bringing up secondary CPUs ...905client # [ 0.281479] ... max period: 00007fffffffffff906lb # [ 0.288263] rcu: Hierarchical SRCU implementation.907worker2 # [ 0.303064] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations908worker1 # [ 0.296060] smp: Brought up 1 node, 1 CPU909client # [ 0.282256] ... global_ctrl mask: 000000000000003f910lb # [ 0.288933] rcu: Max phase no-delay instances is 400.911worker1 # [ 0.296700] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)912client # [ 0.282715] signal: max sigframe size: 3376913worker2 # [ 0.304322] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations914client # [ 0.283509] rcu: Hierarchical SRCU implementation.915worker2 # [ 0.305052] audit: initializing netlink subsys (disabled)916client # [ 0.284238] rcu: Max phase no-delay instances is 400.917worker1 # [ 0.297267] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)918worker2 # [ 0.306043] audit: type=2000 audit(1790000937.271:1): state=initialized audit_enabled=0 res=1919worker1 # [ 0.298326] devtmpfs: initialized920worker1 # [ 0.299155] x86/mm: Memory block size: 128MB921worker2 # [ 0.307248] thermal_sys: Registered thermal governor 'fair_share'922lb # [ 0.293981] smp: Bringing up secondary CPUs ...923worker2 # [ 0.307250] thermal_sys: Registered thermal governor 'bang_bang'924lb # [ 0.294348] smp: Brought up 1 node, 1 CPU925worker2 # [ 0.308042] thermal_sys: Registered thermal governor 'step_wise'926worker1 # [ 0.300970] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)927lb # [ 0.295050] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)928worker2 # [ 0.309043] thermal_sys: Registered thermal governor 'user_space'929client # [ 0.289129] smp: Bringing up secondary CPUs ...930worker1 # [ 0.301923] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).931worker2 # [ 0.309965] thermal_sys: Registered thermal governor 'power_allocator'932client # [ 0.289632] smp: Brought up 1 node, 1 CPU933worker1 # [ 0.303116] pinctrl core: initialized pinctrl subsystem934worker2 # [ 0.310772] cpuidle: using governor menu935client # [ 0.290320] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)936lb # [ 0.295552] Memory: 1454556K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)937worker1 # [ 0.304138] PM: RTC time: 14:28:56, date: 2026-09-21938lb # [ 0.296606] devtmpfs: initialized939worker2 # [ 0.312280] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5940lb # [ 0.297406] x86/mm: Memory block size: 128MB941client # [ 0.290839] Memory: 1454592K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)942worker2 # [ 0.313290] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]943client # [ 0.291835] devtmpfs: initialized944worker2 # [ 0.314044] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry945lb # [ 0.299183] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)946client # [ 0.292705] x86/mm: Memory block size: 128MB947worker1 # [ 0.307821] NET: Registered PF_NETLINK/PF_ROUTE protocol family948worker2 # [ 0.315054] PCI: Using configuration type 1 for base access949lb # [ 0.300182] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).950worker1 # [ 0.309253] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations951worker2 # [ 0.316147] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.952lb # [ 0.301423] pinctrl core: initialized pinctrl subsystem953client # [ 0.294575] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)954worker1 # [ 0.310074] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations955lb # [ 0.302543] PM: RTC time: 14:28:56, date: 2026-09-21956client # [ 0.295491] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).957worker1 # [ 0.311333] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations958client # [ 0.296732] pinctrl core: initialized pinctrl subsystem959worker1 # [ 0.312055] audit: initializing netlink subsys (disabled)960client # [ 0.297912] PM: RTC time: 14:28:56, date: 2026-09-21961worker1 # [ 0.313194] thermal_sys: Registered thermal governor 'fair_share'962lb # [ 0.306218] NET: Registered PF_NETLINK/PF_ROUTE protocol family963worker1 # [ 0.313196] thermal_sys: Registered thermal governor 'bang_bang'964worker1 # [ 0.314047] thermal_sys: Registered thermal governor 'step_wise'965lb # [ 0.307617] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations966worker2 # [ 0.323066] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages967worker1 # [ 0.314992] thermal_sys: Registered thermal governor 'user_space'968client # [ 0.301591] NET: Registered PF_NETLINK/PF_ROUTE protocol family969lb # [ 0.308361] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations970worker2 # [ 0.324042] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page971worker1 # [ 0.315776] audit: type=2000 audit(1790000937.293:1): state=initialized audit_enabled=0 res=1972client # [ 0.302855] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations973lb # [ 0.309631] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations974worker1 # [ 0.317049] thermal_sys: Registered thermal governor 'power_allocator'975worker1 # [ 0.317061] cpuidle: using governor menu976lb # [ 0.310346] audit: initializing netlink subsys (disabled)977client # [ 0.303639] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations978lb # [ 0.311473] thermal_sys: Registered thermal governor 'fair_share'979worker1 # [ 0.319240] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5980client # [ 0.304906] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations981lb # [ 0.311475] thermal_sys: Registered thermal governor 'bang_bang'982client # [ 0.305628] audit: initializing netlink subsys (disabled)983worker2 # [ 0.329043] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages984worker1 # [ 0.320263] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]985lb # [ 0.312336] thermal_sys: Registered thermal governor 'step_wise'986client # [ 0.306721] thermal_sys: Registered thermal governor 'fair_share'987worker2 # [ 0.330042] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page988worker1 # [ 0.321048] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry989lb # [ 0.313270] thermal_sys: Registered thermal governor 'user_space'990client # [ 0.306723] thermal_sys: Registered thermal governor 'bang_bang'991worker1 # [ 0.322057] PCI: Using configuration type 1 for base access992lb # [ 0.314027] audit: type=2000 audit(1790000937.300:1): state=initialized audit_enabled=0 res=1993client # [ 0.307619] thermal_sys: Registered thermal governor 'step_wise'994worker1 # [ 0.323138] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.995lb # [ 0.315338] thermal_sys: Registered thermal governor 'power_allocator'996client # [ 0.308554] thermal_sys: Registered thermal governor 'user_space'997lb # [ 0.315361] cpuidle: using governor menu998client # [ 0.309342] audit: type=2000 audit(1790000937.303:1): state=initialized audit_enabled=0 res=1999lb # [ 0.317590] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51000client # [ 0.310622] thermal_sys: Registered thermal governor 'power_allocator'1001client # [ 0.310643] cpuidle: using governor menu1002lb # [ 0.318576] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1003client # [ 0.312862] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51004lb # [ 0.319338] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1005lb # [ 0.320347] PCI: Using configuration type 1 for base access1006client # [ 0.313848] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1007worker1 # [ 0.330338] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1008client # [ 0.314622] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1009lb # [ 0.321476] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1010worker2 # [ 0.341327] ACPI: Added _OSI(Module Device)1011worker1 # [ 0.331047] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1012client # [ 0.315630] PCI: Using configuration type 1 for base access1013worker2 # [ 0.342044] ACPI: Added _OSI(Processor Device)1014worker2 # [ 0.342816] ACPI: Added _OSI(Processor Aggregator Device)1015client # [ 0.316742] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1016lb # [ 0.326572] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1017worker1 # [ 0.336046] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1018lb # [ 0.327336] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1019worker1 # [ 0.337046] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1020client # [ 0.321733] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1021worker2 # [ 0.348302] ACPI: 1 ACPI AML tables successfully acquired and loaded1022client # [ 0.322619] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1023lb # [ 0.332336] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1024lb # [ 0.333335] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1025worker2 # [ 0.352506] ACPI: Interpreter enabled1026worker2 # [ 0.353064] ACPI: PM: (supports S0 S3 S4 S5)1027client # [ 0.327620] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1028worker2 # [ 0.353782] ACPI: Using IOAPIC for interrupt routing1029client # [ 0.328618] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1030worker1 # [ 0.345710] ACPI: Added _OSI(Module Device)1031worker1 # [ 0.346048] ACPI: Added _OSI(Processor Device)1032worker1 # [ 0.346777] ACPI: Added _OSI(Processor Aggregator Device)1033worker2 # [ 0.356112] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1034worker2 # [ 0.357042] PCI: Using E820 reservations for host bridge windows1035worker2 # [ 0.360164] ACPI: Enabled 2 GPEs in block 00 to 3F1036lb # [ 0.343734] ACPI: Added _OSI(Module Device)1037lb # [ 0.344336] ACPI: Added _OSI(Processor Device)1038worker1 # [ 0.353625] ACPI: 1 ACPI AML tables successfully acquired and loaded1039lb # [ 0.345112] ACPI: Added _OSI(Processor Aggregator Device)1040client # [ 0.338983] ACPI: Added _OSI(Module Device)1041client # [ 0.339620] ACPI: Added _OSI(Processor Device)1042client # [ 0.340405] ACPI: Added _OSI(Processor Aggregator Device)1043worker1 # [ 0.356990] ACPI: Interpreter enabled1044worker1 # [ 0.357513] ACPI: PM: (supports S0 S3 S4 S5)1045worker2 # [ 0.367655] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1046worker1 # [ 0.358047] ACPI: Using IOAPIC for interrupt routing1047worker2 # [ 0.368049] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1048worker1 # [ 0.358900] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1049worker2 # [ 0.369117] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1050worker2 # [ 0.370165] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1051client # [ 0.346545] ACPI: 1 ACPI AML tables successfully acquired and loaded1052lb # [ 0.353899] ACPI: 1 ACPI AML tables successfully acquired and loaded1053worker2 # [ 0.371403] PCI host bridge to bus 0000:001054lb # [ 0.355297] ACPI: Interpreter enabled1055worker2 # [ 0.372046] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1056worker1 # [ 0.364046] PCI: Using E820 reservations for host bridge windows1057lb # [ 0.355797] ACPI: PM: (supports S0 S3 S4 S5)1058worker2 # [ 0.373042] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1059worker1 # [ 0.365133] ACPI: Enabled 2 GPEs in block 00 to 3F1060worker2 # [ 0.374042] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1061worker2 # [ 0.375041] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1062lb # [ 0.358336] ACPI: Using IOAPIC for interrupt routing1063client # [ 0.352566] ACPI: Interpreter enabled1064client # [ 0.353100] ACPI: PM: (supports S0 S3 S4 S5)1065worker2 # [ 0.376042] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1066lb # [ 0.359230] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1067client # [ 0.353620] ACPI: Using IOAPIC for interrupt routing1068worker2 # [ 0.377042] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1069worker2 # [ 0.378042] pci_bus 0000:00: root bus resource [bus 00-ff]1070client # [ 0.354503] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1071lb # [ 0.362335] PCI: Using E820 reservations for host bridge windows1072worker2 # [ 0.379020] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1073lb # [ 0.363452] ACPI: Enabled 2 GPEs in block 00 to 3F1074worker1 # [ 0.373759] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1075worker2 # [ 0.380467] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1076worker1 # [ 0.374467] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1077client # [ 0.359619] PCI: Using E820 reservations for host bridge windows1078client # [ 0.360728] ACPI: Enabled 2 GPEs in block 00 to 3F1079worker1 # [ 0.375135] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1080worker1 # [ 0.376169] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1081worker1 # [ 0.377395] PCI host bridge to bus 0000:001082worker1 # [ 0.378050] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1083worker1 # [ 0.379046] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1084lb # [ 0.372192] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1085worker1 # [ 0.380046] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1086worker2 # [ 0.383132] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1087worker2 # [ 0.384055] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1088worker1 # [ 0.381046] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1089lb # [ 0.373948] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1090worker2 # [ 0.385011] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1091client # [ 0.368605] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1092worker1 # [ 0.382046] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1093lb # [ 0.374422] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1094worker1 # [ 0.383046] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1095client # [ 0.369367] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1096worker2 # [ 0.386031] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1097lb # [ 0.375458] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1098worker1 # [ 0.384046] pci_bus 0000:00: root bus resource [bus 00-ff]1099client # [ 0.370696] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1100lb # [ 0.376687] PCI host bridge to bus 0000:001101worker1 # [ 0.384998] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1102worker2 # [ 0.387823] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1103client # [ 0.371740] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1104lb # [ 0.377339] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1105client # [ 0.372948] PCI host bridge to bus 0000:001106lb # [ 0.378335] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1107worker1 # [ 0.386450] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1108client # [ 0.373619] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1109lb # [ 0.379335] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1110client # [ 0.374619] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1111worker2 # [ 0.391079] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1112lb # [ 0.380335] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1113client # [ 0.375620] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1114worker2 # [ 0.391912] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1115lb # [ 0.381335] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1116worker2 # [ 0.392776] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1117client # [ 0.376619] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1118lb # [ 0.382336] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1119worker2 # [ 0.393048] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1120client # [ 0.377624] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1121lb # [ 0.383335] pci_bus 0000:00: root bus resource [bus 00-ff]1122client # [ 0.378619] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1123lb # [ 0.384322] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1124worker2 # [ 0.394709] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1125client # [ 0.379643] pci_bus 0000:00: root bus resource [bus 00-ff]1126lb # [ 0.385825] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1127client # [ 0.380618] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1128worker1 # [ 0.389259] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1129worker1 # [ 0.390099] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1130client # [ 0.382056] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1131worker2 # [ 0.398148] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1132worker1 # [ 0.391057] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1133worker2 # [ 0.399049] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1134worker1 # [ 0.392371] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1135worker2 # [ 0.400058] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1136worker2 # [ 0.401581] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1137worker1 # [ 0.393785] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1138worker2 # [ 0.404076] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1139lb # [ 0.388483] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1140worker1 # [ 0.397054] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1141worker2 # [ 0.404938] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1142worker1 # [ 0.397888] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1143client # [ 0.384705] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1144lb # [ 0.389388] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1145worker1 # [ 0.398749] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1146client # [ 0.385631] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1147lb # [ 0.390357] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1148worker2 # [ 0.405786] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1149client # [ 0.386578] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1150worker1 # [ 0.399051] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1151lb # [ 0.391623] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1152worker2 # [ 0.406592] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1153client # [ 0.387838] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1154worker1 # [ 0.400594] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1155lb # [ 0.393084] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1156worker2 # [ 0.408079] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1157client # [ 0.389333] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1158worker2 # [ 0.408952] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1159worker1 # [ 0.403054] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1160worker2 # [ 0.409826] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1161worker1 # [ 0.403899] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1162lb # [ 0.396391] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1163worker2 # [ 0.410948] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1164client # [ 0.392664] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1165worker1 # [ 0.404764] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1166lb # [ 0.397266] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1167client # [ 0.393510] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1168worker2 # [ 0.412364] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1169lb # [ 0.398039] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1170worker1 # [ 0.406387] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1171client # [ 0.394350] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1172lb # [ 0.398342] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1173client # [ 0.395450] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1174worker2 # [ 0.414483] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1175worker2 # [ 0.415064] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1176lb # [ 0.399949] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1177client # [ 0.397189] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1178worker1 # [ 0.410062] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1179worker2 # [ 0.416633] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1180worker1 # [ 0.410933] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1181lb # [ 0.402383] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1182client # [ 0.398655] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1183worker1 # [ 0.411786] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1184worker2 # [ 0.418055] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1185lb # [ 0.403223] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1186client # [ 0.399480] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1187worker2 # [ 0.418985] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1188lb # [ 0.404087] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1189worker1 # [ 0.412616] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1190client # [ 0.400379] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1191worker2 # [ 0.420464] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1192client # [ 0.401197] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1193lb # [ 0.404954] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1194worker1 # [ 0.415053] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1195worker1 # [ 0.415898] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1196worker2 # [ 0.422434] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1197worker2 # [ 0.423065] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1198lb # [ 0.407390] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1199client # [ 0.403657] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1200worker1 # [ 0.416796] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1201lb # [ 0.408309] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1202client # [ 0.404509] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1203worker1 # [ 0.417053] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1204worker2 # [ 0.424574] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1205lb # [ 0.409085] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1206client # [ 0.405376] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1207worker2 # [ 0.425861] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1208worker1 # [ 0.418512] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1209lb # [ 0.409898] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1210worker2 # [ 0.426810] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1211client # [ 0.406184] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1212worker1 # [ 0.420058] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1213worker2 # [ 0.428434] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1214worker1 # [ 0.421016] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1215client # [ 0.408385] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1216client # [ 0.408626] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1217worker1 # [ 0.422443] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1218lb # [ 0.412384] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1219worker2 # [ 0.430872] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1220lb # [ 0.414073] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1221client # [ 0.409618] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1222worker2 # [ 0.431919] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1223lb # [ 0.414358] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1224client # [ 0.410652] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1225worker2 # [ 0.432777] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1226worker1 # [ 0.424058] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1227lb # [ 0.415341] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1228worker1 # [ 0.425781] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1229client # [ 0.412165] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1230worker2 # [ 0.433589] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1231lb # [ 0.416959] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1232worker1 # [ 0.426624] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1233client # [ 0.413663] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1234worker2 # [ 0.435081] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1235client # [ 0.414666] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1236lb # [ 0.418348] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1237worker2 # [ 0.435951] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1238worker1 # [ 0.428058] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1239lb # [ 0.419262] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1240worker2 # [ 0.436779] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1241worker1 # [ 0.429004] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1242client # [ 0.416224] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1243worker2 # [ 0.438070] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1244worker1 # [ 0.430433] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1245lb # [ 0.420764] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1246client # [ 0.417663] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1247worker2 # [ 0.439602] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1248client # [ 0.418671] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1249lb # [ 0.423384] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1250worker1 # [ 0.432058] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1251worker2 # [ 0.440248] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1252lb # [ 0.424341] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1253worker1 # [ 0.433775] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1254client # [ 0.420150] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1255worker2 # [ 0.441580] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1256lb # [ 0.425863] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1257worker1 # [ 0.434577] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1258worker2 # [ 0.442286] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1259client # [ 0.421660] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1260client # [ 0.422629] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1261worker2 # [ 0.443636] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1262lb # [ 0.427394] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1263worker1 # [ 0.436823] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1264lb # [ 0.428358] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1265client # [ 0.424154] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1266worker1 # [ 0.437053] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1267worker1 # [ 0.437986] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1268lb # [ 0.430422] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1269client # [ 0.425661] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1270client # [ 0.426642] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1271worker1 # [ 0.439429] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1272lb # [ 0.432381] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1273client # [ 0.428203] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1274lb # [ 0.433357] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1275worker1 # [ 0.441741] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1276worker1 # [ 0.442060] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1277lb # [ 0.434864] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1278client # [ 0.429664] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1279worker1 # [ 0.443003] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1280client # [ 0.430518] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1281client # [ 0.431423] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1282worker1 # [ 0.444435] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1283worker2 # [ 0.444262] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1284lb # [ 0.437387] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1285worker1 # [ 0.445893] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1286lb # [ 0.438343] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1287client # [ 0.433023] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1288lb # [ 0.439293] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1289worker1 # [ 0.446865] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1290worker2 # [ 0.447619] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1291client # [ 0.434656] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1292lb # [ 0.440716] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1293client # [ 0.435504] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1294worker1 # [ 0.449533] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1295worker2 # [ 0.448327] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1296client # [ 0.436355] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1297worker1 # [ 0.450249] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1298worker2 # [ 0.449362] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1299lb # [ 0.443069] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1300client # [ 0.437180] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1301lb # [ 0.443342] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1302worker1 # [ 0.451844] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1303worker2 # [ 0.450294] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1304lb # [ 0.444291] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1305client # [ 0.438399] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1306worker2 # [ 0.452090] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1307client # [ 0.438821] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1308lb # [ 0.445770] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1309worker2 # [ 0.452944] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1310lb # [ 0.447178] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1311client # [ 0.440449] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1312worker2 # [ 0.454206] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1313lb # [ 0.448199] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1314client # [ 0.441435] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1315worker2 # [ 0.455692] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1316lb # [ 0.450083] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1317client # [ 0.443301] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1318lb # [ 0.450547] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1319worker2 # [ 0.459136] ACPI: PCI: Interrupt link LNKA configured for IRQ 101320lb # [ 0.452087] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1321worker2 # [ 0.460149] ACPI: PCI: Interrupt link LNKB configured for IRQ 101322worker1 # [ 0.452877] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1323worker2 # [ 0.461142] ACPI: PCI: Interrupt link LNKC configured for IRQ 111324worker1 # [ 0.454805] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1325worker2 # [ 0.462143] ACPI: PCI: Interrupt link LNKD configured for IRQ 111326worker2 # [ 0.463144] ACPI: PCI: Interrupt link LNKE configured for IRQ 101327worker1 # [ 0.455331] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1328worker2 # [ 0.464138] ACPI: PCI: Interrupt link LNKF configured for IRQ 101329worker1 # [ 0.456513] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1330worker2 # [ 0.465143] ACPI: PCI: Interrupt link LNKG configured for IRQ 111331worker2 # [ 0.466146] ACPI: PCI: Interrupt link LNKH configured for IRQ 111332worker1 # [ 0.457322] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1333worker2 # [ 0.467081] ACPI: PCI: Interrupt link GSIA configured for IRQ 161334worker2 # [ 0.468057] ACPI: PCI: Interrupt link GSIB configured for IRQ 171335client # [ 0.443909] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1336worker1 # [ 0.459074] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1337worker2 # [ 0.469056] ACPI: PCI: Interrupt link GSIC configured for IRQ 181338worker1 # [ 0.459913] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1339client # [ 0.446272] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1340worker2 # [ 0.470056] ACPI: PCI: Interrupt link GSID configured for IRQ 191341worker1 # [ 0.461073] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1342worker2 # [ 0.471054] ACPI: PCI: Interrupt link GSIE configured for IRQ 201343client # [ 0.446958] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1344lb # [ 0.452599] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1345worker2 # [ 0.472050] ACPI: PCI: Interrupt link GSIF configured for IRQ 211346worker1 # [ 0.462939] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1347worker2 # [ 0.473054] ACPI: PCI: Interrupt link GSIG configured for IRQ 221348client # [ 0.447925] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1349worker2 # [ 0.474012] ACPI: PCI: Interrupt link GSIH configured for IRQ 231350client # [ 0.448875] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1351lb # [ 0.456038] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1352worker2 # [ 0.475869] iommu: Default domain type: Translated1353client # [ 0.450647] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1354lb # [ 0.456581] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1355worker2 # [ 0.476702] iommu: DMA domain TLB invalidation policy: lazy mode1356worker1 # [ 0.467357] ACPI: PCI: Interrupt link LNKA configured for IRQ 101357client # [ 0.451498] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1358worker2 # [ 0.477314] ACPI: bus type USB registered1359worker1 # [ 0.468152] ACPI: PCI: Interrupt link LNKB configured for IRQ 101360lb # [ 0.457684] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1361worker2 # [ 0.478067] usbcore: registered new interface driver usbfs1362client # [ 0.452706] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1363worker1 # [ 0.469145] ACPI: PCI: Interrupt link LNKC configured for IRQ 111364worker2 # [ 0.478945] usbcore: registered new interface driver hub1365lb # [ 0.458591] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1366worker2 # [ 0.479651] usbcore: registered new device driver usb1367worker1 # [ 0.470144] ACPI: PCI: Interrupt link LNKD configured for IRQ 111368client # [ 0.454290] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1369worker1 # [ 0.471140] ACPI: PCI: Interrupt link LNKE configured for IRQ 101370worker2 # [ 0.480891] NetLabel: Initializing1371worker1 # [ 0.472141] ACPI: PCI: Interrupt link LNKF configured for IRQ 101372worker2 # [ 0.481517] NetLabel: domain hash size = 1281373lb # [ 0.461410] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1374worker2 # [ 0.482042] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1375worker1 # [ 0.473143] ACPI: PCI: Interrupt link LNKG configured for IRQ 111376lb # [ 0.462262] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1377worker2 # [ 0.482989] NetLabel: unlabeled traffic allowed by default1378worker1 # [ 0.474142] ACPI: PCI: Interrupt link LNKH configured for IRQ 111379worker2 # [ 0.483701] PCI: Using ACPI for IRQ routing1380client # [ 0.458342] ACPI: PCI: Interrupt link LNKA configured for IRQ 101381worker1 # [ 0.475088] ACPI: PCI: Interrupt link GSIA configured for IRQ 161382lb # [ 0.463707] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1383client # [ 0.458724] ACPI: PCI: Interrupt link LNKB configured for IRQ 101384worker1 # [ 0.476065] ACPI: PCI: Interrupt link GSIB configured for IRQ 171385client # [ 0.459720] ACPI: PCI: Interrupt link LNKC configured for IRQ 111386lb # [ 0.465227] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1387worker1 # [ 0.477054] ACPI: PCI: Interrupt link GSIC configured for IRQ 181388client # [ 0.460719] ACPI: PCI: Interrupt link LNKD configured for IRQ 111389worker1 # [ 0.478003] ACPI: PCI: Interrupt link GSID configured for IRQ 191390client # [ 0.461722] ACPI: PCI: Interrupt link LNKE configured for IRQ 101391worker1 # [ 0.478812] ACPI: PCI: Interrupt link GSIE configured for IRQ 201392worker1 # [ 0.479057] ACPI: PCI: Interrupt link GSIF configured for IRQ 211393client # [ 0.462716] ACPI: PCI: Interrupt link LNKF configured for IRQ 101394worker1 # [ 0.480027] ACPI: PCI: Interrupt link GSIG configured for IRQ 221395client # [ 0.463719] ACPI: PCI: Interrupt link LNKG configured for IRQ 111396lb # [ 0.469068] ACPI: PCI: Interrupt link LNKA configured for IRQ 101397worker1 # [ 0.480768] ACPI: PCI: Interrupt link GSIH configured for IRQ 231398lb # [ 0.469439] ACPI: PCI: Interrupt link LNKB configured for IRQ 101399client # [ 0.464714] ACPI: PCI: Interrupt link LNKH configured for IRQ 111400lb # [ 0.470436] ACPI: PCI: Interrupt link LNKC configured for IRQ 111401client # [ 0.465658] ACPI: PCI: Interrupt link GSIA configured for IRQ 161402worker1 # [ 0.482132] iommu: Default domain type: Translated1403lb # [ 0.471435] ACPI: PCI: Interrupt link LNKD configured for IRQ 111404client # [ 0.466635] ACPI: PCI: Interrupt link GSIB configured for IRQ 171405worker1 # [ 0.482910] iommu: DMA domain TLB invalidation policy: lazy mode1406client # [ 0.467635] ACPI: PCI: Interrupt link GSIC configured for IRQ 181407lb # [ 0.472436] ACPI: PCI: Interrupt link LNKE configured for IRQ 101408worker1 # [ 0.484035] ACPI: bus type USB registered1409client # [ 0.468633] ACPI: PCI: Interrupt link GSID configured for IRQ 191410worker1 # [ 0.484571] usbcore: registered new interface driver usbfs1411lb # [ 0.473431] ACPI: PCI: Interrupt link LNKF configured for IRQ 101412worker1 # [ 0.485062] usbcore: registered new interface driver hub1413client # [ 0.469624] ACPI: PCI: Interrupt link GSIE configured for IRQ 201414lb # [ 0.474437] ACPI: PCI: Interrupt link LNKG configured for IRQ 111415worker1 # [ 0.485889] usbcore: registered new device driver usb1416client # [ 0.470577] ACPI: PCI: Interrupt link GSIF configured for IRQ 211417lb # [ 0.475431] ACPI: PCI: Interrupt link LNKH configured for IRQ 111418client # [ 0.471356] ACPI: PCI: Interrupt link GSIG configured for IRQ 221419lb # [ 0.476372] ACPI: PCI: Interrupt link GSIA configured for IRQ 161420worker1 # [ 0.487529] NetLabel: Initializing1421client # [ 0.471630] ACPI: PCI: Interrupt link GSIH configured for IRQ 231422worker1 # [ 0.488046] NetLabel: domain hash size = 1281423lb # [ 0.477327] ACPI: PCI: Interrupt link GSIB configured for IRQ 171424worker1 # [ 0.488750] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1425lb # [ 0.478071] ACPI: PCI: Interrupt link GSIC configured for IRQ 181426worker1 # [ 0.489082] NetLabel: unlabeled traffic allowed by default1427client # [ 0.473607] iommu: Default domain type: Translated1428lb # [ 0.478350] ACPI: PCI: Interrupt link GSID configured for IRQ 191429worker1 # [ 0.489957] PCI: Using ACPI for IRQ routing1430client # [ 0.474421] iommu: DMA domain TLB invalidation policy: lazy mode1431lb # [ 0.479298] ACPI: PCI: Interrupt link GSIE configured for IRQ 201432client # [ 0.475602] ACPI: bus type USB registered1433lb # [ 0.480211] ACPI: PCI: Interrupt link GSIF configured for IRQ 211434client # [ 0.476169] usbcore: registered new interface driver usbfs1435lb # [ 0.481074] ACPI: PCI: Interrupt link GSIG configured for IRQ 221436client # [ 0.476633] usbcore: registered new interface driver hub1437lb # [ 0.481347] ACPI: PCI: Interrupt link GSIH configured for IRQ 231438client # [ 0.477531] usbcore: registered new device driver usb1439lb # [ 0.483374] iommu: Default domain type: Translated1440client # [ 0.479154] NetLabel: Initializing1441client # [ 0.479620] NetLabel: domain hash size = 1281442lb # [ 0.484181] iommu: DMA domain TLB invalidation policy: lazy mode1443client # [ 0.480355] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1444lb # [ 0.485394] ACPI: bus type USB registered1445client # [ 0.480678] NetLabel: unlabeled traffic allowed by default1446lb # [ 0.486159] usbcore: registered new interface driver usbfs1447client # [ 0.481606] PCI: Using ACPI for IRQ routing1448lb # [ 0.487028] usbcore: registered new interface driver hub1449lb # [ 0.487344] usbcore: registered new device driver usb1450lb # [ 0.489095] NetLabel: Initializing1451lb # [ 0.489342] NetLabel: domain hash size = 1281452lb # [ 0.490105] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1453lb # [ 0.490398] NetLabel: unlabeled traffic allowed by default1454lb # [ 0.491303] PCI: Using ACPI for IRQ routing1455worker2 # [ 0.527879] pci 0000:00:01.0: vgaarb: setting as boot VGA device1456worker2 # [ 0.528038] pci 0000:00:01.0: vgaarb: bridge control possible1457worker2 # [ 0.528038] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1458worker2 # [ 0.528048] vgaarb: loaded1459worker2 # [ 0.528801] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01460worker2 # [ 0.529042] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461worker2 # [ 0.534139] clocksource: Switched to clocksource kvm-clock1462worker2 # [ 0.537637] VFS: Disk quotas dquot_6.6.01463worker2 # [ 0.538393] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1464worker2 # [ 0.539799] pnp: PnP ACPI init1465worker2 # [ 0.540646] ACPI: IRQ 4 override to edge(!), high(!)1466worker1 # [ 0.533343] pci 0000:00:01.0: vgaarb: setting as boot VGA device1467worker1 # [ 0.534042] pci 0000:00:01.0: vgaarb: bridge control possible1468worker2 # [ 0.541619] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1469worker1 # [ 0.534042] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1470worker2 # [ 0.543201] pnp: PnP ACPI: found 5 devices1471worker1 # [ 0.534051] vgaarb: loaded1472worker1 # [ 0.534810] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01473worker1 # [ 0.535048] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1474worker1 # [ 0.538137] clocksource: Switched to clocksource kvm-clock1475client # [ 0.525642] pci 0000:00:01.0: vgaarb: setting as boot VGA device1476client # [ 0.526615] pci 0000:00:01.0: vgaarb: bridge control possible1477worker2 # [ 0.550872] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1478client # [ 0.526615] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1479worker2 # [ 0.552349] clocksource: Switched to clocksource acpi_pm1480client # [ 0.526622] vgaarb: loaded1481worker1 # [ 0.541755] VFS: Disk quotas dquot_6.6.01482worker2 # [ 0.553334] NET: Registered PF_INET protocol family1483client # [ 0.527373] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01484worker1 # [ 0.542490] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1485client # [ 0.527620] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1486worker2 # [ 0.554510] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1487worker1 # [ 0.543841] pnp: PnP ACPI init1488worker1 # [ 0.544610] ACPI: IRQ 4 override to edge(!), high(!)1489worker1 # [ 0.545551] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1490lb # [ 0.536392] pci 0000:00:01.0: vgaarb: setting as boot VGA device1491lb # [ 0.537331] pci 0000:00:01.0: vgaarb: bridge control possible1492worker1 # [ 0.547080] pnp: PnP ACPI: found 5 devices1493client # [ 0.530700] clocksource: Switched to clocksource kvm-clock1494lb # [ 0.537331] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1495lb # [ 0.537340] vgaarb: loaded1496lb # [ 0.538104] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01497client # [ 0.534120] VFS: Disk quotas dquot_6.6.01498lb # [ 0.538338] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1499client # [ 0.534839] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1500client # [ 0.536259] pnp: PnP ACPI init1501client # [ 0.537045] ACPI: IRQ 4 override to edge(!), high(!)1502lb # [ 0.541436] clocksource: Switched to clocksource kvm-clock1503client # [ 0.538062] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1504worker1 # [ 0.554686] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1505client # [ 0.539599] pnp: PnP ACPI: found 5 devices1506worker1 # [ 0.556102] clocksource: Switched to clocksource acpi_pm1507worker1 # [ 0.557089] NET: Registered PF_INET protocol family1508worker1 # [ 0.558269] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1509lb # [ 0.545305] VFS: Disk quotas dquot_6.6.01510lb # [ 0.546034] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1511worker2 # [ 0.571072] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1512lb # [ 0.547408] pnp: PnP ACPI init1513worker2 # [ 0.572620] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1514lb # [ 0.548181] ACPI: IRQ 4 override to edge(!), high(!)1515worker2 # [ 0.573951] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1516lb # [ 0.549196] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1517lb # [ 0.550820] pnp: PnP ACPI: found 5 devices1518worker2 # [ 0.575336] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1519client # [ 0.547414] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1520worker2 # [ 0.576585] TCP: Hash tables configured (established 16384 bind 16384)1521client # [ 0.548842] clocksource: Switched to clocksource acpi_pm1522client # [ 0.549825] NET: Registered PF_INET protocol family1523worker2 # [ 0.577718] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1524worker2 # [ 0.579024] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1525client # [ 0.551156] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1526worker2 # [ 0.580107] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1527worker2 # [ 0.581397] NET: Registered PF_UNIX/PF_LOCAL protocol family1528worker2 # [ 0.582400] NET: Registered PF_XDP protocol family1529worker2 # [ 0.583290] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1530lb # [ 0.558458] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1531worker2 # [ 0.584315] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1532lb # [ 0.559928] clocksource: Switched to clocksource acpi_pm1533worker2 # [ 0.585328] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1534lb # [ 0.560927] NET: Registered PF_INET protocol family1535worker2 # [ 0.586440] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1536worker1 # [ 0.575138] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1537lb # [ 0.562151] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1538worker2 # [ 0.587570] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1539worker1 # [ 0.576645] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1540worker2 # [ 0.588657] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1541worker1 # [ 0.577960] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1542worker2 # [ 0.590411] ACPI: \_SB_.GSIA: Enabled at IRQ 161543worker1 # [ 0.579294] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1544worker1 # [ 0.580520] TCP: Hash tables configured (established 16384 bind 16384)1545worker2 # [ 0.592765] ACPI: \_SB_.GSIB: Enabled at IRQ 171546worker1 # [ 0.581628] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1547worker1 # [ 0.582906] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1548worker2 # [ 0.594768] ACPI: \_SB_.GSIC: Enabled at IRQ 181549worker1 # [ 0.583975] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1550client # [ 0.567587] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1551worker1 # [ 0.585150] NET: Registered PF_UNIX/PF_LOCAL protocol family1552worker2 # [ 0.596793] ACPI: \_SB_.GSID: Enabled at IRQ 191553worker1 # [ 0.586098] NET: Registered PF_XDP protocol family1554client # [ 0.569125] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1555worker2 # [ 0.598487] PCI: CLS 0 bytes, default 641556worker1 # [ 0.586926] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1557client # [ 0.570446] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1558worker1 # [ 0.587941] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1559client # [ 0.571767] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1560worker2 # [ 0.599407] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1561worker1 # [ 0.588924] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1562worker2 # [ 0.601127] Trying to unpack rootfs image as initramfs...1563client # [ 0.572975] TCP: Hash tables configured (established 16384 bind 16384)1564worker1 # [ 0.589991] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1565client # [ 0.574116] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1566worker1 # [ 0.591064] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1567client # [ 0.575451] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1568worker1 # [ 0.592142] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1569lb # [ 0.578695] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1570client # [ 0.576521] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1571worker1 # [ 0.593822] ACPI: \_SB_.GSIA: Enabled at IRQ 161572lb # [ 0.580247] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1573client # [ 0.577732] NET: Registered PF_UNIX/PF_LOCAL protocol family1574client # [ 0.578691] NET: Registered PF_XDP protocol family1575lb # [ 0.581538] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1576client # [ 0.579565] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1577worker1 # [ 0.596443] ACPI: \_SB_.GSIB: Enabled at IRQ 171578lb # [ 0.582900] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1579client # [ 0.580588] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1580lb # [ 0.584164] TCP: Hash tables configured (established 16384 bind 16384)1581worker1 # [ 0.598451] ACPI: \_SB_.GSIC: Enabled at IRQ 181582client # [ 0.581745] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1583lb # [ 0.585393] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1584client # [ 0.582964] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1585worker1 # [ 0.600401] ACPI: \_SB_.GSID: Enabled at IRQ 191586lb # [ 0.586732] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1587client # [ 0.584097] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1588lb # [ 0.587895] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1589worker1 # [ 0.602066] PCI: CLS 0 bytes, default 641590client # [ 0.585392] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1591lb # [ 0.589147] NET: Registered PF_UNIX/PF_LOCAL protocol family1592worker1 # [ 0.602922] Trying to unpack rootfs image as initramfs...1593lb # [ 0.590134] NET: Registered PF_XDP protocol family1594client # [ 0.587481] ACPI: \_SB_.GSIA: Enabled at IRQ 161595lb # [ 0.590988] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1596lb # [ 0.592046] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1597client # [ 0.589745] ACPI: \_SB_.GSIB: Enabled at IRQ 171598lb # [ 0.593111] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1599lb # [ 0.594273] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1600worker1 # [ 0.607449] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1601client # [ 0.592067] ACPI: \_SB_.GSIC: Enabled at IRQ 181602lb # [ 0.595351] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1603lb # [ 0.596501] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1604client # [ 0.594416] ACPI: \_SB_.GSID: Enabled at IRQ 191605lb # [ 0.598366] ACPI: \_SB_.GSIA: Enabled at IRQ 161606client # [ 0.596127] PCI: CLS 0 bytes, default 641607client # [ 0.597098] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1608lb # [ 0.601352] ACPI: \_SB_.GSIB: Enabled at IRQ 171609client # [ 0.598881] Trying to unpack rootfs image as initramfs...1610lb # [ 0.603288] ACPI: \_SB_.GSIC: Enabled at IRQ 181611lb # [ 0.605266] ACPI: \_SB_.GSID: Enabled at IRQ 191612lb # [ 0.606978] PCI: CLS 0 bytes, default 641613lb # [ 0.607920] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614lb # [ 0.609750] Trying to unpack rootfs image as initramfs...1615worker2 # [ 0.646252] Initialise system trusted keyrings1616worker2 # [ 0.649896] workingset: timestamp_bits=40 max_order=19 bucket_order=01617worker1 # [ 0.648249] Initialise system trusted keyrings1618worker1 # [ 0.649223] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker2 # [ 0.669041] Key type asymmetric registered1620worker2 # [ 0.669761] Asymmetric key parser 'x509' registered1621client # [ 0.643001] Initialise system trusted keyrings1622worker2 # [ 0.672878] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1623client # [ 0.646254] workingset: timestamp_bits=40 max_order=19 bucket_order=01624worker2 # [ 0.675893] io scheduler mq-deadline registered1625worker2 # [ 0.676698] io scheduler kyber registered1626lb # [ 0.653254] Initialise system trusted keyrings1627worker2 # [ 0.679439] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1628worker2 # [ 0.680769] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1629lb # [ 0.657296] workingset: timestamp_bits=40 max_order=19 bucket_order=01630worker1 # [ 0.672908] Key type asymmetric registered1631worker1 # [ 0.673623] Asymmetric key parser 'x509' registered1632worker1 # [ 0.674481] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1633worker2 # [ 0.686665] Linux agpgart interface v0.1031634worker2 # [ 0.687484] ACPI: bus type drm_connector registered1635worker1 # [ 0.679930] io scheduler mq-deadline registered1636worker1 # [ 0.680682] io scheduler kyber registered1637worker2 # [ 0.691923] usbcore: registered new interface driver usbserial_generic1638worker2 # [ 0.693039] usbserial: USB Serial support registered for generic1639worker1 # [ 0.683407] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1640client # [ 0.667461] Key type asymmetric registered1641worker2 # [ 0.695844] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1642worker2 # [ 0.695844] Please enable it if your BIOS has the CPPC option.1643client # [ 0.670209] Asymmetric key parser 'x509' registered1644worker2 # [ 0.698217] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1645worker1 # [ 0.684774] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1646client # [ 0.671060] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1647worker1 # [ 0.690485] Linux agpgart interface v0.1031648client # [ 0.674381] io scheduler mq-deadline registered1649worker1 # [ 0.691255] ACPI: bus type drm_connector registered1650client # [ 0.675134] io scheduler kyber registered1651worker2 # [ 0.702992] drop_monitor: Initializing network drop monitor service1652lb # [ 0.678579] Key type asymmetric registered1653worker2 # [ 0.704129] NET: Registered PF_INET6 protocol family1654client # [ 0.677794] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1655worker1 # [ 0.694266] usbcore: registered new interface driver usbserial_generic1656lb # [ 0.681256] Asymmetric key parser 'x509' registered1657worker1 # [ 0.695334] usbserial: USB Serial support registered for generic1658worker2 # [ 0.707285] Segment Routing with IPv61659lb # [ 0.682075] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1660client # [ 0.679022] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1661worker2 # [ 0.707969] In-situ OAM (IOAM) with IPv61662lb # [ 0.685368] io scheduler mq-deadline registered1663worker1 # [ 0.698825] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1664lb # [ 0.686117] io scheduler kyber registered1665worker2 # [ 0.711108] IPI shorthand broadcast: enabled1666worker1 # [ 0.698825] Please enable it if your BIOS has the CPPC option.1667client # [ 0.684616] Linux agpgart interface v0.1031668worker1 # [ 0.700991] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1669client # [ 0.685381] ACPI: bus type drm_connector registered1670lb # [ 0.688845] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1671worker1 # [ 0.703965] drop_monitor: Initializing network drop monitor service1672lb # [ 0.690171] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1673worker1 # [ 0.705108] NET: Registered PF_INET6 protocol family1674client # [ 0.688666] usbcore: registered new interface driver usbserial_generic1675client # [ 0.689744] usbserial: USB Serial support registered for generic1676worker1 # [ 0.708295] Segment Routing with IPv61677worker2 # [ 0.719467] sched_clock: Marking stable (584015009, 134815046)->(792632747, -73802692)1678client # [ 0.692218] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1679lb # [ 0.696043] Linux agpgart interface v0.1031680client # [ 0.692218] Please enable it if your BIOS has the CPPC option.1681lb # [ 0.696848] ACPI: bus type drm_connector registered1682worker1 # [ 0.710834] In-situ OAM (IOAM) with IPv61683client # [ 0.694477] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1684worker2 # [ 0.723024] registered taskstats version 11685worker1 # [ 0.711890] IPI shorthand broadcast: enabled1686worker2 # [ 0.724035] Loading compiled-in X.509 certificates1687lb # [ 0.699730] usbcore: registered new interface driver usbserial_generic1688lb # [ 0.700878] usbserial: USB Serial support registered for generic1689client # [ 0.698339] drop_monitor: Initializing network drop monitor service1690client # [ 0.699536] NET: Registered PF_INET6 protocol family1691client # [ 0.702671] Segment Routing with IPv61692lb # [ 0.705253] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1693lb # [ 0.705253] Please enable it if your BIOS has the CPPC option.1694worker1 # [ 0.720224] sched_clock: Marking stable (586020737, 133784756)->(791698122, -71892629)1695lb # [ 0.707554] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1696client # [ 0.705236] In-situ OAM (IOAM) with IPv61697client # [ 0.706343] IPI shorthand broadcast: enabled1698lb # [ 0.710411] drop_monitor: Initializing network drop monitor service1699worker1 # [ 0.724908] registered taskstats version 11700lb # [ 0.711603] NET: Registered PF_INET6 protocol family1701worker1 # [ 0.725857] Loading compiled-in X.509 certificates1702lb # [ 0.714758] Segment Routing with IPv61703lb # [ 0.717280] In-situ OAM (IOAM) with IPv61704worker2 # [ 0.742842] Demotion targets for Node 0: null1705lb # [ 0.718388] IPI shorthand broadcast: enabled1706worker2 # [ 0.743719] Key type .fscrypt registered1707client # [ 0.714693] sched_clock: Marking stable (581013952, 133176714)->(786806926, -72616260)1708worker2 # [ 0.746834] Key type fscrypt-provisioning registered1709worker2 # [ 0.747777] ima: No TPM chip found, activating TPM-bypass!1710worker2 # [ 0.748736] ima: Allocated hash algorithm: sha11711worker2 # [ 0.749582] ima: No architecture policies found1712client # [ 0.722413] registered taskstats version 11713client # [ 0.723453] Loading compiled-in X.509 certificates1714worker2 # [ 0.751841] PM: Magic number: 10:9:4861715lb # [ 0.727042] sched_clock: Marking stable (593014380, 133223854)->(803073384, -76835150)1716worker2 # [ 0.753448] RAS: Correctable Errors collector initialized.1717worker1 # [ 0.743818] Demotion targets for Node 0: null1718worker1 # [ 0.744670] Key type .fscrypt registered1719lb # [ 0.732443] registered taskstats version 11720lb # [ 0.733420] Loading compiled-in X.509 certificates1721worker1 # [ 0.747811] Key type fscrypt-provisioning registered1722worker1 # [ 0.748732] ima: No TPM chip found, activating TPM-bypass!1723worker1 # [ 0.749647] ima: Allocated hash algorithm: sha11724worker1 # [ 0.750463] ima: No architecture policies found1725worker2 # [ 0.762522] clk: Disabling unused clocks1726worker1 # [ 0.752817] PM: Magic number: 10:9:4861727worker1 # [ 0.754429] RAS: Correctable Errors collector initialized.1728worker2 # [ 0.766866] PM: genpd: Disabling unused power domains1729client # [ 0.742209] Demotion targets for Node 0: null1730client # [ 0.743137] Key type .fscrypt registered1731client # [ 0.746196] Key type fscrypt-provisioning registered1732worker1 # [ 0.763447] clk: Disabling unused clocks1733client # [ 0.747172] ima: No TPM chip found, activating TPM-bypass!1734client # [ 0.748081] ima: Allocated hash algorithm: sha11735lb # [ 0.752245] Demotion targets for Node 0: null1736lb # [ 0.753153] Key type .fscrypt registered1737client # [ 0.750225] ima: No architecture policies found1738lb # [ 0.753868] Key type fscrypt-provisioning registered1739client # [ 0.751160] PM: Magic number: 10:9:4861740worker1 # [ 0.767827] PM: genpd: Disabling unused power domains1741lb # [ 0.754826] ima: No TPM chip found, activating TPM-bypass!1742client # [ 0.754734] RAS: Correctable Errors collector initialized.1743lb # [ 0.759249] ima: Allocated hash algorithm: sha11744lb # [ 0.760059] ima: No architecture policies found1745lb # [ 0.762258] PM: Magic number: 10:9:4861746lb # [ 0.763934] RAS: Correctable Errors collector initialized.1747client # [ 0.763355] clk: Disabling unused clocks1748client # [ 0.764081] PM: genpd: Disabling unused power domains1749lb # [ 0.772944] clk: Disabling unused clocks1750lb # [ 0.777272] PM: genpd: Disabling unused power domains1751worker2 # [ 0.932063] Freeing initrd memory: 29060K1752worker2 # [ 0.935243] Freeing unused decrypted memory: 2028K1753worker2 # [ 0.937896] Freeing unused kernel image (initmem) memory: 3652K1754worker2 # [ 0.939007] Write protecting the kernel read-only data: 32768k1755worker2 # [ 0.940879] Freeing unused kernel image (text/rodata gap) memory: 1184K1756worker2 # [ 0.942409] Freeing unused kernel image (rodata/data gap) memory: 720K1757worker1 # [ 0.938876] Freeing initrd memory: 29056K1758worker1 # [ 0.942134] Freeing unused decrypted memory: 2028K1759worker1 # [ 0.944855] Freeing unused kernel image (initmem) memory: 3652K1760worker1 # [ 0.945934] Write protecting the kernel read-only data: 32768k1761worker1 # [ 0.947816] Freeing unused kernel image (text/rodata gap) memory: 1184K1762worker1 # [ 0.949322] Freeing unused kernel image (rodata/data gap) memory: 720K1763lb # [ 0.941154] Freeing initrd memory: 29064K1764client # [ 0.938231] Freeing initrd memory: 29056K1765lb # [ 0.944432] Freeing unused decrypted memory: 2028K1766client # [ 0.941415] Freeing unused decrypted memory: 2028K1767client # [ 0.944040] Freeing unused kernel image (initmem) memory: 3652K1768lb # [ 0.947118] Freeing unused kernel image (initmem) memory: 3652K1769client # [ 0.945239] Write protecting the kernel read-only data: 32768k1770lb # [ 0.948711] Write protecting the kernel read-only data: 32768k1771client # [ 0.947070] Freeing unused kernel image (text/rodata gap) memory: 1184K1772lb # [ 0.950637] Freeing unused kernel image (text/rodata gap) memory: 1184K1773client # [ 0.948632] Freeing unused kernel image (rodata/data gap) memory: 720K1774lb # [ 0.952151] Freeing unused kernel image (rodata/data gap) memory: 720K1775worker2 # [ 0.993204] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776worker2 # [ 0.994319] Run /init as init process1777worker2 # [ 1.005280] systemd[1]: Inserted module 'autofs4'1778worker1 # [ 1.000229] x86/mm: Checked W+X mappings: passed, no W+X pages found.1779worker1 # [ 1.001302] Run /init as init process1780worker1 # [ 1.012856] systemd[1]: Inserted module 'autofs4'1781worker2 # [ 1.025169] fuse: init (API version 7.45)1782client # [ 0.999508] x86/mm: Checked W+X mappings: passed, no W+X pages found.1783client # [ 1.000605] Run /init as init process1784lb # [ 1.003182] x86/mm: Checked W+X mappings: passed, no W+X pages found.1785lb # [ 1.004266] Run /init as init process1786worker2 # [ 1.032531] ACPI: \_SB_.GSIG: Enabled at IRQ 221787worker2 # [ 1.034977] ACPI: \_SB_.GSIH: Enabled at IRQ 231788worker2 # [ 1.038052] ACPI: \_SB_.GSIE: Enabled at IRQ 201789client # [ 1.011666] systemd[1]: Inserted module 'autofs4'1790worker2 # [ 1.040022] ACPI: \_SB_.GSIF: Enabled at IRQ 211791lb # [ 1.015357] systemd[1]: Inserted module 'autofs4'1792worker1 # [ 1.032713] fuse: init (API version 7.45)1793worker2 # [ 1.045157] virtiofs virtio5: discovered new tag: nix-store1794worker2 # [ 1.046749] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1795worker1 # [ 1.040105] ACPI: \_SB_.GSIG: Enabled at IRQ 221796worker2 # [ 1.054179] virtiofs virtio6: discovered new tag: shared1797worker1 # [ 1.042945] ACPI: \_SB_.GSIH: Enabled at IRQ 231798worker2 # [ 1.055781] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1799worker1 # [ 1.046265] ACPI: \_SB_.GSIE: Enabled at IRQ 201800worker2 # [ 1.059031] virtiofs virtio7: discovered new tag: xchg1801client # [ 1.031391] fuse: init (API version 7.45)1802lb # [ 1.035173] fuse: init (API version 7.45)1803worker1 # [ 1.048723] ACPI: \_SB_.GSIF: Enabled at IRQ 211804worker2 # [ 1.060505] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1805worker1 # [ 1.054277] virtiofs virtio5: discovered new tag: nix-store1806client # [ 1.038796] ACPI: \_SB_.GSIG: Enabled at IRQ 221807lb # [ 1.042687] ACPI: \_SB_.GSIG: Enabled at IRQ 221808worker1 # [ 1.056025] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1809client # [ 1.041274] ACPI: \_SB_.GSIH: Enabled at IRQ 231810lb # [ 1.045840] ACPI: \_SB_.GSIH: Enabled at IRQ 231811client # [ 1.044062] ACPI: \_SB_.GSIE: Enabled at IRQ 201812client # [ 1.045989] ACPI: \_SB_.GSIF: Enabled at IRQ 211813worker1 # [ 1.063405] virtiofs virtio6: discovered new tag: shared1814lb # [ 1.049983] ACPI: \_SB_.GSIE: Enabled at IRQ 201815worker1 # [ 1.064946] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1816lb # [ 1.052183] ACPI: \_SB_.GSIF: Enabled at IRQ 211817client # [ 1.051277] virtiofs virtio5: discovered new tag: nix-store1818worker1 # [ 1.068515] virtiofs virtio7: discovered new tag: xchg1819client # [ 1.052835] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1820worker1 # [ 1.069991] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1821lb # [ 1.057535] virtiofs virtio5: discovered new tag: nix-store1822worker2 # [ 1.083969] systemd[1]: Successfully made /usr/ read-only.1823lb # [ 1.059270] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1824client # [ 1.059473] virtiofs virtio6: discovered new tag: shared1825client # [ 1.060894] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1826lb # [ 1.066816] virtiofs virtio6: discovered new tag: shared1827lb # [ 1.068827] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1828client # [ 1.066118] virtiofs virtio7: discovered new tag: xchg1829client # [ 1.067737] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1830lb # [ 1.072260] virtiofs virtio7: discovered new tag: xchg1831lb # [ 1.073735] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1832worker1 # [ 1.089930] systemd[1]: Successfully made /usr/ read-only.1833client # [ 1.090384] systemd[1]: Successfully made /usr/ read-only.1834lb # [ 1.094348] systemd[1]: Successfully made /usr/ read-only.1835worker2 # [ 1.420692] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836worker2 # [ 1.432680] systemd[1]: Detected virtualization kvm.1837worker2 # [ 1.434917] systemd[1]: Detected architecture x86-64.1838worker2 # [ 1.437024] systemd[1]: Running in initrd.1839worker1 # [ 1.425475] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1840worker2 # [ 1.439545] systemd[1]: Initializing machine ID from random generator.1841worker1 # [ 1.430522] systemd[1]: Detected virtualization kvm.1842worker1 # [ 1.431414] systemd[1]: Detected architecture x86-64.1843worker1 # [ 1.432278] systemd[1]: Running in initrd.1844worker2 # [ 1.442475] systemd[1]: Hostname set to <worker2>.1845worker1 # [ 1.433400] systemd[1]: Initializing machine ID from random generator.1846worker1 # [ 1.434662] systemd[1]: Hostname set to <worker1>.1847lb # [ 1.429984] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1848lb # [ 1.438123] systemd[1]: Detected virtualization kvm.1849client # [ 1.426994] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1850lb # [ 1.440368] systemd[1]: Detected architecture x86-64.1851client # [ 1.438937] systemd[1]: Detected virtualization kvm.1852lb # [ 1.442459] systemd[1]: Running in initrd.1853client # [ 1.441095] systemd[1]: Detected architecture x86-64.1854lb # [ 1.444919] systemd[1]: Initializing machine ID from random generator.1855client # [ 1.443265] systemd[1]: Running in initrd.1856lb # [ 1.447778] systemd[1]: Hostname set to <lb>.1857client # [ 1.445791] systemd[1]: Initializing machine ID from random generator.1858client # [ 1.448688] systemd[1]: Hostname set to <client>.1859lb # [ 1.553196] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860lb # [ 1.592353] systemd[1]: Queued start job for default target Initrd Default Target.1861lb # [ 1.595607] systemd[1]: Created slice Slice /system/modprobe.1862lb # [ 1.596810] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1863lb # [ 1.598287] systemd[1]: Expecting device /dev/disk/by-label/nixos...1864lb # [ 1.599486] systemd[1]: Reached target Path Units.1865lb # [ 1.600387] systemd[1]: Reached target Slice Units.1866lb # [ 1.601266] systemd[1]: Reached target Swaps.1867lb # [ 1.602066] systemd[1]: Reached target Timer Units.1868lb # [ 1.603123] systemd[1]: Listening on D-Bus System Message Bus Socket.1869lb # [ 1.604386] systemd[1]: Listening on Journal Socket (/dev/log).1870lb # [ 1.605592] systemd[1]: Listening on Journal Sockets.1871lb # [ 1.606639] systemd[1]: Listening on udev Control Socket.1872lb # [ 1.607708] systemd[1]: Listening on udev Kernel Socket.1873lb # [ 1.608719] systemd[1]: Reached target Socket Units.1874lb # [ 1.610658] systemd[1]: Starting Create List of Static Device Nodes...1875lb # [ 1.615309] systemd[1]: Starting Load Kernel Module configfs...1876worker1 # [ 1.638934] systemd[1]: bpf-restrict-fs: LSM BPF program attached1877lb # [ 1.625334] systemd[1]: Starting Journal Service...1878worker2 # [ 1.668531] systemd[1]: bpf-restrict-fs: LSM BPF program attached1879lb # [ 1.652453] systemd[1]: Starting Load Kernel Modules...1880lb # [ 1.658383] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1881lb # [ 1.665363] systemd[1]: Starting Coldplug All udev Devices...1882worker1 # [ 1.686166] systemd[1]: Queued start job for default target Initrd Default Target.1883client # [ 1.670317] systemd[1]: bpf-restrict-fs: LSM BPF program attached1884lb # [ 1.676355] systemd-journald[66]: Collecting audit messages is disabled.1885worker1 # [ 1.690217] systemd[1]: Created slice Slice /system/modprobe.1886worker1 # [ 1.691401] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1887worker1 # [ 1.692854] systemd[1]: Expecting device /dev/disk/by-label/nixos...1888worker1 # [ 1.693969] systemd[1]: Reached target Path Units.1889lb # [ 1.680302] systemd[1]: Finished Create List of Static Device Nodes.1890worker1 # [ 1.694895] systemd[1]: Reached target Slice Units.1891worker1 # [ 1.695759] systemd[1]: Reached target Swaps.1892worker1 # [ 1.696575] systemd[1]: Reached target Timer Units.1893worker1 # [ 1.697563] systemd[1]: Listening on D-Bus System Message Bus Socket.1894worker1 # [ 1.698889] systemd[1]: Listening on Journal Socket (/dev/log).1895lb # [ 1.685895] systemd[1]: modprobe@configfs.service: Deactivated successfully.1896worker1 # [ 1.700062] systemd[1]: Listening on Journal Sockets.1897worker1 # [ 1.701076] systemd[1]: Listening on udev Control Socket.1898worker1 # [ 1.702116] systemd[1]: Listening on udev Kernel Socket.1899worker1 # [ 1.703079] systemd[1]: Reached target Socket Units.1900worker1 # [ 1.705010] systemd[1]: Starting Create List of Static Device Nodes...1901lb # [ 1.692640] systemd[1]: Finished Load Kernel Module configfs.1902worker1 # [ 1.708897] systemd[1]: Starting Load Kernel Module configfs...1903worker2 # [ 1.723023] systemd[1]: Queued start job for default target Initrd Default Target.1904lb # [ 1.697676] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1905worker2 # [ 1.727257] systemd[1]: Created slice Slice /system/modprobe.1906worker2 # [ 1.728697] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1907worker2 # [ 1.730162] systemd[1]: Expecting device /dev/disk/by-label/nixos...1908worker2 # [ 1.731292] systemd[1]: Reached target Path Units.1909worker2 # [ 1.732230] systemd[1]: Reached target Slice Units.1910worker2 # [ 1.733117] systemd[1]: Reached target Swaps.1911worker2 # [ 1.733944] systemd[1]: Reached target Timer Units.1912worker2 # [ 1.734957] systemd[1]: Listening on D-Bus System Message Bus Socket.1913lb # [ 1.710363] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1914worker2 # [ 1.736214] systemd[1]: Listening on Journal Socket (/dev/log).1915client # [ 1.708453] systemd[1]: Queued start job for default target Initrd Default Target.1916worker2 # [ 1.737385] systemd[1]: Listening on Journal Sockets.1917worker2 # [ 1.738411] systemd[1]: Listening on udev Control Socket.1918worker2 # [ 1.739480] systemd[1]: Listening on udev Kernel Socket.1919worker2 # [ 1.740478] systemd[1]: Reached target Socket Units.1920client # [ 1.712465] systemd[1]: Created slice Slice /system/modprobe.1921worker1 # [ 1.717892] systemd[1]: Starting Journal Service...1922client # [ 1.713610] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1923worker2 # [ 1.742391] systemd[1]: Starting Create List of Static Device Nodes...1924client # [ 1.715023] systemd[1]: Expecting device /dev/disk/by-label/nixos...1925client # [ 1.716134] systemd[1]: Reached target Path Units.1926client # [ 1.717034] systemd[1]: Reached target Slice Units.1927client # [ 1.717943] systemd[1]: Reached target Swaps.1928client # [ 1.718731] systemd[1]: Reached target Timer Units.1929lb # [ 1.720603] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1930worker2 # [ 1.746902] systemd[1]: Starting Load Kernel Module configfs...1931client # [ 1.719667] systemd[1]: Listening on D-Bus System Message Bus Socket.1932client # [ 1.720868] systemd[1]: Listening on Journal Socket (/dev/log).1933client # [ 1.722085] systemd[1]: Listening on Journal Sockets.1934client # [ 1.723117] systemd[1]: Listening on udev Control Socket.1935client # [ 1.724161] systemd[1]: Listening on udev Kernel Socket.1936client # [ 1.725148] systemd[1]: Reached target Socket Units.1937client # [ 1.726904] systemd[1]: Starting Create List of Static Device Nodes...1938lb # [ 1.730278] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1939worker1 # [ 1.745026] systemd[1]: Starting Load Kernel Modules...1940client # [ 1.731262] systemd[1]: Starting Load Kernel Module configfs...1941worker1 # [ 1.748929] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1942worker2 # [ 1.756538] systemd[1]: Starting Journal Service...1943worker1 # [ 1.759052] systemd[1]: Starting Coldplug All udev Devices...1944client # [ 1.740793] systemd[1]: Starting Journal Service...1945worker1 # [ 1.768613] systemd-journald[66]: Collecting audit messages is disabled.1946worker2 # [ 1.781917] systemd[1]: Starting Load Kernel Modules...1947lb # [ 1.756879] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1948worker1 # [ 1.771346] systemd[1]: Finished Create List of Static Device Nodes.1949worker2 # [ 1.785927] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1950worker1 # [ 1.777612] systemd[1]: modprobe@configfs.service: Deactivated successfully.1951lb # [ 1.765463] systemd[1]: Starting Create Static Device Nodes in /dev...1952client # [ 1.765402] systemd[1]: Starting Load Kernel Modules...1953worker2 # [ 1.796129] systemd[1]: Starting Coldplug All udev Devices...1954worker1 # [ 1.785694] systemd[1]: Finished Load Kernel Module configfs.1955client # [ 1.769291] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1956lb # [ 1.776831] systemd[1]: Finished Load Kernel Modules.1957worker2 # [ 1.803428] systemd-journald[66]: Collecting audit messages is disabled.1958client # [ 1.776259] systemd[1]: Starting Coldplug All udev Devices...1959worker1 # [ 1.792277] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1960worker2 # [ 1.809091] systemd[1]: Finished Create List of Static Device Nodes.1961lb # [ 1.786333] systemd[1]: Starting Apply Kernel Variables...1962worker1 # [ 1.802883] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1963worker2 # [ 1.814974] systemd[1]: modprobe@configfs.service: Deactivated successfully.1964client # [ 1.788828] systemd-journald[66]: Collecting audit messages is disabled.1965client # [ 1.792551] systemd[1]: Finished Create List of Static Device Nodes.1966worker2 # [ 1.821880] systemd[1]: Finished Load Kernel Module configfs.1967client # [ 1.794865] systemd[1]: modprobe@configfs.service: Deactivated successfully.1968lb # [ 1.665216] systemd-modules-load[68]: Inserted module 'dm_mod'[ 1.799556] systemd[1]: Started Journal Service.1969lb # 1970lb # [ 1.668325] systemd-modules-load[68]: Inserted module 'virtio_balloon'1971worker1 # [ 1.815214] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1972lb # [ 1.670293] systemd-modules-load[68]: Inserted module 'virtio_gpu'1973worker2 # [ 1.828282] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1974client # [ 1.803922] systemd[1]: Finished Load Kernel Module configfs.1975worker1 # [ 1.824874] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1976client # [ 1.808635] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1977lb # [ 1.680082] systemd[1]: Finished Create Static Device Nodes in /dev.1978worker2 # [ 1.841904] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1979lb # [ 1.688090] systemd[1]: Reached target Preparation for Local File Systems.1980lb # [ 1.689140] systemd[1]: Reached target Local File Systems.1981client # [ 1.820273] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1982worker2 # [ 1.847705] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1983lb # [ 1.692831] systemd[1]: Starting Create System Files and Directories...1984client # [ 1.826099] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1985lb # [ 1.698101] systemd[1]: Starting Rule-based Manager for Device Events and Files...1986worker2 # [ 1.858100] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1987worker1 # [ 1.846905] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1988lb # [ 1.704654] systemd[1]: Finished Apply Kernel Variables.1989client # [ 1.838339] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1990worker1 # [ 1.856180] systemd[1]: Starting Create Static Device Nodes in /dev...1991worker2 # [ 1.880456] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1992worker1 # [ 1.870886] systemd[1]: Finished Load Kernel Modules.1993lb # [ 1.726293] systemd[1]: Finished Create System Files and Directories.1994client # [ 1.859744] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1995worker2 # [ 1.891061] systemd[1]: Starting Create Static Device Nodes in /dev...1996worker1 # [ 1.884896] systemd[1]: Starting Apply Kernel Variables...1997client # [ 1.869394] systemd[1]: Starting Create Static Device Nodes in /dev...1998worker2 # [ 1.901912] systemd[1]: Finished Load Kernel Modules.1999client # [ 1.880716] systemd[1]: Finished Load Kernel Modules.2000lb # [ 1.752368] systemd-udevd[80]: Using default interface naming scheme 'v261'.
loading 2,377 hidden lines…
4378worker2 # [ 24.109769] nix-grpc-daemon[782]: ts=2026-09-21T14:29:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4379worker2 # [ 24.307607] nix-grpc-daemon[782]: ts=2026-09-21T14:29:21Z level=info event=scheduler_connected addr=lb:500514380worker1 # [ 24.299294] nix-grpc-daemon[781]: ts=2026-09-21T14:29:21Z level=info event=scheduler_connected addr=lb:500514381worker2 # [ 24.311591] nix-grpc-daemon[782]: ts=2026-09-21T14:29:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4382worker1 # [ 24.303171] nix-grpc-daemon[781]: ts=2026-09-21T14:29:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4383worker2 # [ 24.512719] nix-grpc-daemon[782]: ts=2026-09-21T14:29:21Z level=info event=scheduler_connected addr=lb:500514384worker1 # [ 24.504085] nix-grpc-daemon[781]: ts=2026-09-21T14:29:21Z level=info event=scheduler_connected addr=lb:500514385worker2 # [ 24.516498] nix-grpc-daemon[782]: ts=2026-09-21T14:29:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4386worker1 # [ 24.507811] nix-grpc-daemon[781]: ts=2026-09-21T14:29:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4387worker1 # [ 24.706122] nix-grpc-daemon[781]: ts=2026-09-21T14:29:21Z level=info event=scheduler_connected addr=lb:500514388worker2 # [ 24.718259] nix-grpc-daemon[782]: ts=2026-09-21T14:29:21Z level=info event=scheduler_connected addr=lb:500514389worker1 # [ 24.710175] nix-grpc-daemon[781]: ts=2026-09-21T14:29:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4390worker2 # [ 24.721344] nix-grpc-daemon[782]: ts=2026-09-21T14:29:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4391lb: must succeed: curl -sf localhost:9901/clusters4392lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4393worker2 # [ 24.919126] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514394worker1 # [ 24.910894] nix-grpc-daemon[781]: ts=2026-09-21T14:29:21Z level=info event=scheduler_connected addr=lb:500514395worker2 # [ 24.922775] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4396worker1 # [ 24.914661] nix-grpc-daemon[781]: ts=2026-09-21T14:29:21Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4397worker2 # [ 25.121988] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514398worker1 # [ 25.114559] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514399worker2 # [ 25.125308] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4400worker1 # [ 25.117535] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4401worker2 # [ 25.323375] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514402worker1 # [ 25.314788] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514403worker2 # [ 25.327383] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4404worker1 # [ 25.318604] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4405worker1 # [ 25.517110] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514406worker2 # [ 25.528736] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514407worker1 # [ 25.521741] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4408worker2 # [ 25.532609] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4409worker2 # [ 25.733798] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514410worker1 # [ 25.722742] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514411worker2 # [ 25.737641] nix-grpc-daemon[782]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4412worker1 # [ 25.727203] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4413lb: must succeed: curl -sf localhost:9901/clusters4414worker2 # [ 25.938588] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514415worker1 # [ 25.927800] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_connected addr=lb:500514416worker2 # [ 25.942752] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4417worker1 # [ 25.932202] nix-grpc-daemon[781]: ts=2026-09-21T14:29:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4418lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4419worker1 # [ 26.133997] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514420worker2 # [ 26.144522] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514421worker1 # [ 26.137822] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4422worker2 # [ 26.148401] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4423worker1 # [ 26.339257] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514424worker2 # [ 26.349798] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514425worker1 # [ 26.343305] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4426worker2 # [ 26.353753] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4427worker1 # [ 26.544233] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514428worker2 # [ 26.554938] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514429worker1 # [ 26.548243] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4430worker2 # [ 26.558856] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4431worker1 # [ 26.749267] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514432worker2 # [ 26.759993] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514433worker1 # [ 26.753431] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4434worker2 # [ 26.764186] nix-grpc-daemon[782]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4435lb: must succeed: curl -sf localhost:9901/clusters4436worker1 # [ 26.953863] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_connected addr=lb:500514437worker2 # [ 26.964633] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514438worker1 # [ 26.957815] nix-grpc-daemon[781]: ts=2026-09-21T14:29:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4439worker2 # [ 26.968545] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4440lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4441worker1 # [ 27.159872] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514442worker2 # [ 27.170252] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514443worker2 # [ 27.173956] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4444worker1 # [ 27.163689] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4445worker2 # [ 27.374931] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514446worker1 # [ 27.364816] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514447worker2 # [ 27.378884] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4448worker1 # [ 27.368788] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4449worker2 # [ 27.580156] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514450worker1 # [ 27.569946] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514451worker2 # [ 27.584116] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4452worker1 # [ 27.573927] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4453worker2 # [ 27.785260] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514454worker1 # [ 27.775219] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514455worker2 # [ 27.789179] nix-grpc-daemon[782]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4456worker1 # [ 27.780121] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4457worker2 # [ 27.989970] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514458worker1 # [ 27.981133] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_connected addr=lb:500514459worker2 # [ 27.994092] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4460worker1 # [ 27.984886] nix-grpc-daemon[781]: ts=2026-09-21T14:29:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4461lb: must succeed: curl -sf localhost:9901/clusters4462lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4463worker2 # [ 28.195929] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514464worker1 # [ 28.186890] nix-grpc-daemon[781]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514465worker2 # [ 28.199931] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4466worker1 # [ 28.190456] nix-grpc-daemon[781]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4467worker2 # [ 28.401304] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514468worker1 # [ 28.391686] nix-grpc-daemon[781]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514469worker2 # [ 28.405283] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4470worker1 # [ 28.395711] nix-grpc-daemon[781]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4471worker2 # [ 28.606729] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514472worker1 # [ 28.596722] nix-grpc-daemon[781]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514473worker2 # [ 28.610712] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4474worker1 # [ 28.600643] nix-grpc-daemon[781]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4475worker1 # [ 28.801612] nix-grpc-daemon[781]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514476worker2 # [ 28.811996] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_connected addr=lb:500514477worker2 # [ 28.815922] nix-grpc-daemon[782]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4478worker1 # [ 28.805546] nix-grpc-daemon[781]: ts=2026-09-21T14:29:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4479worker2 # [ 29.016718] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514480worker1 # [ 29.006379] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514481worker1 # [ 29.010335] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4482worker2 # [ 29.020742] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4483lb: must succeed: curl -sf localhost:9901/clusters4484lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)4485worker2 # [ 29.222599] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514486worker1 # [ 29.212224] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514487worker2 # [ 29.226375] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4488worker1 # [ 29.216129] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4489worker1 # [ 29.417410] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514490worker2 # [ 29.431323] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514491worker1 # [ 29.421473] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4492worker2 # [ 29.435148] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4493worker1 # [ 29.622534] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514494worker2 # [ 29.636268] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514495worker1 # [ 29.626579] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4496worker2 # [ 29.640121] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4497worker1 # [ 29.827651] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514498worker2 # [ 29.841158] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_connected addr=lb:500514499worker1 # [ 29.831610] nix-grpc-daemon[781]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4500worker2 # [ 29.844980] nix-grpc-daemon[782]: ts=2026-09-21T14:29:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4501worker1 # [ 30.032512] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514502worker2 # [ 30.046197] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514503worker1 # [ 30.036334] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4504worker2 # [ 30.050116] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4505lb: must succeed: curl -sf localhost:9901/clusters4506lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4507worker1 # [ 30.238601] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514508worker2 # [ 30.252473] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514509worker1 # [ 30.241889] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4510worker2 # [ 30.255557] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4511worker1 # [ 30.442829] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514512worker2 # [ 30.453391] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514513worker1 # [ 30.446877] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4514worker2 # [ 30.457353] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4515worker1 # [ 30.647912] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514516worker2 # [ 30.658473] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514517worker2 # [ 30.662358] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4518worker1 # [ 30.651958] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4519worker2 # [ 30.863250] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514520worker1 # [ 30.852923] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_connected addr=lb:500514521worker2 # [ 30.867161] nix-grpc-daemon[782]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4522worker1 # [ 30.856816] nix-grpc-daemon[781]: ts=2026-09-21T14:29:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4523worker2 # [ 31.067547] nix-grpc-daemon[782]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514524worker2 # [ 31.069267] nix-grpc-daemon[782]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4525worker1 # [ 31.057673] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514526worker1 # [ 31.061171] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4527lb: must succeed: curl -sf localhost:9901/clusters4528lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4529worker2 # [ 31.271490] nix-grpc-daemon[782]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514530worker1 # [ 31.263783] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514531worker2 # [ 31.275398] nix-grpc-daemon[782]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4532worker1 # [ 31.266870] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4533worker1 # [ 31.464456] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514534worker2 # [ 31.476663] nix-grpc-daemon[782]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514535worker1 # [ 31.468429] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4536worker2 # [ 31.480461] nix-grpc-daemon[782]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4537worker1 # [ 31.669456] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514538worker2 # [ 31.681604] nix-grpc-daemon[782]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514539worker1 # [ 31.673483] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4540worker2 # [ 31.685395] nix-grpc-daemon[782]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4541worker1 # [ 31.874366] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_connected addr=lb:500514542worker2 # [ 31.886477] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514543worker1 # [ 31.878396] nix-grpc-daemon[781]: ts=2026-09-21T14:29:28Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4544worker2 # [ 31.890324] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4545worker1 # [ 32.079162] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514546worker2 # [ 32.091546] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514547worker1 # [ 32.083300] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4548worker2 # [ 32.095329] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4549lb: must succeed: curl -sf localhost:9901/clusters4550worker2 # [ 32.243575] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_take_over4551lb # [ 32.220904] niks3-server[993]: 2026/09/21 14:29:29 INFO lead: acquired remote=192.168.1.4:434704552lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.07 seconds)4553worker1 # [ 32.285499] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514554worker2 # [ 32.297478] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514555worker1 # [ 32.288635] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4556worker2 # [ 32.300525] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4557worker1 # [ 32.486580] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514558worker2 # [ 32.498243] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514559worker1 # [ 32.490568] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4560worker2 # [ 32.502225] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4561worker1 # [ 32.691563] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514562worker2 # [ 32.703603] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514563worker1 # [ 32.695608] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4564worker2 # [ 32.707402] nix-grpc-daemon[782]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4565worker1 # [ 32.896197] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_connected addr=lb:500514566worker2 # [ 32.908402] nix-grpc-daemon[782]: ts=2026-09-21T14:29:30Z level=info event=scheduler_connected addr=lb:500514567worker1 # [ 32.899957] nix-grpc-daemon[781]: ts=2026-09-21T14:29:29Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4568worker2 # [ 32.912263] nix-grpc-daemon[782]: ts=2026-09-21T14:29:30Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"4569worker1 # [ 33.101080] nix-grpc-daemon[781]: ts=2026-09-21T14:29:30Z level=info event=scheduler_connected addr=lb:500514570worker2 # [ 33.114926] nix-grpc-daemon[782]: ts=2026-09-21T14:29:30Z level=info event=scheduler_connected addr=lb:500514571worker2 # [ 33.120499] nix-grpc-daemon[782]: ts=2026-09-21T14:29:30Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:483324572worker2 # [ 33.125539] nix-grpc-daemon[782]: ts=2026-09-21T14:29:30Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04573worker2 # [ 33.138327] nix-grpc-daemon[782]: ts=2026-09-21T14:29:30Z level=info event=oidc_jwks provider=mock keys=14574worker2 # [ 33.141246] nix-grpc-daemon[782]: ts=2026-09-21T14:29:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:483324575worker2 # [ 33.145409] nix-grpc-daemon[782]: ts=2026-09-21T14:29:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=04576lb: must succeed: curl -sf localhost:9901/clusters4577lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4578worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4579worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4580lb: must succeed: curl -sf localhost:9901/clusters4581lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4582lb: must succeed: curl -sf localhost:9901/clusters4583lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4584lb: must succeed: curl -sf localhost:9901/clusters4585lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4586lb: must succeed: curl -sf localhost:9901/clusters4587lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4588worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open'4589worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'event=worker_session_open', in 0.04 seconds)4590worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4591worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4592lb: must succeed: curl -sf localhost:9901/clusters4593lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)4594lb: must succeed: curl -sf localhost:9901/clusters4595lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4596worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4597worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4598worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4599worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4600worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4601worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)4602worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4603worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)4604(finished: subtest: one node schedules and both builders hold a WorkerSession on it, in 10.95 seconds)4605subtest: balancer auth: client cert, bearer token, nothing, unknown CN4606worker2 # [ 33.959814] nix-grpc-daemon[782]: ts=2026-09-21T14:29:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:483424607worker2 # [ 33.970969] systemd[1]: Started Nix Daemon.4608worker2 # [ 34.083433] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4609worker2 # [ 34.097235] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 1/5)4610worker2 # [ 34.185930] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 137 ms (attempt 2/5)4611worker2 # [ 34.324482] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 288 ms (attempt 3/5)4612worker2 # [ 34.614668] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 361 ms (attempt 4/5)4613worker2 # [ 34.979820] nix-grpc-daemon[782]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4614worker2 # [ 34.988464] nix-grpc-daemon[782]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4615worker1 # [ 35.009363] nix-grpc-daemon[781]: ts=2026-09-21T14:29:32Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:517904616worker1 # [ 35.016860] systemd[1]: Started Nix Daemon.4617worker1 # [ 35.135585] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4618worker1 # [ 35.159238] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 1/5)4619worker1 # [ 35.242794] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 2/5)4620worker1 # [ 35.294979] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 3/5)4621worker1 # [ 35.409287] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 667 ms (attempt 4/5)4622worker1 # [ 36.080349] nix-grpc-daemon[781]: disabling binary cache 'https://cache.nixos.org' for 60 seconds4623worker1 # [ 36.084380] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org4624worker1 # [ 36.096789] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=debug event=path_info_miss path=0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test error="error: path '/nix/store/0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test' is required, but there is no substituter that can build it"4625worker2 # [ 36.127365] nix-grpc-daemon[782]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:483424626worker2 # [ 36.139140] nix-grpc-daemon[782]: ts=2026-09-21T14:29:33Z level=debug event=path_info_miss path=0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test error="error: path '/nix/store/0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test' is required, but there is no substituter that can build it"4627client: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt4628client: (finished: must succeed: curl -sfG http://lb:8081/issue --data-urlencode 'aud=grpc://lb:50051' --data-urlencode sub=dev:alice > /root/dev.jwt && test -s /root/dev.jwt, in 0.05 seconds)4629client: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key4630client: (finished: must succeed: install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.pem /var/lib/nix-grpc-store/client.crt && install -D /nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/foreign.key /var/lib/nix-grpc-store/client.key, in 0.02 seconds)4631worker2 # [ 36.364671] nix-grpc-daemon[782]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=QueryMissing cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:483424632worker2 # [ 36.377594] nix-grpc-daemon[782]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:483424633worker2 # [ 36.386557] nix-grpc-daemon[782]: ts=2026-09-21T14:29:33Z level=debug event=path_info_miss path=0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test error="error: path '/nix/store/0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test' is required, but there is no substituter that can build it"4634worker1 # [ 36.390844] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=info event=oidc_jwks provider=mock keys=14635worker1 # [ 36.394442] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=QueryPathInfos cn=oidc:mock:dev:alice peer=ipv4:192.168.1.2:517904636worker1 # [ 36.403408] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=debug event=path_info_miss path=0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test error="error: path '/nix/store/0y74x5k7r56p0shs5cg615fz26fhjrpm-nixos-system-client-test' is required, but there is no substituter that can build it"4637client: must succeed: rm -r /var/lib/nix-grpc-store4638client: (finished: must succeed: rm -r /var/lib/nix-grpc-store, in 0.01 seconds)4639worker2 # [ 36.551745] nix-grpc-daemon[782]: ts=2026-09-21T14:29:33Z level=info event=denied method=QueryMissing cn=- role=none4640worker1 # [ 36.677140] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=info event=denied method=QueryMissing cn=stranger role=none4641(finished: subtest: balancer auth: client cert, bearer token, nothing, unknown CN, in 2.93 seconds)4642subtest: DAG build is scheduled across workers and lands in the cache4643client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24644worker1 # [ 36.812135] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517904645client # don't know how to build these paths:4646client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4647worker1 # [ 36.823671] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517904648worker2 # [ 36.863522] nix-grpc-daemon[782]: ts=2026-09-21T14:29:33Z level=debug event=want client=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv cached=04649worker2 # [ 36.868243] nix-grpc-daemon[782]: ts=2026-09-21T14:29:33Z level=debug event=assigned drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv worker=192.168.1.3:50051 clients=14650worker1 # [ 36.902908] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517904651worker1 # [ 36.918198] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517904652client # copying 1 paths to 'lb:50051'...4653worker1 # [ 36.930760] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517904654client # copying path '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' to 'lb:50051'...4655worker1 # [ 36.952558] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4656worker1 # [ 36.981191] nix-grpc-daemon[781]: ts=2026-09-21T14:29:33Z level=info event=niks3_push_started pid=8594657worker1 # [ 37.134486] nix-grpc-daemon[859]: time=2026-09-21T14:29:34.140Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4658worker1 # [ 37.231241] nix-daemon[847]: accepted connection from pid 872, user nix-grpc-daemon (trusted)4659worker1 # [ 37.244177] nix-daemon[847]: reaped child process 875, status = succeeded4660worker1 # [ 37.333722] nix-daemon[847]: accepted connection from pid 877, user nix-grpc-daemon (trusted)4661worker1 # [ 37.346599] nix-daemon[847]: reaped child process 880, status = succeeded4662lb # [ 37.345587] niks3-server[993]: 2026/09/21 14:29:34 INFO Received uploads request method=POST path=/api/pending_closures4663worker1 # [ 37.374143] nix-grpc-daemon[859]: time=2026-09-21T14:29:34.380Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4664worker1 # [ 37.375485] nix-grpc-daemon[859]: time=2026-09-21T14:29:34.380Z level=INFO msg="Uploading 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv (504B)"4665lb # [ 37.400272] niks3-server[993]: 2026/09/21 14:29:34 INFO Registered completed upload object_key=nar/15vlbxqk65jh0v1qgc8308zz4diydc8wp113i0cc4ylv0zc5w964.nar.zst4666lb # [ 37.406870] niks3-server[993]: 2026/09/21 14:29:34 INFO Received sign narinfos request method=POST path=/api/pending_closures/1/sign4667lb # [ 37.421562] niks3-server[993]: 2026/09/21 14:29:34 INFO Signed narinfos id=1 count=14668worker1 # [ 37.435085] nix-grpc-daemon[859]: time=2026-09-21T14:29:34.441Z level=INFO msg="Uploading 1 narinfos"4669lb # [ 37.432713] niks3-server[993]: 2026/09/21 14:29:34 INFO Received complete upload request method=POST path=/api/pending_closures/1/complete4670lb # [ 37.438841] niks3-server[993]: 2026/09/21 14:29:34 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.ls4671lb # [ 37.445920] niks3-server[993]: 2026/09/21 14:29:34 INFO Registered completed upload object_key=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1.narinfo4672lb # [ 37.449783] niks3-server[993]: 2026/09/21 14:29:34 INFO Completed upload id=14673worker1 # [ 37.463069] nix-grpc-daemon[859]: time=2026-09-21T14:29:34.469Z level=INFO msg="Upload complete. (324ms)"4674worker1 # [ 37.466289] nix-grpc-daemon[781]: ts=2026-09-21T14:29:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_in=5044675worker1 # [ 37.468870] nix-grpc-daemon[781]: ts=2026-09-21T14:29:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517904676worker1 # [ 37.473946] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4677worker1 # [ 37.479819] nix-daemon[847]: reaped child process 857, status = succeeded4678worker1 # [ 37.484107] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4679client # building '/nix/store/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv' on lb:50051...4680client # farm-a> node-a: building 3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4681client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4682client # farm-a> LOG-farm-a-t14683worker1 # [ 39.118327] nix-daemon[847]: reaped child process 885, status = succeeded4684worker1 # [ 39.208662] nix-daemon[847]: accepted connection from pid 901, user nix-grpc-daemon (trusted)4685worker1 # [ 39.222070] nix-daemon[847]: reaped child process 904, status = succeeded4686lb # [ 39.220391] niks3-server[993]: 2026/09/21 14:29:36 INFO Received uploads request method=POST path=/api/pending_closures4687worker1 # [ 39.245977] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.252Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4688worker1 # [ 39.247985] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.253Z level=INFO msg="Uploading qm96yd1cmricgf2hr21k88syh75l317v-farm-a (128B)"4689lb # [ 39.271900] niks3-server[993]: 2026/09/21 14:29:36 INFO Registered completed upload object_key=log/3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv4690lb # [ 39.276748] niks3-server[993]: 2026/09/21 14:29:36 INFO Registered completed upload object_key=nar/1rmil4ja9jzfwn6l5hwbir70b6plx99sgk0pnjjy377ngbc3vz28.nar.zst4691lb # [ 39.291116] niks3-server[993]: 2026/09/21 14:29:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/2/sign4692lb # [ 39.295551] niks3-server[993]: 2026/09/21 14:29:36 INFO Signed narinfos id=2 count=14693worker1 # [ 39.310124] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.315Z level=INFO msg="Uploading 1 narinfos"4694lb # [ 39.301245] niks3-server[993]: 2026/09/21 14:29:36 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.ls4695lb # [ 39.308269] niks3-server[993]: 2026/09/21 14:29:36 INFO Received complete upload request method=POST path=/api/pending_closures/2/complete4696lb # [ 39.380793] niks3-server[993]: 2026/09/21 14:29:36 INFO Registered completed upload object_key=qm96yd1cmricgf2hr21k88syh75l317v.narinfo4697lb # [ 39.390086] niks3-server[993]: 2026/09/21 14:29:36 INFO Completed upload id=24698worker1 # [ 39.403567] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.409Z level=INFO msg="Upload complete. (309ms)"4699worker1 # [ 39.409663] nix-grpc-daemon[781]: ts=2026-09-21T14:29:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 drv=3ijpb6qq7zgbnifk6lc6i1fipl0iqkl1-farm-a.drv assign_id=1 outputs=14700worker2 # [ 39.431788] nix-grpc-daemon[782]: ts=2026-09-21T14:29:36Z level=debug event=want client=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv cached=04701worker1 # [ 39.425770] nix-daemon[847]: reaped child process 883, status = succeeded4702worker2 # [ 39.435970] nix-grpc-daemon[782]: ts=2026-09-21T14:29:36Z level=debug event=assigned drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv worker=192.168.1.3:50051 clients=14703worker1 # [ 39.465160] nix-grpc-daemon[781]: ts=2026-09-21T14:29:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517904704worker1 # [ 39.478148] nix-grpc-daemon[781]: ts=2026-09-21T14:29:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517904705worker1 # [ 39.491115] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.497Z level=INFO msg="All 1 paths already cached"4706client # copying 1 paths to 'lb:50051'...4707worker1 # [ 39.499331] nix-grpc-daemon[781]: ts=2026-09-21T14:29:36Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517904708client # copying path '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' to 'lb:50051'...4709worker1 # [ 39.518184] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4710worker1 # [ 39.625340] nix-daemon[847]: accepted connection from pid 908, user nix-grpc-daemon (trusted)4711worker1 # [ 39.639559] nix-daemon[847]: reaped child process 911, status = succeeded4712worker1 # [ 39.725205] nix-daemon[847]: accepted connection from pid 913, user nix-grpc-daemon (trusted)4713worker1 # [ 39.738785] nix-daemon[847]: reaped child process 916, status = succeeded4714lb # [ 39.733597] niks3-server[993]: 2026/09/21 14:29:36 INFO Received uploads request method=POST path=/api/pending_closures4715worker1 # [ 39.755483] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.761Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4716worker1 # [ 39.756883] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.761Z level=INFO msg="Uploading zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv (624B)"4717lb # [ 39.768926] niks3-server[993]: 2026/09/21 14:29:36 INFO Received sign narinfos request method=POST path=/api/pending_closures/3/sign4718lb # [ 39.776403] niks3-server[993]: 2026/09/21 14:29:36 INFO Registered completed upload object_key=nar/19sdpvl15ladfjss8gp73f6binhg0wgk0323wmn91mihvj530pxq.nar.zst4719worker1 # [ 39.798733] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.805Z level=INFO msg="Uploading 1 narinfos"4720lb # [ 39.785498] niks3-server[993]: 2026/09/21 14:29:36 INFO Signed narinfos id=3 count=14721lb # [ 39.803612] niks3-server[993]: 2026/09/21 14:29:36 INFO Received complete upload request method=POST path=/api/pending_closures/3/complete4722lb # [ 39.807406] niks3-server[993]: 2026/09/21 14:29:36 INFO Completed upload id=34723worker1 # [ 39.822588] nix-grpc-daemon[859]: time=2026-09-21T14:29:36.829Z level=INFO msg="Upload complete. (291ms)"4724lb # [ 39.810616] niks3-server[993]: 2026/09/21 14:29:36 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.narinfo4725lb # [ 39.814745] niks3-server[993]: 2026/09/21 14:29:36 INFO Registered completed upload object_key=zbhcql1mkb05ig74hjj18iqpvqy3rfbp.ls4726worker1 # [ 39.826535] nix-grpc-daemon[781]: ts=2026-09-21T14:29:36Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_in=6244727worker1 # [ 39.832497] nix-grpc-daemon[781]: ts=2026-09-21T14:29:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517904728worker1 # [ 39.841789] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4729worker1 # [ 39.849139] nix-daemon[847]: reaped child process 906, status = succeeded4730client # building '/nix/store/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv' on lb:50051...4731worker1 # [ 39.853694] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4732client # farm-b> node-a: building zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4733client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4734client # farm-b> LOG-farm-b-t14735worker1 # [ 40.886537] nix-daemon[847]: reaped child process 920, status = succeeded4736worker1 # [ 40.971269] nix-daemon[847]: accepted connection from pid 937, user nix-grpc-daemon (trusted)4737worker1 # [ 40.984548] nix-daemon[847]: reaped child process 940, status = succeeded4738lb # [ 40.980891] niks3-server[993]: 2026/09/21 14:29:38 INFO Received uploads request method=POST path=/api/pending_closures4739worker1 # [ 41.006483] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.012Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4740worker1 # [ 41.007904] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.014Z level=INFO msg="Uploading jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b (176B)"4741lb # [ 41.028236] niks3-server[993]: 2026/09/21 14:29:38 INFO Registered completed upload object_key=nar/0n7l6rw4y681bymb8bwra2nh13s7ybxcm2l7y8iwy0k5mmnaznjg.nar.zst4742lb # [ 41.033567] niks3-server[993]: 2026/09/21 14:29:38 INFO Registered completed upload object_key=log/zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv4743lb # [ 41.042566] niks3-server[993]: 2026/09/21 14:29:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/4/sign4744lb # [ 41.048716] niks3-server[993]: 2026/09/21 14:29:38 INFO Signed narinfos id=4 count=14745worker1 # [ 41.064706] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.071Z level=INFO msg="Uploading 1 narinfos"4746lb # [ 41.053183] niks3-server[993]: 2026/09/21 14:29:38 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.ls4747lb # [ 41.062389] niks3-server[993]: 2026/09/21 14:29:38 INFO Received complete upload request method=POST path=/api/pending_closures/4/complete4748lb # [ 41.068866] niks3-server[993]: 2026/09/21 14:29:38 INFO Completed upload id=44749worker1 # [ 41.082302] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.088Z level=INFO msg="Upload complete. (221ms)"4750lb # [ 41.070754] niks3-server[993]: 2026/09/21 14:29:38 INFO Registered completed upload object_key=jicn175r0p4806ghnhxdvn7fsa1akkr0.narinfo4751worker1 # [ 41.087899] nix-grpc-daemon[781]: ts=2026-09-21T14:29:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 drv=zbhcql1mkb05ig74hjj18iqpvqy3rfbp-farm-b.drv assign_id=2 outputs=14752worker2 # [ 41.105826] nix-grpc-daemon[782]: ts=2026-09-21T14:29:38Z level=debug event=want client=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=04753worker2 # [ 41.110206] nix-grpc-daemon[782]: ts=2026-09-21T14:29:38Z level=debug event=assigned drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv worker=192.168.1.3:50051 clients=14754worker1 # [ 41.104129] nix-daemon[847]: reaped child process 918, status = succeeded4755worker1 # [ 41.139741] nix-grpc-daemon[781]: ts=2026-09-21T14:29:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517904756worker1 # [ 41.152375] nix-grpc-daemon[781]: ts=2026-09-21T14:29:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517904757worker1 # [ 41.165391] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.171Z level=INFO msg="All 2 paths already cached"4758client # copying 1 paths to 'lb:50051'...4759worker1 # [ 41.178996] nix-grpc-daemon[781]: ts=2026-09-21T14:29:38Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517904760client # copying path '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' to 'lb:50051'...4761worker1 # [ 41.198289] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4762worker1 # [ 41.304467] nix-daemon[847]: accepted connection from pid 944, user nix-grpc-daemon (trusted)4763worker1 # [ 41.319577] nix-daemon[847]: reaped child process 947, status = succeeded4764worker1 # [ 41.403697] nix-daemon[847]: accepted connection from pid 949, user nix-grpc-daemon (trusted)4765worker1 # [ 41.418359] nix-daemon[847]: reaped child process 952, status = succeeded4766lb # [ 41.411640] niks3-server[993]: 2026/09/21 14:29:38 INFO Received uploads request method=POST path=/api/pending_closures4767worker1 # [ 41.434368] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.440Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4768worker1 # [ 41.436279] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.442Z level=INFO msg="Uploading c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv (752B)"4769lb # [ 41.445532] niks3-server[993]: 2026/09/21 14:29:38 INFO Registered completed upload object_key=nar/1nf01qbgzkrsqfjd6d2bx60xxkfkbrljmfrvypvjci1c69ad7knd.nar.zst4770lb # [ 41.451567] niks3-server[993]: 2026/09/21 14:29:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/5/sign4771worker1 # [ 41.473471] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.479Z level=INFO msg="Uploading 1 narinfos"4772lb # [ 41.459863] niks3-server[993]: 2026/09/21 14:29:38 INFO Signed narinfos id=5 count=14773lb # [ 41.469444] niks3-server[993]: 2026/09/21 14:29:38 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.ls4774lb # [ 41.480136] niks3-server[993]: 2026/09/21 14:29:38 INFO Received complete upload request method=POST path=/api/pending_closures/5/complete4775lb # [ 41.492872] niks3-server[993]: 2026/09/21 14:29:38 INFO Completed upload id=54776lb # [ 41.495288] niks3-server[993]: 2026/09/21 14:29:38 INFO Registered completed upload object_key=c7ysj984j3hnalhcvn7z0328xf5fr0zd.narinfo4777worker1 # [ 41.507557] nix-grpc-daemon[859]: time=2026-09-21T14:29:38.513Z level=INFO msg="Upload complete. (296ms)"4778worker1 # [ 41.511480] nix-grpc-daemon[781]: ts=2026-09-21T14:29:38Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_in=7524779worker1 # [ 41.516282] nix-grpc-daemon[781]: ts=2026-09-21T14:29:38Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517904780worker1 # [ 41.527143] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4781worker1 # [ 41.534822] nix-daemon[847]: reaped child process 942, status = succeeded4782worker1 # [ 41.537767] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4783client # building '/nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv' on lb:50051...4784client # farm-top> node-a: building c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4785client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4786client # farm-top> LOG-farm-top-t14787worker1 # [ 42.981907] nix-daemon[847]: reaped child process 956, status = succeeded4788worker1 # [ 43.059717] nix-daemon[847]: accepted connection from pid 972, user nix-grpc-daemon (trusted)4789worker1 # [ 43.074232] nix-daemon[847]: reaped child process 975, status = succeeded4790lb # [ 43.069904] niks3-server[993]: 2026/09/21 14:29:40 INFO Received uploads request method=POST path=/api/pending_closures4791worker1 # [ 43.091586] nix-grpc-daemon[859]: time=2026-09-21T14:29:40.097Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4792worker1 # [ 43.094063] nix-grpc-daemon[859]: time=2026-09-21T14:29:40.099Z level=INFO msg="Uploading 5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top (232B)"4793lb # [ 43.106709] niks3-server[993]: 2026/09/21 14:29:40 INFO Registered completed upload object_key=log/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4794lb # [ 43.120960] niks3-server[993]: 2026/09/21 14:29:40 INFO Received sign narinfos request method=POST path=/api/pending_closures/6/sign4795lb # [ 43.127934] niks3-server[993]: 2026/09/21 14:29:40 INFO Signed narinfos id=6 count=14796worker1 # [ 43.144234] nix-grpc-daemon[859]: time=2026-09-21T14:29:40.150Z level=INFO msg="Uploading 1 narinfos"4797lb # [ 43.131440] niks3-server[993]: 2026/09/21 14:29:40 INFO Registered completed upload object_key=nar/0pq8hp2qdfznpnf4pn4my9c4nd9yx6cfjn0x0bgv1rpk0536kpjy.nar.zst4798lb # [ 43.142376] niks3-server[993]: 2026/09/21 14:29:40 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.ls4799lb # [ 43.147385] niks3-server[993]: 2026/09/21 14:29:40 INFO Received complete upload request method=POST path=/api/pending_closures/6/complete4800lb # [ 43.151882] niks3-server[993]: 2026/09/21 14:29:40 INFO Completed upload id=64801lb # [ 43.154079] niks3-server[993]: 2026/09/21 14:29:40 INFO Registered completed upload object_key=5qgbfb7awggfq3smlyfxgz8zixnqxk2n.narinfo4802worker1 # [ 43.166488] nix-grpc-daemon[859]: time=2026-09-21T14:29:40.172Z level=INFO msg="Upload complete. (220ms)"4803worker1 # [ 43.172460] nix-grpc-daemon[781]: ts=2026-09-21T14:29:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv assign_id=3 outputs=14804worker1 # [ 43.187630] nix-daemon[847]: reaped child process 954, status = succeeded4805client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 6.51 seconds)4806client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4807client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.11 seconds)4808client: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4809client: (finished: must fail: test -e /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.01 seconds)4810client: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4811client # copying 3 paths...4812client # copying path '/nix/store/qm96yd1cmricgf2hr21k88syh75l317v-farm-a' from 'http://lb:5751'...4813client # copying path '/nix/store/jicn175r0p4806ghnhxdvn7fsa1akkr0-farm-b' from 'http://lb:5751'...4814client # copying path '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top' from 'http://lb:5751'...4815client: (finished: must succeed: nix copy --from http://lb:5751 --no-check-sigs /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top && grep farm-top-t1 /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4816(finished: subtest: DAG build is scheduled across workers and lands in the cache, in 6.86 seconds)4817subtest: repeat is answered Cached by the scheduler without building4818worker1: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4819worker1 # finding garbage collector roots...4820worker1 # deleting specified paths...4821worker1 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4822worker1 # deleting unused links...4823worker1 # note: hard linking is currently saving 0.0 KiB4824worker1: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.16 seconds)4825worker2: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top4826worker2 # finding garbage collector roots...4827worker2 # deleting specified paths...4828worker2 # deleting '/nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top'4829worker2 # deleting unused links...4830worker2 # note: hard linking is currently saving 0.0 KiB4831worker2: (finished: must succeed: nix-store --delete /nix/store/5qgbfb7awggfq3smlyfxgz8zixnqxk2n-farm-top, in 0.17 seconds)4832client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&24833worker2 # [ 44.039954] nix-grpc-daemon[782]: ts=2026-09-21T14:29:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:483424834client # don't know how to build these paths:4835client # /nix/store/c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv4836worker2 # [ 44.048845] nix-grpc-daemon[782]: ts=2026-09-21T14:29:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483424837worker2 # [ 44.076709] nix-grpc-daemon[782]: ts=2026-09-21T14:29:41Z level=debug event=want client=2 drv=c7ysj984j3hnalhcvn7z0328xf5fr0zd-farm-top.drv cached=14838client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 >&2, in 0.20 seconds)4839client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath4840client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 outPath, in 0.10 seconds)4841lb: must succeed: curl -sf localhost:9901/clusters4842lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4843lb: must succeed: curl -sf localhost:9901/clusters4844lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4845worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4846worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4847(finished: subtest: repeat is answered Cached by the scheduler without building, in 0.81 seconds)4848subtest: inputs already in the cache are not sent to the scheduler4849lb: must succeed: curl -sf localhost:9901/clusters4850lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4851lb: must succeed: curl -sf localhost:9901/clusters4852lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)4853worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4854worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4855client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&24856worker1 # [ 44.605855] nix-grpc-daemon[781]: ts=2026-09-21T14:29:41Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517904857client # don't know how to build these paths:4858client # /nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4859worker1 # [ 44.614267] nix-grpc-daemon[781]: ts=2026-09-21T14:29:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517904860worker1 # [ 44.623382] nix-grpc-daemon[859]: time=2026-09-21T14:29:41.629Z level=INFO msg="All 2 paths already cached"4861worker2 # [ 44.645896] nix-grpc-daemon[782]: ts=2026-09-21T14:29:41Z level=debug event=want client=3 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv cached=04862worker2 # [ 44.650373] nix-grpc-daemon[782]: ts=2026-09-21T14:29:41Z level=debug event=assigned drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv worker=192.168.1.3:50051 clients=14863worker1 # [ 44.647423] nix-grpc-daemon[781]: ts=2026-09-21T14:29:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517904864worker1 # [ 44.659895] nix-grpc-daemon[781]: ts=2026-09-21T14:29:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517904865worker1 # [ 44.671945] nix-grpc-daemon[859]: time=2026-09-21T14:29:41.678Z level=INFO msg="All 2 paths already cached"4866client # copying 1 paths to 'lb:50051'...4867client # copying path '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' to 'lb:50051'...4868worker1 # [ 44.679594] nix-grpc-daemon[781]: ts=2026-09-21T14:29:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517904869worker1 # [ 44.691478] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4870worker1 # [ 44.792829] nix-daemon[847]: accepted connection from pid 992, user nix-grpc-daemon (trusted)4871worker1 # [ 44.806896] nix-daemon[847]: reaped child process 995, status = succeeded4872worker1 # [ 44.894742] nix-daemon[847]: accepted connection from pid 997, user nix-grpc-daemon (trusted)4873worker1 # [ 44.909132] nix-daemon[847]: reaped child process 1000, status = succeeded4874lb # [ 44.904515] niks3-server[993]: 2026/09/21 14:29:42 INFO Received uploads request method=POST path=/api/pending_closures4875worker1 # [ 44.925912] nix-grpc-daemon[859]: time=2026-09-21T14:29:41.932Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4876worker1 # [ 44.927476] nix-grpc-daemon[859]: time=2026-09-21T14:29:41.933Z level=INFO msg="Uploading y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv (752B)"4877lb # [ 44.941344] niks3-server[993]: 2026/09/21 14:29:42 INFO Received sign narinfos request method=POST path=/api/pending_closures/7/sign4878lb # [ 44.948433] niks3-server[993]: 2026/09/21 14:29:42 INFO Signed narinfos id=7 count=14879worker1 # [ 44.964311] nix-grpc-daemon[859]: time=2026-09-21T14:29:41.970Z level=INFO msg="Uploading 1 narinfos"4880lb # [ 44.956174] niks3-server[993]: 2026/09/21 14:29:42 INFO Registered completed upload object_key=nar/0p14b74h5654i3y6qv7zaaib0ya421b01knaq0m00r5ypjyh2q1d.nar.zst4881lb # [ 44.963532] niks3-server[993]: 2026/09/21 14:29:42 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.ls4882lb # [ 44.974164] niks3-server[993]: 2026/09/21 14:29:42 INFO Received complete upload request method=POST path=/api/pending_closures/7/complete4883lb # [ 44.978650] niks3-server[993]: 2026/09/21 14:29:42 INFO Completed upload id=74884worker1 # [ 44.991781] nix-grpc-daemon[859]: time=2026-09-21T14:29:41.998Z level=INFO msg="Upload complete. (288ms)"4885lb # [ 44.980727] niks3-server[993]: 2026/09/21 14:29:42 INFO Registered completed upload object_key=y2wm8aizyyg449ia2pcscrjc2pklsrz4.narinfo4886worker1 # [ 44.994901] nix-grpc-daemon[781]: ts=2026-09-21T14:29:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_in=7524887worker1 # [ 45.000282] nix-grpc-daemon[781]: ts=2026-09-21T14:29:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517904888worker1 # [ 45.009499] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4889worker1 # [ 45.017590] nix-daemon[847]: reaped child process 990, status = succeeded4890worker1 # [ 45.021891] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)4891client # building '/nix/store/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv' on lb:50051...4892client # farm-top> node-a: building y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4893lb # [ 45.646484] envoy[906]: conn peer=[2001:db8:1::1]:38444 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=732 tx=407 ms=22134894lb # [ 45.650280] envoy[906]: conn peer=[2001:db8:1::1]:38452 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=2596 tx=364 ms=584895lb # [ 45.653557] envoy[906]: conn peer=[2001:db8:1::1]:38454 tls=TLSv1.3 subject="-" sni=- flags=- tls_fail="-" rx=423 tx=198 ms=114896lb # [ 45.656788] envoy[906]: conn peer=[2001:db8:1::1]:38464 tls=TLSv1.3 subject="CN=stranger" sni=- flags=- tls_fail="-" rx=423 tx=181 ms=134897lb # [ 45.660825] envoy[906]: conn peer=[2001:db8:1::1]:38480 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9511 tx=4070 ms=63794898lb # [ 45.664354] envoy[906]: conn peer=[2001:db8:1::1]:43406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=534899client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4900client # farm-top> LOG-farm-top-t1b4901worker1 # [ 46.259660] nix-daemon[847]: reaped child process 1004, status = succeeded4902worker1 # [ 46.346928] nix-daemon[847]: accepted connection from pid 1020, user nix-grpc-daemon (trusted)4903worker1 # [ 46.359476] nix-daemon[847]: reaped child process 1023, status = succeeded4904lb # [ 46.355789] niks3-server[993]: 2026/09/21 14:29:43 INFO Received uploads request method=POST path=/api/pending_closures4905worker1 # [ 46.385259] nix-grpc-daemon[859]: time=2026-09-21T14:29:43.391Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4906worker1 # [ 46.390153] nix-grpc-daemon[859]: time=2026-09-21T14:29:43.393Z level=INFO msg="Uploading v688vnrkfcqiwaygym5ifmnr3yawh4xd-farm-top (232B)"4907lb # [ 46.401621] niks3-server[993]: 2026/09/21 14:29:43 INFO Registered completed upload object_key=nar/1j36mvvk44wddjhqvk302zl80d251m1g0d4kc4gfqy00xyklwzbf.nar.zst4908lb # [ 46.415907] niks3-server[993]: 2026/09/21 14:29:43 INFO Received sign narinfos request method=POST path=/api/pending_closures/8/sign4909lb # [ 46.423517] niks3-server[993]: 2026/09/21 14:29:43 INFO Signed narinfos id=8 count=14910worker1 # [ 46.436345] nix-grpc-daemon[859]: time=2026-09-21T14:29:43.442Z level=INFO msg="Uploading 1 narinfos"4911lb # [ 46.429612] niks3-server[993]: 2026/09/21 14:29:43 INFO Registered completed upload object_key=log/y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv4912lb # [ 46.433318] niks3-server[993]: 2026/09/21 14:29:43 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4913lb # [ 46.442315] niks3-server[993]: 2026/09/21 14:29:43 INFO Received complete upload request method=POST path=/api/pending_closures/8/complete4914lb # [ 46.445826] niks3-server[993]: 2026/09/21 14:29:43 INFO Completed upload id=84915lb # [ 46.447614] niks3-server[993]: 2026/09/21 14:29:43 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4916worker1 # [ 46.460350] nix-grpc-daemon[859]: time=2026-09-21T14:29:43.466Z level=INFO msg="Upload complete. (230ms)"4917worker1 # [ 46.467428] nix-grpc-daemon[781]: ts=2026-09-21T14:29:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14918worker1 # [ 46.480969] nix-daemon[847]: reaped child process 1002, status = succeeded4919client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 2.03 seconds)4920client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4921client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.10 seconds)4922lb: must succeed: curl -sf localhost:9901/clusters4923lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4924lb: must succeed: curl -sf localhost:9901/clusters4925lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4926worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4927worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4928(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.35 seconds)4929subtest: two clients wanting the same drv build it once4930client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4931client # [ 46.725847] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4932client # Running as unit: dedup1.service; invocation ID: a46714b35e4e4259ae5350f0bbca8f754933client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4934client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4935worker1 # [ 46.868095] nix-grpc-daemon[781]: ts=2026-09-21T14:29:43Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517904936client # [ 46.858656] nix[881]: don't know how to build these paths:4937client # [ 46.859962] nix[881]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4938worker2 # [ 46.893863] nix-grpc-daemon[782]: ts=2026-09-21T14:29:44Z level=debug event=want client=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04939worker2 # [ 46.899485] nix-grpc-daemon[782]: ts=2026-09-21T14:29:44Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14940worker2 # [ 46.906825] nix-grpc-daemon[782]: ts=2026-09-21T14:29:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483424941worker2 # [ 46.917560] nix-grpc-daemon[782]: ts=2026-09-21T14:29:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483424942client # [ 46.901075] nix[881]: copying 1 paths to 'lb:50051'...4943client # [ 46.901895] nix[881]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4944worker2 # [ 46.928678] nix-grpc-daemon[782]: ts=2026-09-21T14:29:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483424945worker2 # [ 46.939296] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4946worker2 # [ 46.958713] nix-grpc-daemon[782]: ts=2026-09-21T14:29:44Z level=info event=niks3_push_started pid=9204947worker2 # [ 47.089024] nix-grpc-daemon[920]: time=2026-09-21T14:29:44.205Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""4948worker2 # [ 47.180906] nix-daemon[867]: accepted connection from pid 933, user nix-grpc-daemon (trusted)4949worker2 # [ 47.194214] nix-daemon[867]: reaped child process 936, status = succeeded4950worker2 # [ 47.279110] nix-daemon[867]: accepted connection from pid 938, user nix-grpc-daemon (trusted)4951worker2 # [ 47.291934] nix-daemon[867]: reaped child process 941, status = succeeded4952lb # [ 47.274887] niks3-server[993]: 2026/09/21 14:29:44 INFO Received uploads request method=POST path=/api/pending_closures4953worker2 # [ 47.307071] nix-grpc-daemon[920]: time=2026-09-21T14:29:44.423Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4954worker2 # [ 47.309601] nix-grpc-daemon[920]: time=2026-09-21T14:29:44.425Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4955lb # [ 47.309529] niks3-server[993]: 2026/09/21 14:29:44 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4956lb # [ 47.319218] niks3-server[993]: 2026/09/21 14:29:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/9/sign4957worker2 # [ 47.351994] nix-grpc-daemon[920]: time=2026-09-21T14:29:44.468Z level=INFO msg="Uploading 1 narinfos"4958lb # [ 47.328881] niks3-server[993]: 2026/09/21 14:29:44 INFO Signed narinfos id=9 count=14959lb # [ 47.338383] niks3-server[993]: 2026/09/21 14:29:44 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4960lb # [ 47.347194] niks3-server[993]: 2026/09/21 14:29:44 INFO Received complete upload request method=POST path=/api/pending_closures/9/complete4961lb # [ 47.356424] niks3-server[993]: 2026/09/21 14:29:44 INFO Completed upload id=94962lb # [ 47.357681] niks3-server[993]: 2026/09/21 14:29:44 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4963worker2 # [ 47.380219] nix-grpc-daemon[920]: time=2026-09-21T14:29:44.496Z level=INFO msg="Upload complete. (286ms)"4964worker2 # [ 47.384646] nix-grpc-daemon[782]: ts=2026-09-21T14:29:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_in=4324965worker2 # [ 47.389362] nix-grpc-daemon[782]: ts=2026-09-21T14:29:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483424966worker2 # [ 47.397107] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4967worker2 # [ 47.409977] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)4968client # [ 47.383469] nix[881]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4969worker2 # [ 47.413378] nix-daemon[867]: reaped child process 918, status = succeeded4970client # [ 47.779689] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4971client # Running as unit: dedup2.service; invocation ID: e904691cd5dd4390af2763ceadfb5df34972client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4973client # [ 47.936407] nix[897]: don't know how to build these paths:4974client # [ 47.937343] nix[897]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4975worker1 # [ 47.951672] nix-grpc-daemon[781]: ts=2026-09-21T14:29:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517904976worker2 # [ 47.972402] nix-grpc-daemon[782]: ts=2026-09-21T14:29:45Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04977client # [ 47.951794] nix[897]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4978worker2 # [ 47.977797] nix-grpc-daemon[782]: ts=2026-09-21T14:29:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483424979client: must succeed: sleep 24980client: (finished: must succeed: sleep 2, in 2.04 seconds)4981client: waiting for success: ! systemctl is-active dedup1 dedup24982worker2 # [ 51.227574] nix-daemon[867]: reaped child process 945, status = succeeded4983worker2 # [ 51.301819] nix-daemon[867]: accepted connection from pid 977, user nix-grpc-daemon (trusted)4984worker2 # [ 51.313145] nix-daemon[867]: reaped child process 980, status = succeeded4985lb # [ 51.296925] niks3-server[993]: 2026/09/21 14:29:48 INFO Received uploads request method=POST path=/api/pending_closures4986worker2 # [ 51.331762] nix-grpc-daemon[920]: time=2026-09-21T14:29:48.448Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4987worker2 # [ 51.333298] nix-grpc-daemon[920]: time=2026-09-21T14:29:48.449Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4988lb # [ 51.334863] niks3-server[993]: 2026/09/21 14:29:48 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4989lb # [ 51.356188] niks3-server[993]: 2026/09/21 14:29:48 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4990lb # [ 51.363538] niks3-server[993]: 2026/09/21 14:29:48 INFO Received sign narinfos request method=POST path=/api/pending_closures/10/sign4991lb # [ 51.366925] niks3-server[993]: 2026/09/21 14:29:48 INFO Signed narinfos id=10 count=14992worker2 # [ 51.394135] nix-grpc-daemon[920]: time=2026-09-21T14:29:48.509Z level=INFO msg="Uploading 1 narinfos"4993lb # [ 51.372473] niks3-server[993]: 2026/09/21 14:29:48 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4994lb # [ 51.382357] niks3-server[993]: 2026/09/21 14:29:48 INFO Received complete upload request method=POST path=/api/pending_closures/10/complete4995lb # [ 51.386026] niks3-server[993]: 2026/09/21 14:29:48 INFO Completed upload id=104996worker2 # [ 51.409765] nix-grpc-daemon[920]: time=2026-09-21T14:29:48.526Z level=INFO msg="Upload complete. (188ms)"4997lb # [ 51.388576] niks3-server[993]: 2026/09/21 14:29:48 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4998worker2 # [ 51.419266] nix-grpc-daemon[782]: ts=2026-09-21T14:29:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=14999worker2 # [ 51.425334] nix-grpc-daemon[782]: ts=2026-09-21T14:29:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=5 outputs=15000worker2 # [ 51.436906] nix-daemon[867]: reaped child process 943, status = succeeded5001client # [ 51.432592] systemd[1]: dedup1.service: Deactivated successfully.5002client # [ 51.435989] systemd[1]: dedup1.service: Consumed 116ms CPU time over 4.709s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.5K outgoing IP traffic.5003client # [ 51.442562] systemd[1]: dedup2.service: Deactivated successfully.5004client # [ 51.446194] systemd[1]: dedup2.service: Consumed 134ms CPU time over 3.666s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5005client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.08 seconds)5006client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }5007client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)5008worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5009worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5010worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true5011worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)5012worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5013worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5014worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5015worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5016(finished: subtest: two clients wanting the same drv build it once, in 5.74 seconds)5017subtest: build hook: nix-daemon with builders = grpc://lb5018client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5019client # [ 52.561621] systemd[1]: Started Nix Daemon.5020client # [ 52.648346] nix-daemon[939]: accepted connection from pid 936, user root (trusted)5021worker2 # [ 52.791832] nix-grpc-daemon[782]: ts=2026-09-21T14:29:49Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:483425022worker1 # [ 52.785395] nix-grpc-daemon[781]: ts=2026-09-21T14:29:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517905023worker2 # [ 52.811104] nix-grpc-daemon[782]: ts=2026-09-21T14:29:49Z level=debug event=want client=6 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05024worker2 # [ 52.815341] nix-grpc-daemon[782]: ts=2026-09-21T14:29:49Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15025worker1 # [ 52.844719] nix-grpc-daemon[781]: ts=2026-09-21T14:29:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517905026worker1 # [ 52.856616] nix-grpc-daemon[781]: ts=2026-09-21T14:29:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517905027worker1 # [ 52.868822] nix-grpc-daemon[781]: ts=2026-09-21T14:29:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517905028worker1 # [ 52.880590] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5029worker1 # [ 52.985813] nix-daemon[847]: accepted connection from pid 1048, user nix-grpc-daemon (trusted)5030worker1 # [ 53.001339] nix-daemon[847]: reaped child process 1051, status = succeeded5031worker1 # [ 53.083616] nix-daemon[847]: accepted connection from pid 1053, user nix-grpc-daemon (trusted)5032worker1 # [ 53.097969] nix-daemon[847]: reaped child process 1056, status = succeeded5033lb # [ 53.091884] niks3-server[993]: 2026/09/21 14:29:50 INFO Received uploads request method=POST path=/api/pending_closures5034worker1 # [ 53.122242] nix-grpc-daemon[859]: time=2026-09-21T14:29:50.128Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5035worker1 # [ 53.123839] nix-grpc-daemon[859]: time=2026-09-21T14:29:50.130Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5036lb # [ 53.133922] niks3-server[993]: 2026/09/21 14:29:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/11/sign5037lb # [ 53.140663] niks3-server[993]: 2026/09/21 14:29:50 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5038worker1 # [ 53.159197] nix-grpc-daemon[859]: time=2026-09-21T14:29:50.165Z level=INFO msg="Uploading 1 narinfos"5039lb # [ 53.145932] niks3-server[993]: 2026/09/21 14:29:50 INFO Signed narinfos id=11 count=15040lb # [ 53.159295] niks3-server[993]: 2026/09/21 14:29:50 INFO Received complete upload request method=POST path=/api/pending_closures/11/complete5041lb # [ 53.166076] niks3-server[993]: 2026/09/21 14:29:50 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5042lb # [ 53.174236] niks3-server[993]: 2026/09/21 14:29:50 INFO Completed upload id=115043worker1 # [ 53.189989] nix-grpc-daemon[859]: time=2026-09-21T14:29:50.196Z level=INFO msg="Upload complete. (297ms)"5044lb # [ 53.178749] niks3-server[993]: 2026/09/21 14:29:50 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5045worker1 # [ 53.193833] nix-grpc-daemon[781]: ts=2026-09-21T14:29:50Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_in=5125046worker1 # [ 53.200396] nix-grpc-daemon[781]: ts=2026-09-21T14:29:50Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517905047worker1 # [ 53.212178] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5048worker1 # [ 53.222216] nix-daemon[847]: reaped child process 1046, status = succeeded5049worker1 # [ 53.226232] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5050worker1 # [ 54.041959] nix-daemon[847]: reaped child process 1060, status = succeeded5051worker1 # [ 54.117203] nix-daemon[847]: accepted connection from pid 1076, user nix-grpc-daemon (trusted)5052worker1 # [ 54.131174] nix-daemon[847]: reaped child process 1079, status = succeeded5053lb # [ 54.125411] niks3-server[993]: 2026/09/21 14:29:51 INFO Received uploads request method=POST path=/api/pending_closures5054worker1 # [ 54.146474] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.152Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5055worker1 # [ 54.147967] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.154Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5056lb # [ 54.160402] niks3-server[993]: 2026/09/21 14:29:51 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5057lb # [ 54.176172] niks3-server[993]: 2026/09/21 14:29:51 INFO Received sign narinfos request method=POST path=/api/pending_closures/12/sign5058lb # [ 54.184805] niks3-server[993]: 2026/09/21 14:29:51 INFO Signed narinfos id=12 count=15059worker1 # [ 54.200765] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.207Z level=INFO msg="Uploading 1 narinfos"5060lb # [ 54.188916] niks3-server[993]: 2026/09/21 14:29:51 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5061lb # [ 54.196476] niks3-server[993]: 2026/09/21 14:29:51 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5062lb # [ 54.205198] niks3-server[993]: 2026/09/21 14:29:51 INFO Received complete upload request method=POST path=/api/pending_closures/12/complete5063lb # [ 54.209303] niks3-server[993]: 2026/09/21 14:29:51 INFO Completed upload id=125064worker1 # [ 54.222355] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.228Z level=INFO msg="Upload complete. (189ms)"5065lb # [ 54.210953] niks3-server[993]: 2026/09/21 14:29:51 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5066worker1 # [ 54.228524] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=6 outputs=15067worker2 # [ 54.242976] nix-grpc-daemon[782]: ts=2026-09-21T14:29:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:483425068worker1 # [ 54.240194] nix-daemon[847]: reaped child process 1058, status = succeeded5069worker2 # [ 54.252522] nix-grpc-daemon[782]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5070worker1 # [ 54.267242] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:517905071worker1 # [ 54.271619] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_out=1285072worker1 # [ 54.476893] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:517905073worker1 # [ 54.483352] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517905074worker1 # [ 54.488380] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.493Z level=INFO msg="All 1 paths already cached"5075worker2 # [ 54.508621] nix-grpc-daemon[782]: ts=2026-09-21T14:29:51Z level=debug event=want client=7 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05076worker2 # [ 54.513564] nix-grpc-daemon[782]: ts=2026-09-21T14:29:51Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15077worker1 # [ 54.505544] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517905078worker1 # [ 54.517130] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517905079worker1 # [ 54.528972] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.535Z level=INFO msg="All 1 paths already cached"5080worker1 # [ 54.536240] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517905081worker1 # [ 54.546181] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5082worker1 # [ 54.646655] nix-daemon[847]: accepted connection from pid 1083, user nix-grpc-daemon (trusted)5083worker1 # [ 54.661223] nix-daemon[847]: reaped child process 1086, status = succeeded5084worker1 # [ 54.744283] nix-daemon[847]: accepted connection from pid 1088, user nix-grpc-daemon (trusted)5085worker1 # [ 54.758706] nix-daemon[847]: reaped child process 1091, status = succeeded5086lb # [ 54.752662] niks3-server[993]: 2026/09/21 14:29:51 INFO Received uploads request method=POST path=/api/pending_closures5087worker1 # [ 54.854314] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.860Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5088worker1 # [ 54.857778] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.864Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5089lb # [ 54.870755] niks3-server[993]: 2026/09/21 14:29:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/13/sign5090lb # [ 54.877872] niks3-server[993]: 2026/09/21 14:29:52 INFO Signed narinfos id=13 count=15091worker1 # [ 54.894623] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.900Z level=INFO msg="Uploading 1 narinfos"5092lb # [ 54.885249] niks3-server[993]: 2026/09/21 14:29:52 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5093lb # [ 54.890856] niks3-server[993]: 2026/09/21 14:29:52 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5094lb # [ 54.901313] niks3-server[993]: 2026/09/21 14:29:52 INFO Received complete upload request method=POST path=/api/pending_closures/13/complete5095lb # [ 54.910170] niks3-server[993]: 2026/09/21 14:29:52 INFO Completed upload id=135096worker1 # [ 54.923278] nix-grpc-daemon[859]: time=2026-09-21T14:29:51.929Z level=INFO msg="Upload complete. (363ms)"5097lb # [ 54.912831] niks3-server[993]: 2026/09/21 14:29:52 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5098worker1 # [ 54.928232] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_in=6245099worker1 # [ 54.935133] nix-grpc-daemon[781]: ts=2026-09-21T14:29:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517905100worker1 # [ 54.946591] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5101worker1 # [ 54.960698] nix-daemon[847]: reaped child process 1081, status = succeeded5102worker1 # [ 54.968281] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5103lb # [ 55.647441] envoy[906]: conn peer=[2001:db8:1::1]:43410 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4195 tx=1783 ms=18735104lb # [ 55.651308] envoy[906]: conn peer=[2001:db8:1::1]:43412 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3011 tx=1451 ms=45495105lb # [ 55.654810] envoy[906]: conn peer=[2001:db8:1::1]:43420 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1082 ms=34725106lb # [ 55.658388] envoy[906]: conn peer=[2001:db8:1::1]:44568 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=565107lb # [ 55.661842] envoy[906]: conn peer=[2001:db8:1::1]:43428 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3421 tx=1852 ms=15435108worker1 # [ 56.412674] nix-daemon[847]: reaped child process 1095, status = succeeded5109worker1 # [ 56.492103] nix-daemon[847]: accepted connection from pid 1111, user nix-grpc-daemon (trusted)5110worker1 # [ 56.506251] nix-daemon[847]: reaped child process 1114, status = succeeded5111lb # [ 56.499491] niks3-server[993]: 2026/09/21 14:29:53 INFO Received uploads request method=POST path=/api/pending_closures5112worker1 # [ 56.521220] nix-grpc-daemon[859]: time=2026-09-21T14:29:53.527Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5113worker1 # [ 56.522683] nix-grpc-daemon[859]: time=2026-09-21T14:29:53.529Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5114lb # [ 56.541340] niks3-server[993]: 2026/09/21 14:29:53 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5115lb # [ 56.553427] niks3-server[993]: 2026/09/21 14:29:53 INFO Received sign narinfos request method=POST path=/api/pending_closures/14/sign5116lb # [ 56.561503] niks3-server[993]: 2026/09/21 14:29:53 INFO Signed narinfos id=14 count=15117worker1 # [ 56.574853] nix-grpc-daemon[859]: time=2026-09-21T14:29:53.581Z level=INFO msg="Uploading 1 narinfos"5118lb # [ 56.565682] niks3-server[993]: 2026/09/21 14:29:53 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5119lb # [ 56.570556] niks3-server[993]: 2026/09/21 14:29:53 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5120lb # [ 56.580372] niks3-server[993]: 2026/09/21 14:29:53 INFO Received complete upload request method=POST path=/api/pending_closures/14/complete5121lb # [ 56.583254] niks3-server[993]: 2026/09/21 14:29:53 INFO Completed upload id=145122worker1 # [ 56.596251] nix-grpc-daemon[859]: time=2026-09-21T14:29:53.602Z level=INFO msg="Upload complete. (215ms)"5123lb # [ 56.585946] niks3-server[993]: 2026/09/21 14:29:53 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5124worker1 # [ 56.603139] nix-grpc-daemon[781]: ts=2026-09-21T14:29:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=7 outputs=15125worker1 # [ 56.610233] nix-grpc-daemon[781]: ts=2026-09-21T14:29:53Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:517905126worker1 # [ 56.621770] nix-daemon[847]: reaped child process 1093, status = succeeded5127worker2 # [ 56.633329] nix-grpc-daemon[782]: ts=2026-09-21T14:29:53Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:483425128worker2 # [ 56.642483] nix-grpc-daemon[782]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5129worker2 # [ 56.653541] nix-grpc-daemon[782]: ts=2026-09-21T14:29:53Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_out=1765130worker2 # [ 56.862137] nix-grpc-daemon[782]: ts=2026-09-21T14:29:53Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:483425131worker2 # [ 56.868933] nix-grpc-daemon[782]: ts=2026-09-21T14:29:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483425132worker2 # [ 56.874971] nix-grpc-daemon[920]: time=2026-09-21T14:29:53.990Z level=INFO msg="All 2 paths already cached"5133worker2 # [ 56.884824] nix-grpc-daemon[782]: ts=2026-09-21T14:29:54Z level=debug event=want client=8 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05134worker2 # [ 56.889125] nix-grpc-daemon[782]: ts=2026-09-21T14:29:54Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15135worker1 # [ 56.919506] nix-grpc-daemon[781]: ts=2026-09-21T14:29:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517905136worker1 # [ 56.932664] nix-grpc-daemon[781]: ts=2026-09-21T14:29:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517905137worker1 # [ 56.946644] nix-grpc-daemon[859]: time=2026-09-21T14:29:53.953Z level=INFO msg="All 2 paths already cached"5138worker1 # [ 56.953922] nix-grpc-daemon[781]: ts=2026-09-21T14:29:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517905139worker1 # [ 56.964166] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5140worker1 # [ 57.067164] nix-daemon[847]: accepted connection from pid 1118, user nix-grpc-daemon (trusted)5141worker1 # [ 57.081743] nix-daemon[847]: reaped child process 1121, status = succeeded5142worker1 # [ 57.165337] nix-daemon[847]: accepted connection from pid 1123, user nix-grpc-daemon (trusted)5143worker1 # [ 57.179674] nix-daemon[847]: reaped child process 1126, status = succeeded5144lb # [ 57.172648] niks3-server[993]: 2026/09/21 14:29:54 INFO Received uploads request method=POST path=/api/pending_closures5145worker1 # [ 57.194814] nix-grpc-daemon[859]: time=2026-09-21T14:29:54.201Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5146worker1 # [ 57.196586] nix-grpc-daemon[859]: time=2026-09-21T14:29:54.202Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5147lb # [ 57.202594] niks3-server[993]: 2026/09/21 14:29:54 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5148lb # [ 57.214352] niks3-server[993]: 2026/09/21 14:29:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/15/sign5149lb # [ 57.218673] niks3-server[993]: 2026/09/21 14:29:54 INFO Signed narinfos id=15 count=15150worker1 # [ 57.234874] nix-grpc-daemon[859]: time=2026-09-21T14:29:54.241Z level=INFO msg="Uploading 1 narinfos"5151lb # [ 57.232554] niks3-server[993]: 2026/09/21 14:29:54 INFO Received complete upload request method=POST path=/api/pending_closures/15/complete5152lb # [ 57.236056] niks3-server[993]: 2026/09/21 14:29:54 INFO Completed upload id=155153worker1 # [ 57.249199] nix-grpc-daemon[859]: time=2026-09-21T14:29:54.255Z level=INFO msg="Upload complete. (273ms)"5154lb # [ 57.238562] niks3-server[993]: 2026/09/21 14:29:54 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5155worker1 # [ 57.251356] nix-grpc-daemon[781]: ts=2026-09-21T14:29:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_in=7525156worker1 # [ 57.255033] nix-grpc-daemon[781]: ts=2026-09-21T14:29:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517905157lb # [ 57.243851] niks3-server[993]: 2026/09/21 14:29:54 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5158worker1 # [ 57.262399] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5159worker1 # [ 57.264835] nix-daemon[847]: reaped child process 1116, status = succeeded5160worker1 # [ 57.269892] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5161worker1 # [ 58.486091] nix-daemon[847]: reaped child process 1130, status = succeeded5162worker1 # [ 58.564107] nix-daemon[847]: accepted connection from pid 1146, user nix-grpc-daemon (trusted)5163worker1 # [ 58.576449] nix-daemon[847]: reaped child process 1149, status = succeeded5164lb # [ 58.571882] niks3-server[993]: 2026/09/21 14:29:55 INFO Received uploads request method=POST path=/api/pending_closures5165worker1 # [ 58.593736] nix-grpc-daemon[859]: time=2026-09-21T14:29:55.600Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5166worker1 # [ 58.595275] nix-grpc-daemon[859]: time=2026-09-21T14:29:55.601Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5167lb # [ 58.615449] niks3-server[993]: 2026/09/21 14:29:55 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5168lb # [ 58.626997] niks3-server[993]: 2026/09/21 14:29:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/16/sign5169worker1 # [ 58.650400] nix-grpc-daemon[859]: time=2026-09-21T14:29:55.656Z level=INFO msg="Uploading 1 narinfos"5170lb # [ 58.636877] niks3-server[993]: 2026/09/21 14:29:55 INFO Signed narinfos id=16 count=15171lb # [ 58.640817] niks3-server[993]: 2026/09/21 14:29:55 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5172lb # [ 58.645220] niks3-server[993]: 2026/09/21 14:29:55 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5173lb # [ 58.654279] niks3-server[993]: 2026/09/21 14:29:55 INFO Received complete upload request method=POST path=/api/pending_closures/16/complete5174lb # [ 58.659884] niks3-server[993]: 2026/09/21 14:29:55 INFO Completed upload id=165175worker1 # [ 58.673216] nix-grpc-daemon[859]: time=2026-09-21T14:29:55.679Z level=INFO msg="Upload complete. (193ms)"5176lb # [ 58.661976] niks3-server[993]: 2026/09/21 14:29:55 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5177worker1 # [ 58.679539] nix-grpc-daemon[781]: ts=2026-09-21T14:29:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=8 outputs=15178worker2 # [ 58.696137] nix-grpc-daemon[782]: ts=2026-09-21T14:29:55Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:483425179worker1 # [ 58.694975] nix-daemon[847]: reaped child process 1128, status = succeeded5180worker2 # [ 58.704412] nix-grpc-daemon[782]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5181worker2 # [ 58.726994] nix-grpc-daemon[782]: ts=2026-09-21T14:29:55Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:483425182worker2 # [ 58.730804] nix-grpc-daemon[782]: ts=2026-09-21T14:29:55Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_out=2325183client # [ 58.813537] nix-daemon[939]: reaped child process 941, status = succeeded5184client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 6.38 seconds)5185client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5186client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5187client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5188client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5189client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5190client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5191(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 6.44 seconds)5192subtest: an input only one worker has still reaches the builder5193client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5194client # this derivation will be built:5195client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5196client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5197client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.23 seconds)5198client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5199client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.09 seconds)5200worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5201worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5202worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5203worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5204client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25205client # [ 59.471409] nix-daemon[939]: accepted connection from pid 1022, user root (trusted)5206client # this derivation will be built:5207client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5208worker1 # [ 59.603199] nix-grpc-daemon[781]: ts=2026-09-21T14:29:56Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:517905209client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5210worker2 # [ 59.617165] nix-grpc-daemon[782]: ts=2026-09-21T14:29:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483425211client # copying 1 paths...5212client # copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' to 'grpc://lb:50051'...5213worker2 # [ 59.629468] nix-grpc-daemon[782]: ts=2026-09-21T14:29:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483425214worker2 # [ 59.640186] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5215worker2 # [ 59.747725] nix-daemon[867]: accepted connection from pid 1005, user nix-grpc-daemon (trusted)5216worker2 # [ 59.765383] nix-daemon[867]: reaped child process 1008, status = succeeded5217lb # [ 59.746930] niks3-server[993]: 2026/09/21 14:29:56 INFO Received uploads request method=POST path=/api/pending_closures5218worker2 # [ 59.779735] nix-grpc-daemon[920]: time=2026-09-21T14:29:56.896Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5219worker2 # [ 59.781569] nix-grpc-daemon[920]: time=2026-09-21T14:29:56.897Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5220lb # [ 59.779980] niks3-server[993]: 2026/09/21 14:29:56 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5221lb # [ 59.789623] niks3-server[993]: 2026/09/21 14:29:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/17/sign5222lb # [ 59.793949] niks3-server[993]: 2026/09/21 14:29:56 INFO Signed narinfos id=17 count=15223worker2 # [ 59.821548] nix-grpc-daemon[920]: time=2026-09-21T14:29:56.938Z level=INFO msg="Uploading 1 narinfos"5224lb # [ 59.801718] niks3-server[993]: 2026/09/21 14:29:56 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5225lb # [ 59.812968] niks3-server[993]: 2026/09/21 14:29:56 INFO Received complete upload request method=POST path=/api/pending_closures/17/complete5226lb # [ 59.820329] niks3-server[993]: 2026/09/21 14:29:57 INFO Completed upload id=175227worker2 # [ 59.845501] nix-grpc-daemon[920]: time=2026-09-21T14:29:56.962Z level=INFO msg="Upload complete. (194ms)"5228lb # [ 59.823883] niks3-server[993]: 2026/09/21 14:29:57 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5229worker2 # [ 59.850621] nix-grpc-daemon[782]: ts=2026-09-21T14:29:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_in=1205230worker2 # [ 59.863093] nix-grpc-daemon[782]: ts=2026-09-21T14:29:56Z level=debug event=want client=9 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05231worker2 # [ 59.867181] nix-grpc-daemon[782]: ts=2026-09-21T14:29:56Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15232worker2 # [ 59.875295] nix-daemon[867]: reaped child process 1003, status = succeeded5233worker2 # [ 59.907162] nix-grpc-daemon[782]: ts=2026-09-21T14:29:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483425234worker2 # [ 59.919602] nix-grpc-daemon[782]: ts=2026-09-21T14:29:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483425235client # copying 2 paths to 'lb:50051'...5236client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5237worker2 # [ 59.936436] nix-grpc-daemon[782]: ts=2026-09-21T14:29:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483425238client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5239worker2 # [ 59.947906] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5240worker2 # [ 60.044662] nix-daemon[867]: accepted connection from pid 1014, user nix-grpc-daemon (trusted)5241worker2 # [ 60.059172] nix-daemon[867]: reaped child process 1017, status = succeeded5242worker2 # [ 60.140090] nix-daemon[867]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)5243worker2 # [ 60.156583] nix-daemon[867]: reaped child process 1022, status = succeeded5244lb # [ 60.138812] niks3-server[993]: 2026/09/21 14:29:57 INFO Received uploads request method=POST path=/api/pending_closures5245lb # [ 60.147453] niks3-server[993]: 2026/09/21 14:29:57 INFO Received uploads request method=POST path=/api/pending_closures5246worker2 # [ 60.180546] nix-grpc-daemon[920]: time=2026-09-21T14:29:57.297Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5247worker2 # [ 60.182814] nix-grpc-daemon[920]: time=2026-09-21T14:29:57.298Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5248worker2 # [ 60.184464] nix-grpc-daemon[920]: time=2026-09-21T14:29:57.298Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5249lb # [ 60.186322] niks3-server[993]: 2026/09/21 14:29:57 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5250lb # [ 60.191108] niks3-server[993]: 2026/09/21 14:29:57 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5251lb # [ 60.203130] niks3-server[993]: 2026/09/21 14:29:57 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5252lb # [ 60.210289] niks3-server[993]: 2026/09/21 14:29:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/18/sign5253lb # [ 60.212393] niks3-server[993]: 2026/09/21 14:29:57 INFO Signed narinfos id=18 count=15254lb # [ 60.214958] niks3-server[993]: 2026/09/21 14:29:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/19/sign5255lb # [ 60.218175] niks3-server[993]: 2026/09/21 14:29:57 INFO Signed narinfos id=19 count=25256worker2 # [ 60.242837] nix-grpc-daemon[920]: time=2026-09-21T14:29:57.358Z level=INFO msg="Uploading 3 narinfos"5257lb # [ 60.225725] niks3-server[993]: 2026/09/21 14:29:57 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5258lb # [ 60.230760] niks3-server[993]: 2026/09/21 14:29:57 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5259lb # [ 60.237862] niks3-server[993]: 2026/09/21 14:29:57 INFO Received complete upload request method=POST path=/api/pending_closures/18/complete5260lb # [ 60.240307] niks3-server[993]: 2026/09/21 14:29:57 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5261lb # [ 60.243925] niks3-server[993]: 2026/09/21 14:29:57 INFO Completed upload id=185262lb # [ 60.246296] niks3-server[993]: 2026/09/21 14:29:57 INFO Received complete upload request method=POST path=/api/pending_closures/19/complete5263lb # [ 60.250967] niks3-server[993]: 2026/09/21 14:29:57 INFO Completed upload id=195264lb # [ 60.252384] niks3-server[993]: 2026/09/21 14:29:57 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5265worker2 # [ 60.274697] nix-grpc-daemon[920]: time=2026-09-21T14:29:57.391Z level=INFO msg="Upload complete. (312ms)"5266worker2 # [ 60.279595] nix-grpc-daemon[782]: ts=2026-09-21T14:29:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=2 nar_bytes_in=9765267worker2 # [ 60.285533] nix-grpc-daemon[782]: ts=2026-09-21T14:29:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483425268worker2 # [ 60.297221] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5269worker2 # [ 60.305557] nix-daemon[867]: reaped child process 1012, status = succeeded5270client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5271worker2 # [ 60.314330] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5272client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5273client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5274worker2 # [ 61.129253] nix-daemon[867]: reaped child process 1026, status = succeeded5275worker2 # [ 61.202384] nix-daemon[867]: accepted connection from pid 1042, user nix-grpc-daemon (trusted)5276worker2 # [ 61.214718] nix-daemon[867]: reaped child process 1045, status = succeeded5277lb # [ 61.199254] niks3-server[993]: 2026/09/21 14:29:58 INFO Received uploads request method=POST path=/api/pending_closures5278worker2 # [ 61.231154] nix-grpc-daemon[920]: time=2026-09-21T14:29:58.347Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5279lb # [ 61.230939] niks3-server[993]: 2026/09/21 14:29:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/20/sign5280worker2 # [ 61.262446] nix-grpc-daemon[920]: time=2026-09-21T14:29:58.379Z level=INFO msg="Uploading 1 narinfos"5281lb # [ 61.239498] niks3-server[993]: 2026/09/21 14:29:58 INFO Signed narinfos id=20 count=15282lb # [ 61.246384] niks3-server[993]: 2026/09/21 14:29:58 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5283lb # [ 61.251933] niks3-server[993]: 2026/09/21 14:29:58 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5284lb # [ 61.264715] niks3-server[993]: 2026/09/21 14:29:58 INFO Received complete upload request method=POST path=/api/pending_closures/20/complete5285lb # [ 61.276669] niks3-server[993]: 2026/09/21 14:29:58 INFO Completed upload id=205286worker2 # [ 61.300324] nix-grpc-daemon[920]: time=2026-09-21T14:29:58.416Z level=INFO msg="Upload complete. (178ms)"5287lb # [ 61.279423] niks3-server[993]: 2026/09/21 14:29:58 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5288worker2 # [ 61.304747] nix-grpc-daemon[782]: ts=2026-09-21T14:29:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=9 outputs=15289worker2 # [ 61.317810] nix-daemon[867]: reaped child process 1024, status = succeeded5290client # copying 1 paths...5291worker1 # [ 61.345850] nix-grpc-daemon[781]: ts=2026-09-21T14:29:58Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:517905292worker1 # [ 61.358188] nix-grpc-daemon[781]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5293client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5294worker2 # [ 61.402275] nix-grpc-daemon[782]: ts=2026-09-21T14:29:58Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:483425295worker2 # [ 61.406678] nix-grpc-daemon[782]: ts=2026-09-21T14:29:58Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_out=1205296client # [ 61.452954] nix-daemon[939]: reaped child process 1025, status = succeeded5297client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 2.10 seconds)5298client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25299client # [ 61.553872] nix-daemon[939]: accepted connection from pid 1044, user root (trusted)5300client # this derivation will be built:5301client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5302worker1 # [ 61.683161] nix-grpc-daemon[781]: ts=2026-09-21T14:29:58Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:517905303client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5304worker2 # [ 61.701659] nix-grpc-daemon[782]: ts=2026-09-21T14:29:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483425305worker2 # [ 61.710078] nix-grpc-daemon[920]: time=2026-09-21T14:29:58.826Z level=INFO msg="All 1 paths already cached"5306worker2 # [ 61.724232] nix-grpc-daemon[782]: ts=2026-09-21T14:29:58Z level=debug event=want client=10 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05307worker1 # [ 61.716163] nix-grpc-daemon[781]: ts=2026-09-21T14:29:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517905308worker2 # [ 61.728665] nix-grpc-daemon[782]: ts=2026-09-21T14:29:58Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15309worker1 # [ 61.728475] nix-grpc-daemon[781]: ts=2026-09-21T14:29:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517905310client # copying 1 paths to 'lb:50051'...5311client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5312worker1 # [ 61.745360] nix-grpc-daemon[781]: ts=2026-09-21T14:29:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:517905313worker1 # [ 61.757291] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5314worker1 # [ 61.773235] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 25 ms (attempt 1/5)5315worker1 # [ 61.805805] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 121 ms (attempt 2/5)5316worker1 # [ 61.929374] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 62 ms (attempt 3/5)5317worker1 # [ 61.993534] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 517 ms (attempt 4/5)5318worker1 # [ 62.513790] nix-grpc-daemon[781]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5319worker1 # [ 62.517657] nix-grpc-daemon[781]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5320worker1 # [ 62.522821] nix-grpc-daemon[781]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5321worker1 # [ 62.642238] nix-daemon[847]: accepted connection from pid 1164, user nix-grpc-daemon (trusted)5322worker1 # [ 62.660064] nix-daemon[847]: reaped child process 1167, status = succeeded5323worker1 # [ 62.751588] nix-daemon[847]: accepted connection from pid 1169, user nix-grpc-daemon (trusted)5324worker1 # [ 62.772187] nix-daemon[847]: reaped child process 1172, status = succeeded5325lb # [ 62.763648] niks3-server[993]: 2026/09/21 14:29:59 INFO Received uploads request method=POST path=/api/pending_closures5326worker1 # [ 62.786565] nix-grpc-daemon[859]: time=2026-09-21T14:29:59.792Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5327worker1 # [ 62.788722] nix-grpc-daemon[859]: time=2026-09-21T14:29:59.794Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5328lb # [ 62.793250] niks3-server[993]: 2026/09/21 14:29:59 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5329lb # [ 62.805730] niks3-server[993]: 2026/09/21 14:29:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/21/sign5330lb # [ 62.810514] niks3-server[993]: 2026/09/21 14:29:59 INFO Signed narinfos id=21 count=15331worker1 # [ 62.826334] nix-grpc-daemon[859]: time=2026-09-21T14:29:59.832Z level=INFO msg="Uploading 1 narinfos"5332lb # [ 62.820906] niks3-server[993]: 2026/09/21 14:30:00 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5333lb # [ 62.830069] niks3-server[993]: 2026/09/21 14:30:00 INFO Received complete upload request method=POST path=/api/pending_closures/21/complete5334lb # [ 62.832861] niks3-server[993]: 2026/09/21 14:30:00 INFO Completed upload id=215335worker1 # [ 62.845856] nix-grpc-daemon[859]: time=2026-09-21T14:29:59.852Z level=INFO msg="Upload complete. (304ms)"5336lb # [ 62.834571] niks3-server[993]: 2026/09/21 14:30:00 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5337worker1 # [ 62.848072] nix-grpc-daemon[781]: ts=2026-09-21T14:29:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 paths=1 nar_bytes_in=5605338worker1 # [ 62.852552] nix-grpc-daemon[781]: ts=2026-09-21T14:29:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:517905339worker1 # [ 62.859112] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5340worker1 # [ 62.865164] nix-daemon[847]: reaped child process 1157, status = succeeded5341worker1 # [ 62.868087] nix-daemon[847]: accepted connection from pid 781, user nix-grpc-daemon (trusted)5342client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5343client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5344client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5345worker1 # [ 63.699852] nix-daemon[847]: reaped child process 1176, status = succeeded5346worker1 # [ 63.783730] nix-daemon[847]: accepted connection from pid 1192, user nix-grpc-daemon (trusted)5347worker1 # [ 63.796573] nix-daemon[847]: reaped child process 1195, status = succeeded5348lb # [ 63.791723] niks3-server[993]: 2026/09/21 14:30:00 INFO Received uploads request method=POST path=/api/pending_closures5349worker1 # [ 63.812823] nix-grpc-daemon[859]: time=2026-09-21T14:30:00.819Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5350lb # [ 63.824386] niks3-server[993]: 2026/09/21 14:30:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/22/sign5351worker1 # [ 63.845911] nix-grpc-daemon[859]: time=2026-09-21T14:30:00.852Z level=INFO msg="Uploading 1 narinfos"5352lb # [ 63.833466] niks3-server[993]: 2026/09/21 14:30:01 INFO Signed narinfos id=22 count=15353lb # [ 63.837944] niks3-server[993]: 2026/09/21 14:30:01 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5354lb # [ 63.844629] niks3-server[993]: 2026/09/21 14:30:01 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5355lb # [ 63.856550] niks3-server[993]: 2026/09/21 14:30:01 INFO Received complete upload request method=POST path=/api/pending_closures/22/complete5356lb # [ 63.863862] niks3-server[993]: 2026/09/21 14:30:01 INFO Completed upload id=225357lb # [ 63.866378] niks3-server[993]: 2026/09/21 14:30:01 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5358worker1 # [ 63.876816] nix-grpc-daemon[859]: time=2026-09-21T14:30:00.883Z level=INFO msg="Upload complete. (200ms)"5359client # copying 1 paths...5360worker1 # [ 63.883320] nix-grpc-daemon[781]: ts=2026-09-21T14:30:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=10 outputs=15361worker2 # [ 63.896599] nix-grpc-daemon[782]: ts=2026-09-21T14:30:01Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:483425362worker1 # [ 63.896638] nix-daemon[847]: reaped child process 1174, status = succeeded5363worker2 # [ 63.906071] nix-grpc-daemon[782]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5364client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5365worker1 # [ 63.923152] nix-grpc-daemon[781]: ts=2026-09-21T14:30:00Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:517905366worker1 # [ 63.927937] nix-grpc-daemon[781]: ts=2026-09-21T14:30:00Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:51790 duration_s=0 paths=1 nar_bytes_out=1205367client # [ 63.979402] nix-daemon[939]: reaped child process 1047, status = succeeded5368client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.53 seconds)5369client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25370client # [ 64.080709] nix-daemon[939]: accepted connection from pid 1066, user root (trusted)5371client # this derivation will be built:5372client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5373worker1 # [ 64.211758] nix-grpc-daemon[781]: ts=2026-09-21T14:30:01Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:517905374client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5375worker1 # [ 64.217737] nix-grpc-daemon[781]: ts=2026-09-21T14:30:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:517905376worker1 # [ 64.222628] nix-grpc-daemon[859]: time=2026-09-21T14:30:01.228Z level=INFO msg="All 1 paths already cached"5377worker2 # [ 64.244205] nix-grpc-daemon[782]: ts=2026-09-21T14:30:01Z level=debug event=want client=11 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05378worker2 # [ 64.252262] nix-grpc-daemon[782]: ts=2026-09-21T14:30:01Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15379worker2 # [ 64.257400] nix-grpc-daemon[782]: ts=2026-09-21T14:30:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483425380worker2 # [ 64.270569] nix-grpc-daemon[782]: ts=2026-09-21T14:30:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:483425381worker2 # [ 64.284759] nix-grpc-daemon[920]: time=2026-09-21T14:30:01.400Z level=INFO msg="All 1 paths already cached"5382client # copying 1 paths to 'lb:50051'...5383client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5384worker2 # [ 64.293087] nix-grpc-daemon[782]: ts=2026-09-21T14:30:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:483425385worker2 # [ 64.304296] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5386worker2 # [ 64.402840] nix-daemon[867]: accepted connection from pid 1054, user nix-grpc-daemon (trusted)5387worker2 # [ 64.417427] nix-daemon[867]: reaped child process 1057, status = succeeded5388worker2 # [ 64.499989] nix-daemon[867]: accepted connection from pid 1059, user nix-grpc-daemon (trusted)5389worker2 # [ 64.513151] nix-daemon[867]: reaped child process 1062, status = succeeded5390lb # [ 64.496923] niks3-server[993]: 2026/09/21 14:30:01 INFO Received uploads request method=POST path=/api/pending_closures5391worker2 # [ 64.528854] nix-grpc-daemon[920]: time=2026-09-21T14:30:01.645Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5392worker2 # [ 64.530194] nix-grpc-daemon[920]: time=2026-09-21T14:30:01.645Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5393lb # [ 64.529724] niks3-server[993]: 2026/09/21 14:30:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/23/sign5394lb # [ 64.536565] niks3-server[993]: 2026/09/21 14:30:01 INFO Signed narinfos id=23 count=15395worker2 # [ 64.563444] nix-grpc-daemon[920]: time=2026-09-21T14:30:01.679Z level=INFO msg="Uploading 1 narinfos"5396lb # [ 64.541805] niks3-server[993]: 2026/09/21 14:30:01 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5397lb # [ 64.551251] niks3-server[993]: 2026/09/21 14:30:01 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5398lb # [ 64.562854] niks3-server[993]: 2026/09/21 14:30:01 INFO Received complete upload request method=POST path=/api/pending_closures/23/complete5399lb # [ 64.569340] niks3-server[993]: 2026/09/21 14:30:01 INFO Completed upload id=235400worker2 # [ 64.593126] nix-grpc-daemon[920]: time=2026-09-21T14:30:01.709Z level=INFO msg="Upload complete. (276ms)"5401lb # [ 64.574180] niks3-server[993]: 2026/09/21 14:30:01 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5402worker2 # [ 64.597195] nix-grpc-daemon[782]: ts=2026-09-21T14:30:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_in=5605403worker2 # [ 64.603303] nix-grpc-daemon[782]: ts=2026-09-21T14:30:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:483425404worker2 # [ 64.612554] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5405worker2 # [ 64.620499] nix-daemon[867]: reaped child process 1052, status = succeeded5406worker2 # [ 64.623840] nix-daemon[867]: accepted connection from pid 782, user nix-grpc-daemon (trusted)5407client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5408client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5409client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5410worker2 # [ 65.439622] nix-daemon[867]: reaped child process 1066, status = succeeded5411worker2 # [ 65.514129] nix-daemon[867]: accepted connection from pid 1082, user nix-grpc-daemon (trusted)5412worker2 # [ 65.527746] nix-daemon[867]: reaped child process 1085, status = succeeded5413lb # [ 65.512619] niks3-server[993]: 2026/09/21 14:30:02 INFO Received uploads request method=POST path=/api/pending_closures5414worker2 # [ 65.544473] nix-grpc-daemon[920]: time=2026-09-21T14:30:02.660Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5415lb # [ 65.544248] niks3-server[993]: 2026/09/21 14:30:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/24/sign5416lb # [ 65.554420] niks3-server[993]: 2026/09/21 14:30:02 INFO Signed narinfos id=24 count=15417worker2 # [ 65.581189] nix-grpc-daemon[920]: time=2026-09-21T14:30:02.697Z level=INFO msg="Uploading 1 narinfos"5418lb # [ 65.559764] niks3-server[993]: 2026/09/21 14:30:02 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5419lb # [ 65.565589] niks3-server[993]: 2026/09/21 14:30:02 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5420lb # [ 65.581391] niks3-server[993]: 2026/09/21 14:30:02 INFO Received complete upload request method=POST path=/api/pending_closures/24/complete5421lb # [ 65.588430] niks3-server[993]: 2026/09/21 14:30:02 INFO Completed upload id=245422lb # [ 65.591931] niks3-server[993]: 2026/09/21 14:30:02 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5423worker2 # [ 65.614863] nix-grpc-daemon[920]: time=2026-09-21T14:30:02.731Z level=INFO msg="Upload complete. (184ms)"5424worker2 # [ 65.619847] nix-grpc-daemon[782]: ts=2026-09-21T14:30:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=11 outputs=15425worker2 # [ 65.632253] nix-daemon[867]: reaped child process 1064, status = succeeded5426client # copying 1 paths...5427worker2 # [ 65.665564] nix-grpc-daemon[782]: ts=2026-09-21T14:30:02Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:483425428client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5429lb # [ 65.647948] envoy[906]: conn peer=[2001:db8:1::1]:44584 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3893 tx=1932 ms=22185430lb # [ 65.651648] envoy[906]: conn peer=[2001:db8:1::1]:44586 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=435 tx=379 ms=735431lb # [ 65.657125] envoy[906]: conn peer=[2001:db8:1::1]:44590 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4327 tx=2301 ms=19185432lb # [ 65.660615] envoy[906]: conn peer=[2001:db8:1::1]:44592 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=410 ms=535433lb # [ 65.663970] envoy[906]: conn peer=[2001:db8:1::1]:44600 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4201 tx=1943 ms=18105434lb # [ 65.667506] envoy[906]: conn peer=[2001:db8:1::1]:44610 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=235435lb # [ 65.670868] envoy[906]: conn peer=[2001:db8:1::1]:44614 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3740 tx=2007 ms=22565436lb # [ 65.674907] envoy[906]: conn peer=[2001:db8:1::1]:50214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=175437worker2 # [ 65.705242] nix-grpc-daemon[782]: ts=2026-09-21T14:30:02Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:483425438worker2 # [ 65.709874] nix-grpc-daemon[782]: ts=2026-09-21T14:30:02Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:48342 duration_s=0 paths=1 nar_bytes_out=1205439client # [ 65.763758] nix-daemon[939]: reaped child process 1069, status = succeeded5440client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.78 seconds)5441(finished: subtest: an input only one worker has still reaches the builder, in 6.89 seconds)5442subtest: upload whose reference the worker lacks is completed from the cache5443client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5444client # this derivation will be built:5445client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5446client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5447client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.23 seconds)5448client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5449client # this derivation will be built:5450client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5451client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5452client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.23 seconds)5453client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5454client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.09 seconds)5455worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5456worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.14 seconds)5457worker2 # [ 66.538361] systemd[1]: Stopping Nix worker-protocol over gRPC...5458worker2 # [ 66.874307] nix-grpc-daemon[782]: ts=2026-09-21T14:30:03Z level=info event=signal_exit5459lb # [ 67.218791] niks3-server[993]: 2026/09/21 14:30:04 INFO lead: released remote=192.168.1.4:434705460worker1 # [ 68.232982] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=info event=scheduler_restarting addr=lb:500515461worker2 # [ 68.247381] nix-grpc-daemon[782]: ts=2026-09-21T14:30:05Z level=info event=scheduler_restarting5462worker2 # [ 68.250593] nix-grpc-daemon[782]: ts=2026-09-21T14:30:05Z level=info event=scheduler_restarting addr=lb:500515463worker2 # [ 68.254260] nix-grpc-daemon[782]: ts=2026-09-21T14:30:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:483325464worker2 # [ 68.258587] nix-grpc-daemon[782]: ts=2026-09-21T14:30:05Z level=info event=worker_gone addr=192.168.1.3:500515465worker2 # [ 68.261750] nix-grpc-daemon[782]: ts=2026-09-21T14:30:05Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:483325466worker2 # [ 68.265362] nix-grpc-daemon[782]: ts=2026-09-21T14:30:05Z level=info event=worker_gone addr=192.168.1.4:500515467worker2 # [ 68.285421] nix-daemon[867]: reaped child process 869, status = succeeded5468worker1 # [ 68.433129] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=info event=scheduler_connected addr=lb:500515469worker1 # [ 68.437374] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5470worker2 # [ 68.454671] nix-grpc-daemon[782]: ts=2026-09-21T14:30:05Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5471worker2 # [ 68.510390] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5472worker2 # [ 68.514215] systemd[1]: Stopped Nix worker-protocol over gRPC.5473worker2 # [ 68.518965] systemd[1]: nix-grpc-daemon.service: Consumed 1.501s CPU time over 52.794s wall clock time, 58.8M memory peak, 64.1K incoming IP traffic, 206.1K outgoing IP traffic.5474worker2 # [ 68.529695] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5475worker2 # [ 68.533239] systemd[1]: Closed Nix worker-protocol over gRPC.5476worker2 # [ 68.535846] systemd[1]: nix-grpc-daemon.socket: Consumed 303us CPU time over 1min 875ms wall clock time, 256K memory peak, 70K incoming IP traffic, 59.4K outgoing IP traffic.5477lb: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy5478lb: must succeed: curl -sf localhost:9901/clusters5479lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5480lb: (finished: waiting until exactly ['worker2'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5481client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25482worker1 # [ 68.642224] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=info event=scheduler_connected addr=lb:500515483worker1 # [ 68.646349] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5484worker1 # [ 68.714247] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:517905485worker1 # [ 68.720124] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:517905486client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5487client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.16 seconds)5488worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5489worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5490worker2 # [ 68.815962] systemd[1]: Listening on Nix worker-protocol over gRPC.5491worker1 # [ 68.847315] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=info event=scheduler_connected addr=lb:500515492worker1 # [ 68.853086] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5493worker1 # [ 68.858797] systemd[1]: Stopping Nix worker-protocol over gRPC...5494worker1 # [ 68.862457] nix-grpc-daemon[781]: ts=2026-09-21T14:30:05Z level=info event=signal_exit5495worker2 # [ 69.012182] systemd[1]: Starting scheduler-token.service...5496worker1 # [ 69.053138] nix-grpc-daemon[781]: ts=2026-09-21T14:30:06Z level=info event=scheduler_connected addr=lb:500515497worker1 # [ 69.057250] nix-grpc-daemon[781]: ts=2026-09-21T14:30:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5498worker2 # [ 69.073961] systemd[1]: scheduler-token.service: Deactivated successfully.5499worker2 # [ 69.077522] systemd[1]: Finished scheduler-token.service.5500worker2 # [ 69.080617] systemd[1]: scheduler-token.service: Consumed 34ms CPU time over 64ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5501worker2 # [ 69.089081] systemd[1]: Starting Nix worker-protocol over gRPC...5502worker2 # [ 69.171149] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5503worker2 # [ 69.173170] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5504worker2 # [ 69.176442] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15505worker2 # [ 69.180703] nix-grpc-daemon[1109]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5506worker2 # [ 69.182070] nix-grpc-daemon[1109]: E0000 00:00:1790001006.297213 1109 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5507worker2 # [ 69.188133] nix-grpc-daemon[1109]: E0000 00:00:1790001006.302656 1109 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5508worker2 # [ 69.192074] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5509worker2 # [ 69.193700] systemd[1]: Started Nix worker-protocol over gRPC.5510worker2 # [ 69.199068] nix-grpc-daemon[1109]: E0000 00:00:1790001006.315013 1150 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5511worker2 # [ 69.202123] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=scheduler_take_over5512lb # [ 69.178285] niks3-server[993]: 2026/09/21 14:30:06 INFO lead: acquired remote=192.168.1.4:383225513worker2 # [ 69.214135] nix-grpc-daemon[1109]: E0000 00:00:1790001006.330782 1150 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5514worker2 # [ 69.219054] nix-grpc-daemon[1109]: E0000 00:00:1790001006.333179 1150 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5515worker2 # [ 69.228373] nix-grpc-daemon[1109]: E0000 00:00:1790001006.345007 1150 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5516worker2 # [ 69.237403] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=scheduler_connected addr=lb:500515517worker2 # [ 69.243973] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=oidc_jwks provider=mock keys=15518worker2 # [ 69.245439] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:565025519worker2 # [ 69.247034] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05520worker2 # [ 69.269445] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:565025521worker1 # [ 69.258667] nix-grpc-daemon[781]: ts=2026-09-21T14:30:06Z level=info event=scheduler_connected addr=lb:500515522worker2 # [ 69.270929] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:06Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05523worker1 # [ 70.239266] nix-grpc-daemon[781]: ts=2026-09-21T14:30:07Z level=info event=scheduler_restarting5524worker2 # [ 70.259202] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:07Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:565025525worker2 # [ 70.262966] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:07Z level=info event=worker_gone addr=192.168.1.3:500515526worker1 # [ 70.266248] nix-grpc-daemon[781]: ts=2026-09-21T14:30:07Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5527worker1 # [ 70.278616] nix-daemon[847]: reaped child process 849, status = succeeded5528worker1 # [ 70.462422] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5529worker1 # [ 70.466487] systemd[1]: Stopped Nix worker-protocol over gRPC.5530worker1 # [ 70.470779] systemd[1]: nix-grpc-daemon.service: Consumed 2.637s CPU time over 54.804s wall clock time, 63.7M memory peak, 91.9K incoming IP traffic, 174.9K outgoing IP traffic.5531worker1 # [ 70.482840] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5532worker1 # [ 70.487284] systemd[1]: Closed Nix worker-protocol over gRPC.5533worker1 # [ 70.490845] systemd[1]: nix-grpc-daemon.socket: Consumed 324us CPU time over 1min 2.831s wall clock time, 256K memory peak, 73K incoming IP traffic, 61.3K outgoing IP traffic.5534lb: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy5535lb: must succeed: curl -sf localhost:9901/clusters5536lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5537lb: (finished: waiting until exactly ['worker1'] endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5538client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25539worker2 # [ 70.709161] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565065540worker2 # [ 70.721112] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5541worker2 # [ 70.737214] nix-grpc-daemon[1109]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 17 ms (attempt 1/5)5542worker2 # [ 70.761357] nix-grpc-daemon[1109]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 137 ms (attempt 2/5)5543worker2 # [ 70.900621] nix-grpc-daemon[1109]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 49 ms (attempt 3/5)5544worker2 # [ 70.951494] nix-grpc-daemon[1109]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 517 ms (attempt 4/5)5545worker2 # [ 71.478154] nix-grpc-daemon[1109]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5546worker2 # [ 71.481298] nix-grpc-daemon[1109]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5547client # copying 2 paths...5548client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5549worker2 # [ 71.504183] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565065550client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5551worker2 # [ 71.515497] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5552worker2 # [ 71.539630] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:08Z level=info event=niks3_push_started pid=11695553worker2 # [ 71.643546] nix-grpc-daemon[1169]: time=2026-09-21T14:30:08.759Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5554worker2 # [ 71.732797] nix-daemon[867]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5555worker2 # [ 71.744580] nix-daemon[867]: reaped child process 1185, status = succeeded5556lb # [ 71.732516] niks3-server[993]: 2026/09/21 14:30:08 INFO Received uploads request method=POST path=/api/pending_closures5557lb # [ 71.739813] niks3-server[993]: 2026/09/21 14:30:08 INFO Received uploads request method=POST path=/api/pending_closures5558worker2 # [ 71.770547] nix-grpc-daemon[1169]: time=2026-09-21T14:30:08.887Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5559worker2 # [ 71.772080] nix-grpc-daemon[1169]: time=2026-09-21T14:30:08.888Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5560worker2 # [ 71.775185] nix-grpc-daemon[1169]: time=2026-09-21T14:30:08.890Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5561lb # [ 71.789199] niks3-server[993]: 2026/09/21 14:30:08 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5562lb # [ 71.800525] niks3-server[993]: 2026/09/21 14:30:08 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5563lb # [ 71.804722] niks3-server[993]: 2026/09/21 14:30:08 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5564lb # [ 71.811596] niks3-server[993]: 2026/09/21 14:30:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/25/sign5565lb # [ 71.813562] niks3-server[993]: 2026/09/21 14:30:08 INFO Signed narinfos id=25 count=15566lb # [ 71.815423] niks3-server[993]: 2026/09/21 14:30:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/26/sign5567lb # [ 71.818022] niks3-server[993]: 2026/09/21 14:30:09 INFO Signed narinfos id=26 count=25568worker2 # [ 71.843156] nix-grpc-daemon[1169]: time=2026-09-21T14:30:08.958Z level=INFO msg="Uploading 3 narinfos"5569lb # [ 71.822069] niks3-server[993]: 2026/09/21 14:30:09 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5570lb # [ 71.835618] niks3-server[993]: 2026/09/21 14:30:09 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5571lb # [ 71.838055] niks3-server[993]: 2026/09/21 14:30:09 INFO Received complete upload request method=POST path=/api/pending_closures/25/complete5572lb # [ 71.840229] niks3-server[993]: 2026/09/21 14:30:09 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5573lb # [ 71.843791] niks3-server[993]: 2026/09/21 14:30:09 INFO Completed upload id=255574lb # [ 71.845525] niks3-server[993]: 2026/09/21 14:30:09 INFO Received complete upload request method=POST path=/api/pending_closures/26/complete5575lb # [ 71.848973] niks3-server[993]: 2026/09/21 14:30:09 INFO Completed upload id=265576worker2 # [ 71.872922] nix-grpc-daemon[1169]: time=2026-09-21T14:30:08.989Z level=INFO msg="Upload complete. (226ms)"5577lb # [ 71.850994] niks3-server[993]: 2026/09/21 14:30:09 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5578worker2 # [ 71.878460] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:08Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56506 duration_s=0 paths=2 nar_bytes_in=3125579worker2 # [ 71.891126] nix-daemon[867]: reaped child process 1167, status = succeeded5580client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.35 seconds)5581worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5582worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5583worker1 # [ 71.966569] systemd[1]: Listening on Nix worker-protocol over gRPC.5584worker1 # [ 71.977122] systemd[1]: Starting Nix worker-protocol over gRPC...5585worker1 # [ 72.066271] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:09Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5586worker1 # [ 72.070914] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:09Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5587worker1 # [ 72.075121] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:09Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15588worker1 # [ 72.077058] nix-grpc-daemon[1218]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5589worker1 # [ 72.078357] nix-grpc-daemon[1218]: E0000 00:00:1790001009.080203 1218 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5590worker1 # [ 72.087719] nix-grpc-daemon[1218]: E0000 00:00:1790001009.087407 1218 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5591worker1 # [ 72.091836] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:09Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket5592worker1 # [ 72.094946] systemd[1]: Started Nix worker-protocol over gRPC.5593worker1 # [ 72.097560] nix-grpc-daemon[1218]: E0000 00:00:1790001009.103247 1260 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5594worker1 # [ 72.111087] nix-grpc-daemon[1218]: E0000 00:00:1790001009.117567 1260 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5595worker1 # [ 72.115157] nix-grpc-daemon[1218]: E0000 00:00:1790001009.119885 1260 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5596lb: waiting until exactly no endpoints of x86_64-linux are unhealthy5597lb: must succeed: curl -sf localhost:9901/clusters5598worker1 # [ 72.126081] nix-grpc-daemon[1218]: E0000 00:00:1790001009.132492 1260 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5599worker1 # [ 72.138367] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:09Z level=info event=scheduler_connected addr=lb:500515600worker2 # [ 72.150774] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:09Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:565025601worker2 # [ 72.154978] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:09Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05602lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5603lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 0.05 seconds)5604(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 6.38 seconds)5605subtest: a niks3 restart does not cost the leader its role5606lb: must succeed: curl -sf localhost:9901/clusters5607lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5608lb: must succeed: curl -sf localhost:9901/clusters5609lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5610lb # [ 72.228542] systemd[1]: Stopping niks3 server...5611lb # [ 72.229632] niks3-server[993]: 2026/09/21 14:30:09 INFO Shutdown signal received, draining in-flight requests timeout=10s5612worker2 # [ 72.252658] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:09Z level=info event=scheduler_lock_lost status=2005613worker1 # [ 72.242172] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:09Z level=info event=scheduler_lock_lost status=2005614lb # [ 72.232238] niks3-server[993]: 2026/09/21 14:30:09 INFO lead: released remote=192.168.1.4:383225615lb # [ 75.648308] envoy[906]: conn peer=[2001:db8:1::1]:50228 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3755 tx=1876 ms=15115616lb # [ 75.652173] envoy[906]: conn peer=[2001:db8:1::1]:50232 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=447 tx=328 ms=495617lb # [ 75.655586] envoy[906]: conn peer=[2001:db8:1::4]:60120 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=71106 tx=3254 ms=525415618lb # [ 75.659475] envoy[906]: conn peer=[2001:db8:1::1]:50248 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=195619lb # [ 75.662939] envoy[906]: conn peer=[2001:db8:1::3]:36538 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=19840 tx=3789 ms=543725620lb # [ 75.666639] envoy[906]: conn peer=[2001:db8:1::1]:50262 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1003 tx=282 ms=11835621lb # [ 77.059733] systemd[1]: niks3.service: Deactivated successfully.5622lb # [ 77.064428] systemd[1]: Stopped niks3 server.5623lb # [ 77.066402] systemd[1]: niks3.service: Consumed 600ms CPU time over 55.080s wall clock time, 26.5M memory peak, 70.3K incoming IP traffic, 100.8K outgoing IP traffic.5624lb # [ 77.084531] systemd[1]: Starting niks3 server...5625lb # [ 77.149148] niks3-server[1129]: 2026/09/21 14:30:14 goose: no migrations to run. current version: 202609200000005626lb # [ 77.156901] niks3-server[1129]: 2026/09/21 14:30:14 OK 1_commit_pending_closure.sql (6.02ms)5627lb # [ 77.160616] niks3-server[1129]: 2026/09/21 14:30:14 OK 2_object_stats_trigger.sql (3.4ms)5628lb # [ 77.161731] niks3-server[1129]: 2026/09/21 14:30:14 goose: up to current file version: 25629lb # [ 77.163335] niks3-server[1129]: 2026/09/21 14:30:14 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5630lb # [ 77.174184] niks3-server[1129]: 2026/09/21 14:30:14 INFO Read proxy enabled — serving cache objects from S35631lb # [ 77.175465] niks3-server[1129]: 2026/09/21 14:30:14 INFO Using socket-activated listener address=0.0.0.0:57515632lb # [ 77.177244] niks3-server[1129]: 2026/09/21 14:30:14 INFO systemd watchdog enabled interval=15s5633lb # [ 77.179160] niks3-server[1129]: 2026/09/21 14:30:14 INFO Starting HTTP server address=0.0.0.0:57515634lb # [ 77.180300] systemd[1]: Started niks3 server.5635lb # [ 77.186083] niks3-server[1129]: 2026/09/21 14:30:14 INFO lead: acquired remote=192.168.1.4:492485636lb: waiting for unit niks3.service5637lb: (finished: waiting for unit niks3.service, in 0.03 seconds)5638lb: must succeed: curl -sf localhost:9901/clusters5639lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5640worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5641worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5642lb: must succeed: curl -sf localhost:9901/clusters5643lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5644lb: must succeed: curl -sf localhost:9901/clusters5645lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5646worker2: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5647worker2: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5648client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25649worker1 # [ 77.535400] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:14Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:599985650worker1 # [ 77.547194] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5651client # don't know how to build these paths:5652client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5653worker1 # [ 77.561933] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599985654worker1 # [ 77.572243] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 84 ms (attempt 1/5)5655worker1 # [ 77.663878] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 113 ms (attempt 2/5)5656worker1 # [ 77.785242] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 56 ms (attempt 3/5)5657worker1 # [ 77.843080] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 592 ms (attempt 4/5)5658worker1 # [ 78.437793] nix-grpc-daemon[1218]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5659worker1 # [ 78.441554] nix-grpc-daemon[1218]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5660worker2 # [ 78.493074] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:15Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05661worker2 # [ 78.497259] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:15Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.4:50051 clients=15662worker2 # [ 78.536795] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565065663worker2 # [ 78.551130] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565065664client # copying 1 paths to 'lb:50051'...5665client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5666worker2 # [ 78.563708] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565065667worker2 # [ 78.572719] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5668worker2 # [ 78.678048] nix-daemon[867]: accepted connection from pid 1204, user nix-grpc-daemon (trusted)5669worker2 # [ 78.691729] nix-daemon[867]: reaped child process 1207, status = succeeded5670worker2 # [ 78.772681] nix-daemon[867]: accepted connection from pid 1209, user nix-grpc-daemon (trusted)5671worker2 # [ 78.785431] nix-daemon[867]: reaped child process 1212, status = succeeded5672lb # [ 78.771813] niks3-server[1129]: 2026/09/21 14:30:15 INFO Received uploads request method=POST path=/api/pending_closures5673worker2 # [ 78.806304] nix-grpc-daemon[1169]: time=2026-09-21T14:30:15.922Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5674worker2 # [ 78.807704] nix-grpc-daemon[1169]: time=2026-09-21T14:30:15.922Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5675lb # [ 78.811287] niks3-server[1129]: 2026/09/21 14:30:15 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5676lb # [ 78.822442] niks3-server[1129]: 2026/09/21 14:30:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/27/sign5677lb # [ 78.832713] niks3-server[1129]: 2026/09/21 14:30:16 INFO Signed narinfos id=27 count=15678worker2 # [ 78.859454] nix-grpc-daemon[1169]: time=2026-09-21T14:30:15.975Z level=INFO msg="Uploading 1 narinfos"5679lb # [ 78.842834] niks3-server[1129]: 2026/09/21 14:30:16 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5680lb # [ 78.846632] niks3-server[1129]: 2026/09/21 14:30:16 INFO Received complete upload request method=POST path=/api/pending_closures/27/complete5681lb # [ 78.855052] niks3-server[1129]: 2026/09/21 14:30:16 INFO Completed upload id=275682worker2 # [ 78.878810] nix-grpc-daemon[1169]: time=2026-09-21T14:30:15.995Z level=INFO msg="Upload complete. (292ms)"5683worker2 # [ 78.883169] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56506 duration_s=0 paths=1 nar_bytes_in=5365684worker2 # [ 78.889710] nix-grpc-daemon[1169]: time=2026-09-21T14:30:15.999Z level=WARN msg="Failed to register uploaded object" key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo error="server returned 404: object is not pending upload\n"5685worker2 # [ 78.895333] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565065686worker2 # [ 78.902215] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5687worker2 # [ 78.910962] nix-daemon[867]: reaped child process 1202, status = succeeded5688worker2 # [ 78.915869] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5689client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5690client # farm-a> worker2: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5691client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5692client # farm-a> LOG-farm-a-after-niks3-restart5693worker2 # [ 80.127971] nix-daemon[867]: reaped child process 1216, status = succeeded5694worker2 # [ 80.207250] nix-daemon[867]: accepted connection from pid 1232, user nix-grpc-daemon (trusted)5695worker2 # [ 80.220896] nix-daemon[867]: reaped child process 1235, status = succeeded5696lb # [ 80.205376] niks3-server[1129]: 2026/09/21 14:30:17 INFO Received uploads request method=POST path=/api/pending_closures5697worker2 # [ 80.240658] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.357Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5698worker2 # [ 80.242134] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.358Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5699lb # [ 80.257721] niks3-server[1129]: 2026/09/21 14:30:17 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5700lb # [ 80.262130] niks3-server[1129]: 2026/09/21 14:30:17 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5701lb # [ 80.268500] niks3-server[1129]: 2026/09/21 14:30:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/28/sign5702lb # [ 80.273700] niks3-server[1129]: 2026/09/21 14:30:17 INFO Signed narinfos id=28 count=15703worker2 # [ 80.300700] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.416Z level=INFO msg="Uploading 1 narinfos"5704lb # [ 80.280510] niks3-server[1129]: 2026/09/21 14:30:17 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5705lb # [ 80.284262] niks3-server[1129]: 2026/09/21 14:30:17 INFO Received complete upload request method=POST path=/api/pending_closures/28/complete5706lb # [ 80.287224] niks3-server[1129]: 2026/09/21 14:30:17 INFO Completed upload id=285707worker2 # [ 80.310907] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.427Z level=INFO msg="Upload complete. (186ms)"5708lb # [ 80.292975] niks3-server[1129]: 2026/09/21 14:30:17 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5709worker2 # [ 80.322326] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56506 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15710worker2 # [ 80.328728] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:17Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05711worker2 # [ 80.333157] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:17Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.4:50051 clients=15712worker2 # [ 80.341101] nix-daemon[867]: reaped child process 1214, status = succeeded5713worker2 # [ 80.372812] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565065714worker2 # [ 80.384696] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:17Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565065715worker2 # [ 80.398299] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.514Z level=INFO msg="All 1 paths already cached"5716client # copying 1 paths to 'lb:50051'...5717client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5718worker2 # [ 80.406735] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:17Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565065719worker2 # [ 80.415244] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5720worker2 # [ 80.511470] nix-daemon[867]: accepted connection from pid 1241, user nix-grpc-daemon (trusted)5721worker2 # [ 80.524495] nix-daemon[867]: reaped child process 1244, status = succeeded5722worker2 # [ 80.607474] nix-daemon[867]: accepted connection from pid 1246, user nix-grpc-daemon (trusted)5723worker2 # [ 80.620560] nix-daemon[867]: reaped child process 1249, status = succeeded5724lb # [ 80.603856] niks3-server[1129]: 2026/09/21 14:30:17 INFO Received uploads request method=POST path=/api/pending_closures5725worker2 # [ 80.637437] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.754Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5726worker2 # [ 80.639030] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.755Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5727lb # [ 80.633584] niks3-server[1129]: 2026/09/21 14:30:17 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5728lb # [ 80.645232] niks3-server[1129]: 2026/09/21 14:30:17 INFO Received sign narinfos request method=POST path=/api/pending_closures/29/sign5729lb # [ 80.652705] niks3-server[1129]: 2026/09/21 14:30:17 INFO Signed narinfos id=29 count=15730worker2 # [ 80.679594] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.796Z level=INFO msg="Uploading 1 narinfos"5731lb # [ 80.661200] niks3-server[1129]: 2026/09/21 14:30:17 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5732lb # [ 80.675260] niks3-server[1129]: 2026/09/21 14:30:17 INFO Received complete upload request method=POST path=/api/pending_closures/29/complete5733lb # [ 80.685392] niks3-server[1129]: 2026/09/21 14:30:17 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5734lb # [ 80.690373] niks3-server[1129]: 2026/09/21 14:30:17 INFO Completed upload id=295735worker2 # [ 80.714170] nix-grpc-daemon[1169]: time=2026-09-21T14:30:17.830Z level=INFO msg="Upload complete. (286ms)"5736worker2 # [ 80.718684] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:17Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56506 duration_s=0 paths=1 nar_bytes_in=6565737worker2 # [ 80.725348] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:17Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565065738worker2 # [ 80.736534] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5739worker2 # [ 80.745964] nix-daemon[867]: reaped child process 1239, status = succeeded5740worker2 # [ 80.753265] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5741client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5742client # farm-b> worker2: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5743client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5744client # farm-b> LOG-farm-b-after-niks3-restart5745worker2 # [ 81.968247] nix-daemon[867]: reaped child process 1253, status = succeeded5746worker2 # [ 82.043621] nix-daemon[867]: accepted connection from pid 1269, user nix-grpc-daemon (trusted)5747worker2 # [ 82.055989] nix-daemon[867]: reaped child process 1272, status = succeeded5748lb # [ 82.041688] niks3-server[1129]: 2026/09/21 14:30:19 INFO Received uploads request method=POST path=/api/pending_closures5749worker2 # [ 82.074374] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.190Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5750worker2 # [ 82.075922] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.192Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5751lb # [ 82.077934] niks3-server[1129]: 2026/09/21 14:30:19 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5752lb # [ 82.094554] niks3-server[1129]: 2026/09/21 14:30:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/30/sign5753lb # [ 82.101455] niks3-server[1129]: 2026/09/21 14:30:19 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5754lb # [ 82.110215] niks3-server[1129]: 2026/09/21 14:30:19 INFO Signed narinfos id=30 count=15755worker2 # [ 82.133746] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.250Z level=INFO msg="Uploading 1 narinfos"5756lb # [ 82.121351] niks3-server[1129]: 2026/09/21 14:30:19 INFO Received complete upload request method=POST path=/api/pending_closures/30/complete5757lb # [ 82.129067] niks3-server[1129]: 2026/09/21 14:30:19 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5758lb # [ 82.135417] niks3-server[1129]: 2026/09/21 14:30:19 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5759lb # [ 82.138287] niks3-server[1129]: 2026/09/21 14:30:19 INFO Completed upload id=305760worker2 # [ 82.162190] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.278Z level=INFO msg="Upload complete. (201ms)"5761worker2 # [ 82.174881] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56506 duration_s=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15762worker2 # [ 82.180770] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:19Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05763worker2 # [ 82.184691] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:19Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.4:50051 clients=15764worker2 # [ 82.195747] nix-daemon[867]: reaped child process 1251, status = succeeded5765worker2 # [ 82.218357] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565065766worker2 # [ 82.231154] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565065767worker2 # [ 82.246258] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.362Z level=INFO msg="All 2 paths already cached"5768client # copying 1 paths to 'lb:50051'...5769client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5770worker2 # [ 82.254663] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565065771worker2 # [ 82.262894] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5772worker2 # [ 82.359127] nix-daemon[867]: accepted connection from pid 1278, user nix-grpc-daemon (trusted)5773worker2 # [ 82.372162] nix-daemon[867]: reaped child process 1281, status = succeeded5774worker2 # [ 82.455805] nix-daemon[867]: accepted connection from pid 1283, user nix-grpc-daemon (trusted)5775worker2 # [ 82.470125] nix-daemon[867]: reaped child process 1286, status = succeeded5776lb # [ 82.453163] niks3-server[1129]: 2026/09/21 14:30:19 INFO Received uploads request method=POST path=/api/pending_closures5777worker2 # [ 82.486544] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.603Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5778worker2 # [ 82.488453] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.604Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5779lb # [ 82.484918] niks3-server[1129]: 2026/09/21 14:30:19 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5780lb # [ 82.499417] niks3-server[1129]: 2026/09/21 14:30:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/31/sign5781worker2 # [ 82.530056] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.646Z level=INFO msg="Uploading 1 narinfos"5782lb # [ 82.506489] niks3-server[1129]: 2026/09/21 14:30:19 INFO Signed narinfos id=31 count=15783lb # [ 82.509758] niks3-server[1129]: 2026/09/21 14:30:19 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5784lb # [ 82.523065] niks3-server[1129]: 2026/09/21 14:30:19 INFO Received complete upload request method=POST path=/api/pending_closures/31/complete5785lb # [ 82.530143] niks3-server[1129]: 2026/09/21 14:30:19 INFO Completed upload id=315786worker2 # [ 82.553908] nix-grpc-daemon[1169]: time=2026-09-21T14:30:19.670Z level=INFO msg="Upload complete. (278ms)"5787lb # [ 82.533481] niks3-server[1129]: 2026/09/21 14:30:19 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5788worker2 # [ 82.558258] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56506 duration_s=0 paths=1 nar_bytes_in=7845789worker2 # [ 82.562844] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565065790worker2 # [ 82.574109] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5791worker2 # [ 82.580684] nix-daemon[867]: reaped child process 1276, status = succeeded5792worker2 # [ 82.585668] nix-daemon[867]: accepted connection from pid 1109, user nix-grpc-daemon (trusted)5793client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5794client # farm-top> worker2: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5795client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5796client # farm-top> LOG-farm-top-after-niks3-restart5797worker2 # [ 83.599812] nix-daemon[867]: reaped child process 1290, status = succeeded5798worker2 # [ 83.673739] nix-daemon[867]: accepted connection from pid 1306, user nix-grpc-daemon (trusted)5799worker2 # [ 83.686099] nix-daemon[867]: reaped child process 1309, status = succeeded5800lb # [ 83.670858] niks3-server[1129]: 2026/09/21 14:30:20 INFO Received uploads request method=POST path=/api/pending_closures5801worker2 # [ 83.703769] nix-grpc-daemon[1169]: time=2026-09-21T14:30:20.820Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5802worker2 # [ 83.705560] nix-grpc-daemon[1169]: time=2026-09-21T14:30:20.821Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5803lb # [ 83.710310] niks3-server[1129]: 2026/09/21 14:30:20 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5804lb # [ 83.720760] niks3-server[1129]: 2026/09/21 14:30:20 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5805lb # [ 83.729817] niks3-server[1129]: 2026/09/21 14:30:20 INFO Received sign narinfos request method=POST path=/api/pending_closures/32/sign5806lb # [ 83.733613] niks3-server[1129]: 2026/09/21 14:30:20 INFO Signed narinfos id=32 count=15807worker2 # [ 83.759834] nix-grpc-daemon[1169]: time=2026-09-21T14:30:20.876Z level=INFO msg="Uploading 1 narinfos"5808lb # [ 83.743601] niks3-server[1129]: 2026/09/21 14:30:20 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5809lb # [ 83.751149] niks3-server[1129]: 2026/09/21 14:30:20 INFO Received complete upload request method=POST path=/api/pending_closures/32/complete5810lb # [ 83.754123] niks3-server[1129]: 2026/09/21 14:30:20 INFO Completed upload id=325811worker2 # [ 83.777857] nix-grpc-daemon[1169]: time=2026-09-21T14:30:20.894Z level=INFO msg="Upload complete. (184ms)"5812lb # [ 83.759510] niks3-server[1129]: 2026/09/21 14:30:20 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5813worker2 # [ 83.784403] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:20Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56506 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15814worker2 # [ 83.798911] nix-daemon[867]: reaped child process 1288, status = succeeded5815client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 6.42 seconds)5816client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5817client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.11 seconds)5818(finished: subtest: a niks3 restart does not cost the leader its role, in 11.76 seconds)5819subtest: scheduler down: the other node takes the lock and keeps it5820lb: must succeed: curl -sf localhost:9901/clusters5821lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5822lb: must succeed: curl -sf localhost:9901/clusters5823lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5824lb: must succeed: curl -sf localhost:9901/clusters5825lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5826lb: must succeed: curl -sf localhost:9901/clusters5827lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5828worker2 # [ 84.101723] systemd[1]: Stopping Nix worker-protocol over gRPC...5829worker2 # [ 84.194166] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:21Z level=info event=signal_exit5830lb # [ 84.189188] niks3-server[1129]: 2026/09/21 14:30:21 INFO lead: released remote=192.168.1.4:492485831worker1 # [ 85.204505] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:22Z level=info event=scheduler_restarting addr=lb:500515832worker2 # [ 85.217877] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:22Z level=info event=scheduler_restarting5833worker2 # [ 85.222614] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:22Z level=info event=scheduler_restarting addr=lb:500515834worker2 # [ 85.226222] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:565025835worker2 # [ 85.230283] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:22Z level=info event=worker_gone addr=192.168.1.4:500515836worker2 # [ 85.233591] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:22Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:565025837worker2 # [ 85.237277] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:22Z level=info event=worker_gone addr=192.168.1.3:500515838worker2 # [ 85.245827] nix-grpc-daemon[1109]: E0000 00:00:1790001022.362238 1154 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 17, started true, shutdown true5839worker2 # [ 85.250582] nix-grpc-daemon[1109]: E0000 00:00:1790001022.362420 1154 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 9, started true, shutdown true5840worker2 # [ 85.265456] nix-daemon[867]: reaped child process 1161, status = succeeded5841worker1 # [ 85.404700] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:22Z level=info event=scheduler_connected addr=lb:500515842worker1 # [ 85.408952] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5843worker2 # [ 85.426544] nix-grpc-daemon[1109]: ts=2026-09-21T14:30:22Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5844worker1 # [ 85.609923] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:22Z level=info event=scheduler_connected addr=lb:500515845worker1 # [ 85.614103] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5846lb # [ 85.648489] envoy[906]: conn peer=[2001:db8:1::1]:58550 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9666 tx=4101 ms=62635847lb # [ 85.652355] envoy[906]: conn peer=[2001:db8:1::4]:48010 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=1755 tx=425 ms=161845848worker1 # [ 85.815107] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:22Z level=info event=scheduler_connected addr=lb:500515849worker1 # [ 85.819282] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:22Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5850worker1 # [ 86.020480] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_connected addr=lb:500515851worker1 # [ 86.024635] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5852worker1 # [ 86.226416] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_connected addr=lb:500515853worker1 # [ 86.230336] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5854worker1 # [ 86.430984] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_connected addr=lb:500515855worker1 # [ 86.435151] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5856worker1 # [ 86.635941] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_connected addr=lb:500515857worker1 # [ 86.640217] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5858worker1 # [ 86.841211] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_connected addr=lb:500515859worker1 # [ 86.845343] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:23Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5860worker1 # [ 87.046263] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_connected addr=lb:500515861worker1 # [ 87.050401] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5862worker1 # [ 87.200253] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_take_over5863lb # [ 87.187288] niks3-server[1129]: 2026/09/21 14:30:24 INFO lead: acquired remote=192.168.1.3:494985864worker1 # [ 87.251976] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_connected addr=lb:500515865worker1 # [ 87.255877] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5866worker2 # [ 87.345171] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5867worker2 # [ 87.348136] systemd[1]: Stopped Nix worker-protocol over gRPC.5868worker2 # [ 87.352488] systemd[1]: nix-grpc-daemon.service: Consumed 1.050s CPU time over 18.256s wall clock time, 38.6M memory peak, 45.2K incoming IP traffic, 77.6K outgoing IP traffic.5869worker2 # [ 87.363237] systemd[1]: nix-grpc-daemon.socket: Deactivated successfully.5870worker2 # [ 87.367540] systemd[1]: Closed Nix worker-protocol over gRPC.5871worker2 # [ 87.370895] systemd[1]: nix-grpc-daemon.socket: Consumed 446us CPU time over 18.555s wall clock time, 256K memory peak, 38.7K incoming IP traffic, 28.4K outgoing IP traffic.5872worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5873worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 0.06 seconds)5874worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5875worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5876worker1 # [ 87.456264] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_connected addr=lb:500515877worker1 # [ 87.458070] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5878worker1 # [ 87.658867] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_connected addr=lb:500515879worker1 # [ 87.662917] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5880worker1 # [ 87.863902] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_connected addr=lb:500515881worker1 # [ 87.868176] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:24Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5882worker1 # [ 88.069294] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=info event=scheduler_connected addr=lb:500515883worker1 # [ 88.073473] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5884worker1 # [ 88.275612] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=info event=scheduler_connected addr=lb:500515885worker1 # [ 88.283152] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:407385886worker1 # [ 88.286716] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05887worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5888worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5889client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25890worker1 # [ 88.638853] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:599985891client # don't know how to build these paths:5892client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5893worker1 # [ 88.647473] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599985894worker1 # [ 88.670484] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05895worker1 # [ 88.675209] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.3:50051 clients=15896worker1 # [ 88.715275] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599985897worker1 # [ 88.728211] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599985898client # copying 1 paths to 'lb:50051'...5899client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5900worker1 # [ 88.740811] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599985901worker1 # [ 88.752906] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5902worker1 # [ 88.773243] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:25Z level=info event=niks3_push_started pid=12945903worker1 # [ 88.877843] nix-grpc-daemon[1294]: time=2026-09-21T14:30:25.883Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5904worker1 # [ 88.969073] nix-daemon[847]: accepted connection from pid 1307, user nix-grpc-daemon (trusted)5905worker1 # [ 88.981126] nix-daemon[847]: reaped child process 1310, status = succeeded5906worker1 # [ 89.068477] nix-daemon[847]: accepted connection from pid 1312, user nix-grpc-daemon (trusted)5907worker1 # [ 89.081531] nix-daemon[847]: reaped child process 1315, status = succeeded5908lb # [ 89.078612] niks3-server[1129]: 2026/09/21 14:30:26 INFO Received uploads request method=POST path=/api/pending_closures5909worker1 # [ 89.101924] nix-grpc-daemon[1294]: time=2026-09-21T14:30:26.108Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5910worker1 # [ 89.103439] nix-grpc-daemon[1294]: time=2026-09-21T14:30:26.109Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5911lb # [ 89.118071] niks3-server[1129]: 2026/09/21 14:30:26 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5912lb # [ 89.130801] niks3-server[1129]: 2026/09/21 14:30:26 INFO Received sign narinfos request method=POST path=/api/pending_closures/33/sign5913worker1 # [ 89.152667] nix-grpc-daemon[1294]: time=2026-09-21T14:30:26.158Z level=INFO msg="Uploading 1 narinfos"5914lb # [ 89.139277] niks3-server[1129]: 2026/09/21 14:30:26 INFO Signed narinfos id=33 count=15915lb # [ 89.148764] niks3-server[1129]: 2026/09/21 14:30:26 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5916lb # [ 89.159315] niks3-server[1129]: 2026/09/21 14:30:26 INFO Received complete upload request method=POST path=/api/pending_closures/33/complete5917lb # [ 89.163464] niks3-server[1129]: 2026/09/21 14:30:26 INFO Completed upload id=335918worker1 # [ 89.177908] nix-grpc-daemon[1294]: time=2026-09-21T14:30:26.184Z level=INFO msg="Upload complete. (296ms)"5919lb # [ 89.165963] niks3-server[1129]: 2026/09/21 14:30:26 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5920worker1 # [ 89.182277] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:26Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59998 duration_s=0 paths=1 nar_bytes_in=5205921worker1 # [ 89.188521] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599985922worker1 # [ 89.199122] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5923worker1 # [ 89.204675] nix-daemon[847]: reaped child process 1292, status = succeeded5924worker1 # [ 89.209583] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5925client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5926client # farm-a> node-a: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5927client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5928client # farm-a> LOG-farm-a-on-standby5929worker1 # [ 90.047872] nix-daemon[847]: reaped child process 1319, status = succeeded5930worker1 # [ 90.135773] nix-daemon[847]: accepted connection from pid 1335, user nix-grpc-daemon (trusted)5931worker1 # [ 90.150993] nix-daemon[847]: reaped child process 1338, status = succeeded5932lb # [ 90.145875] niks3-server[1129]: 2026/09/21 14:30:27 INFO Received uploads request method=POST path=/api/pending_closures5933worker1 # [ 90.169354] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.175Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5934worker1 # [ 90.173145] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.177Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"5935lb # [ 90.181171] niks3-server[1129]: 2026/09/21 14:30:27 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5936lb # [ 90.197642] niks3-server[1129]: 2026/09/21 14:30:27 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst5937lb # [ 90.206156] niks3-server[1129]: 2026/09/21 14:30:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/34/sign5938lb # [ 90.210268] niks3-server[1129]: 2026/09/21 14:30:27 INFO Signed narinfos id=34 count=15939worker1 # [ 90.226626] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.232Z level=INFO msg="Uploading 1 narinfos"5940lb # [ 90.218558] niks3-server[1129]: 2026/09/21 14:30:27 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls5941lb # [ 90.223628] niks3-server[1129]: 2026/09/21 14:30:27 INFO Received complete upload request method=POST path=/api/pending_closures/34/complete5942lb # [ 90.227072] niks3-server[1129]: 2026/09/21 14:30:27 INFO Completed upload id=345943worker1 # [ 90.240300] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.246Z level=INFO msg="Upload complete. (221ms)"5944lb # [ 90.231063] niks3-server[1129]: 2026/09/21 14:30:27 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo5945worker1 # [ 90.250864] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:27Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59998 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=15946worker1 # [ 90.260101] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:27Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=05947worker1 # [ 90.263193] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:27Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.3:50051 clients=15948worker1 # [ 90.270310] nix-daemon[847]: reaped child process 1317, status = succeeded5949worker1 # [ 90.302884] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599985950worker1 # [ 90.314522] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:27Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599985951worker1 # [ 90.328989] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.335Z level=INFO msg="All 1 paths already cached"5952client # copying 1 paths to 'lb:50051'...5953client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...5954worker1 # [ 90.337320] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599985955worker1 # [ 90.345634] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5956worker1 # [ 90.443857] nix-daemon[847]: accepted connection from pid 1344, user nix-grpc-daemon (trusted)5957worker1 # [ 90.457930] nix-daemon[847]: reaped child process 1347, status = succeeded5958worker1 # [ 90.543977] nix-daemon[847]: accepted connection from pid 1349, user nix-grpc-daemon (trusted)5959worker1 # [ 90.558646] nix-daemon[847]: reaped child process 1352, status = succeeded5960lb # [ 90.551592] niks3-server[1129]: 2026/09/21 14:30:27 INFO Received uploads request method=POST path=/api/pending_closures5961worker1 # [ 90.573566] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.580Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5962worker1 # [ 90.575730] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.581Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"5963lb # [ 90.584797] niks3-server[1129]: 2026/09/21 14:30:27 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst5964lb # [ 90.593925] niks3-server[1129]: 2026/09/21 14:30:27 INFO Received sign narinfos request method=POST path=/api/pending_closures/35/sign5965worker1 # [ 90.611975] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.618Z level=INFO msg="Uploading 1 narinfos"5966lb # [ 90.598863] niks3-server[1129]: 2026/09/21 14:30:27 INFO Signed narinfos id=35 count=15967lb # [ 90.606067] niks3-server[1129]: 2026/09/21 14:30:27 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls5968lb # [ 90.620539] niks3-server[1129]: 2026/09/21 14:30:27 INFO Received complete upload request method=POST path=/api/pending_closures/35/complete5969lb # [ 90.626932] niks3-server[1129]: 2026/09/21 14:30:27 INFO Completed upload id=355970worker1 # [ 90.640191] nix-grpc-daemon[1294]: time=2026-09-21T14:30:27.646Z level=INFO msg="Upload complete. (282ms)"5971worker1 # [ 90.643539] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59998 duration_s=0 paths=1 nar_bytes_in=6405972lb # [ 90.630996] niks3-server[1129]: 2026/09/21 14:30:27 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo5973worker1 # [ 90.648905] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599985974worker1 # [ 90.659147] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5975worker1 # [ 90.666156] nix-daemon[847]: reaped child process 1342, status = succeeded5976worker1 # [ 90.669801] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)5977client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...5978client # farm-b> node-a: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5979client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5980client # farm-b> LOG-farm-b-on-standby5981worker1 # [ 92.108623] nix-daemon[847]: reaped child process 1356, status = succeeded5982worker1 # [ 92.192462] nix-daemon[847]: accepted connection from pid 1372, user nix-grpc-daemon (trusted)5983worker1 # [ 92.206791] nix-daemon[847]: reaped child process 1375, status = succeeded5984lb # [ 92.203443] niks3-server[1129]: 2026/09/21 14:30:29 INFO Received uploads request method=POST path=/api/pending_closures5985worker1 # [ 92.225118] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.231Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5986worker1 # [ 92.226646] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.233Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"5987lb # [ 92.237823] niks3-server[1129]: 2026/09/21 14:30:29 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv5988lb # [ 92.256214] niks3-server[1129]: 2026/09/21 14:30:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/36/sign5989lb # [ 92.262715] niks3-server[1129]: 2026/09/21 14:30:29 INFO Signed narinfos id=36 count=15990worker1 # [ 92.276133] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.282Z level=INFO msg="Uploading 1 narinfos"5991lb # [ 92.271409] niks3-server[1129]: 2026/09/21 14:30:29 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst5992lb # [ 92.275147] niks3-server[1129]: 2026/09/21 14:30:29 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls5993lb # [ 92.280557] niks3-server[1129]: 2026/09/21 14:30:29 INFO Received complete upload request method=POST path=/api/pending_closures/36/complete5994lb # [ 92.284741] niks3-server[1129]: 2026/09/21 14:30:29 INFO Completed upload id=365995worker1 # [ 92.297980] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.304Z level=INFO msg="Upload complete. (219ms)"5996lb # [ 92.286747] niks3-server[1129]: 2026/09/21 14:30:29 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo5997worker1 # [ 92.310470] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:29Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59998 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=15998worker1 # [ 92.314996] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:29Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=05999worker1 # [ 92.319197] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:29Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.3:50051 clients=16000worker1 # [ 92.327899] nix-daemon[847]: reaped child process 1354, status = succeeded6001worker1 # [ 92.353744] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599986002worker1 # [ 92.366454] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599986003worker1 # [ 92.379756] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.386Z level=INFO msg="All 2 paths already cached"6004client # copying 1 paths to 'lb:50051'...6005client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6006worker1 # [ 92.388210] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599986007worker1 # [ 92.396557] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)6008worker1 # [ 92.493983] nix-daemon[847]: accepted connection from pid 1381, user nix-grpc-daemon (trusted)6009worker1 # [ 92.508054] nix-daemon[847]: reaped child process 1384, status = succeeded6010worker1 # [ 92.593266] nix-daemon[847]: accepted connection from pid 1386, user nix-grpc-daemon (trusted)6011worker1 # [ 92.607976] nix-daemon[847]: reaped child process 1389, status = succeeded6012lb # [ 92.600873] niks3-server[1129]: 2026/09/21 14:30:29 INFO Received uploads request method=POST path=/api/pending_closures6013worker1 # [ 92.623233] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.629Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6014worker1 # [ 92.625148] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.631Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6015lb # [ 92.634633] niks3-server[1129]: 2026/09/21 14:30:29 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6016lb # [ 92.641425] niks3-server[1129]: 2026/09/21 14:30:29 INFO Received sign narinfos request method=POST path=/api/pending_closures/37/sign6017lb # [ 92.646687] niks3-server[1129]: 2026/09/21 14:30:29 INFO Signed narinfos id=37 count=16018worker1 # [ 92.662760] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.669Z level=INFO msg="Uploading 1 narinfos"6019lb # [ 92.661516] niks3-server[1129]: 2026/09/21 14:30:29 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6020lb # [ 92.668274] niks3-server[1129]: 2026/09/21 14:30:29 INFO Received complete upload request method=POST path=/api/pending_closures/37/complete6021lb # [ 92.675421] niks3-server[1129]: 2026/09/21 14:30:29 INFO Completed upload id=376022worker1 # [ 92.688691] nix-grpc-daemon[1294]: time=2026-09-21T14:30:29.695Z level=INFO msg="Upload complete. (279ms)"6023lb # [ 92.678135] niks3-server[1129]: 2026/09/21 14:30:29 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6024worker1 # [ 92.692977] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59998 duration_s=0 paths=1 nar_bytes_in=7686025worker1 # [ 92.698284] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599986026worker1 # [ 92.711067] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)6027worker1 # [ 92.713965] nix-daemon[847]: reaped child process 1379, status = succeeded6028worker1 # [ 92.719185] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)6029client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6030client # farm-top> node-a: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6031client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6032client # farm-top> LOG-farm-top-on-standby6033worker1 # [ 93.358871] nix-daemon[847]: reaped child process 1393, status = succeeded6034worker1 # [ 93.440110] nix-daemon[847]: accepted connection from pid 1409, user nix-grpc-daemon (trusted)6035worker1 # [ 93.453344] nix-daemon[847]: reaped child process 1412, status = succeeded6036lb # [ 93.450490] niks3-server[1129]: 2026/09/21 14:30:30 INFO Received uploads request method=POST path=/api/pending_closures6037worker1 # [ 93.471215] nix-grpc-daemon[1294]: time=2026-09-21T14:30:30.477Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6038worker1 # [ 93.474178] nix-grpc-daemon[1294]: time=2026-09-21T14:30:30.479Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6039lb # [ 93.487166] niks3-server[1129]: 2026/09/21 14:30:30 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6040lb # [ 93.497529] niks3-server[1129]: 2026/09/21 14:30:30 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6041lb # [ 93.505554] niks3-server[1129]: 2026/09/21 14:30:30 INFO Received sign narinfos request method=POST path=/api/pending_closures/38/sign6042lb # [ 93.511244] niks3-server[1129]: 2026/09/21 14:30:30 INFO Signed narinfos id=38 count=16043worker1 # [ 93.526866] nix-grpc-daemon[1294]: time=2026-09-21T14:30:30.533Z level=INFO msg="Uploading 1 narinfos"6044lb # [ 93.519405] niks3-server[1129]: 2026/09/21 14:30:30 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6045lb # [ 93.527429] niks3-server[1129]: 2026/09/21 14:30:30 INFO Received complete upload request method=POST path=/api/pending_closures/38/complete6046lb # [ 93.533567] niks3-server[1129]: 2026/09/21 14:30:30 INFO Completed upload id=386047worker1 # [ 93.546974] nix-grpc-daemon[1294]: time=2026-09-21T14:30:30.553Z level=INFO msg="Upload complete. (221ms)"6048lb # [ 93.535863] niks3-server[1129]: 2026/09/21 14:30:30 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6049worker1 # [ 93.554862] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:30Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59998 duration_s=0 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16050worker1 # [ 93.569169] nix-daemon[847]: reaped child process 1391, status = succeeded6051client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 5.09 seconds)6052client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6053client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6054worker2 # [ 93.762584] systemd[1]: Listening on Nix worker-protocol over gRPC.6055worker2 # [ 93.769133] systemd[1]: Starting scheduler-token.service...6056worker2 # [ 93.812501] systemd[1]: scheduler-token.service: Deactivated successfully.6057worker2 # [ 93.814170] systemd[1]: Finished scheduler-token.service.6058worker2 # [ 93.815625] systemd[1]: scheduler-token.service: Consumed 21ms CPU time over 45ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6059worker2 # [ 93.819849] systemd[1]: Starting Nix worker-protocol over gRPC...6060worker2 # [ 93.895117] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6061worker2 # [ 93.897094] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6062worker2 # [ 93.900406] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16063worker2 # [ 93.904707] nix-grpc-daemon[1330]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6064worker2 # [ 93.905995] nix-grpc-daemon[1330]: E0000 00:00:1790001031.021002 1330 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6065worker2 # [ 93.911778] nix-grpc-daemon[1330]: E0000 00:00:1790001031.026266 1330 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6066worker2 # [ 93.915960] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:31Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6067worker2 # [ 93.918884] systemd[1]: Started Nix worker-protocol over gRPC.6068worker2 # [ 93.923037] nix-grpc-daemon[1330]: E0000 00:00:1790001031.036133 1371 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6069worker2 # [ 93.934468] nix-grpc-daemon[1330]: E0000 00:00:1790001031.051149 1371 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6070worker2 # [ 93.937142] nix-grpc-daemon[1330]: E0000 00:00:1790001031.053417 1371 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6071lb: must succeed: curl -sf localhost:9901/clusters6072worker2 # [ 93.949157] nix-grpc-daemon[1330]: E0000 00:00:1790001031.065841 1371 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6073worker2 # [ 93.960536] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:31Z level=info event=scheduler_connected addr=lb:500516074worker1 # [ 93.960209] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:30Z level=info event=oidc_jwks provider=mock keys=16075worker1 # [ 93.964229] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:30Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:407386076worker1 # [ 93.968164] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:30Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06077lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6078worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6079worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6080lb: must succeed: curl -sf localhost:9901/clusters6081lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6082lb: must succeed: curl -sf localhost:9901/clusters6083lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6084worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6085worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6086(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 10.21 seconds)6087subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6088client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6089client # [ 94.167386] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6090client # Running as unit: rs.service; invocation ID: cff1013a5a7747679e9a7a18f7b3f95e6091client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6092worker2 # [ 94.332751] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:31Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:589726093worker2 # [ 94.343506] nix-daemon[867]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)6094client # [ 94.327238] nix[1193]: don't know how to build these paths:6095client # [ 94.328190] nix[1193]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6096worker1 # [ 94.350616] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:31Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06097worker1 # [ 94.355871] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:31Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.3:50051 clients=16098worker1 # [ 94.395483] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599986099worker1 # [ 94.408173] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:31Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:599986100client # [ 94.402193] nix[1193]: copying 1 paths to 'lb:50051'...6101client # [ 94.404835] nix[1193]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6102worker1 # [ 94.421680] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:31Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:599986103worker1 # [ 94.432402] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)6104worker1 # [ 94.535650] nix-daemon[847]: accepted connection from pid 1425, user nix-grpc-daemon (trusted)6105worker1 # [ 94.550751] nix-daemon[847]: reaped child process 1428, status = succeeded6106worker1 # [ 94.634448] nix-daemon[847]: accepted connection from pid 1430, user nix-grpc-daemon (trusted)6107worker1 # [ 94.648312] nix-daemon[847]: reaped child process 1433, status = succeeded6108lb # [ 94.643364] niks3-server[1129]: 2026/09/21 14:30:31 INFO Received uploads request method=POST path=/api/pending_closures6109worker1 # [ 94.665890] nix-grpc-daemon[1294]: time=2026-09-21T14:30:31.671Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6110worker1 # [ 94.667353] nix-grpc-daemon[1294]: time=2026-09-21T14:30:31.671Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6111lb # [ 94.672200] niks3-server[1129]: 2026/09/21 14:30:31 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6112lb # [ 94.680945] niks3-server[1129]: 2026/09/21 14:30:31 INFO Received sign narinfos request method=POST path=/api/pending_closures/39/sign6113worker1 # [ 94.703327] nix-grpc-daemon[1294]: time=2026-09-21T14:30:31.709Z level=INFO msg="Uploading 1 narinfos"6114lb # [ 94.690162] niks3-server[1129]: 2026/09/21 14:30:31 INFO Signed narinfos id=39 count=16115lb # [ 94.693677] niks3-server[1129]: 2026/09/21 14:30:31 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6116lb # [ 94.707131] niks3-server[1129]: 2026/09/21 14:30:31 INFO Received complete upload request method=POST path=/api/pending_closures/39/complete6117lb # [ 94.714910] niks3-server[1129]: 2026/09/21 14:30:31 INFO Completed upload id=396118worker1 # [ 94.727598] nix-grpc-daemon[1294]: time=2026-09-21T14:30:31.734Z level=INFO msg="Upload complete. (283ms)"6119lb # [ 94.716846] niks3-server[1129]: 2026/09/21 14:30:31 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6120worker1 # [ 94.732366] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:31Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:59998 duration_s=0 paths=1 nar_bytes_in=4166121worker1 # [ 94.737127] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:599986122worker1 # [ 94.748067] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)6123worker1 # [ 94.751545] nix-daemon[847]: reaped child process 1423, status = succeeded6124worker1 # [ 94.756405] nix-daemon[847]: accepted connection from pid 1218, user nix-grpc-daemon (trusted)6125client # [ 94.740403] nix[1193]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6126lb: must succeed: curl -sf localhost:9901/clusters6127lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6128lb: must succeed: curl -sf localhost:9901/clusters6129lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6130lb: must succeed: curl -sf localhost:9901/clusters6131lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6132lb: must succeed: curl -sf localhost:9901/clusters6133lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6134worker2: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6135worker2 # [ 95.774458] systemd-journald[284]: Received client request to rotate journal, rotating.6136worker2 # [ 95.794412] systemd-journald[284]: Vacuuming done, freed 0B of archived journals from /var/log/journal/45538d3f52864602867ade9a001be247.6137lb # [ 95.648628] envoy[906]: conn peer=[2001:db8:1::1]:35696 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9547 tx=4008 ms=49256138worker2: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6139worker1: must succeed: systemctl restart --no-block nix-grpc-daemon.service6140worker1 # [ 95.695723] systemd[1]: Stopping Nix worker-protocol over gRPC...6141worker1: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6142client: must succeed: sleep 5.06143worker1 # [ 96.092193] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:33Z level=info event=draining6144client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6145worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6146worker1 # [ 100.993486] nix-daemon[847]: reaped child process 1437, status = succeeded6147worker1 # [ 101.072105] nix-daemon[847]: accepted connection from pid 1474, user nix-grpc-daemon (trusted)6148worker1 # [ 101.084835] nix-daemon[847]: reaped child process 1477, status = succeeded6149lb # [ 101.080945] niks3-server[1129]: 2026/09/21 14:30:38 INFO Received uploads request method=POST path=/api/pending_closures6150worker1 # [ 101.103339] nix-grpc-daemon[1294]: time=2026-09-21T14:30:38.109Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6151worker1 # [ 101.105280] nix-grpc-daemon[1294]: time=2026-09-21T14:30:38.111Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6152lb # [ 101.125866] niks3-server[1129]: 2026/09/21 14:30:38 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6153lb # [ 101.131787] niks3-server[1129]: 2026/09/21 14:30:38 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6154lb # [ 101.146235] niks3-server[1129]: 2026/09/21 14:30:38 INFO Received sign narinfos request method=POST path=/api/pending_closures/40/sign6155lb # [ 101.151582] niks3-server[1129]: 2026/09/21 14:30:38 INFO Signed narinfos id=40 count=16156worker1 # [ 101.165416] nix-grpc-daemon[1294]: time=2026-09-21T14:30:38.171Z level=INFO msg="Uploading 1 narinfos"6157lb # [ 101.159510] niks3-server[1129]: 2026/09/21 14:30:38 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6158lb # [ 101.163516] niks3-server[1129]: 2026/09/21 14:30:38 INFO Received complete upload request method=POST path=/api/pending_closures/40/complete6159lb # [ 101.167845] niks3-server[1129]: 2026/09/21 14:30:38 INFO Completed upload id=406160worker1 # [ 101.181070] nix-grpc-daemon[1294]: time=2026-09-21T14:30:38.187Z level=INFO msg="Upload complete. (212ms)"6161lb # [ 101.170145] niks3-server[1129]: 2026/09/21 14:30:38 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6162worker1 # [ 101.190179] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:59998 duration_s=6 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16163worker1 # [ 101.203754] nix-daemon[847]: reaped child process 1435, status = succeeded6164client # [ 101.202222] systemd[1]: rs.service: Deactivated successfully.6165client # [ 101.206139] systemd[1]: rs.service: Consumed 128ms CPU time over 7.037s wall clock time, 41.4M memory peak, 7.2K incoming IP traffic, 8.4K outgoing IP traffic.6166worker1 # [ 102.094247] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:39Z level=info event=signal_exit6167lb # [ 102.187860] niks3-server[1129]: 2026/09/21 14:30:39 INFO lead: released remote=192.168.1.3:494986168worker2 # [ 103.213606] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:40Z level=info event=scheduler_restarting addr=lb:500516169worker1 # [ 103.206698] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:40Z level=info event=scheduler_restarting6170worker1 # [ 103.210357] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:40Z level=info event=scheduler_restarting addr=lb:500516171worker1 # [ 103.213645] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:407386172worker1 # [ 103.217370] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:40Z level=info event=worker_gone addr=192.168.1.4:500516173worker1 # [ 103.220862] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:40Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:407386174worker1 # [ 103.224449] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:40Z level=info event=worker_gone addr=192.168.1.3:500516175worker1 # [ 103.242791] nix-daemon[847]: reaped child process 1272, status = succeeded6176worker2 # [ 103.413876] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:40Z level=info event=scheduler_connected addr=lb:500516177worker2 # [ 103.418195] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6178worker1 # [ 103.415627] nix-grpc-daemon[1218]: ts=2026-09-21T14:30:40Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6179worker2 # [ 103.619072] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:40Z level=info event=scheduler_connected addr=lb:500516180worker2 # [ 103.623133] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6181worker2 # [ 103.824157] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:40Z level=info event=scheduler_connected addr=lb:500516182worker2 # [ 103.828270] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:40Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6183worker2 # [ 103.924147] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=scheduler_take_over6184lb # [ 103.900949] niks3-server[1129]: 2026/09/21 14:30:41 INFO lead: acquired remote=192.168.1.4:573346185worker2 # [ 104.029996] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=scheduler_connected addr=lb:500516186worker2 # [ 104.034162] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6187worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 3.16 seconds)6188worker1 # [ 104.066942] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6189client: waiting for success: ! systemctl is-active rs6190worker1 # [ 104.070680] systemd[1]: Stopped Nix worker-protocol over gRPC.6191worker1 # [ 104.072713] systemd[1]: nix-grpc-daemon.service: Consumed 1.367s CPU time over 32.093s wall clock time, 39.5M memory peak, 48.3K incoming IP traffic, 99.8K outgoing IP traffic.6192worker1 # [ 104.087489] systemd[1]: Starting Nix worker-protocol over gRPC...6193client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6194client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6195client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6196client: must fail: journalctl -u rs | grep -q 'reconnecting'6197client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.02 seconds)6198lb: must succeed: curl -sf localhost:9901/clusters6199worker1 # [ 104.182777] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:41Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6200worker1 # [ 104.187243] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:41Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6201worker1 # [ 104.191907] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:41Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16202worker1 # [ 104.197153] nix-grpc-daemon[1484]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6203worker1 # [ 104.198517] nix-grpc-daemon[1484]: E0000 00:00:1790001041.196452 1484 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6204worker1 # [ 104.200656] nix-grpc-daemon[1484]: E0000 00:00:1790001041.198172 1484 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6205worker1 # [ 104.202712] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:41Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6206worker1 # [ 104.209239] systemd[1]: Started Nix worker-protocol over gRPC.6207lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6208worker1 # [ 104.212076] nix-grpc-daemon[1484]: E0000 00:00:1790001041.218062 1526 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6209worker1 # [ 104.224357] nix-grpc-daemon[1484]: E0000 00:00:1790001041.230835 1526 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6210worker2 # [ 104.236702] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=scheduler_connected addr=lb:500516211worker1 # [ 104.229061] nix-grpc-daemon[1484]: E0000 00:00:1790001041.233190 1526 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6212worker2 # [ 104.240129] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6213worker1 # [ 104.238482] nix-grpc-daemon[1484]: E0000 00:00:1790001041.244950 1526 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6214worker1 # [ 104.247366] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:41Z level=info event=scheduler_connected addr=lb:500516215worker1 # [ 104.248906] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:41Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6216worker2 # [ 104.437793] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=scheduler_connected addr=lb:500516217worker2 # [ 104.454141] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=oidc_jwks provider=mock keys=16218worker2 # [ 104.458174] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:513566219worker1 # [ 104.450370] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:41Z level=info event=scheduler_connected addr=lb:500516220worker2 # [ 104.463095] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06221worker2 # [ 104.468784] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:513566222worker2 # [ 104.472304] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:41Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06223lb: must succeed: curl -sf localhost:9901/clusters6224lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6225worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6226worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6227(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 11.17 seconds)6228subtest: scheduler restart mid-build: build finishes, no double build6229client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6230client # [ 105.334431] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6231client # Running as unit: mid.service; invocation ID: 0e089a583a79455ca0e31352da5f4e4c6232client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6233worker2 # [ 105.497264] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:589726234client # [ 105.477410] nix[1229]: don't know how to build these paths:6235client # [ 105.478337] nix[1229]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6236worker2 # [ 105.511228] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:42Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06237worker2 # [ 105.515675] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:42Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16238worker2 # [ 105.559425] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589726239worker2 # [ 105.570238] nix-grpc-daemon[1330]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 59 ms (attempt 1/5)6240worker2 # [ 105.636869] nix-grpc-daemon[1330]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 139 ms (attempt 2/5)6241lb # [ 105.649723] envoy[906]: conn peer=[2001:db8:1::1]:59338 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2973 tx=1404 ms=68796242lb # [ 105.653678] envoy[906]: conn peer=[2001:db8:1::3]:52078 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=4382 tx=1078 ms=312756243worker2 # [ 105.778132] nix-grpc-daemon[1330]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 162 ms (attempt 3/5)6244worker2 # [ 105.941950] nix-grpc-daemon[1330]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 579 ms (attempt 4/5)6245worker2 # [ 106.524782] nix-grpc-daemon[1330]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6246worker2 # [ 106.528379] nix-grpc-daemon[1330]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6247worker2 # [ 106.548735] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:43Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:589726248client # [ 106.534884] nix[1229]: copying 1 paths to 'lb:50051'...6249client # [ 106.538570] nix[1229]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6250worker2 # [ 106.565256] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:43Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:589726251worker2 # [ 106.579423] nix-daemon[867]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)6252worker2 # [ 106.600388] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:43Z level=info event=niks3_push_started pid=14556253worker2 # [ 106.689655] nix-grpc-daemon[1455]: time=2026-09-21T14:30:43.805Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6254worker2 # [ 106.772304] nix-daemon[867]: accepted connection from pid 1468, user nix-grpc-daemon (trusted)6255worker2 # [ 106.785713] nix-daemon[867]: reaped child process 1471, status = succeeded6256worker2 # [ 106.865498] nix-daemon[867]: accepted connection from pid 1473, user nix-grpc-daemon (trusted)6257worker2 # [ 106.877581] nix-daemon[867]: reaped child process 1476, status = succeeded6258lb # [ 106.862473] niks3-server[1129]: 2026/09/21 14:30:44 INFO Received uploads request method=POST path=/api/pending_closures6259worker2 # [ 106.894469] nix-grpc-daemon[1455]: time=2026-09-21T14:30:44.011Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6260worker2 # [ 106.897136] nix-grpc-daemon[1455]: time=2026-09-21T14:30:44.012Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6261lb # [ 106.901164] niks3-server[1129]: 2026/09/21 14:30:44 INFO Received sign narinfos request method=POST path=/api/pending_closures/41/sign6262lb # [ 106.912312] niks3-server[1129]: 2026/09/21 14:30:44 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6263lb # [ 106.932963] niks3-server[1129]: 2026/09/21 14:30:44 INFO Signed narinfos id=41 count=16264worker2 # [ 106.956887] nix-grpc-daemon[1455]: time=2026-09-21T14:30:44.073Z level=INFO msg="Uploading 1 narinfos"6265lb # [ 106.941274] niks3-server[1129]: 2026/09/21 14:30:44 INFO Received complete upload request method=POST path=/api/pending_closures/41/complete6266lb # [ 106.945831] niks3-server[1129]: 2026/09/21 14:30:44 INFO Completed upload id=416267worker2 # [ 106.969353] nix-grpc-daemon[1455]: time=2026-09-21T14:30:44.085Z level=INFO msg="Upload complete. (276ms)"6268lb # [ 106.951186] niks3-server[1129]: 2026/09/21 14:30:44 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6269worker2 # [ 106.973427] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58972 duration_s=0 paths=1 nar_bytes_in=4246270lb # [ 106.953542] niks3-server[1129]: 2026/09/21 14:30:44 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6271worker2 # [ 106.979438] nix-grpc-daemon[1330]: ts=2026-09-21T14:30:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:589726272worker2 # [ 106.989112] nix-daemon[867]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)6273worker2 # [ 106.996621] nix-daemon[867]: reaped child process 1453, status = succeeded6274worker2 # [ 107.000997] nix-daemon[867]: accepted connection from pid 1330, user nix-grpc-daemon (trusted)6275client # [ 106.974719] nix[1229]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6276lb: must succeed: curl -sf localhost:9901/clusters6277lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6278lb: must succeed: curl -sf localhost:9901/clusters6279lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6280worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6281worker2 # [ 107.891397] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1330 (nix-grpc-daemon) on client request.6282worker2 # [ 107.893037] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6283lb # [ 107.875751] niks3-server[1129]: 2026/09/21 14:30:45 INFO lead: released remote=192.168.1.4:573346284worker2 # [ 107.906102] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6285client # [ 107.877298] nix[1229]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6286worker2 # [ 107.907627] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6287worker1 # [ 107.892946] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:44Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6288client # [ 107.881884] nix[1229]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6289worker2 # [ 107.910133] systemd[1]: nix-grpc-daemon.service: Consumed 416ms CPU time over 14.084s wall clock time, 35.6M memory peak, 15.8K incoming IP traffic, 28.4K outgoing IP traffic.6290worker2 # [ 107.918839] systemd[1]: Starting scheduler-token.service...6291worker2 # [ 107.943412] nix-daemon[867]: reaped child process 1478, status = succeeded6292worker2 # [ 107.945135] nix-daemon[867]: reaped child process 1389, status = succeeded6293worker2 # [ 107.953190] nix-daemon[1480]: unexpected Nix daemon error: error: interrupted by the user6294worker2 # [ 107.961087] nix-daemon[867]: reaped child process 1480, status = failed with exit code 16295worker2 # [ 107.975872] systemd[1]: scheduler-token.service: Deactivated successfully.6296worker2 # [ 107.977445] systemd[1]: Finished scheduler-token.service.6297worker2 # [ 107.978849] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 58ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6298worker2 # [ 107.981599] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6299worker2 # [ 107.985136] systemd[1]: Starting scheduler-token.service...6300worker2 # [ 108.022376] systemd[1]: scheduler-token.service: Deactivated successfully.6301worker2 # [ 108.023938] systemd[1]: Finished scheduler-token.service.6302worker2 # [ 108.025278] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 38ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6303worker2 # [ 108.029738] systemd[1]: Starting Nix worker-protocol over gRPC...6304worker2 # [ 108.105374] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6305worker1 # [ 108.093394] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:45Z level=info event=scheduler_connected addr=lb:500516306worker2 # [ 108.110189] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6307worker2 # [ 108.113305] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16308worker2 # [ 108.118105] nix-grpc-daemon[1519]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6309worker2 # [ 108.122109] nix-grpc-daemon[1519]: E0000 00:00:1790001045.232416 1519 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6310worker2 # [ 108.125613] nix-grpc-daemon[1519]: E0000 00:00:1790001045.233724 1519 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6311worker2 # [ 108.127770] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6312worker2 # [ 108.131655] systemd[1]: Started Nix worker-protocol over gRPC.6313worker2 # [ 108.134930] nix-grpc-daemon[1519]: E0000 00:00:1790001045.249491 1559 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6314lb # [ 108.111426] niks3-server[1129]: 2026/09/21 14:30:45 INFO lead: acquired remote=192.168.1.4:493326315worker2 # [ 108.138169] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=scheduler_take_over6316worker2 # [ 108.141707] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:513706317worker2 # [ 108.145107] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06318worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.28 seconds)6319client: must succeed: sleep 5.06320worker2 # [ 108.153255] nix-grpc-daemon[1519]: E0000 00:00:1790001045.269945 1559 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6321worker2 # [ 108.156194] nix-grpc-daemon[1519]: E0000 00:00:1790001045.272078 1559 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6322worker2 # [ 108.167451] nix-grpc-daemon[1519]: E0000 00:00:1790001045.284086 1559 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6323worker2 # [ 108.176448] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=scheduler_connected addr=lb:500516324worker2 # [ 108.181761] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=oidc_jwks provider=mock keys=16325worker2 # [ 108.183139] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:513706326worker2 # [ 108.184775] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06327worker2 # [ 108.414789] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06328worker2 # [ 108.419221] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:45Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16329worker1 # [ 108.457760] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564246330worker1 # [ 108.469828] nix-daemon[847]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)6331worker1 # [ 108.486509] nix-grpc-daemon[1484]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 20 ms (attempt 1/5)6332worker1 # [ 108.514092] nix-grpc-daemon[1484]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 200 ms (attempt 2/5)6333worker1 # [ 108.716409] nix-grpc-daemon[1484]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 229 ms (attempt 3/5)6334worker1 # [ 108.948761] nix-grpc-daemon[1484]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 567 ms (attempt 4/5)6335worker1 # [ 109.517751] nix-grpc-daemon[1484]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6336worker1 # [ 109.521304] nix-grpc-daemon[1484]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6337worker1 # [ 109.537139] nix-grpc-daemon[1484]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6338worker1 # [ 109.561880] nix-daemon[847]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)6339worker1 # [ 109.575276] nix-daemon[847]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)6340client # [ 109.559066] nix[1229]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6341client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6342client: waiting for success: ! systemctl is-active mid6343worker1 # [ 113.397070] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:50Z level=info event=niks3_push_started pid=15866344worker1 # [ 113.411680] nix-daemon[847]: reaped child process 1561, status = succeeded6345worker1 # [ 113.498710] nix-grpc-daemon[1586]: time=2026-09-21T14:30:50.504Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6346worker1 # [ 113.587964] nix-daemon[847]: accepted connection from pid 1599, user nix-grpc-daemon (trusted)6347worker1 # [ 113.602202] nix-daemon[847]: reaped child process 1602, status = succeeded6348lb # [ 113.598829] niks3-server[1129]: 2026/09/21 14:30:50 INFO Received uploads request method=POST path=/api/pending_closures6349worker1 # [ 113.624202] nix-grpc-daemon[1586]: time=2026-09-21T14:30:50.630Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6350worker1 # [ 113.625704] nix-grpc-daemon[1586]: time=2026-09-21T14:30:50.632Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6351lb # [ 113.649965] niks3-server[1129]: 2026/09/21 14:30:50 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6352lb # [ 113.656488] niks3-server[1129]: 2026/09/21 14:30:50 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6353lb # [ 113.670077] niks3-server[1129]: 2026/09/21 14:30:50 INFO Received sign narinfos request method=POST path=/api/pending_closures/42/sign6354lb # [ 113.673168] niks3-server[1129]: 2026/09/21 14:30:50 INFO Signed narinfos id=42 count=16355worker1 # [ 113.686492] nix-grpc-daemon[1586]: time=2026-09-21T14:30:50.692Z level=INFO msg="Uploading 1 narinfos"6356lb # [ 113.676656] niks3-server[1129]: 2026/09/21 14:30:50 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6357lb # [ 113.682067] niks3-server[1129]: 2026/09/21 14:30:50 INFO Received complete upload request method=POST path=/api/pending_closures/42/complete6358lb # [ 113.690624] niks3-server[1129]: 2026/09/21 14:30:50 INFO Completed upload id=426359lb # [ 113.692858] niks3-server[1129]: 2026/09/21 14:30:50 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6360worker1 # [ 113.703838] nix-grpc-daemon[1586]: time=2026-09-21T14:30:50.710Z level=INFO msg="Upload complete. (202ms)"6361worker1 # [ 113.710189] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:50Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56424 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16362worker1 # [ 113.721877] nix-daemon[847]: reaped child process 1559, status = succeeded6363client # [ 113.725524] systemd[1]: mid.service: Deactivated successfully.6364client # [ 113.729271] systemd[1]: mid.service: Consumed 141ms CPU time over 8.395s wall clock time, 41.4M memory peak, 8.7K incoming IP traffic, 10.4K outgoing IP traffic.6365client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6366client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6367client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6368(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.16 seconds)6369subtest: interrupting the client stops the build on the worker6370client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6371client # [ 114.467195] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6372client # Running as unit: intr.service; invocation ID: 1b71a77097e44852bd7fbaa19f8d361b6373client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6374client # [ 114.589759] nix[1265]: don't know how to build these paths:6375client # [ 114.590694] nix[1265]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6376worker1 # [ 114.604675] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564246377worker2 # [ 114.626790] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:51Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06378worker2 # [ 114.630891] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:51Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16379worker2 # [ 114.676769] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403426380worker2 # [ 114.687544] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6381worker2 # [ 114.703464] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 76 ms (attempt 1/5)6382worker2 # [ 114.785945] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 136 ms (attempt 2/5)6383worker2 # [ 114.924133] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 21 ms (attempt 3/5)6384worker2 # [ 114.946392] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 112 ms (attempt 4/5)6385worker2 # [ 115.066298] nix-grpc-daemon[1519]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6386worker2 # [ 115.069827] nix-grpc-daemon[1519]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6387worker2 # [ 115.086990] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:52Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403426388client # [ 115.071187] nix[1265]: copying 1 paths to 'lb:50051'...6389client # [ 115.075177] nix[1265]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6390worker2 # [ 115.101804] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:52Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403426391worker2 # [ 115.109918] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6392worker2 # [ 115.132598] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:52Z level=info event=niks3_push_started pid=15956393worker2 # [ 115.232937] nix-grpc-daemon[1595]: time=2026-09-21T14:30:52.348Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6394worker2 # [ 115.315324] nix-daemon[867]: accepted connection from pid 1607, user nix-grpc-daemon (trusted)6395worker2 # [ 115.328871] nix-daemon[867]: reaped child process 1610, status = succeeded6396worker2 # [ 115.409899] nix-daemon[867]: accepted connection from pid 1612, user nix-grpc-daemon (trusted)6397worker2 # [ 115.422450] nix-daemon[867]: reaped child process 1615, status = succeeded6398lb # [ 115.406331] niks3-server[1129]: 2026/09/21 14:30:52 INFO Received uploads request method=POST path=/api/pending_closures6399worker2 # [ 115.438459] nix-grpc-daemon[1595]: time=2026-09-21T14:30:52.555Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6400worker2 # [ 115.440293] nix-grpc-daemon[1595]: time=2026-09-21T14:30:52.556Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6401lb # [ 115.439651] niks3-server[1129]: 2026/09/21 14:30:52 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6402lb # [ 115.450252] niks3-server[1129]: 2026/09/21 14:30:52 INFO Received sign narinfos request method=POST path=/api/pending_closures/43/sign6403worker2 # [ 115.479749] nix-grpc-daemon[1595]: time=2026-09-21T14:30:52.596Z level=INFO msg="Uploading 1 narinfos"6404lb # [ 115.456272] niks3-server[1129]: 2026/09/21 14:30:52 INFO Signed narinfos id=43 count=16405lb # [ 115.471715] niks3-server[1129]: 2026/09/21 14:30:52 INFO Received complete upload request method=POST path=/api/pending_closures/43/complete6406lb # [ 115.479661] niks3-server[1129]: 2026/09/21 14:30:52 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6407lb # [ 115.484650] niks3-server[1129]: 2026/09/21 14:30:52 INFO Completed upload id=436408worker2 # [ 115.508276] nix-grpc-daemon[1595]: time=2026-09-21T14:30:52.624Z level=INFO msg="Upload complete. (272ms)"6409worker2 # [ 115.513130] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:52Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=0 paths=1 nar_bytes_in=4246410worker2 # [ 115.518168] nix-grpc-daemon[1595]: time=2026-09-21T14:30:52.634Z level=WARN msg="Failed to register uploaded object" key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo error="server returned 404: object is not pending upload\n"6411worker2 # [ 115.522603] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:52Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403426412worker2 # [ 115.531790] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6413worker2 # [ 115.533256] nix-daemon[867]: reaped child process 1593, status = succeeded6414worker2 # [ 115.538457] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6415client # [ 115.511951] nix[1265]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6416lb # [ 115.649632] envoy[906]: conn peer=[2001:db8:1::4]:44328 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=6171 tx=511 ms=139486417lb # [ 115.653413] envoy[906]: conn peer=[2001:db8:1::1]:44742 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1853 ms=82386418client: must succeed: systemctl kill -s INT intr6419worker2 # [ 116.824969] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:53Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6420client # [ 116.797612] systemd[1]: intr.service: Sent signal SIGINT to main process 1265 (nix) on client request.6421client # [ 116.803667] nix[1265]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6422client: (finished: must succeed: systemctl kill -s INT intr, in 0.04 seconds)6423client # [ 116.810444] nix[1265]: error: interrupted by the user6424client # [ 116.835077] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6425client # [ 116.837944] systemd[1]: intr.service: Failed with result 'exit-code'.6426client # [ 116.841468] systemd[1]: intr.service: Consumed 114ms CPU time over 2.370s wall clock time, 43.6M memory peak, 6.5K incoming IP traffic, 8.4K outgoing IP traffic.6427worker2 # [ 116.955801] nix-daemon[867]: reaped child process 1617, status = succeeded6428worker2 # [ 116.957616] nix-daemon[1619]: unexpected Nix daemon error: error: interrupted by the user6429worker2 # [ 116.962359] nix-daemon[867]: reaped child process 1619, status = failed with exit code 16430(finished: subtest: interrupting the client stops the build on the worker, in 3.55 seconds)6431subtest: a deploy mid-build drains: the build finishes, the new generation takes over6432client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6433client # [ 118.038926] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6434client # Running as unit: sw.service; invocation ID: d74f80a3324643a998bcb8ed7bb262166435client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6436worker2 # [ 118.201653] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:55Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:403426437client # [ 118.181465] nix[1286]: don't know how to build these paths:6438client # [ 118.182334] nix[1286]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6439worker2 # [ 118.220225] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:55Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06440worker2 # [ 118.224689] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:55Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16441worker1 # [ 118.254324] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564246442worker1 # [ 118.268274] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564246443client # [ 118.263256] nix[1286]: copying 1 paths to 'lb:50051'...6444worker1 # [ 118.280390] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:564246445client # [ 118.266196] nix[1286]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6446worker1 # [ 118.293460] nix-daemon[847]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)6447worker1 # [ 118.397103] nix-daemon[847]: accepted connection from pid 1630, user nix-grpc-daemon (trusted)6448worker1 # [ 118.410424] nix-daemon[847]: reaped child process 1633, status = succeeded6449worker1 # [ 118.497457] nix-daemon[847]: accepted connection from pid 1635, user nix-grpc-daemon (trusted)6450worker1 # [ 118.511124] nix-daemon[847]: reaped child process 1638, status = succeeded6451lb # [ 118.506895] niks3-server[1129]: 2026/09/21 14:30:55 INFO Received uploads request method=POST path=/api/pending_closures6452worker1 # [ 118.534106] nix-grpc-daemon[1586]: time=2026-09-21T14:30:55.539Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6453worker1 # [ 118.535487] nix-grpc-daemon[1586]: time=2026-09-21T14:30:55.539Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6454lb # [ 118.546726] niks3-server[1129]: 2026/09/21 14:30:55 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6455lb # [ 118.556259] niks3-server[1129]: 2026/09/21 14:30:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/44/sign6456lb # [ 118.562637] niks3-server[1129]: 2026/09/21 14:30:55 INFO Signed narinfos id=44 count=16457worker1 # [ 118.578970] nix-grpc-daemon[1586]: time=2026-09-21T14:30:55.585Z level=INFO msg="Uploading 1 narinfos"6458lb # [ 118.568622] niks3-server[1129]: 2026/09/21 14:30:55 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6459lb # [ 118.581802] niks3-server[1129]: 2026/09/21 14:30:55 INFO Received complete upload request method=POST path=/api/pending_closures/44/complete6460lb # [ 118.586054] niks3-server[1129]: 2026/09/21 14:30:55 INFO Completed upload id=446461worker1 # [ 118.599476] nix-grpc-daemon[1586]: time=2026-09-21T14:30:55.605Z level=INFO msg="Upload complete. (294ms)"6462lb # [ 118.588680] niks3-server[1129]: 2026/09/21 14:30:55 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6463worker1 # [ 118.603481] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56424 duration_s=0 paths=1 nar_bytes_in=4166464worker1 # [ 118.609302] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:564246465worker1 # [ 118.618827] nix-daemon[847]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)6466worker1 # [ 118.628126] nix-daemon[847]: reaped child process 1628, status = succeeded6467worker1 # [ 118.631153] nix-daemon[847]: accepted connection from pid 1484, user nix-grpc-daemon (trusted)6468client # [ 118.615273] nix[1286]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6469worker1: must succeed: systemctl reload nix-grpc-daemon.service6470worker1 # [ 120.488862] systemd[1]: Reloading Nix worker-protocol over gRPC...6471worker1 # [ 120.520400] systemd[1]: Reloaded Nix worker-protocol over gRPC.6472worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.07 seconds)6473worker1: must succeed: pgrep -f 'read -t [9]0 x'6474worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.04 seconds)6475client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26476worker1 # [ 120.705890] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:564246477client # don't know how to build these paths:6478client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6479worker1 # [ 120.711371] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:564246480worker2 # [ 120.744783] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:57Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06481worker2 # [ 120.748796] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:57Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16482worker2 # [ 120.793666] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403426483worker2 # [ 120.806975] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403426484client # copying 1 paths to 'lb:50051'...6485worker2 # [ 120.819842] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403426486client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6487worker2 # [ 120.844824] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6488worker2 # [ 120.948550] nix-daemon[867]: accepted connection from pid 1674, user nix-grpc-daemon (trusted)6489worker2 # [ 120.960822] nix-daemon[867]: reaped child process 1677, status = succeeded6490worker2 # [ 121.041499] nix-daemon[867]: accepted connection from pid 1679, user nix-grpc-daemon (trusted)6491worker2 # [ 121.054332] nix-daemon[867]: reaped child process 1682, status = succeeded6492lb # [ 121.037772] niks3-server[1129]: 2026/09/21 14:30:58 INFO Received uploads request method=POST path=/api/pending_closures6493worker2 # [ 121.070248] nix-grpc-daemon[1595]: time=2026-09-21T14:30:58.186Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6494worker2 # [ 121.071805] nix-grpc-daemon[1595]: time=2026-09-21T14:30:58.188Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6495lb # [ 121.074799] niks3-server[1129]: 2026/09/21 14:30:58 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6496lb # [ 121.085349] niks3-server[1129]: 2026/09/21 14:30:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/45/sign6497worker2 # [ 121.115580] nix-grpc-daemon[1595]: time=2026-09-21T14:30:58.232Z level=INFO msg="Uploading 1 narinfos"6498lb # [ 121.092059] niks3-server[1129]: 2026/09/21 14:30:58 INFO Signed narinfos id=45 count=16499lb # [ 121.107895] niks3-server[1129]: 2026/09/21 14:30:58 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6500lb # [ 121.109630] niks3-server[1129]: 2026/09/21 14:30:58 INFO Received complete upload request method=POST path=/api/pending_closures/45/complete6501lb # [ 121.113349] niks3-server[1129]: 2026/09/21 14:30:58 INFO Completed upload id=456502worker2 # [ 121.138280] nix-grpc-daemon[1595]: time=2026-09-21T14:30:58.254Z level=INFO msg="Upload complete. (279ms)"6503lb # [ 121.116569] niks3-server[1129]: 2026/09/21 14:30:58 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6504worker2 # [ 121.142053] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=0 paths=1 nar_bytes_in=5286505worker2 # [ 121.146781] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403426506worker2 # [ 121.156195] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6507worker2 # [ 121.163252] nix-daemon[867]: reaped child process 1672, status = succeeded6508worker2 # [ 121.167789] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6509client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6510client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6511worker1 # [ 121.205401] nix-grpc-daemon[1484]: ts=2026-09-21T14:30:58Z level=info event=draining6512client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6513client # farm-a> LOG-farm-a-during-drain6514worker2 # [ 122.005264] nix-daemon[867]: reaped child process 1686, status = succeeded6515worker2 # [ 122.087086] nix-daemon[867]: accepted connection from pid 1702, user nix-grpc-daemon (trusted)6516worker2 # [ 122.098753] nix-daemon[867]: reaped child process 1705, status = succeeded6517lb # [ 122.084236] niks3-server[1129]: 2026/09/21 14:30:59 INFO Received uploads request method=POST path=/api/pending_closures6518worker2 # [ 122.117933] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.234Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6519worker2 # [ 122.119443] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.236Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6520lb # [ 122.120875] niks3-server[1129]: 2026/09/21 14:30:59 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6521lb # [ 122.128945] niks3-server[1129]: 2026/09/21 14:30:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/46/sign6522lb # [ 122.131192] niks3-server[1129]: 2026/09/21 14:30:59 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6523lb # [ 122.136070] niks3-server[1129]: 2026/09/21 14:30:59 INFO Signed narinfos id=46 count=16524worker2 # [ 122.159773] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.276Z level=INFO msg="Uploading 1 narinfos"6525lb # [ 122.138320] niks3-server[1129]: 2026/09/21 14:30:59 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6526lb # [ 122.145617] niks3-server[1129]: 2026/09/21 14:30:59 INFO Received complete upload request method=POST path=/api/pending_closures/46/complete6527lb # [ 122.149308] niks3-server[1129]: 2026/09/21 14:30:59 INFO Completed upload id=466528worker2 # [ 122.172901] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.289Z level=INFO msg="Upload complete. (197ms)"6529lb # [ 122.152535] niks3-server[1129]: 2026/09/21 14:30:59 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6530worker2 # [ 122.176291] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:59Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16531worker2 # [ 122.181757] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:59Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06532worker2 # [ 122.184257] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:59Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16533worker2 # [ 122.188328] nix-daemon[867]: reaped child process 1684, status = succeeded6534worker2 # [ 122.225554] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403426535worker2 # [ 122.236729] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:59Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403426536worker2 # [ 122.248156] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.364Z level=INFO msg="All 1 paths already cached"6537client # copying 1 paths to 'lb:50051'...6538client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6539worker2 # [ 122.256355] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:59Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403426540worker2 # [ 122.270132] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6541worker2 # [ 122.363297] nix-daemon[867]: accepted connection from pid 1711, user nix-grpc-daemon (trusted)6542worker2 # [ 122.375939] nix-daemon[867]: reaped child process 1714, status = succeeded6543worker2 # [ 122.455483] nix-daemon[867]: accepted connection from pid 1716, user nix-grpc-daemon (trusted)6544worker2 # [ 122.467786] nix-daemon[867]: reaped child process 1719, status = succeeded6545lb # [ 122.451407] niks3-server[1129]: 2026/09/21 14:30:59 INFO Received uploads request method=POST path=/api/pending_closures6546worker2 # [ 122.484690] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.601Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6547worker2 # [ 122.486213] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.602Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6548lb # [ 122.483064] niks3-server[1129]: 2026/09/21 14:30:59 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6549lb # [ 122.496154] niks3-server[1129]: 2026/09/21 14:30:59 INFO Received sign narinfos request method=POST path=/api/pending_closures/47/sign6550worker2 # [ 122.524895] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.641Z level=INFO msg="Uploading 1 narinfos"6551lb # [ 122.502149] niks3-server[1129]: 2026/09/21 14:30:59 INFO Signed narinfos id=47 count=16552lb # [ 122.507279] niks3-server[1129]: 2026/09/21 14:30:59 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6553lb # [ 122.519531] niks3-server[1129]: 2026/09/21 14:30:59 INFO Received complete upload request method=POST path=/api/pending_closures/47/complete6554lb # [ 122.527129] niks3-server[1129]: 2026/09/21 14:30:59 INFO Completed upload id=476555worker2 # [ 122.550712] nix-grpc-daemon[1595]: time=2026-09-21T14:30:59.667Z level=INFO msg="Upload complete. (269ms)"6556lb # [ 122.529608] niks3-server[1129]: 2026/09/21 14:30:59 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6557worker2 # [ 122.554431] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:59Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=0 paths=1 nar_bytes_in=6406558worker2 # [ 122.559285] nix-grpc-daemon[1519]: ts=2026-09-21T14:30:59Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403426559worker2 # [ 122.569111] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6560worker2 # [ 122.576468] nix-daemon[867]: reaped child process 1709, status = succeeded6561worker2 # [ 122.581256] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6562client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6563client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6564client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6565client # farm-b> LOG-farm-b-during-drain6566worker2 # [ 123.420219] nix-daemon[867]: reaped child process 1723, status = succeeded6567worker2 # [ 123.500191] nix-daemon[867]: accepted connection from pid 1739, user nix-grpc-daemon (trusted)6568worker2 # [ 123.513333] nix-daemon[867]: reaped child process 1742, status = succeeded6569lb # [ 123.496591] niks3-server[1129]: 2026/09/21 14:31:00 INFO Received uploads request method=POST path=/api/pending_closures6570worker2 # [ 123.528883] nix-grpc-daemon[1595]: time=2026-09-21T14:31:00.645Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6571worker2 # [ 123.530465] nix-grpc-daemon[1595]: time=2026-09-21T14:31:00.647Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6572lb # [ 123.534250] niks3-server[1129]: 2026/09/21 14:31:00 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6573lb # [ 123.544226] niks3-server[1129]: 2026/09/21 14:31:00 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6574lb # [ 123.554263] niks3-server[1129]: 2026/09/21 14:31:00 INFO Received sign narinfos request method=POST path=/api/pending_closures/48/sign6575lb # [ 123.558956] niks3-server[1129]: 2026/09/21 14:31:00 INFO Signed narinfos id=48 count=16576worker2 # [ 123.585568] nix-grpc-daemon[1595]: time=2026-09-21T14:31:00.701Z level=INFO msg="Uploading 1 narinfos"6577lb # [ 123.564749] niks3-server[1129]: 2026/09/21 14:31:00 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6578lb # [ 123.574701] niks3-server[1129]: 2026/09/21 14:31:00 INFO Received complete upload request method=POST path=/api/pending_closures/48/complete6579lb # [ 123.578482] niks3-server[1129]: 2026/09/21 14:31:00 INFO Completed upload id=486580lb # [ 123.581150] niks3-server[1129]: 2026/09/21 14:31:00 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6581worker2 # [ 123.603657] nix-grpc-daemon[1595]: time=2026-09-21T14:31:00.720Z level=INFO msg="Upload complete. (215ms)"6582worker2 # [ 123.609882] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=1 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16583worker2 # [ 123.622441] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:00Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06584worker2 # [ 123.625876] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:00Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16585worker2 # [ 123.635356] nix-daemon[867]: reaped child process 1721, status = succeeded6586worker2 # [ 123.665661] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403426587worker2 # [ 123.678673] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403426588worker2 # [ 123.693418] nix-grpc-daemon[1595]: time=2026-09-21T14:31:00.809Z level=INFO msg="All 2 paths already cached"6589client # copying 1 paths to 'lb:50051'...6590client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6591worker2 # [ 123.701936] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403426592worker2 # [ 123.716143] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6593worker2 # [ 123.809724] nix-daemon[867]: accepted connection from pid 1749, user nix-grpc-daemon (trusted)6594worker2 # [ 123.822805] nix-daemon[867]: reaped child process 1752, status = succeeded6595worker2 # [ 123.901959] nix-daemon[867]: accepted connection from pid 1754, user nix-grpc-daemon (trusted)6596worker2 # [ 123.914677] nix-daemon[867]: reaped child process 1757, status = succeeded6597lb # [ 123.897529] niks3-server[1129]: 2026/09/21 14:31:01 INFO Received uploads request method=POST path=/api/pending_closures6598worker2 # [ 123.929499] nix-grpc-daemon[1595]: time=2026-09-21T14:31:01.046Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6599worker2 # [ 123.930963] nix-grpc-daemon[1595]: time=2026-09-21T14:31:01.047Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6600lb # [ 123.928948] niks3-server[1129]: 2026/09/21 14:31:01 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6601lb # [ 123.937769] niks3-server[1129]: 2026/09/21 14:31:01 INFO Received sign narinfos request method=POST path=/api/pending_closures/49/sign6602lb # [ 123.943385] niks3-server[1129]: 2026/09/21 14:31:01 INFO Signed narinfos id=49 count=16603worker2 # [ 123.970521] nix-grpc-daemon[1595]: time=2026-09-21T14:31:01.087Z level=INFO msg="Uploading 1 narinfos"6604lb # [ 123.959114] niks3-server[1129]: 2026/09/21 14:31:01 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6605lb # [ 123.965512] niks3-server[1129]: 2026/09/21 14:31:01 INFO Received complete upload request method=POST path=/api/pending_closures/49/complete6606lb # [ 123.972082] niks3-server[1129]: 2026/09/21 14:31:01 INFO Completed upload id=496607worker2 # [ 123.995784] nix-grpc-daemon[1595]: time=2026-09-21T14:31:01.112Z level=INFO msg="Upload complete. (268ms)"6608lb # [ 123.975771] niks3-server[1129]: 2026/09/21 14:31:01 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6609worker2 # [ 123.999932] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=0 paths=1 nar_bytes_in=7686610worker2 # [ 124.005260] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403426611worker2 # [ 124.014203] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6612worker2 # [ 124.023098] nix-daemon[867]: reaped child process 1746, status = succeeded6613worker2 # [ 124.025963] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)6614client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6615client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6616client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6617client # farm-top> LOG-farm-top-during-drain6618worker2 # [ 125.241236] nix-daemon[867]: reaped child process 1761, status = succeeded6619worker2 # [ 125.315087] nix-daemon[867]: accepted connection from pid 1777, user nix-grpc-daemon (trusted)6620worker2 # [ 125.327366] nix-daemon[867]: reaped child process 1780, status = succeeded6621lb # [ 125.310647] niks3-server[1129]: 2026/09/21 14:31:02 INFO Received uploads request method=POST path=/api/pending_closures6622worker2 # [ 125.342900] nix-grpc-daemon[1595]: time=2026-09-21T14:31:02.459Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6623worker2 # [ 125.346061] nix-grpc-daemon[1595]: time=2026-09-21T14:31:02.461Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6624lb # [ 125.348217] niks3-server[1129]: 2026/09/21 14:31:02 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6625lb # [ 125.357307] niks3-server[1129]: 2026/09/21 14:31:02 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6626lb # [ 125.368330] niks3-server[1129]: 2026/09/21 14:31:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/50/sign6627worker2 # [ 125.396174] nix-grpc-daemon[1595]: time=2026-09-21T14:31:02.512Z level=INFO msg="Uploading 1 narinfos"6628lb # [ 125.372653] niks3-server[1129]: 2026/09/21 14:31:02 INFO Signed narinfos id=50 count=16629lb # [ 125.382992] niks3-server[1129]: 2026/09/21 14:31:02 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6630lb # [ 125.386887] niks3-server[1129]: 2026/09/21 14:31:02 INFO Received complete upload request method=POST path=/api/pending_closures/50/complete6631lb # [ 125.390870] niks3-server[1129]: 2026/09/21 14:31:02 INFO Completed upload id=506632worker2 # [ 125.414615] nix-grpc-daemon[1595]: time=2026-09-21T14:31:02.531Z level=INFO msg="Upload complete. (181ms)"6633lb # [ 125.392848] niks3-server[1129]: 2026/09/21 14:31:02 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6634worker2 # [ 125.421967] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16635worker2 # [ 125.435754] nix-daemon[867]: reaped child process 1759, status = succeeded6636client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.88 seconds)6637client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6638client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6639worker1: must succeed: pkill -f 'read -t [9]0 x'6640worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6641client: waiting for success: ! systemctl is-active sw6642worker1 # [ 125.653929] nix-daemon[847]: reaped child process 1642, status = succeeded6643lb # [ 125.650455] envoy[906]: conn peer=[2001:db8:1::1]:46670 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2993 tx=829 ms=22356644lb # [ 125.654143] envoy[906]: conn peer=[2001:db8:1::1]:46684 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9542 tx=4040 ms=47146645worker1 # [ 125.732100] nix-daemon[847]: accepted connection from pid 1684, user nix-grpc-daemon (trusted)6646worker1 # [ 125.744721] nix-daemon[847]: reaped child process 1687, status = succeeded6647lb # [ 125.739685] niks3-server[1129]: 2026/09/21 14:31:02 INFO Received uploads request method=POST path=/api/pending_closures6648worker1 # [ 125.761724] nix-grpc-daemon[1586]: time=2026-09-21T14:31:02.767Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6649worker1 # [ 125.763373] nix-grpc-daemon[1586]: time=2026-09-21T14:31:02.769Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6650lb # [ 125.779283] niks3-server[1129]: 2026/09/21 14:31:02 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6651lb # [ 125.792970] niks3-server[1129]: 2026/09/21 14:31:02 INFO Received sign narinfos request method=POST path=/api/pending_closures/51/sign6652lb # [ 125.797467] niks3-server[1129]: 2026/09/21 14:31:02 INFO Signed narinfos id=51 count=16653worker1 # [ 125.811201] nix-grpc-daemon[1586]: time=2026-09-21T14:31:02.817Z level=INFO msg="Uploading 1 narinfos"6654lb # [ 125.801891] niks3-server[1129]: 2026/09/21 14:31:02 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6655lb # [ 125.814995] niks3-server[1129]: 2026/09/21 14:31:02 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6656lb # [ 125.816608] niks3-server[1129]: 2026/09/21 14:31:02 INFO Received complete upload request method=POST path=/api/pending_closures/51/complete6657lb # [ 125.821431] niks3-server[1129]: 2026/09/21 14:31:03 INFO Completed upload id=516658worker1 # [ 125.834591] nix-grpc-daemon[1586]: time=2026-09-21T14:31:02.841Z level=INFO msg="Upload complete. (190ms)"6659lb # [ 125.822832] niks3-server[1129]: 2026/09/21 14:31:03 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6660worker1 # [ 125.841167] nix-grpc-daemon[1484]: ts=2026-09-21T14:31:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56424 duration_s=7 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16661worker1 # [ 125.852827] nix-daemon[847]: reaped child process 1640, status = succeeded6662client # [ 125.854410] systemd[1]: sw.service: Deactivated successfully.6663client # [ 125.858250] systemd[1]: sw.service: Consumed 127ms CPU time over 7.818s wall clock time, 41.4M memory peak, 7.1K incoming IP traffic, 8.4K outgoing IP traffic.6664worker1 # [ 126.207790] nix-grpc-daemon[1484]: ts=2026-09-21T14:31:03Z level=info event=signal_exit6665client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6666client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6667client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.03 seconds)6668worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6669worker2 # [ 127.229386] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:513706670worker2 # [ 127.230981] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:04Z level=info event=worker_gone addr=192.168.1.3:500516671worker1 # [ 127.225324] nix-grpc-daemon[1484]: ts=2026-09-21T14:31:04Z level=info event=scheduler_restarting6672worker1 # [ 127.233718] nix-grpc-daemon[1484]: ts=2026-09-21T14:31:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6673worker1 # [ 127.252936] nix-daemon[847]: reaped child process 1552, status = succeeded6674worker1 # [ 127.262242] nix-grpc-daemon[1484]: E0000 00:00:1790001064.268501 1529 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 16, started true, shutdown true6675worker1 # [ 127.267370] nix-grpc-daemon[1484]: E0000 00:00:1790001064.268705 1529 external_connection_acceptor_impl.cc:72] NOT handling external connection with fd 7, started true, shutdown true6676worker1 # [ 128.230573] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6677worker1 # [ 128.234448] systemd[1]: nix-grpc-daemon.service: Consumed 636ms CPU time over 24.145s wall clock time, 35.8M memory peak, 21K incoming IP traffic, 27.7K outgoing IP traffic.6678worker1 # [ 128.250974] systemd[1]: Starting Nix worker-protocol over gRPC...6679worker1 # [ 128.344766] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:05Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6680worker1 # [ 128.346815] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:05Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6681worker1 # [ 128.350042] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:05Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16682worker1 # [ 128.355063] nix-grpc-daemon[1701]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6683worker1 # [ 128.357178] nix-grpc-daemon[1701]: E0000 00:00:1790001065.361408 1701 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6684worker1 # [ 128.362074] nix-grpc-daemon[1701]: E0000 00:00:1790001065.366391 1701 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6685worker1 # [ 128.366342] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:05Z level=info event=startup listen=systemd proxy_socket=/nix/var/nix/daemon-socket/socket6686worker1 # [ 128.370554] systemd[1]: Started Nix worker-protocol over gRPC.6687worker1 # [ 128.374146] nix-grpc-daemon[1701]: E0000 00:00:1790001065.377336 1742 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6688worker1 # [ 128.386082] nix-grpc-daemon[1701]: E0000 00:00:1790001065.391884 1742 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6689worker1 # [ 128.391132] nix-grpc-daemon[1701]: E0000 00:00:1790001065.392009 1742 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6690worker1 # [ 128.400983] nix-grpc-daemon[1701]: E0000 00:00:1790001065.407462 1742 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6691worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 1.71 seconds)6692lb: waiting until exactly no endpoints of x86_64-linux are unhealthy6693lb: must succeed: curl -sf localhost:9901/clusters6694worker1 # [ 128.410840] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:05Z level=info event=scheduler_connected addr=lb:500516695worker2 # [ 128.424555] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:05Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:513706696worker2 # [ 128.428379] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:05Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06697lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6698lb: must succeed: curl -sf localhost:9901/clusters6699lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6700lb: (finished: waiting until exactly no endpoints of x86_64-linux are unhealthy, in 1.10 seconds)6701lb: must succeed: curl -sf localhost:9901/clusters6702lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6703worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6704worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6705(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 11.56 seconds)6706subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6707client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16708worker1 # [ 129.721760] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544126709worker1 # [ 129.733980] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6710worker2 # [ 129.761318] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:06Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403426711worker2 # [ 129.803904] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:06Z level=debug event=want client=5 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06712worker2 # [ 129.808372] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:06Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16713worker2 # [ 129.812691] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:06Z level=debug event=want client=5 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06714worker2 # [ 129.816561] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:06Z level=debug event=want client=5 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06715worker1 # [ 129.838533] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126716worker1 # [ 129.850929] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 52 ms (attempt 1/5)6717worker1 # [ 129.910517] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 73 ms (attempt 2/5)6718worker1 # [ 129.985799] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 260 ms (attempt 3/5)6719worker1 # [ 130.248085] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 254 ms (attempt 4/5)6720worker1 # [ 130.516327] nix-grpc-daemon[1701]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6721worker1 # [ 130.520136] nix-grpc-daemon[1701]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6722worker1 # [ 130.538487] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544126723worker1 # [ 130.550809] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544126724worker1 # [ 130.561302] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6725worker1 # [ 130.582658] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:07Z level=info event=niks3_push_started pid=17616726worker1 # [ 130.690613] nix-grpc-daemon[1761]: time=2026-09-21T14:31:07.696Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6727worker1 # [ 130.782951] nix-daemon[847]: accepted connection from pid 1774, user nix-grpc-daemon (trusted)6728worker1 # [ 130.795670] nix-daemon[847]: reaped child process 1777, status = succeeded6729worker1 # [ 130.883803] nix-daemon[847]: accepted connection from pid 1779, user nix-grpc-daemon (trusted)6730worker1 # [ 130.897785] nix-daemon[847]: reaped child process 1782, status = succeeded6731lb # [ 130.893250] niks3-server[1129]: 2026/09/21 14:31:08 INFO Received uploads request method=POST path=/api/pending_closures6732worker1 # [ 130.914615] nix-grpc-daemon[1761]: time=2026-09-21T14:31:07.921Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6733worker1 # [ 130.917430] nix-grpc-daemon[1761]: time=2026-09-21T14:31:07.922Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6734lb # [ 130.928535] niks3-server[1129]: 2026/09/21 14:31:08 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6735lb # [ 130.943126] niks3-server[1129]: 2026/09/21 14:31:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/52/sign6736worker1 # [ 130.961452] nix-grpc-daemon[1761]: time=2026-09-21T14:31:07.967Z level=INFO msg="Uploading 1 narinfos"6737lb # [ 130.948143] niks3-server[1129]: 2026/09/21 14:31:08 INFO Signed narinfos id=52 count=16738lb # [ 130.960471] niks3-server[1129]: 2026/09/21 14:31:08 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6739lb # [ 130.969849] niks3-server[1129]: 2026/09/21 14:31:08 INFO Received complete upload request method=POST path=/api/pending_closures/52/complete6740lb # [ 130.973477] niks3-server[1129]: 2026/09/21 14:31:08 INFO Completed upload id=526741worker1 # [ 130.986826] nix-grpc-daemon[1761]: time=2026-09-21T14:31:07.993Z level=INFO msg="Upload complete. (292ms)"6742lb # [ 130.976574] niks3-server[1129]: 2026/09/21 14:31:08 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6743worker1 # [ 130.992443] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=5046744worker1 # [ 130.997202] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126745worker1 # [ 131.006983] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6746worker1 # [ 131.011380] nix-daemon[847]: reaped child process 1759, status = succeeded6747worker1 # [ 131.016116] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6748worker1 # [ 131.632871] nix-daemon[847]: reaped child process 1786, status = succeeded6749worker1 # [ 131.710187] nix-daemon[847]: accepted connection from pid 1802, user nix-grpc-daemon (trusted)6750worker1 # [ 131.723594] nix-daemon[847]: reaped child process 1805, status = succeeded6751lb # [ 131.719645] niks3-server[1129]: 2026/09/21 14:31:08 INFO Received uploads request method=POST path=/api/pending_closures6752worker1 # [ 131.742168] nix-grpc-daemon[1761]: time=2026-09-21T14:31:08.748Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6753worker1 # [ 131.746797] nix-grpc-daemon[1761]: time=2026-09-21T14:31:08.750Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"6754lb # [ 131.760144] niks3-server[1129]: 2026/09/21 14:31:08 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst6755lb # [ 131.766588] niks3-server[1129]: 2026/09/21 14:31:08 INFO Received sign narinfos request method=POST path=/api/pending_closures/53/sign6756lb # [ 131.772923] niks3-server[1129]: 2026/09/21 14:31:08 INFO Signed narinfos id=53 count=16757worker1 # [ 131.790282] nix-grpc-daemon[1761]: time=2026-09-21T14:31:08.796Z level=INFO msg="Uploading 1 narinfos"6758lb # [ 131.778745] niks3-server[1129]: 2026/09/21 14:31:08 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv6759lb # [ 131.781184] niks3-server[1129]: 2026/09/21 14:31:08 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls6760lb # [ 131.786522] niks3-server[1129]: 2026/09/21 14:31:08 INFO Received complete upload request method=POST path=/api/pending_closures/53/complete6761lb # [ 131.790720] niks3-server[1129]: 2026/09/21 14:31:08 INFO Completed upload id=536762worker1 # [ 131.803713] nix-grpc-daemon[1761]: time=2026-09-21T14:31:08.810Z level=INFO msg="Upload complete. (180ms)"6763lb # [ 131.793785] niks3-server[1129]: 2026/09/21 14:31:08 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo6764worker1 # [ 131.806104] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:08Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=7 outputs=16765worker2 # [ 131.818568] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:08Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=16766worker1 # [ 131.814779] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:08Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126767worker1 # [ 131.817394] nix-daemon[847]: reaped child process 1784, status = succeeded6768worker2 # [ 131.827810] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:08Z level=debug event=want client=5 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=06769worker1 # [ 131.823078] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:08Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544126770worker1 # [ 131.828498] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:08Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544126771worker1 # [ 131.833543] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6772worker1 # [ 131.920691] nix-daemon[847]: accepted connection from pid 1809, user nix-grpc-daemon (trusted)6773worker1 # [ 131.934789] nix-daemon[847]: reaped child process 1812, status = succeeded6774worker1 # [ 132.020843] nix-daemon[847]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)6775worker1 # [ 132.034682] nix-daemon[847]: reaped child process 1817, status = succeeded6776lb # [ 132.032337] niks3-server[1129]: 2026/09/21 14:31:09 INFO Received uploads request method=POST path=/api/pending_closures6777worker1 # [ 132.052405] nix-grpc-daemon[1761]: time=2026-09-21T14:31:09.058Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6778worker1 # [ 132.053862] nix-grpc-daemon[1761]: time=2026-09-21T14:31:09.058Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"6779lb # [ 132.054835] niks3-server[1129]: 2026/09/21 14:31:09 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst6780lb # [ 132.069070] niks3-server[1129]: 2026/09/21 14:31:09 INFO Received sign narinfos request method=POST path=/api/pending_closures/54/sign6781worker1 # [ 132.086946] nix-grpc-daemon[1761]: time=2026-09-21T14:31:09.093Z level=INFO msg="Uploading 1 narinfos"6782lb # [ 132.073834] niks3-server[1129]: 2026/09/21 14:31:09 INFO Signed narinfos id=54 count=16783lb # [ 132.080804] niks3-server[1129]: 2026/09/21 14:31:09 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls6784lb # [ 132.088489] niks3-server[1129]: 2026/09/21 14:31:09 INFO Received complete upload request method=POST path=/api/pending_closures/54/complete6785lb # [ 132.091983] niks3-server[1129]: 2026/09/21 14:31:09 INFO Completed upload id=546786worker1 # [ 132.106336] nix-grpc-daemon[1761]: time=2026-09-21T14:31:09.112Z level=INFO msg="Upload complete. (268ms)"6787lb # [ 132.095840] niks3-server[1129]: 2026/09/21 14:31:09 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo6788worker1 # [ 132.109909] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=5046789worker1 # [ 132.115287] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126790worker1 # [ 132.123707] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6791worker1 # [ 132.126775] nix-daemon[847]: reaped child process 1807, status = succeeded6792worker1 # [ 132.131928] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6793worker1 # [ 133.173691] nix-daemon[847]: reaped child process 1821, status = succeeded6794worker1 # [ 133.261063] nix-daemon[847]: accepted connection from pid 1837, user nix-grpc-daemon (trusted)6795worker1 # [ 133.274281] nix-daemon[847]: reaped child process 1840, status = succeeded6796lb # [ 133.268616] niks3-server[1129]: 2026/09/21 14:31:10 INFO Received uploads request method=POST path=/api/pending_closures6797worker1 # [ 133.290244] nix-grpc-daemon[1761]: time=2026-09-21T14:31:10.296Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6798worker1 # [ 133.291761] nix-grpc-daemon[1761]: time=2026-09-21T14:31:10.298Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"6799lb # [ 133.302136] niks3-server[1129]: 2026/09/21 14:31:10 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst6800lb # [ 133.320832] niks3-server[1129]: 2026/09/21 14:31:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/55/sign6801lb # [ 133.323997] niks3-server[1129]: 2026/09/21 14:31:10 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv6802lb # [ 133.331119] niks3-server[1129]: 2026/09/21 14:31:10 INFO Signed narinfos id=55 count=16803worker1 # [ 133.344411] nix-grpc-daemon[1761]: time=2026-09-21T14:31:10.350Z level=INFO msg="Uploading 1 narinfos"6804lb # [ 133.336831] niks3-server[1129]: 2026/09/21 14:31:10 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls6805lb # [ 133.345352] niks3-server[1129]: 2026/09/21 14:31:10 INFO Received complete upload request method=POST path=/api/pending_closures/55/complete6806lb # [ 133.350531] niks3-server[1129]: 2026/09/21 14:31:10 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo6807lb # [ 133.353593] niks3-server[1129]: 2026/09/21 14:31:10 INFO Completed upload id=556808worker1 # [ 133.366908] nix-grpc-daemon[1761]: time=2026-09-21T14:31:10.373Z level=INFO msg="Upload complete. (226ms)"6809worker2 # [ 133.383851] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:10Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=16810worker1 # [ 133.374240] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=8 outputs=16811worker1 # [ 133.382455] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126812worker2 # [ 133.393252] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:10Z level=debug event=want client=5 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=06813worker1 # [ 133.394553] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544126814worker1 # [ 133.400599] nix-daemon[847]: reaped child process 1819, status = succeeded6815worker1 # [ 133.405629] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544126816worker1 # [ 133.413760] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6817worker1 # [ 133.501292] nix-daemon[847]: accepted connection from pid 1844, user nix-grpc-daemon (trusted)6818worker1 # [ 133.515436] nix-daemon[847]: reaped child process 1847, status = succeeded6819worker1 # [ 133.601239] nix-daemon[847]: accepted connection from pid 1849, user nix-grpc-daemon (trusted)6820worker1 # [ 133.615356] nix-daemon[847]: reaped child process 1852, status = succeeded6821lb # [ 133.609400] niks3-server[1129]: 2026/09/21 14:31:10 INFO Received uploads request method=POST path=/api/pending_closures6822worker1 # [ 133.631373] nix-grpc-daemon[1761]: time=2026-09-21T14:31:10.637Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6823worker1 # [ 133.632807] nix-grpc-daemon[1761]: time=2026-09-21T14:31:10.637Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"6824lb # [ 133.634941] niks3-server[1129]: 2026/09/21 14:31:10 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst6825lb # [ 133.646674] niks3-server[1129]: 2026/09/21 14:31:10 INFO Received sign narinfos request method=POST path=/api/pending_closures/56/sign6826lb # [ 133.651323] niks3-server[1129]: 2026/09/21 14:31:10 INFO Signed narinfos id=56 count=16827worker1 # [ 133.667845] nix-grpc-daemon[1761]: time=2026-09-21T14:31:10.674Z level=INFO msg="Uploading 1 narinfos"6828lb # [ 133.666057] niks3-server[1129]: 2026/09/21 14:31:10 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls6829lb # [ 133.673075] niks3-server[1129]: 2026/09/21 14:31:10 INFO Received complete upload request method=POST path=/api/pending_closures/56/complete6830lb # [ 133.686881] niks3-server[1129]: 2026/09/21 14:31:10 INFO Completed upload id=566831worker1 # [ 133.699985] nix-grpc-daemon[1761]: time=2026-09-21T14:31:10.706Z level=INFO msg="Upload complete. (281ms)"6832lb # [ 133.689308] niks3-server[1129]: 2026/09/21 14:31:10 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo6833worker1 # [ 133.704156] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=5046834worker1 # [ 133.712363] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126835worker1 # [ 133.721671] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6836worker1 # [ 133.731122] nix-daemon[847]: reaped child process 1842, status = succeeded6837worker1 # [ 133.736758] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6838worker1 # [ 134.980317] nix-daemon[847]: reaped child process 1856, status = succeeded6839worker1 # [ 135.061872] nix-daemon[847]: accepted connection from pid 1872, user nix-grpc-daemon (trusted)6840worker1 # [ 135.077094] nix-daemon[847]: reaped child process 1875, status = succeeded6841lb # [ 135.071716] niks3-server[1129]: 2026/09/21 14:31:12 INFO Received uploads request method=POST path=/api/pending_closures6842worker1 # [ 135.093234] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.099Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6843worker1 # [ 135.094728] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.101Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"6844lb # [ 135.106465] niks3-server[1129]: 2026/09/21 14:31:12 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst6845lb # [ 135.121605] niks3-server[1129]: 2026/09/21 14:31:12 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv6846lb # [ 135.131783] niks3-server[1129]: 2026/09/21 14:31:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/57/sign6847lb # [ 135.135799] niks3-server[1129]: 2026/09/21 14:31:12 INFO Signed narinfos id=57 count=16848worker1 # [ 135.148966] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.155Z level=INFO msg="Uploading 1 narinfos"6849lb # [ 135.142623] niks3-server[1129]: 2026/09/21 14:31:12 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls6850lb # [ 135.151759] niks3-server[1129]: 2026/09/21 14:31:12 INFO Received complete upload request method=POST path=/api/pending_closures/57/complete6851lb # [ 135.154562] niks3-server[1129]: 2026/09/21 14:31:12 INFO Completed upload id=576852worker1 # [ 135.167826] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.174Z level=INFO msg="Upload complete. (223ms)"6853lb # [ 135.157074] niks3-server[1129]: 2026/09/21 14:31:12 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo6854worker1 # [ 135.173817] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=9 outputs=16855worker2 # [ 135.185297] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:12Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=16856worker1 # [ 135.182575] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126857worker2 # [ 135.194478] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:12Z level=debug event=want client=5 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=06858worker1 # [ 135.195552] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544126859worker1 # [ 135.202719] nix-daemon[847]: reaped child process 1854, status = succeeded6860worker1 # [ 135.205048] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.211Z level=INFO msg="All 1 paths already cached"6861worker1 # [ 135.207189] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544126862worker1 # [ 135.211286] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6863worker1 # [ 135.297734] nix-daemon[847]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)6864worker1 # [ 135.312212] nix-daemon[847]: reaped child process 1882, status = succeeded6865worker1 # [ 135.394393] nix-daemon[847]: accepted connection from pid 1884, user nix-grpc-daemon (trusted)6866worker1 # [ 135.407870] nix-daemon[847]: reaped child process 1887, status = succeeded6867lb # [ 135.402200] niks3-server[1129]: 2026/09/21 14:31:12 INFO Received uploads request method=POST path=/api/pending_closures6868worker1 # [ 135.423317] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.429Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6869worker1 # [ 135.424723] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.429Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"6870lb # [ 135.429170] niks3-server[1129]: 2026/09/21 14:31:12 INFO Received sign narinfos request method=POST path=/api/pending_closures/58/sign6871worker1 # [ 135.449970] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.456Z level=INFO msg="Uploading 1 narinfos"6872lb # [ 135.438228] niks3-server[1129]: 2026/09/21 14:31:12 INFO Signed narinfos id=58 count=16873lb # [ 135.443184] niks3-server[1129]: 2026/09/21 14:31:12 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst6874lb # [ 135.449348] niks3-server[1129]: 2026/09/21 14:31:12 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls6875lb # [ 135.461770] niks3-server[1129]: 2026/09/21 14:31:12 INFO Received complete upload request method=POST path=/api/pending_closures/58/complete6876worker1 # [ 135.481595] nix-grpc-daemon[1761]: time=2026-09-21T14:31:12.488Z level=INFO msg="Upload complete. (264ms)"6877lb # [ 135.468639] niks3-server[1129]: 2026/09/21 14:31:12 INFO Completed upload id=586878worker1 # [ 135.483792] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=6246879lb # [ 135.473739] niks3-server[1129]: 2026/09/21 14:31:12 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo6880worker1 # [ 135.489585] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126881worker1 # [ 135.499475] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6882worker1 # [ 135.501990] nix-daemon[847]: reaped child process 1877, status = succeeded6883worker1 # [ 135.506051] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6884lb # [ 135.651298] envoy[906]: conn peer=[2001:db8:1::1]:46672 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=1428 ms=76596885lb # [ 135.654985] envoy[906]: conn peer=[2001:db8:1::3]:54680 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1313 tx=515 ms=229806886worker1 # [ 136.548092] nix-daemon[847]: reaped child process 1891, status = succeeded6887worker1 # [ 136.623118] nix-daemon[847]: accepted connection from pid 1907, user nix-grpc-daemon (trusted)6888worker1 # [ 136.636115] nix-daemon[847]: reaped child process 1910, status = succeeded6889lb # [ 136.631759] niks3-server[1129]: 2026/09/21 14:31:13 INFO Received uploads request method=POST path=/api/pending_closures6890worker1 # [ 136.651832] nix-grpc-daemon[1761]: time=2026-09-21T14:31:13.658Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6891worker1 # [ 136.653322] nix-grpc-daemon[1761]: time=2026-09-21T14:31:13.659Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"6892lb # [ 136.669524] niks3-server[1129]: 2026/09/21 14:31:13 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv6893lb # [ 136.680408] niks3-server[1129]: 2026/09/21 14:31:13 INFO Received sign narinfos request method=POST path=/api/pending_closures/59/sign6894lb # [ 136.687975] niks3-server[1129]: 2026/09/21 14:31:13 INFO Signed narinfos id=59 count=16895worker1 # [ 136.704824] nix-grpc-daemon[1761]: time=2026-09-21T14:31:13.711Z level=INFO msg="Uploading 1 narinfos"6896lb # [ 136.694888] niks3-server[1129]: 2026/09/21 14:31:13 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst6897lb # [ 136.701342] niks3-server[1129]: 2026/09/21 14:31:13 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls6898lb # [ 136.710978] niks3-server[1129]: 2026/09/21 14:31:13 INFO Received complete upload request method=POST path=/api/pending_closures/59/complete6899lb # [ 136.715583] niks3-server[1129]: 2026/09/21 14:31:13 INFO Completed upload id=596900lb # [ 136.716950] niks3-server[1129]: 2026/09/21 14:31:13 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo6901worker1 # [ 136.728784] nix-grpc-daemon[1761]: time=2026-09-21T14:31:13.735Z level=INFO msg="Upload complete. (215ms)"6902worker1 # [ 136.735545] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=10 outputs=16903worker2 # [ 136.745957] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:13Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=16904worker1 # [ 136.743356] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126905worker2 # [ 136.755591] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:13Z level=debug event=want client=5 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=06906worker1 # [ 136.754593] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544126907worker1 # [ 136.764246] nix-daemon[847]: reaped child process 1889, status = succeeded6908worker1 # [ 136.766395] nix-grpc-daemon[1761]: time=2026-09-21T14:31:13.772Z level=INFO msg="All 1 paths already cached"6909worker1 # [ 136.768709] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544126910worker1 # [ 136.772616] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6911worker1 # [ 136.857798] nix-daemon[847]: accepted connection from pid 1914, user nix-grpc-daemon (trusted)6912worker1 # [ 136.872623] nix-daemon[847]: reaped child process 1917, status = succeeded6913worker1 # [ 136.953350] nix-daemon[847]: accepted connection from pid 1919, user nix-grpc-daemon (trusted)6914worker1 # [ 136.968397] nix-daemon[847]: reaped child process 1922, status = succeeded6915lb # [ 136.959833] niks3-server[1129]: 2026/09/21 14:31:14 INFO Received uploads request method=POST path=/api/pending_closures6916worker1 # [ 136.982586] nix-grpc-daemon[1761]: time=2026-09-21T14:31:13.988Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6917worker1 # [ 136.984070] nix-grpc-daemon[1761]: time=2026-09-21T14:31:13.988Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"6918lb # [ 136.988652] niks3-server[1129]: 2026/09/21 14:31:14 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst6919lb # [ 136.997738] niks3-server[1129]: 2026/09/21 14:31:14 INFO Received sign narinfos request method=POST path=/api/pending_closures/60/sign6920lb # [ 137.002364] niks3-server[1129]: 2026/09/21 14:31:14 INFO Signed narinfos id=60 count=16921worker1 # [ 137.018113] nix-grpc-daemon[1761]: time=2026-09-21T14:31:14.024Z level=INFO msg="Uploading 1 narinfos"6922lb # [ 137.012219] niks3-server[1129]: 2026/09/21 14:31:14 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls6923lb # [ 137.018891] niks3-server[1129]: 2026/09/21 14:31:14 INFO Received complete upload request method=POST path=/api/pending_closures/60/complete6924lb # [ 137.023358] niks3-server[1129]: 2026/09/21 14:31:14 INFO Completed upload id=606925worker1 # [ 137.036410] nix-grpc-daemon[1761]: time=2026-09-21T14:31:14.042Z level=INFO msg="Upload complete. (257ms)"6926worker1 # [ 137.038422] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:14Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=6246927lb # [ 137.026122] niks3-server[1129]: 2026/09/21 14:31:14 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo6928worker1 # [ 137.041341] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126929worker1 # [ 137.048995] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6930worker1 # [ 137.054797] nix-daemon[847]: reaped child process 1912, status = succeeded6931worker1 # [ 137.057678] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6932worker1 # [ 137.696841] nix-daemon[847]: reaped child process 1926, status = succeeded6933worker1 # [ 137.764693] nix-daemon[847]: accepted connection from pid 1943, user nix-grpc-daemon (trusted)6934worker1 # [ 137.779388] nix-daemon[847]: reaped child process 1946, status = succeeded6935lb # [ 137.773761] niks3-server[1129]: 2026/09/21 14:31:14 INFO Received uploads request method=POST path=/api/pending_closures6936worker1 # [ 137.797796] nix-grpc-daemon[1761]: time=2026-09-21T14:31:14.804Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6937worker1 # [ 137.799992] nix-grpc-daemon[1761]: time=2026-09-21T14:31:14.805Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"6938lb # [ 137.808156] niks3-server[1129]: 2026/09/21 14:31:14 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv6939lb # [ 137.824354] niks3-server[1129]: 2026/09/21 14:31:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/61/sign6940worker1 # [ 137.842184] nix-grpc-daemon[1761]: time=2026-09-21T14:31:14.848Z level=INFO msg="Uploading 1 narinfos"6941lb # [ 137.829643] niks3-server[1129]: 2026/09/21 14:31:15 INFO Signed narinfos id=61 count=16942lb # [ 137.831680] niks3-server[1129]: 2026/09/21 14:31:15 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst6943lb # [ 137.839352] niks3-server[1129]: 2026/09/21 14:31:15 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls6944lb # [ 137.842792] niks3-server[1129]: 2026/09/21 14:31:15 INFO Received complete upload request method=POST path=/api/pending_closures/61/complete6945lb # [ 137.845776] niks3-server[1129]: 2026/09/21 14:31:15 INFO Completed upload id=616946worker1 # [ 137.859084] nix-grpc-daemon[1761]: time=2026-09-21T14:31:14.865Z level=INFO msg="Upload complete. (194ms)"6947lb # [ 137.848428] niks3-server[1129]: 2026/09/21 14:31:15 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo6948worker1 # [ 137.864308] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=11 outputs=16949worker2 # [ 137.879082] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:14Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=16950worker1 # [ 137.871618] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:14Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126951worker2 # [ 137.884845] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:15Z level=debug event=want client=5 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=06952worker1 # [ 137.880339] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:14Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544126953worker1 # [ 137.885115] nix-daemon[847]: reaped child process 1924, status = succeeded6954worker1 # [ 137.886932] nix-grpc-daemon[1761]: time=2026-09-21T14:31:14.893Z level=INFO msg="All 1 paths already cached"6955worker1 # [ 137.890737] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:14Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544126956worker1 # [ 137.894855] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6957worker1 # [ 137.983148] nix-daemon[847]: accepted connection from pid 1950, user nix-grpc-daemon (trusted)6958worker1 # [ 137.998529] nix-daemon[847]: reaped child process 1953, status = succeeded6959worker1 # [ 138.080267] nix-daemon[847]: accepted connection from pid 1955, user nix-grpc-daemon (trusted)6960worker1 # [ 138.094712] nix-daemon[847]: reaped child process 1958, status = succeeded6961lb # [ 138.087584] niks3-server[1129]: 2026/09/21 14:31:15 INFO Received uploads request method=POST path=/api/pending_closures6962worker1 # [ 138.110677] nix-grpc-daemon[1761]: time=2026-09-21T14:31:15.116Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6963worker1 # [ 138.112174] nix-grpc-daemon[1761]: time=2026-09-21T14:31:15.116Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"6964lb # [ 138.119520] niks3-server[1129]: 2026/09/21 14:31:15 INFO Received sign narinfos request method=POST path=/api/pending_closures/62/sign6965lb # [ 138.124091] niks3-server[1129]: 2026/09/21 14:31:15 INFO Signed narinfos id=62 count=16966worker1 # [ 138.139647] nix-grpc-daemon[1761]: time=2026-09-21T14:31:15.145Z level=INFO msg="Uploading 1 narinfos"6967lb # [ 138.131495] niks3-server[1129]: 2026/09/21 14:31:15 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst6968lb # [ 138.142967] niks3-server[1129]: 2026/09/21 14:31:15 INFO Received complete upload request method=POST path=/api/pending_closures/62/complete6969lb # [ 138.146302] niks3-server[1129]: 2026/09/21 14:31:15 INFO Completed upload id=626970worker1 # [ 138.159184] nix-grpc-daemon[1761]: time=2026-09-21T14:31:15.165Z level=INFO msg="Upload complete. (257ms)"6971worker1 # [ 138.161440] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=6246972lb # [ 138.149373] niks3-server[1129]: 2026/09/21 14:31:15 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls6973lb # [ 138.151706] niks3-server[1129]: 2026/09/21 14:31:15 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo6974worker1 # [ 138.165132] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126975worker1 # [ 138.172143] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6976worker1 # [ 138.175758] nix-daemon[847]: reaped child process 1948, status = succeeded6977worker1 # [ 138.180882] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)6978worker1 # [ 139.000349] nix-daemon[847]: reaped child process 1962, status = succeeded6979worker1 # [ 139.071246] nix-daemon[847]: accepted connection from pid 1978, user nix-grpc-daemon (trusted)6980worker1 # [ 139.087048] nix-daemon[847]: reaped child process 1981, status = succeeded6981lb # [ 139.079478] niks3-server[1129]: 2026/09/21 14:31:16 INFO Received uploads request method=POST path=/api/pending_closures6982worker1 # [ 139.102076] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.108Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6983worker1 # [ 139.103581] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.110Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"6984lb # [ 139.125924] niks3-server[1129]: 2026/09/21 14:31:16 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv6985lb # [ 139.130901] niks3-server[1129]: 2026/09/21 14:31:16 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst6986lb # [ 139.141865] niks3-server[1129]: 2026/09/21 14:31:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/63/sign6987worker1 # [ 139.158614] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.164Z level=INFO msg="Uploading 1 narinfos"6988lb # [ 139.146065] niks3-server[1129]: 2026/09/21 14:31:16 INFO Signed narinfos id=63 count=16989lb # [ 139.153045] niks3-server[1129]: 2026/09/21 14:31:16 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls6990lb # [ 139.161636] niks3-server[1129]: 2026/09/21 14:31:16 INFO Received complete upload request method=POST path=/api/pending_closures/63/complete6991lb # [ 139.166609] niks3-server[1129]: 2026/09/21 14:31:16 INFO Completed upload id=636992worker1 # [ 139.179728] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.186Z level=INFO msg="Upload complete. (192ms)"6993lb # [ 139.170135] niks3-server[1129]: 2026/09/21 14:31:16 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo6994worker1 # [ 139.182300] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:16Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=12 outputs=16995worker2 # [ 139.194467] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:16Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=16996worker1 # [ 139.190450] nix-daemon[847]: reaped child process 1960, status = succeeded6997worker1 # [ 139.226299] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544126998worker2 # [ 139.238442] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:16Z level=debug event=want client=5 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=06999worker1 # [ 139.234608] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:16Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544127000worker1 # [ 139.243726] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.250Z level=INFO msg="All 2 paths already cached"7001worker1 # [ 139.248040] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:16Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544127002worker1 # [ 139.253101] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7003worker1 # [ 139.340048] nix-daemon[847]: accepted connection from pid 1985, user nix-grpc-daemon (trusted)7004worker1 # [ 139.354961] nix-daemon[847]: reaped child process 1988, status = succeeded7005worker1 # [ 139.436307] nix-daemon[847]: accepted connection from pid 1990, user nix-grpc-daemon (trusted)7006worker1 # [ 139.450203] nix-daemon[847]: reaped child process 1993, status = succeeded7007lb # [ 139.444324] niks3-server[1129]: 2026/09/21 14:31:16 INFO Received uploads request method=POST path=/api/pending_closures7008worker1 # [ 139.465868] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.472Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7009worker1 # [ 139.467702] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.473Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7010lb # [ 139.471965] niks3-server[1129]: 2026/09/21 14:31:16 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7011lb # [ 139.483642] niks3-server[1129]: 2026/09/21 14:31:16 INFO Received sign narinfos request method=POST path=/api/pending_closures/64/sign7012lb # [ 139.489487] niks3-server[1129]: 2026/09/21 14:31:16 INFO Signed narinfos id=64 count=17013worker1 # [ 139.505928] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.512Z level=INFO msg="Uploading 1 narinfos"7014lb # [ 139.500467] niks3-server[1129]: 2026/09/21 14:31:16 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7015lb # [ 139.511786] niks3-server[1129]: 2026/09/21 14:31:16 INFO Received complete upload request method=POST path=/api/pending_closures/64/complete7016lb # [ 139.517710] niks3-server[1129]: 2026/09/21 14:31:16 INFO Completed upload id=647017worker1 # [ 139.532497] nix-grpc-daemon[1761]: time=2026-09-21T14:31:16.538Z level=INFO msg="Upload complete. (273ms)"7018lb # [ 139.520895] niks3-server[1129]: 2026/09/21 14:31:16 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7019worker1 # [ 139.536568] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:16Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=7527020worker1 # [ 139.542714] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:16Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544127021worker1 # [ 139.554137] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7022worker1 # [ 139.562794] nix-daemon[847]: reaped child process 1983, status = succeeded7023worker1 # [ 139.567413] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7024worker1 # [ 140.806543] nix-daemon[847]: reaped child process 1997, status = succeeded7025worker1 # [ 140.890101] nix-daemon[847]: accepted connection from pid 2013, user nix-grpc-daemon (trusted)7026worker1 # [ 140.903523] nix-daemon[847]: reaped child process 2016, status = succeeded7027lb # [ 140.897416] niks3-server[1129]: 2026/09/21 14:31:18 INFO Received uploads request method=POST path=/api/pending_closures7028worker1 # [ 140.919122] nix-grpc-daemon[1761]: time=2026-09-21T14:31:17.925Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7029worker1 # [ 140.920655] nix-grpc-daemon[1761]: time=2026-09-21T14:31:17.927Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7030lb # [ 140.939115] niks3-server[1129]: 2026/09/21 14:31:18 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7031lb # [ 140.950472] niks3-server[1129]: 2026/09/21 14:31:18 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7032lb # [ 140.955319] niks3-server[1129]: 2026/09/21 14:31:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/65/sign7033lb # [ 140.960423] niks3-server[1129]: 2026/09/21 14:31:18 INFO Signed narinfos id=65 count=17034worker1 # [ 140.976637] nix-grpc-daemon[1761]: time=2026-09-21T14:31:17.982Z level=INFO msg="Uploading 1 narinfos"7035lb # [ 140.967621] niks3-server[1129]: 2026/09/21 14:31:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7036lb # [ 140.979308] niks3-server[1129]: 2026/09/21 14:31:18 INFO Received complete upload request method=POST path=/api/pending_closures/65/complete7037lb # [ 140.985061] niks3-server[1129]: 2026/09/21 14:31:18 INFO Completed upload id=657038worker1 # [ 140.998509] nix-grpc-daemon[1761]: time=2026-09-21T14:31:18.004Z level=INFO msg="Upload complete. (223ms)"7039lb # [ 140.987736] niks3-server[1129]: 2026/09/21 14:31:18 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7040worker2 # [ 141.015825] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:18Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17041worker1 # [ 141.006248] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:18Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=13 outputs=17042worker1 # [ 141.020119] nix-daemon[847]: reaped child process 1995, status = succeeded7043worker1 # [ 141.048599] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544127044worker1 # [ 141.061113] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544127045worker1 # [ 141.073393] nix-grpc-daemon[1761]: time=2026-09-21T14:31:18.079Z level=INFO msg="All 2 paths already cached"7046worker1 # [ 141.080386] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544127047worker1 # [ 141.090182] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7048worker1 # [ 141.186850] nix-daemon[847]: accepted connection from pid 2020, user nix-grpc-daemon (trusted)7049worker1 # [ 141.201924] nix-daemon[847]: reaped child process 2023, status = succeeded7050worker1 # [ 141.283428] nix-daemon[847]: accepted connection from pid 2025, user nix-grpc-daemon (trusted)7051worker1 # [ 141.298633] nix-daemon[847]: reaped child process 2028, status = succeeded7052lb # [ 141.290692] niks3-server[1129]: 2026/09/21 14:31:18 INFO Received uploads request method=POST path=/api/pending_closures7053worker1 # [ 141.312158] nix-grpc-daemon[1761]: time=2026-09-21T14:31:18.318Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7054worker1 # [ 141.313540] nix-grpc-daemon[1761]: time=2026-09-21T14:31:18.318Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7055lb # [ 141.319663] niks3-server[1129]: 2026/09/21 14:31:18 INFO Received sign narinfos request method=POST path=/api/pending_closures/66/sign7056lb # [ 141.328555] niks3-server[1129]: 2026/09/21 14:31:18 INFO Signed narinfos id=66 count=17057worker1 # [ 141.344910] nix-grpc-daemon[1761]: time=2026-09-21T14:31:18.351Z level=INFO msg="Uploading 1 narinfos"7058lb # [ 141.335062] niks3-server[1129]: 2026/09/21 14:31:18 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7059lb # [ 141.340950] niks3-server[1129]: 2026/09/21 14:31:18 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7060lb # [ 141.354132] niks3-server[1129]: 2026/09/21 14:31:18 INFO Received complete upload request method=POST path=/api/pending_closures/66/complete7061lb # [ 141.359825] niks3-server[1129]: 2026/09/21 14:31:18 INFO Completed upload id=667062worker1 # [ 141.375146] nix-grpc-daemon[1761]: time=2026-09-21T14:31:18.381Z level=INFO msg="Upload complete. (273ms)"7063lb # [ 141.365748] niks3-server[1129]: 2026/09/21 14:31:18 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7064worker1 # [ 141.378855] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=7527065worker1 # [ 141.384324] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544127066worker1 # [ 141.391918] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7067worker1 # [ 141.400224] nix-daemon[847]: reaped child process 2018, status = succeeded7068worker1 # [ 141.404207] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7069worker1 # [ 142.247617] nix-daemon[847]: reaped child process 2032, status = succeeded7070worker1 # [ 142.323571] nix-daemon[847]: accepted connection from pid 2048, user nix-grpc-daemon (trusted)7071worker1 # [ 142.337612] nix-daemon[847]: reaped child process 2051, status = succeeded7072lb # [ 142.332765] niks3-server[1129]: 2026/09/21 14:31:19 INFO Received uploads request method=POST path=/api/pending_closures7073worker1 # [ 142.357261] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.363Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7074worker1 # [ 142.359440] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.365Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7075lb # [ 142.375125] niks3-server[1129]: 2026/09/21 14:31:19 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7076lb # [ 142.380842] niks3-server[1129]: 2026/09/21 14:31:19 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7077lb # [ 142.392891] niks3-server[1129]: 2026/09/21 14:31:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/67/sign7078lb # [ 142.399167] niks3-server[1129]: 2026/09/21 14:31:19 INFO Signed narinfos id=67 count=17079worker1 # [ 142.412650] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.418Z level=INFO msg="Uploading 1 narinfos"7080lb # [ 142.411070] niks3-server[1129]: 2026/09/21 14:31:19 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7081lb # [ 142.417145] niks3-server[1129]: 2026/09/21 14:31:19 INFO Received complete upload request method=POST path=/api/pending_closures/67/complete7082lb # [ 142.421419] niks3-server[1129]: 2026/09/21 14:31:19 INFO Completed upload id=677083worker1 # [ 142.434627] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.441Z level=INFO msg="Upload complete. (224ms)"7084lb # [ 142.424787] niks3-server[1129]: 2026/09/21 14:31:19 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7085worker2 # [ 142.452935] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:19Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17086worker1 # [ 142.441570] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:19Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=14 outputs=17087worker1 # [ 142.449378] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544127088worker1 # [ 142.462627] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:19Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:544127089worker1 # [ 142.470276] nix-daemon[847]: reaped child process 2030, status = succeeded7090worker1 # [ 142.473158] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.478Z level=INFO msg="All 2 paths already cached"7091worker1 # [ 142.476802] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:19Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:544127092worker1 # [ 142.481616] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7093worker1 # [ 142.567500] nix-daemon[847]: accepted connection from pid 2055, user nix-grpc-daemon (trusted)7094worker1 # [ 142.581656] nix-daemon[847]: reaped child process 2058, status = succeeded7095worker1 # [ 142.663764] nix-daemon[847]: accepted connection from pid 2060, user nix-grpc-daemon (trusted)7096worker1 # [ 142.677702] nix-daemon[847]: reaped child process 2063, status = succeeded7097lb # [ 142.671140] niks3-server[1129]: 2026/09/21 14:31:19 INFO Received uploads request method=POST path=/api/pending_closures7098worker1 # [ 142.697230] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.703Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7099worker1 # [ 142.699117] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.705Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7100lb # [ 142.709206] niks3-server[1129]: 2026/09/21 14:31:19 INFO Received sign narinfos request method=POST path=/api/pending_closures/68/sign7101worker1 # [ 142.728368] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.734Z level=INFO msg="Uploading 1 narinfos"7102lb # [ 142.715120] niks3-server[1129]: 2026/09/21 14:31:19 INFO Signed narinfos id=68 count=17103lb # [ 142.721917] niks3-server[1129]: 2026/09/21 14:31:19 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7104lb # [ 142.727601] niks3-server[1129]: 2026/09/21 14:31:19 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7105lb # [ 142.741732] niks3-server[1129]: 2026/09/21 14:31:19 INFO Received complete upload request method=POST path=/api/pending_closures/68/complete7106lb # [ 142.748140] niks3-server[1129]: 2026/09/21 14:31:19 INFO Completed upload id=687107worker1 # [ 142.761266] nix-grpc-daemon[1761]: time=2026-09-21T14:31:19.767Z level=INFO msg="Upload complete. (273ms)"7108lb # [ 142.751603] niks3-server[1129]: 2026/09/21 14:31:19 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7109worker1 # [ 142.765333] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:19Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=0 paths=1 nar_bytes_in=7527110worker1 # [ 142.771139] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:19Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:544127111worker1 # [ 142.780256] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7112worker1 # [ 142.790120] nix-daemon[847]: reaped child process 2053, status = succeeded7113worker1 # [ 142.793652] nix-daemon[847]: accepted connection from pid 1701, user nix-grpc-daemon (trusted)7114worker1 # [ 143.832635] nix-daemon[847]: reaped child process 2067, status = succeeded7115worker1 # [ 143.914190] nix-daemon[847]: accepted connection from pid 2083, user nix-grpc-daemon (trusted)7116worker1 # [ 143.927316] nix-daemon[847]: reaped child process 2086, status = succeeded7117lb # [ 143.922470] niks3-server[1129]: 2026/09/21 14:31:21 INFO Received uploads request method=POST path=/api/pending_closures7118worker1 # [ 143.944458] nix-grpc-daemon[1761]: time=2026-09-21T14:31:20.950Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7119worker1 # [ 143.946055] nix-grpc-daemon[1761]: time=2026-09-21T14:31:20.952Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7120lb # [ 143.963078] niks3-server[1129]: 2026/09/21 14:31:21 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7121lb # [ 143.973556] niks3-server[1129]: 2026/09/21 14:31:21 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7122lb # [ 143.981389] niks3-server[1129]: 2026/09/21 14:31:21 INFO Received sign narinfos request method=POST path=/api/pending_closures/69/sign7123worker1 # [ 143.999894] nix-grpc-daemon[1761]: time=2026-09-21T14:31:21.006Z level=INFO msg="Uploading 1 narinfos"7124lb # [ 143.987880] niks3-server[1129]: 2026/09/21 14:31:21 INFO Signed narinfos id=69 count=17125lb # [ 143.992744] niks3-server[1129]: 2026/09/21 14:31:21 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7126lb # [ 144.003373] niks3-server[1129]: 2026/09/21 14:31:21 INFO Received complete upload request method=POST path=/api/pending_closures/69/complete7127lb # [ 144.006021] niks3-server[1129]: 2026/09/21 14:31:21 INFO Completed upload id=697128worker1 # [ 144.019350] nix-grpc-daemon[1761]: time=2026-09-21T14:31:21.025Z level=INFO msg="Upload complete. (217ms)"7129lb # [ 144.009544] niks3-server[1129]: 2026/09/21 14:31:21 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7130worker1 # [ 144.026332] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:21Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54412 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=15 outputs=17131worker1 # [ 144.040131] nix-daemon[847]: reaped child process 2065, status = succeeded7132client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 14.50 seconds)7133worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7134worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7135client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7136client # [ 144.143864] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7137client # Running as unit: gpu.service; invocation ID: 2aa0b5a390814dcba027b4ed2eead0127138client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7139lb: must succeed: curl -sf localhost:9901/clusters7140lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7141lb: must succeed: curl -sf localhost:9901/clusters7142lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7143worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7144worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7145client # [ 144.277845] nix[1366]: don't know how to build these paths:7146client # [ 144.278717] nix[1366]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7147worker1 # [ 144.292822] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:21Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544127148worker2 # [ 144.306755] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:21Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403427149worker2 # [ 144.324544] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:21Z level=debug event=want client=6 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07150client # [ 144.303552] nix[1366]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, waiting up to 30s for a worker7151lb: must succeed: curl -sf localhost:9901/clusters7152lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7153lb: must succeed: curl -sf localhost:9901/clusters7154lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7155worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7156worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7157client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7158lb # [ 145.652181] envoy[906]: conn peer=[2001:db8:1::1]:45720 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=26960 tx=11324 ms=143217159client # [ 174.304765] nix[1366]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}7160client # [ 174.313677] nix[1366]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7161client # [ 174.345982] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7162client # [ 174.349829] systemd[1]: gpu.service: Failed with result 'exit-code'.7163client # [ 174.353821] systemd[1]: gpu.service: Consumed 145ms CPU time over 30.203s wall clock time, 37.6M memory peak, 5.1K incoming IP traffic, 5.1K outgoing IP traffic.7164client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.42 seconds)7165client: must succeed: journalctl -u gpu -o cat7166client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7167lb: must succeed: curl -sf localhost:9901/clusters7168lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7169lb: must succeed: curl -sf localhost:9901/clusters7170lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7171worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7172worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7173(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 45.40 seconds)7174subtest: low disk drains a worker and builds go to the other7175worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7176lb # [ 175.654304] envoy[906]: conn peer=[2001:db8:1::1]:45462 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=895 tx=521 ms=300507177worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.93 seconds)7178worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7179worker1 # [ 176.382634] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:53Z level=info event=unhealthy reason=min_free7180worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.09 seconds)7181client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27182worker1 # [ 177.132900] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:54Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:544127183client # don't know how to build these paths:7184client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7185worker2 # [ 177.154182] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403427186worker2 # [ 177.173758] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:54Z level=debug event=want client=7 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07187worker2 # [ 177.178742] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:54Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17188worker2 # [ 177.222587] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403427189worker2 # [ 177.235380] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403427190client # copying 1 paths to 'lb:50051'...7191client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7192worker2 # [ 177.248420] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403427193worker2 # [ 177.259279] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7194worker2 # [ 177.359913] nix-daemon[867]: accepted connection from pid 1814, user nix-grpc-daemon (trusted)7195worker2 # [ 177.373290] nix-daemon[867]: reaped child process 1817, status = succeeded7196worker2 # [ 177.456448] nix-daemon[867]: accepted connection from pid 1819, user nix-grpc-daemon (trusted)7197worker2 # [ 177.469707] nix-daemon[867]: reaped child process 1822, status = succeeded7198lb # [ 177.453562] niks3-server[1129]: 2026/09/21 14:31:54 INFO Received uploads request method=POST path=/api/pending_closures7199worker2 # [ 177.487071] nix-grpc-daemon[1595]: time=2026-09-21T14:31:54.603Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7200worker2 # [ 177.488607] nix-grpc-daemon[1595]: time=2026-09-21T14:31:54.603Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7201lb # [ 177.484871] niks3-server[1129]: 2026/09/21 14:31:54 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7202lb # [ 177.499480] niks3-server[1129]: 2026/09/21 14:31:54 INFO Received sign narinfos request method=POST path=/api/pending_closures/70/sign7203lb # [ 177.506611] niks3-server[1129]: 2026/09/21 14:31:54 INFO Signed narinfos id=70 count=17204worker2 # [ 177.533200] nix-grpc-daemon[1595]: time=2026-09-21T14:31:54.649Z level=INFO msg="Uploading 1 narinfos"7205lb # [ 177.517483] niks3-server[1129]: 2026/09/21 14:31:54 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7206lb # [ 177.527260] niks3-server[1129]: 2026/09/21 14:31:54 INFO Received complete upload request method=POST path=/api/pending_closures/70/complete7207lb # [ 177.532148] niks3-server[1129]: 2026/09/21 14:31:54 INFO Completed upload id=707208lb # [ 177.534853] niks3-server[1129]: 2026/09/21 14:31:54 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7209worker2 # [ 177.557478] nix-grpc-daemon[1595]: time=2026-09-21T14:31:54.674Z level=INFO msg="Upload complete. (286ms)"7210worker2 # [ 177.562194] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=0 paths=1 nar_bytes_in=5127211worker2 # [ 177.568338] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403427212worker2 # [ 177.578692] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7213worker2 # [ 177.588946] nix-daemon[867]: reaped child process 1812, status = succeeded7214worker2 # [ 177.593344] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7215client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7216client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7217client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7218client # farm-a> LOG-farm-a-drain7219worker2 # [ 178.627260] nix-daemon[867]: reaped child process 1826, status = succeeded7220worker2 # [ 178.707092] nix-daemon[867]: accepted connection from pid 1842, user nix-grpc-daemon (trusted)7221worker2 # [ 178.719196] nix-daemon[867]: reaped child process 1845, status = succeeded7222lb # [ 178.705207] niks3-server[1129]: 2026/09/21 14:31:55 INFO Received uploads request method=POST path=/api/pending_closures7223worker2 # [ 178.737489] nix-grpc-daemon[1595]: time=2026-09-21T14:31:55.853Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7224worker2 # [ 178.739705] nix-grpc-daemon[1595]: time=2026-09-21T14:31:55.855Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7225lb # [ 178.747335] niks3-server[1129]: 2026/09/21 14:31:55 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7226lb # [ 178.756670] niks3-server[1129]: 2026/09/21 14:31:55 INFO Received sign narinfos request method=POST path=/api/pending_closures/71/sign7227worker2 # [ 178.784679] nix-grpc-daemon[1595]: time=2026-09-21T14:31:55.901Z level=INFO msg="Uploading 1 narinfos"7228lb # [ 178.761430] niks3-server[1129]: 2026/09/21 14:31:55 INFO Signed narinfos id=71 count=17229lb # [ 178.766651] niks3-server[1129]: 2026/09/21 14:31:55 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7230lb # [ 178.771581] niks3-server[1129]: 2026/09/21 14:31:55 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7231lb # [ 178.776831] niks3-server[1129]: 2026/09/21 14:31:55 INFO Received complete upload request method=POST path=/api/pending_closures/71/complete7232lb # [ 178.779824] niks3-server[1129]: 2026/09/21 14:31:55 INFO Completed upload id=717233worker2 # [ 178.803581] nix-grpc-daemon[1595]: time=2026-09-21T14:31:55.920Z level=INFO msg="Upload complete. (201ms)"7234lb # [ 178.783223] niks3-server[1129]: 2026/09/21 14:31:55 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7235worker2 # [ 178.807954] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:55Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=16 outputs=17236worker2 # [ 178.818713] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:55Z level=debug event=want client=7 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07237worker2 # [ 178.823120] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:55Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17238worker2 # [ 178.831714] nix-daemon[867]: reaped child process 1824, status = succeeded7239worker2 # [ 178.863297] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403427240worker2 # [ 178.875208] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:55Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403427241worker2 # [ 178.887622] nix-grpc-daemon[1595]: time=2026-09-21T14:31:56.004Z level=INFO msg="All 1 paths already cached"7242client # copying 1 paths to 'lb:50051'...7243client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7244worker2 # [ 178.895681] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403427245worker2 # [ 178.906129] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7246worker2 # [ 179.001317] nix-daemon[867]: accepted connection from pid 1851, user nix-grpc-daemon (trusted)7247worker2 # [ 179.015804] nix-daemon[867]: reaped child process 1854, status = succeeded7248worker2 # [ 179.094093] nix-daemon[867]: accepted connection from pid 1856, user nix-grpc-daemon (trusted)7249worker2 # [ 179.106529] nix-daemon[867]: reaped child process 1859, status = succeeded7250lb # [ 179.089771] niks3-server[1129]: 2026/09/21 14:31:56 INFO Received uploads request method=POST path=/api/pending_closures7251worker2 # [ 179.123096] nix-grpc-daemon[1595]: time=2026-09-21T14:31:56.239Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7252worker2 # [ 179.124642] nix-grpc-daemon[1595]: time=2026-09-21T14:31:56.241Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7253lb # [ 179.120700] niks3-server[1129]: 2026/09/21 14:31:56 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7254lb # [ 179.124933] niks3-server[1129]: 2026/09/21 14:31:56 INFO Received sign narinfos request method=POST path=/api/pending_closures/72/sign7255lb # [ 179.127317] niks3-server[1129]: 2026/09/21 14:31:56 INFO Signed narinfos id=72 count=17256worker2 # [ 179.151154] nix-grpc-daemon[1595]: time=2026-09-21T14:31:56.267Z level=INFO msg="Uploading 1 narinfos"7257lb # [ 179.133655] niks3-server[1129]: 2026/09/21 14:31:56 INFO Received complete upload request method=POST path=/api/pending_closures/72/complete7258lb # [ 179.137496] niks3-server[1129]: 2026/09/21 14:31:56 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7259lb # [ 179.141477] niks3-server[1129]: 2026/09/21 14:31:56 INFO Completed upload id=727260worker2 # [ 179.164947] nix-grpc-daemon[1595]: time=2026-09-21T14:31:56.281Z level=INFO msg="Upload complete. (248ms)"7261worker2 # [ 179.166920] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=0 paths=1 nar_bytes_in=6247262lb # [ 179.145045] niks3-server[1129]: 2026/09/21 14:31:56 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7263worker2 # [ 179.171239] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403427264worker2 # [ 179.177141] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7265worker2 # [ 179.179434] nix-daemon[867]: reaped child process 1849, status = succeeded7266worker2 # [ 179.184518] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7267client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7268client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7269client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7270client # farm-b> LOG-farm-b-drain7271worker2 # [ 180.226258] nix-daemon[867]: reaped child process 1863, status = succeeded7272worker2 # [ 180.298084] nix-daemon[867]: accepted connection from pid 1879, user nix-grpc-daemon (trusted)7273worker2 # [ 180.310429] nix-daemon[867]: reaped child process 1882, status = succeeded7274lb # [ 180.292730] niks3-server[1129]: 2026/09/21 14:31:57 INFO Received uploads request method=POST path=/api/pending_closures7275worker2 # [ 180.324750] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.441Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7276worker2 # [ 180.326539] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.443Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7277lb # [ 180.337388] niks3-server[1129]: 2026/09/21 14:31:57 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7278lb # [ 180.343863] niks3-server[1129]: 2026/09/21 14:31:57 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7279lb # [ 180.359104] niks3-server[1129]: 2026/09/21 14:31:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/73/sign7280lb # [ 180.365699] niks3-server[1129]: 2026/09/21 14:31:57 INFO Signed narinfos id=73 count=17281worker2 # [ 180.389735] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.506Z level=INFO msg="Uploading 1 narinfos"7282lb # [ 180.369180] niks3-server[1129]: 2026/09/21 14:31:57 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7283lb # [ 180.377771] niks3-server[1129]: 2026/09/21 14:31:57 INFO Received complete upload request method=POST path=/api/pending_closures/73/complete7284lb # [ 180.381602] niks3-server[1129]: 2026/09/21 14:31:57 INFO Completed upload id=737285lb # [ 180.384310] niks3-server[1129]: 2026/09/21 14:31:57 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7286worker2 # [ 180.406802] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.523Z level=INFO msg="Upload complete. (214ms)"7287worker2 # [ 180.417301] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=17 outputs=17288worker2 # [ 180.425344] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:57Z level=debug event=want client=7 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07289worker2 # [ 180.430648] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:57Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17290worker2 # [ 180.434528] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403427291worker2 # [ 180.443223] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:403427292worker2 # [ 180.450605] nix-daemon[867]: reaped child process 1861, status = succeeded7293worker2 # [ 180.453424] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.570Z level=INFO msg="All 2 paths already cached"7294client # copying 1 paths to 'lb:50051'...7295worker2 # [ 180.458870] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:403427296client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7297worker2 # [ 180.463545] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7298worker2 # [ 180.549494] nix-daemon[867]: accepted connection from pid 1888, user nix-grpc-daemon (trusted)7299worker2 # [ 180.563555] nix-daemon[867]: reaped child process 1891, status = succeeded7300worker2 # [ 180.641064] nix-daemon[867]: accepted connection from pid 1893, user nix-grpc-daemon (trusted)7301worker2 # [ 180.653885] nix-daemon[867]: reaped child process 1896, status = succeeded7302lb # [ 180.636501] niks3-server[1129]: 2026/09/21 14:31:57 INFO Received uploads request method=POST path=/api/pending_closures7303worker2 # [ 180.668227] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.784Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7304worker2 # [ 180.669666] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.784Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7305lb # [ 180.662058] niks3-server[1129]: 2026/09/21 14:31:57 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7306lb # [ 180.675873] niks3-server[1129]: 2026/09/21 14:31:57 INFO Received sign narinfos request method=POST path=/api/pending_closures/74/sign7307lb # [ 180.681675] niks3-server[1129]: 2026/09/21 14:31:57 INFO Signed narinfos id=74 count=17308worker2 # [ 180.708751] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.825Z level=INFO msg="Uploading 1 narinfos"7309lb # [ 180.689112] niks3-server[1129]: 2026/09/21 14:31:57 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7310lb # [ 180.700513] niks3-server[1129]: 2026/09/21 14:31:57 INFO Received complete upload request method=POST path=/api/pending_closures/74/complete7311lb # [ 180.707371] niks3-server[1129]: 2026/09/21 14:31:57 INFO Completed upload id=747312worker2 # [ 180.731041] nix-grpc-daemon[1595]: time=2026-09-21T14:31:57.847Z level=INFO msg="Upload complete. (261ms)"7313lb # [ 180.708904] niks3-server[1129]: 2026/09/21 14:31:57 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7314worker2 # [ 180.734782] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=0 paths=1 nar_bytes_in=7527315worker2 # [ 180.739635] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:403427316worker2 # [ 180.749084] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7317worker2 # [ 180.755058] nix-daemon[867]: reaped child process 1886, status = succeeded7318worker2 # [ 180.759276] nix-daemon[867]: accepted connection from pid 1519, user nix-grpc-daemon (trusted)7319client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7320client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7321client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7322client # farm-top> LOG-farm-top-drain7323worker2 # [ 181.600256] nix-daemon[867]: reaped child process 1900, status = succeeded7324worker2 # [ 181.674389] nix-daemon[867]: accepted connection from pid 1916, user nix-grpc-daemon (trusted)7325worker2 # [ 181.686531] nix-daemon[867]: reaped child process 1919, status = succeeded7326lb # [ 181.671940] niks3-server[1129]: 2026/09/21 14:31:58 INFO Received uploads request method=POST path=/api/pending_closures7327worker2 # [ 181.703994] nix-grpc-daemon[1595]: time=2026-09-21T14:31:58.820Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7328worker2 # [ 181.705463] nix-grpc-daemon[1595]: time=2026-09-21T14:31:58.822Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7329lb # [ 181.707927] niks3-server[1129]: 2026/09/21 14:31:58 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7330lb # [ 181.721922] niks3-server[1129]: 2026/09/21 14:31:58 INFO Received sign narinfos request method=POST path=/api/pending_closures/75/sign7331worker2 # [ 181.752613] nix-grpc-daemon[1595]: time=2026-09-21T14:31:58.869Z level=INFO msg="Uploading 1 narinfos"7332lb # [ 181.730414] niks3-server[1129]: 2026/09/21 14:31:58 INFO Signed narinfos id=75 count=17333lb # [ 181.734623] niks3-server[1129]: 2026/09/21 14:31:58 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7334lb # [ 181.739950] niks3-server[1129]: 2026/09/21 14:31:58 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7335lb # [ 181.750967] niks3-server[1129]: 2026/09/21 14:31:58 INFO Received complete upload request method=POST path=/api/pending_closures/75/complete7336lb # [ 181.753925] niks3-server[1129]: 2026/09/21 14:31:58 INFO Completed upload id=757337worker2 # [ 181.777541] nix-grpc-daemon[1595]: time=2026-09-21T14:31:58.894Z level=INFO msg="Upload complete. (211ms)"7338lb # [ 181.756359] niks3-server[1129]: 2026/09/21 14:31:58 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7339worker2 # [ 181.786100] nix-grpc-daemon[1519]: ts=2026-09-21T14:31:58Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:40342 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=18 outputs=17340worker2 # [ 181.798888] nix-daemon[867]: reaped child process 1898, status = succeeded7341client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 4.81 seconds)7342client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7343client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.12 seconds)7344worker1: must succeed: rm /nix/.rw-store/fill7345worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7346worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7347worker1 # [ 182.389434] nix-grpc-daemon[1701]: ts=2026-09-21T14:31:59Z level=info event=healthy reason=min_free7348worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.09 seconds)7349(finished: subtest: low disk drains a worker and builds go to the other, in 8.21 seconds)7350(finished: run the VM test script, in 184.09 seconds)7351test script finished in 184.25s7352cleanup7353kill QemuMachine (pid 49)7354client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7355client # [2026-09-21T14:32:00Z INFO virtiofsd] Client disconnected, shutting down7356client # [2026-09-21T14:32:00Z INFO virtiofsd] Client disconnected, shutting down7357client # [2026-09-21T14:32:00Z INFO virtiofsd] Client disconnected, shutting down7358kill QemuMachine (pid 50)7359lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7360lb # [2026-09-21T14:32:00Z INFO virtiofsd] Client disconnected, shutting down7361lb # [2026-09-21T14:32:00Z INFO virtiofsd] Client disconnected, shutting down7362lb # [2026-09-21T14:32:00Z INFO virtiofsd] Client disconnected, shutting down7363kill QemuMachine (pid 51)7364worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7365worker1 # [2026-09-21T14:32:01Z INFO virtiofsd] Client disconnected, shutting down7366worker1 # [2026-09-21T14:32:01Z INFO virtiofsd] Client disconnected, shutting down7367worker1 # [2026-09-21T14:32:01Z INFO virtiofsd] Client disconnected, shutting down7368kill QemuMachine (pid 52)7369worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7370worker2 # [2026-09-21T14:32:01Z INFO virtiofsd] Client disconnected, shutting down7371worker2 # [2026-09-21T14:32:01Z INFO virtiofsd] Client disconnected, shutting down7372worker2 # [2026-09-21T14:32:01Z INFO virtiofsd] Client disconnected, shutting down7373(finished: cleanup, in 1.07 seconds)7374additionally exposed symbols:7375 client, lb, worker1, worker2,7376 vlan1,7377 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh