vm-test-run-nix-grpc-farm
checks.x86_64-linux.farm
· build #550
· raw
1tribuchet: building on jamie2Machine state will be reset. To keep it, pass --keep-machine-state3start all VLans4(finished: start all VLans, in 0.00 seconds)5Test will time out and terminate in 900.0 seconds6run the VM test script7start all VMs8client: starting vm9lb: starting vm10worker1: starting vm11worker2: starting vm12worker2 # Disk image does not exist, creating the virtualisation disk image...13worker2 # Formatting '/build/vm-state-worker2/tmp.D117ProZ2Z', fmt=raw size=107374182414worker2: QEMU running (pid 51)15worker2 # mke2fs 1.47.4 (6-Mar-2025)16worker2 # Discarding device blocks: 0/262144 done17worker2 # Creating filesystem with 262144 4k blocks and 65536 inodes18worker2 # Filesystem UUID: a8f85e1c-405a-449f-9531-25a26b98f32019worker2 # Superblock backups stored on blocks:20worker2 # 32768, 98304, 163840, 22937621worker2 # 22lb: QEMU running (pid 50)23worker2 # Allocating group tables: 0/8 done24lb # Disk image does not exist, creating the virtualisation disk image...25worker2 # Writing inode tables: 0/8 done26lb # Formatting '/build/vm-state-lb/tmp.OvS0VFU8XX', fmt=raw size=107374182427worker2 # Creating journal (8192 blocks): done28lb # mke2fs 1.47.4 (6-Mar-2025)29worker1 # Disk image does not exist, creating the virtualisation disk image...30worker1: QEMU running (pid 52)31worker2 # Writing superblocks and filesystem accounting information: 0/8 done32lb # Discarding device blocks: 0/262144 done33worker2 # 34lb # Creating filesystem with 262144 4k blocks and 65536 inodes35worker2 # Virtualisation disk image created.36lb # Filesystem UUID: b936f428-3736-426c-879d-7fe12bc3751237worker2 # Starting virtiofs daemons...38lb # Superblock backups stored on blocks:39worker2 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)40lb # 32768, 98304, 163840, 22937641worker2 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether42lb # 43worker2 # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...44lb # Allocating group tables: 0/8 done45worker2 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)46lb # Writing inode tables: 0/8 done47worker2 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether48lb # Creating journal (8192 blocks): done49worker2 # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...50lb # Writing superblocks and filesystem accounting information: 0/8 done51worker2 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)52lb # 53worker2 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether54lb # Virtualisation disk image created.55worker2 # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...56lb # Starting virtiofs daemons...57worker2 # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests58lb # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)59worker2 # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests60lb # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether61worker2 # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests62lb # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...63worker1 # Formatting '/build/vm-state-worker1/tmp.sRVpRCinmA', fmt=raw size=107374182464worker1 # mke2fs 1.47.4 (6-Mar-2025)65lb # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)66worker1 # Discarding device blocks: 0/262144 done67lb # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether68worker1 # Creating filesystem with 262144 4k blocks and 65536 inodes69lb # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...70worker1 # Filesystem UUID: d1018570-471f-49ae-95f2-1301a0d1bdd371lb # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)72worker1 # Superblock backups stored on blocks:73lb # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether74worker1 # 32768, 98304, 163840, 22937675lb # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...76worker1 # 77lb # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests78worker1 # Allocating group tables: 0/8 done79lb # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests80client # Disk image does not exist, creating the virtualisation disk image...81lb # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests82worker1 # Writing inode tables: 0/8 done83client: QEMU running (pid 49)84worker1 # Creating journal (8192 blocks): done85client # Formatting '/build/vm-state-client/tmp.zf031sj4lC', fmt=raw size=107374182486worker1 # Writing superblocks and filesystem accounting information: 0/8 done87client # mke2fs 1.47.4 (6-Mar-2025)88worker1 # 89(finished: start all VMs, in 0.22 seconds)90worker1 # Virtualisation disk image created.91worker1 # Starting virtiofs daemons...92client # Discarding device blocks: 0/262144 done93worker1 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)94client # Creating filesystem with 262144 4k blocks and 65536 inodes95worker1 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether96client # Filesystem UUID: 038f677c-2411-412e-8a1b-e71cfb1006a897worker1 # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...98client # Superblock backups stored on blocks:99worker1 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)100client # 32768, 98304, 163840, 229376101worker1 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether102client # 103worker1 # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...104client # Allocating group tables: 0/8 done105worker1 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)106client # Writing inode tables: 0/8 done107worker1 # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether108client # Creating journal (8192 blocks): done109worker1 # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...110client # Writing superblocks and filesystem accounting information: 0/8 done111worker1 # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests112client # 113worker1 # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests114client # Virtualisation disk image created.115worker1 # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests116client # Starting virtiofs daemons...117client: waiting for unit network-addresses-eth1.service118client # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)119client: waiting for the VM to finish booting120client # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether121client # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...122client # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)123client # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether124client # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...125client # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] Failed to open file handle for the root node: Operation not permitted (os error 1)126client # [2026-09-24T11:47:10Z WARN virtiofsd::passthrough] File handles do not appear safe to use, disabling file handles altogether127client # [2026-09-24T11:47:10Z INFO virtiofsd] Waiting for vhost-user socket connection...128client # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests129client # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests130client # [2026-09-24T11:47:10Z INFO virtiofsd] Client connected, servicing requests131worker1 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)132worker2 # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)133lb # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)134client # SeaBIOS (version rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org)135worker1 # 136worker1 # 137worker1 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00138worker1 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...139worker1 # 140worker1 # 141worker1 # 142worker1 # 143worker1 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00144worker2 # 145worker2 # 146worker2 # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00147worker1 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...148worker1 # 149worker1 # 150lb # 151lb # 152worker2 # Press Ctrl-B to configure iPXE (PCI 00:02.0)...153worker2 # 154worker2 # 155worker2 # 156worker2 # 157lb # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00158worker2 # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00159lb # Press Ctrl-B to configure iPXE (PCI 00:02.0)...160lb # 161lb # 162lb # 163lb # 164lb # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00165worker2 # Press Ctrl-B to configure iPXE (PCI 00:05.0)...166worker2 # 167worker2 # 168lb # Press Ctrl-B to configure iPXE (PCI 00:05.0)...169lb # 170lb # 171client # 172client # 173client # iPXE (http://ipxe.org) 00:02.0 CA00 PCI2.10 PnP PMM+5EFCC730+5EF2C730 CA00174client # Press Ctrl-B to configure iPXE (PCI 00:02.0)...175client # 176client # 177client # 178client # 179client # iPXE (http://ipxe.org) 00:05.0 CB00 PCI2.10 PnP PMM 5EFCC730 5EF2C730 CB00180worker1 # Booting from ROM...181client # Press Ctrl-B to configure iPXE (PCI 00:05.0)...182client # 183client # 184worker2 # Booting from ROM...185lb # Booting from ROM...186client # Booting from ROM...187worker1 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026188worker1 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mkwkxgwhpjysj2mq8k0s3wqr9pysn9qv-nixos-system-worker1-test/init regInfo=/nix/store/vdlc5q44vmfzpwjrsj2jbidhd4q68s1p-closure-info/registration console=ttyS0,115200n8 console=tty0189worker1 # [ 0.000000] BIOS-provided physical RAM map:190worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable191worker1 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved192worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved193worker1 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable194worker1 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved195worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved196worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved197worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved198worker1 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved199worker1 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved200worker1 # [ 0.000000] NX (Execute Disable) protection: active201worker1 # [ 0.000000] APIC: Static calls initialized202worker1 # [ 0.000000] SMBIOS 2.8 present.203worker1 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014204worker1 # [ 0.000000] DMI: Memory slots populated: 1/1205worker1 # [ 0.000000] Hypervisor detected: KVM206worker1 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000207worker1 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00208worker1 # [ 0.000001] kvm-clock: using sched offset of 473690971 cycles209worker2 # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026210lb # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026211worker1 # [ 0.000003] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns212worker1 # [ 0.000005] tsc: Detected 2400.012 MHz processor213worker1 # [ 0.000814] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000214worker1 # [ 0.000841] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs215worker2 # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n3dssf07312zabh0pyrmzd08hx0vrwvx-nixos-system-worker2-test/init regInfo=/nix/store/m6vyv4sx44r8zyx41gfnkmi5ql2ysdna-closure-info/registration console=ttyS0,115200n8 console=tty0216worker1 # [ 0.000844] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT217lb # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0218worker2 # [ 0.000000] BIOS-provided physical RAM map:219lb # [ 0.000000] BIOS-provided physical RAM map:220worker1 # [ 0.002738] found SMP MP-table at [mem 0x000f5450-0x000f545f]221worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable222worker1 # [ 0.002749] Using GB pages for direct mapping223lb # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable224worker1 # [ 0.002837] RAMDISK: [mem 0x5e370000-0x5ffcffff]225worker2 # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved226lb # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved227worker1 # [ 0.002845] ACPI: Early table checksum verification disabled228worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved229lb # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved230worker1 # [ 0.002848] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )231worker2 # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable232lb # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable233worker1 # [ 0.002851] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)234worker2 # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved235lb # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved236worker1 # [ 0.002856] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)237worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved238lb # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved239worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved240lb # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved241worker1 # [ 0.002863] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)242worker1 # [ 0.002865] ACPI: FACS 0x000000005FFE0000 000040243worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved244lb # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved245worker1 # [ 0.002866] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)246lb # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved247worker2 # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved248lb # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved249worker2 # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved250worker1 # [ 0.002868] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)251worker2 # [ 0.000000] NX (Execute Disable) protection: active252lb # [ 0.000000] NX (Execute Disable) protection: active253worker2 # [ 0.000000] APIC: Static calls initialized254lb # [ 0.000000] APIC: Static calls initialized255worker1 # [ 0.002869] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)256lb # [ 0.000000] SMBIOS 2.8 present.257worker2 # [ 0.000000] SMBIOS 2.8 present.258worker1 # [ 0.002871] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)259lb # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014260worker2 # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014261lb # [ 0.000000] DMI: Memory slots populated: 1/1262worker1 # [ 0.002872] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]263worker2 # [ 0.000000] DMI: Memory slots populated: 1/1264lb # [ 0.000000] Hypervisor detected: KVM265worker2 # [ 0.000000] Hypervisor detected: KVM266worker1 # [ 0.002873] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]267lb # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000268worker2 # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000269worker1 # [ 0.002874] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]270lb # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00271worker2 # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00272lb # [ 0.000000] kvm-clock: using sched offset of 473912230 cycles273worker1 # [ 0.002874] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]274worker2 # [ 0.000000] kvm-clock: using sched offset of 482665046 cycles275worker1 # [ 0.002875] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]276lb # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns277worker2 # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns278lb # [ 0.000004] tsc: Detected 2400.012 MHz processor279worker1 # [ 0.002875] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]280worker2 # [ 0.000004] tsc: Detected 2400.012 MHz processor281lb # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000282worker2 # [ 0.000804] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000283worker1 # [ 0.002876] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]284worker1 # [ 0.003105] No NUMA configuration found285lb # [ 0.000837] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs286worker2 # [ 0.000831] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs287worker1 # [ 0.003105] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]288lb # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT289worker2 # [ 0.000834] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT290worker1 # [ 0.003108] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]291lb # [ 0.002741] found SMP MP-table at [mem 0x000f5450-0x000f545f]292worker1 # [ 0.005343] Zone ranges:293worker2 # [ 0.002725] found SMP MP-table at [mem 0x000f5450-0x000f545f]294lb # [ 0.002751] Using GB pages for direct mapping295worker2 # [ 0.002736] Using GB pages for direct mapping296worker1 # [ 0.005344] DMA [mem 0x0000000000001000-0x0000000000ffffff]297lb # [ 0.002859] RAMDISK: [mem 0x5e36e000-0x5ffcffff]298worker2 # [ 0.002867] RAMDISK: [mem 0x5e36f000-0x5ffcffff]299worker1 # [ 0.005345] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]300lb # [ 0.002866] ACPI: Early table checksum verification disabled301worker1 # [ 0.005346] Normal empty302worker2 # [ 0.002875] ACPI: Early table checksum verification disabled303worker1 # [ 0.005347] Device empty304lb # [ 0.002868] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )305worker2 # [ 0.002877] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )306worker1 # [ 0.005347] Movable zone start for each node307worker1 # [ 0.005348] Early memory node ranges308lb # [ 0.002871] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)309worker2 # [ 0.002881] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)310worker1 # [ 0.005349] node 0: [mem 0x0000000000001000-0x000000000009efff]311lb # [ 0.002875] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)312worker2 # [ 0.002885] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)313worker1 # [ 0.005350] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]314lb # [ 0.002882] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)315worker1 # [ 0.005351] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]316worker2 # [ 0.002891] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)317lb # [ 0.002884] ACPI: FACS 0x000000005FFE0000 000040318worker1 # [ 0.005355] On node 0, zone DMA: 1 pages in unavailable ranges319worker2 # [ 0.002893] ACPI: FACS 0x000000005FFE0000 000040320lb # [ 0.002886] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)321worker1 # [ 0.005369] On node 0, zone DMA: 97 pages in unavailable ranges322worker2 # [ 0.002895] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)323worker1 # [ 0.029957] On node 0, zone DMA32: 40 pages in unavailable ranges324client # Probing EDD (edd=off to disable)... ok[ 0.000000] Linux version 6.18.52 (nixbld@localhost) (gcc (GCC) 15.3.0, GNU ld (GNU Binutils) 2.46) #1-NixOS SMP PREEMPT_DYNAMIC Mon Sep 14 11:36:19 UTC 2026325lb # [ 0.002888] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)326worker1 # [ 0.030413] ACPI: PM-Timer IO Port: 0x608327worker2 # [ 0.002896] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)328lb # [ 0.002889] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)329worker1 # [ 0.030424] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])330worker2 # [ 0.002898] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)331worker1 # [ 0.030450] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23332lb # [ 0.002891] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)333worker2 # [ 0.002899] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)334worker1 # [ 0.030453] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)335lb # [ 0.002892] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]336worker2 # [ 0.002900] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]337worker1 # [ 0.030454] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)338client # [ 0.000000] Command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/v0nhx8gbxswzgx7y6lz7p1zivqn69vd6-nixos-system-client-test/init regInfo=/nix/store/s4r52xgxrfs3dsrmxl8i8fdi7hfx5bfk-closure-info/registration console=ttyS0,115200n8 console=tty0339lb # [ 0.002893] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]340client # [ 0.000000] BIOS-provided physical RAM map:341worker1 # [ 0.030455] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)342worker2 # [ 0.002901] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]343lb # [ 0.002894] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]344client # [ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable345worker1 # [ 0.030456] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)346worker2 # [ 0.002902] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]347lb # [ 0.002894] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]348client # [ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved349worker1 # [ 0.030457] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)350worker2 # [ 0.002902] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]351lb # [ 0.002895] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]352client # [ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved353worker1 # [ 0.030468] ACPI: Using ACPI (MADT) for SMP configuration information354worker2 # [ 0.002903] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]355lb # [ 0.002895] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]356worker1 # [ 0.030469] ACPI: HPET id: 0x8086a201 base: 0xfed00000357client # [ 0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000005ffd7fff] usable358worker2 # [ 0.002903] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]359worker1 # [ 0.030472] TSC deadline timer available360lb # [ 0.002895] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]361client # [ 0.000000] BIOS-e820: [mem 0x000000005ffd8000-0x000000005fffffff] reserved362worker1 # [ 0.030476] CPU topo: Max. logical packages: 1363lb # [ 0.003120] No NUMA configuration found364worker2 # [ 0.002904] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]365worker1 # [ 0.030476] CPU topo: Max. logical dies: 1366client # [ 0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved367worker2 # [ 0.003137] No NUMA configuration found368lb # [ 0.003121] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]369worker1 # [ 0.030477] CPU topo: Max. dies per package: 1370worker2 # [ 0.003138] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]371lb # [ 0.003123] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]372worker1 # [ 0.030480] CPU topo: Max. threads per core: 1373client # [ 0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved374lb # [ 0.005436] Zone ranges:375worker2 # [ 0.003141] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]376worker1 # [ 0.030481] CPU topo: Num. cores per package: 1377client # [ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved378worker2 # [ 0.005236] Zone ranges:379lb # [ 0.005436] DMA [mem 0x0000000000001000-0x0000000000ffffff]380worker1 # [ 0.030481] CPU topo: Num. threads per package: 1381worker2 # [ 0.005237] DMA [mem 0x0000000000001000-0x0000000000ffffff]382client # [ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved383lb # [ 0.005438] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]384worker1 # [ 0.030482] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs385lb # [ 0.005439] Normal empty386worker2 # [ 0.005238] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]387client # [ 0.000000] BIOS-e820: [mem 0x000000fd00000000-0x000000ffffffffff] reserved388lb # [ 0.005439] Device empty389worker1 # [ 0.030499] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()390worker2 # [ 0.005239] Normal empty391lb # [ 0.005440] Movable zone start for each node392worker2 # [ 0.005240] Device empty393client # [ 0.000000] NX (Execute Disable) protection: active394lb # [ 0.005441] Early memory node ranges395client # [ 0.000000] APIC: Static calls initialized396worker2 # [ 0.005241] Movable zone start for each node397worker1 # [ 0.030526] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]398client # [ 0.000000] SMBIOS 2.8 present.399worker2 # [ 0.005241] Early memory node ranges400lb # [ 0.005441] node 0: [mem 0x0000000000001000-0x000000000009efff]401worker1 # [ 0.030528] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]402worker2 # [ 0.005241] node 0: [mem 0x0000000000001000-0x000000000009efff]403lb # [ 0.005442] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]404client # [ 0.000000] DMI: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.17.0-0-gb52ca86e094d-prebuilt.qemu.org 04/01/2014405worker1 # [ 0.030529] [mem 0x60000000-0xafffffff] available for PCI devices406worker2 # [ 0.005242] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]407lb # [ 0.005443] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]408client # [ 0.000000] DMI: Memory slots populated: 1/1409worker1 # [ 0.030531] Booting paravirtualized kernel on KVM410client # [ 0.000000] Hypervisor detected: KVM411lb # [ 0.005462] On node 0, zone DMA: 1 pages in unavailable ranges412worker2 # [ 0.005244] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]413client # [ 0.000000] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000414worker2 # [ 0.005262] On node 0, zone DMA: 1 pages in unavailable ranges415worker1 # [ 0.030533] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns416lb # [ 0.005726] On node 0, zone DMA: 97 pages in unavailable ranges417client # [ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00418lb # [ 0.031017] On node 0, zone DMA32: 40 pages in unavailable ranges419worker2 # [ 0.005514] On node 0, zone DMA: 97 pages in unavailable ranges420client # [ 0.000000] kvm-clock: using sched offset of 492443228 cycles421worker1 # [ 0.035013] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1422lb # [ 0.031474] ACPI: PM-Timer IO Port: 0x608423worker2 # [ 0.030557] On node 0, zone DMA32: 40 pages in unavailable ranges424worker1 # [ 0.035132] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152425lb # [ 0.031485] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])426worker2 # [ 0.031017] ACPI: PM-Timer IO Port: 0x608427client # [ 0.000002] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns428worker1 # [ 0.035176] kvm-guest: PV spinlocks disabled, single CPU429lb # [ 0.031510] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23430client # [ 0.000005] tsc: Detected 2400.012 MHz processor431worker2 # [ 0.031028] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])432lb # [ 0.031513] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)433client # [ 0.000811] last_pfn = 0x5ffd8 max_arch_pfn = 0x10000000000434worker2 # [ 0.031053] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23435lb # [ 0.031515] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)436client # [ 0.000838] MTRR map: 4 entries (3 fixed + 1 variable; max 19), built from 8 variable MTRRs437worker2 # [ 0.031055] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)438lb # [ 0.031516] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)439client # [ 0.000840] x86/PAT: Configuration [0-7]: WB WC UC- UC WB WP UC- WT440worker2 # [ 0.031057] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)441lb # [ 0.031517] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)442client # [ 0.002729] found SMP MP-table at [mem 0x000f5450-0x000f545f]443worker1 # [ 0.035178] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/mkwkxgwhpjysj2mq8k0s3wqr9pysn9qv-nixos-system-worker1-test/init regInfo=/nix/store/vdlc5q44vmfzpwjrsj2jbidhd4q68s1p-closure-info/registration console=ttyS0,115200n8 console=tty0444worker2 # [ 0.031058] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)445client # [ 0.002741] Using GB pages for direct mapping446lb # [ 0.031517] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)447client # [ 0.002814] RAMDISK: [mem 0x5e370000-0x5ffcffff]448worker2 # [ 0.031059] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)449lb # [ 0.031520] ACPI: Using ACPI (MADT) for SMP configuration information450client # [ 0.002823] ACPI: Early table checksum verification disabled451worker2 # [ 0.031060] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)452lb # [ 0.031521] ACPI: HPET id: 0x8086a201 base: 0xfed00000453worker1 # [ 0.035267] Unknown kernel command line parameters "regInfo=/nix/store/vdlc5q44vmfzpwjrsj2jbidhd4q68s1p-closure-info/registration", will be passed to user space.454worker1 # [ 0.035280] random: crng init done455lb # [ 0.031524] TSC deadline timer available456client # [ 0.002825] ACPI: RSDP 0x00000000000F5270 000014 (v00 BOCHS )457worker2 # [ 0.031063] ACPI: Using ACPI (MADT) for SMP configuration information458lb # [ 0.031528] CPU topo: Max. logical packages: 1459worker1 # [ 0.035280] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes460worker2 # [ 0.031064] ACPI: HPET id: 0x8086a201 base: 0xfed00000461client # [ 0.002829] ACPI: RSDT 0x000000005FFE2422 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)462lb # [ 0.031528] CPU topo: Max. logical dies: 1463worker2 # [ 0.031067] TSC deadline timer available464lb # [ 0.031529] CPU topo: Max. dies per package: 1465worker1 # [ 0.035372] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)466worker2 # [ 0.031071] CPU topo: Max. logical packages: 1467client # [ 0.002833] ACPI: FACP 0x000000005FFE221A 0000F4 (v03 BOCHS BXPC 00000001 BXPC 00000001)468lb # [ 0.031532] CPU topo: Max. threads per core: 1469worker2 # [ 0.031071] CPU topo: Max. logical dies: 1470worker1 # [ 0.036405] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)471lb # [ 0.031533] CPU topo: Num. cores per package: 1472worker2 # [ 0.031072] CPU topo: Max. dies per package: 1473client # [ 0.002840] ACPI: DSDT 0x000000005FFE0040 0021DA (v01 BOCHS BXPC 00000001 BXPC 00000001)474worker1 # [ 0.036435] Fallback order for Node 0: 0475lb # [ 0.031533] CPU topo: Num. threads per package: 1476worker2 # [ 0.031075] CPU topo: Max. threads per core: 1477client # [ 0.002842] ACPI: FACS 0x000000005FFE0000 000040478worker1 # [ 0.036437] Built 1 zonelists, mobility grouping on. Total pages: 393078479lb # [ 0.031533] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs480worker2 # [ 0.031076] CPU topo: Num. cores per package: 1481worker1 # [ 0.036438] Policy zone: DMA32482client # [ 0.002843] ACPI: APIC 0x000000005FFE230E 000078 (v03 BOCHS BXPC 00000001 BXPC 00000001)483worker2 # [ 0.031076] CPU topo: Num. threads per package: 1484lb # [ 0.031555] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()485worker1 # [ 0.036840] mem auto-init: stack:all(zero), heap alloc:on, heap free:off486worker2 # [ 0.031076] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs487client # [ 0.002845] ACPI: HPET 0x000000005FFE2386 000038 (v01 BOCHS BXPC 00000001 BXPC 00000001)488lb # [ 0.031584] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]489worker1 # [ 0.039755] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1490worker2 # [ 0.031092] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()491worker1 # [ 0.042924] allocated 3145728 bytes of page_ext492lb # [ 0.031585] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]493client # [ 0.002846] ACPI: MCFG 0x000000005FFE23BE 00003C (v01 BOCHS BXPC 00000001 BXPC 00000001)494worker1 # [ 0.052631] ftrace: allocating 48787 entries in 192 pages495worker2 # [ 0.031118] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]496lb # [ 0.031587] [mem 0x60000000-0xafffffff] available for PCI devices497client # [ 0.002848] ACPI: WAET 0x000000005FFE23FA 000028 (v01 BOCHS BXPC 00000001 BXPC 00000001)498worker1 # [ 0.052632] ftrace: allocated 192 pages with 2 groups499lb # [ 0.031588] Booting paravirtualized kernel on KVM500worker2 # [ 0.031120] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]501worker1 # [ 0.053661] Dynamic Preempt: lazy502client # [ 0.002849] ACPI: Reserving FACP table memory at [mem 0x5ffe221a-0x5ffe230d]503worker2 # [ 0.031122] [mem 0x60000000-0xafffffff] available for PCI devices504worker1 # [ 0.053840] rcu: Preemptible hierarchical RCU implementation.505lb # [ 0.031591] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns506client # [ 0.002850] ACPI: Reserving DSDT table memory at [mem 0x5ffe0040-0x5ffe2219]507worker1 # [ 0.053840] rcu: RCU event tracing is enabled.508worker2 # [ 0.031123] Booting paravirtualized kernel on KVM509lb # [ 0.036040] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1510worker1 # [ 0.053841] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.511client # [ 0.002850] ACPI: Reserving FACS table memory at [mem 0x5ffe0000-0x5ffe003f]512worker2 # [ 0.031126] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns513worker1 # [ 0.053842] Trampoline variant of Tasks RCU enabled.514lb # [ 0.038107] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152515client # [ 0.002851] ACPI: Reserving APIC table memory at [mem 0x5ffe230e-0x5ffe2385]516worker1 # [ 0.053843] Rude variant of Tasks RCU enabled.517lb # [ 0.038149] kvm-guest: PV spinlocks disabled, single CPU518worker2 # [ 0.035592] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1519client # [ 0.002851] ACPI: Reserving HPET table memory at [mem 0x5ffe2386-0x5ffe23bd]520worker1 # [ 0.053843] Tracing variant of Tasks RCU enabled.521worker2 # [ 0.037785] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152522client # [ 0.002852] ACPI: Reserving MCFG table memory at [mem 0x5ffe23be-0x5ffe23f9]523worker1 # [ 0.053844] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.524worker2 # [ 0.037828] kvm-guest: PV spinlocks disabled, single CPU525client # [ 0.002852] ACPI: Reserving WAET table memory at [mem 0x5ffe23fa-0x5ffe2421]526worker1 # [ 0.053845] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1527client # [ 0.003076] No NUMA configuration found528worker1 # [ 0.053904] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.529client # [ 0.003077] Faking a node at [mem 0x0000000000000000-0x000000005ffd7fff]530lb # [ 0.038151] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/b57s96hna1n87xmygcaz6qx4i7f2zyzx-nixos-system-lb-test/init regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration console=ttyS0,115200n8 console=tty0531client # [ 0.003080] NODE_DATA(0) allocated [mem 0x5ffd2780-0x5ffd7cff]532worker1 # [ 0.053906] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.533client # [ 0.005333] Zone ranges:534client # [ 0.005334] DMA [mem 0x0000000000001000-0x0000000000ffffff]535worker2 # [ 0.037829] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/n3dssf07312zabh0pyrmzd08hx0vrwvx-nixos-system-worker2-test/init regInfo=/nix/store/m6vyv4sx44r8zyx41gfnkmi5ql2ysdna-closure-info/registration console=ttyS0,115200n8 console=tty0536worker1 # [ 0.053907] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.537lb # [ 0.038242] Unknown kernel command line parameters "regInfo=/nix/store/jpd2qccg07hyc3sd9msmabx1jknjxkc6-closure-info/registration", will be passed to user space.538client # [ 0.005335] DMA32 [mem 0x0000000001000000-0x000000005ffd7fff]539lb # [ 0.038254] random: crng init done540worker1 # [ 0.058311] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16541client # [ 0.005336] Normal empty542client # [ 0.005337] Device empty543lb # [ 0.038255] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes544worker1 # [ 0.058586] rcu: srcu_init: Setting srcu_struct sizes based on contention.545worker2 # [ 0.037919] Unknown kernel command line parameters "regInfo=/nix/store/m6vyv4sx44r8zyx41gfnkmi5ql2ysdna-closure-info/registration", will be passed to user space.546client # [ 0.005338] Movable zone start for each node547worker2 # [ 0.037932] random: crng init done548client # [ 0.005339] Early memory node ranges549lb # [ 0.040280] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)550worker1 # [ 0.058591] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns551client # [ 0.005339] node 0: [mem 0x0000000000001000-0x000000000009efff]552worker2 # [ 0.037932] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes553lb # [ 0.041443] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)554lb # [ 0.041472] Fallback order for Node 0: 0555client # [ 0.005340] node 0: [mem 0x0000000000100000-0x000000005ffd7fff]556worker2 # [ 0.039958] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)557worker1 # [ 0.058693] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)558worker1 # [ 0.062246] Console: colour VGA+ 80x25559lb # [ 0.041475] Built 1 zonelists, mobility grouping on. Total pages: 393078560client # [ 0.005341] Initmem setup node 0 [mem 0x0000000000001000-0x000000005ffd7fff]561lb # [ 0.041476] Policy zone: DMA32562worker2 # [ 0.040963] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)563worker1 # [ 0.062249] printk: legacy console [tty0] enabled564client # [ 0.005360] On node 0, zone DMA: 1 pages in unavailable ranges565worker2 # [ 0.040993] Fallback order for Node 0: 0566worker1 # [ 0.091861] printk: legacy console [ttyS0] enabled567lb # [ 0.043855] mem auto-init: stack:all(zero), heap alloc:on, heap free:off568client # [ 0.005623] On node 0, zone DMA: 97 pages in unavailable ranges569worker2 # [ 0.040996] Built 1 zonelists, mobility grouping on. Total pages: 393078570worker1 # [ 0.190087] ACPI: Core revision 20250807571lb # [ 0.046761] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1572worker2 # [ 0.040997] Policy zone: DMA32573client # [ 0.032094] On node 0, zone DMA32: 40 pages in unavailable ranges574lb # [ 0.049911] allocated 3145728 bytes of page_ext575client # [ 0.032560] ACPI: PM-Timer IO Port: 0x608576worker2 # [ 0.043350] mem auto-init: stack:all(zero), heap alloc:on, heap free:off577lb # [ 0.059472] ftrace: allocating 48787 entries in 192 pages578client # [ 0.032571] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])579worker1 # [ 0.191001] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns580lb # [ 0.059474] ftrace: allocated 192 pages with 2 groups581worker2 # [ 0.046266] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1582lb # [ 0.060288] Dynamic Preempt: lazy583worker2 # [ 0.049552] allocated 3145728 bytes of page_ext584client # [ 0.032600] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23585worker1 # [ 0.192507] APIC: Switch to symmetric I/O mode setup586lb # [ 0.060402] rcu: Preemptible hierarchical RCU implementation.587worker2 # [ 0.059324] ftrace: allocating 48787 entries in 192 pages588client # [ 0.032603] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)589worker1 # [ 0.193430] x2apic enabled590lb # [ 0.060402] rcu: RCU event tracing is enabled.591worker2 # [ 0.059328] ftrace: allocated 192 pages with 2 groups592client # [ 0.032604] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)593worker2 # [ 0.060406] Dynamic Preempt: lazy594lb # [ 0.060403] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.595worker1 # [ 0.194169] APIC: Switched APIC routing to: physical x2apic596worker2 # [ 0.060556] rcu: Preemptible hierarchical RCU implementation.597client # [ 0.032605] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)598lb # [ 0.060405] Trampoline variant of Tasks RCU enabled.599worker2 # [ 0.060557] rcu: RCU event tracing is enabled.600lb # [ 0.060405] Rude variant of Tasks RCU enabled.601client # [ 0.032606] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)602lb # [ 0.060405] Tracing variant of Tasks RCU enabled.603worker2 # [ 0.060558] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.604client # [ 0.032607] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)605worker1 # [ 0.196113] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1606worker2 # [ 0.060559] Trampoline variant of Tasks RCU enabled.607lb # [ 0.060406] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.608client # [ 0.032610] ACPI: Using ACPI (MADT) for SMP configuration information609worker2 # [ 0.060560] Rude variant of Tasks RCU enabled.610lb # [ 0.060407] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1611client # [ 0.032611] ACPI: HPET id: 0x8086a201 base: 0xfed00000612worker2 # [ 0.060560] Tracing variant of Tasks RCU enabled.613worker1 # [ 0.197062] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns614client # [ 0.032614] TSC deadline timer available615lb # [ 0.060431] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.616worker2 # [ 0.060560] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.617client # [ 0.032618] CPU topo: Max. logical packages: 1618client # [ 0.032619] CPU topo: Max. logical dies: 1619worker2 # [ 0.060561] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1620worker1 # [ 0.198680] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)621lb # [ 0.060433] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.622client # [ 0.032619] CPU topo: Max. dies per package: 1623client # [ 0.032623] CPU topo: Max. threads per core: 1624worker2 # [ 0.060617] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.625worker1 # [ 0.199764] x86/cpu: User Mode Instruction Prevention (UMIP) activated626lb # [ 0.060434] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.627client # [ 0.032623] CPU topo: Num. cores per package: 1628lb # [ 0.064823] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16629client # [ 0.032624] CPU topo: Num. threads per package: 1630worker2 # [ 0.060619] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.631worker1 # [ 0.200811] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127632lb # [ 0.065256] rcu: srcu_init: Setting srcu_struct sizes based on contention.633client # [ 0.032624] CPU topo: Allowing 1 present CPUs plus 0 hotplug CPUs634worker2 # [ 0.060620] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.635worker1 # [ 0.201677] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0636client # [ 0.032643] kvm-guest: APIC: eoi() replaced with kvm_guest_apic_eoi_write()637lb # [ 0.065262] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns638worker2 # [ 0.064979] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16639client # [ 0.032675] PM: hibernation: Registered nosave memory: [mem 0x00000000-0x00000fff]640worker2 # [ 0.065271] rcu: srcu_init: Setting srcu_struct sizes based on contention.641worker1 # [ 0.202681] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto642lb # [ 0.065361] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)643lb # [ 0.068881] Console: colour VGA+ 80x25644client # [ 0.032676] PM: hibernation: Registered nosave memory: [mem 0x0009f000-0x000fffff]645worker2 # [ 0.065278] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns646lb # [ 0.068885] printk: legacy console [tty0] enabled647client # [ 0.032678] [mem 0x60000000-0xafffffff] available for PCI devices648worker1 # [ 0.204677] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl649lb # [ 0.098338] printk: legacy console [ttyS0] enabled650client # [ 0.032679] Booting paravirtualized kernel on KVM651worker2 # [ 0.065389] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)652lb # [ 0.192556] ACPI: Core revision 20250807653worker1 # [ 0.205677] Transient Scheduler Attacks: Vulnerable: No microcode654worker2 # [ 0.068890] Console: colour VGA+ 80x25655client # [ 0.032682] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1910969940391419 ns656worker2 # [ 0.068893] printk: legacy console [tty0] enabled657worker1 # [ 0.206677] Spectre V2 : Mitigation: Enhanced / Automatic IBRS658worker2 # [ 0.098301] printk: legacy console [ttyS0] enabled659client # [ 0.037163] setup_percpu: NR_CPUS:384 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1660lb # [ 0.193418] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns661worker1 # [ 0.207677] Speculative Return Stack Overflow: Mitigation: Safe RET662worker2 # [ 0.195007] ACPI: Core revision 20250807663client # [ 0.039328] percpu: Embedded 98 pages/cpu s278528 r8192 d114688 u2097152664lb # [ 0.194892] APIC: Switch to symmetric I/O mode setup665client # [ 0.039374] kvm-guest: PV spinlocks disabled, single CPU666lb # [ 0.195813] x2apic enabled667worker2 # [ 0.195873] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns668lb # [ 0.196536] APIC: Switched APIC routing to: physical x2apic669worker2 # [ 0.197393] APIC: Switch to symmetric I/O mode setup670worker2 # [ 0.198370] x2apic enabled671worker1 # [ 0.208677] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization672lb # [ 0.198331] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1673worker2 # [ 0.199142] APIC: Switched APIC routing to: physical x2apic674client # [ 0.039375] Kernel command line: console=ttyS0 console=tty0 panic=1 boot.panic_on_fail clocksource=acpi_pm root=fstab loglevel=7 net.ifnames=0 lsm=landlock,yama,bpf init=/nix/store/v0nhx8gbxswzgx7y6lz7p1zivqn69vd6-nixos-system-client-test/init regInfo=/nix/store/s4r52xgxrfs3dsrmxl8i8fdi7hfx5bfk-closure-info/registration console=ttyS0,115200n8 console=tty0675worker1 # [ 0.210684] Spectre V2 : Enabling IBPB for BPF676lb # [ 0.199285] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns677worker1 # [ 0.211678] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier678worker2 # [ 0.200904] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1679client # [ 0.039469] Unknown kernel command line parameters "regInfo=/nix/store/s4r52xgxrfs3dsrmxl8i8fdi7hfx5bfk-closure-info/registration", will be passed to user space.680client # [ 0.039481] random: crng init done681worker1 # [ 0.212677] active return thunk: srso_alias_return_thunk682lb # [ 0.200882] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)683client # [ 0.039482] printk: log buffer data + meta data: 262144 + 917504 = 1179648 bytes684worker2 # [ 0.201865] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns685lb # [ 0.201966] x86/cpu: User Mode Instruction Prevention (UMIP) activated686worker1 # [ 0.213698] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'687client # [ 0.041606] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)688worker1 # [ 0.214677] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'689worker2 # [ 0.203524] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)690lb # [ 0.203008] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127691client # [ 0.042839] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)692client # [ 0.042870] Fallback order for Node 0: 0693lb # [ 0.203879] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0694worker1 # [ 0.216381] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'695worker2 # [ 0.204607] x86/cpu: User Mode Instruction Prevention (UMIP) activated696client # [ 0.042873] Built 1 zonelists, mobility grouping on. Total pages: 393078697client # [ 0.042874] Policy zone: DMA32698worker1 # [ 0.216677] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'699worker2 # [ 0.205650] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127700lb # [ 0.204882] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto701client # [ 0.045378] mem auto-init: stack:all(zero), heap alloc:on, heap free:off702worker1 # [ 0.218412] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'703worker2 # [ 0.206521] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0704client # [ 0.048573] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1705lb # [ 0.205879] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl706client # [ 0.051892] allocated 3145728 bytes of page_ext707worker1 # [ 0.219395] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'708client # [ 0.061568] ftrace: allocating 48787 entries in 192 pages709lb # [ 0.206879] Transient Scheduler Attacks: Vulnerable: No microcode710worker2 # [ 0.207524] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto711client # [ 0.061570] ftrace: allocated 192 pages with 2 groups712worker1 # [ 0.219677] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'713client # [ 0.062464] Dynamic Preempt: lazy714lb # [ 0.207878] Spectre V2 : Mitigation: Enhanced / Automatic IBRS715worker2 # [ 0.208521] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl716worker1 # [ 0.221677] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256717client # [ 0.062612] rcu: Preemptible hierarchical RCU implementation.718lb # [ 0.208879] Speculative Return Stack Overflow: Mitigation: Safe RET719client # [ 0.062613] rcu: RCU event tracing is enabled.720worker1 # [ 0.222578] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64721worker2 # [ 0.209521] Transient Scheduler Attacks: Vulnerable: No microcode722client # [ 0.062614] rcu: RCU restricting CPUs from NR_CPUS=384 to nr_cpu_ids=1.723worker1 # [ 0.223385] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512724worker2 # [ 0.210521] Spectre V2 : Mitigation: Enhanced / Automatic IBRS725client # [ 0.062616] Trampoline variant of Tasks RCU enabled.726client # [ 0.062616] Rude variant of Tasks RCU enabled.727worker1 # [ 0.224384] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024728worker2 # [ 0.211521] Speculative Return Stack Overflow: Mitigation: Safe RET729client # [ 0.062616] Tracing variant of Tasks RCU enabled.730worker1 # [ 0.225374] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8731client # [ 0.062617] rcu: RCU calculated value of scheduler-enlistment delay is 100 jiffies.732lb # [ 0.209879] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization733client # [ 0.062618] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1734lb # [ 0.211858] Spectre V2 : Enabling IBPB for BPF735worker1 # [ 0.225677] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.736client # [ 0.062628] RCU Tasks: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.737lb # [ 0.212389] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier738worker2 # [ 0.212521] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization739client # [ 0.062629] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.740lb # [ 0.213879] active return thunk: srso_alias_return_thunk741worker2 # [ 0.214528] Spectre V2 : Enabling IBPB for BPF742client # [ 0.062630] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=1.743lb # [ 0.214707] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'744client # [ 0.067260] NR_IRQS: 24832, nr_irqs: 256, preallocated irqs: 16745worker2 # [ 0.215522] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier746lb # [ 0.215879] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'747client # [ 0.067535] rcu: srcu_init: Setting srcu_struct sizes based on contention.748worker2 # [ 0.216521] active return thunk: srso_alias_return_thunk749lb # [ 0.216878] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'750client # [ 0.067543] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns751worker2 # [ 0.217542] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'752lb # [ 0.217879] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'753client # [ 0.067665] kfence: initialized - using 2097152 bytes for 255 objects at 0x(____ptrval____)-0x(____ptrval____)754worker2 # [ 0.218521] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'755client # [ 0.071216] Console: colour VGA+ 80x25756lb # [ 0.218878] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'757client # [ 0.071218] printk: legacy console [tty0] enabled758worker2 # [ 0.219521] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'759lb # [ 0.219878] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'760client # [ 0.100687] printk: legacy console [ttyS0] enabled761worker2 # [ 0.220521] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'762client # [ 0.196839] ACPI: Core revision 20250807763lb # [ 0.220878] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'764worker2 # [ 0.221521] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'765lb # [ 0.221879] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256766client # [ 0.197697] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604467 ns767worker2 # [ 0.222521] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'768lb # [ 0.222878] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64769client # [ 0.199228] APIC: Switch to symmetric I/O mode setup770worker2 # [ 0.223521] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'771client # [ 0.200175] x2apic enabled772lb # [ 0.223878] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512773worker2 # [ 0.224521] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256774lb # [ 0.224878] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024775client # [ 0.200924] APIC: Switched APIC routing to: physical x2apic776worker2 # [ 0.226245] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64777lb # [ 0.225878] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8778worker2 # [ 0.227249] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512779client # [ 0.202736] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1780lb # [ 0.226879] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.781worker2 # [ 0.228243] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024782worker2 # [ 0.228521] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8783client # [ 0.203772] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns784worker2 # [ 0.229521] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.785client # [ 0.205505] Calibrating delay loop (skipped) preset value.. 4800.02 BogoMIPS (lpj=2400012)786client # [ 0.206590] x86/cpu: User Mode Instruction Prevention (UMIP) activated787client # [ 0.208504] Last level iTLB entries: 4KB 512, 2MB 255, 4MB 127788client # [ 0.209479] Last level dTLB entries: 4KB 512, 2MB 255, 4MB 127, 1GB 0789client # [ 0.210505] mitigations: Enabled attack vectors: user_kernel, user_user, guest_host, guest_guest, SMT mitigations: auto790client # [ 0.211501] Speculative Store Bypass: Mitigation: Speculative Store Bypass disabled via prctl791client # [ 0.213502] Transient Scheduler Attacks: Vulnerable: No microcode792client # [ 0.214501] Spectre V2 : Mitigation: Enhanced / Automatic IBRS793client # [ 0.215501] Speculative Return Stack Overflow: Mitigation: Safe RET794client # [ 0.216501] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization795client # [ 0.218514] Spectre V2 : Enabling IBPB for BPF796worker1 # [ 0.260518] Freeing SMP alternatives memory: 44K797worker1 # [ 0.261259] pid_max: default: 32768 minimum: 301798client # [ 0.219502] Spectre V2 : mitigation: Enabling conditional Indirect Branch Prediction Barrier799worker1 # [ 0.261768] LSM: initializing lsm=capability,landlock,yama,bpf,ima800client # [ 0.220502] active return thunk: srso_alias_return_thunk801worker1 # [ 0.262771] landlock: Up and running.802client # [ 0.221523] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'803worker1 # [ 0.263678] Yama: becoming mindful.804worker1 # [ 0.264815] LSM support for eBPF active805client # [ 0.222502] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'806client # [ 0.224269] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'807worker1 # [ 0.265585] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)808client # [ 0.225280] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'809worker1 # [ 0.266715] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)810client # [ 0.226290] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'811client # [ 0.227292] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'812worker1 # [ 0.269135] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)813client # [ 0.228501] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User registers'814client # [ 0.229502] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256815worker1 # [ 0.270179] Performance Events: Fam17h+ core perfctr, AMD PMU driver.816worker1 # [ 0.270682] ... version: 2817client # [ 0.230501] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64818worker1 # [ 0.271434] ... bit width: 48819client # [ 0.231501] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512820worker1 # [ 0.271680] ... generic counters: 6821client # [ 0.232501] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024822worker1 # [ 0.272398] ... generic bitmap: 000000000000003f823client # [ 0.233501] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]: 8824worker1 # [ 0.272724] ... fixed-purpose counters: 0825lb # [ 0.261116] Freeing SMP alternatives memory: 44K826worker1 # [ 0.273463] ... fixed-purpose bitmap: 0000000000000000827client # [ 0.234501] x86/fpu: Enabled xstate features 0x2e7, context size is 2440 bytes, using 'compacted' format.828lb # [ 0.261877] pid_max: default: 32768 minimum: 301829worker1 # [ 0.273680] ... value mask: 0000ffffffffffff830lb # [ 0.262502] LSM: initializing lsm=capability,landlock,yama,bpf,ima831worker1 # [ 0.274601] ... max period: 00007fffffffffff832lb # [ 0.263678] landlock: Up and running.833worker1 # [ 0.275404] ... global_ctrl mask: 000000000000003f834worker2 # [ 0.264607] Freeing SMP alternatives memory: 44K835lb # [ 0.264298] Yama: becoming mindful.836worker2 # [ 0.265384] pid_max: default: 32768 minimum: 301837worker1 # [ 0.275789] signal: max sigframe size: 3376838lb # [ 0.265089] LSM support for eBPF active839worker1 # [ 0.276587] rcu: Hierarchical SRCU implementation.840worker2 # [ 0.266156] LSM: initializing lsm=capability,landlock,yama,bpf,ima841lb # [ 0.265953] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)842worker1 # [ 0.277260] rcu: Max phase no-delay instances is 400.843worker2 # [ 0.266618] landlock: Up and running.844worker2 # [ 0.267521] Yama: becoming mindful.845lb # [ 0.266916] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)846worker2 # [ 0.268733] LSM support for eBPF active847worker2 # [ 0.269576] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)848lb # [ 0.269466] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)849worker2 # [ 0.270558] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)850lb # [ 0.270398] Performance Events: Fam17h+ core perfctr, AMD PMU driver.851worker1 # [ 0.282175] smp: Bringing up secondary CPUs ...852lb # [ 0.270883] ... version: 2853worker1 # [ 0.282693] smp: Brought up 1 node, 1 CPU854lb # [ 0.271584] ... bit width: 48855worker2 # [ 0.273117] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)856worker1 # [ 0.283341] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)857lb # [ 0.271881] ... generic counters: 6858lb # [ 0.272553] ... generic bitmap: 000000000000003f859worker2 # [ 0.274036] Performance Events: Fam17h+ core perfctr, AMD PMU driver.860lb # [ 0.272897] ... fixed-purpose counters: 0861worker2 # [ 0.274525] ... version: 2862worker1 # [ 0.283895] Memory: 1454584K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)863worker2 # [ 0.275246] ... bit width: 48864lb # [ 0.273597] ... fixed-purpose bitmap: 0000000000000000865worker1 # [ 0.284892] devtmpfs: initialized866worker2 # [ 0.275523] ... generic counters: 6867lb # [ 0.273881] ... value mask: 0000ffffffffffff868worker1 # [ 0.285725] x86/mm: Memory block size: 128MB869worker2 # [ 0.276211] ... generic bitmap: 000000000000003f870lb # [ 0.274758] ... max period: 00007fffffffffff871worker2 # [ 0.276569] ... fixed-purpose counters: 0872lb # [ 0.275546] ... global_ctrl mask: 000000000000003f873worker2 # [ 0.277286] ... fixed-purpose bitmap: 0000000000000000874lb # [ 0.275990] signal: max sigframe size: 3376875worker1 # [ 0.287494] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)876worker2 # [ 0.277523] ... value mask: 0000ffffffffffff877lb # [ 0.276772] rcu: Hierarchical SRCU implementation.878worker2 # [ 0.278419] ... max period: 00007fffffffffff879lb # [ 0.277462] rcu: Max phase no-delay instances is 400.880worker1 # [ 0.288628] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).881worker2 # [ 0.279210] ... global_ctrl mask: 000000000000003f882worker1 # [ 0.289769] pinctrl core: initialized pinctrl subsystem883worker2 # [ 0.279645] signal: max sigframe size: 3376884worker1 # [ 0.290945] PM: RTC time: 11:47:10, date: 2026-09-24885worker2 # [ 0.280413] rcu: Hierarchical SRCU implementation.886worker2 # [ 0.281099] rcu: Max phase no-delay instances is 400.887lb # [ 0.282424] smp: Bringing up secondary CPUs ...888worker1 # [ 0.294325] NET: Registered PF_NETLINK/PF_ROUTE protocol family889lb # [ 0.282900] smp: Brought up 1 node, 1 CPU890lb # [ 0.283564] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)891worker1 # [ 0.295151] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations892worker2 # [ 0.286002] smp: Bringing up secondary CPUs ...893worker1 # [ 0.295700] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations894worker2 # [ 0.286536] smp: Brought up 1 node, 1 CPU895lb # [ 0.284131] Memory: 1454604K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109316K reserved, 0K cma-reserved)896lb # [ 0.285106] devtmpfs: initialized897worker2 # [ 0.287221] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)898worker1 # [ 0.296938] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations899lb # [ 0.285837] x86/mm: Memory block size: 128MB900worker1 # [ 0.297698] audit: initializing netlink subsys (disabled)901worker2 # [ 0.287646] Memory: 1454616K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109312K reserved, 0K cma-reserved)902worker1 # [ 0.298681] audit: type=2000 audit(1790250431.594:1): state=initialized audit_enabled=0 res=1903lb # [ 0.287455] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)904worker2 # [ 0.288775] devtmpfs: initialized905worker1 # [ 0.299859] thermal_sys: Registered thermal governor 'fair_share'906client # [ 0.268408] Freeing SMP alternatives memory: 44K907worker2 # [ 0.289598] x86/mm: Memory block size: 128MB908lb # [ 0.287904] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).909client # [ 0.269048] pid_max: default: 32768 minimum: 301910worker1 # [ 0.299861] thermal_sys: Registered thermal governor 'bang_bang'911lb # [ 0.288965] pinctrl core: initialized pinctrl subsystem912client # [ 0.269592] LSM: initializing lsm=capability,landlock,yama,bpf,ima913worker1 # [ 0.300681] thermal_sys: Registered thermal governor 'step_wise'914worker2 # [ 0.291344] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)915lb # [ 0.290021] PM: RTC time: 11:47:10, date: 2026-09-24916client # [ 0.270595] landlock: Up and running.917worker1 # [ 0.301626] thermal_sys: Registered thermal governor 'user_space'918client # [ 0.271502] Yama: becoming mindful.919worker1 # [ 0.302418] thermal_sys: Registered thermal governor 'power_allocator'920worker2 # [ 0.292416] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).921client # [ 0.272532] LSM support for eBPF active922worker1 # [ 0.302702] cpuidle: using governor menu923worker2 # [ 0.293597] pinctrl core: initialized pinctrl subsystem924client # [ 0.273290] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)925worker2 # [ 0.294758] PM: RTC time: 11:47:10, date: 2026-09-24926lb # [ 0.293371] NET: Registered PF_NETLINK/PF_ROUTE protocol family927worker1 # [ 0.304932] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5928client # [ 0.274531] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)929lb # [ 0.294379] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations930worker1 # [ 0.305942] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]931lb # [ 0.294900] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations932worker1 # [ 0.306682] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry933client # [ 0.276777] smpboot: CPU0: AMD EPYC 9654 96-Core Processor (family: 0x19, model: 0x11, stepping: 0x1)934worker2 # [ 0.298254] NET: Registered PF_NETLINK/PF_ROUTE protocol family935worker1 # [ 0.307691] PCI: Using configuration type 1 for base access936lb # [ 0.296142] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations937client # [ 0.278065] Performance Events: Fam17h+ core perfctr, AMD PMU driver.938lb # [ 0.296890] audit: initializing netlink subsys (disabled)939worker2 # [ 0.299017] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations940client # [ 0.278511] ... version: 2941worker1 # [ 0.308802] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.942client # [ 0.279202] ... bit width: 48943lb # [ 0.298036] thermal_sys: Registered thermal governor 'fair_share'944worker2 # [ 0.299544] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations945client # [ 0.279504] ... generic counters: 6946lb # [ 0.298038] thermal_sys: Registered thermal governor 'bang_bang'947client # [ 0.280222] ... generic bitmap: 000000000000003f948worker2 # [ 0.300797] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations949lb # [ 0.298882] thermal_sys: Registered thermal governor 'step_wise'950client # [ 0.280504] ... fixed-purpose counters: 0951worker2 # [ 0.301534] audit: initializing netlink subsys (disabled)952client # [ 0.281186] ... fixed-purpose bitmap: 0000000000000000953lb # [ 0.299819] thermal_sys: Registered thermal governor 'user_space'954client # [ 0.281503] ... value mask: 0000ffffffffffff955worker2 # [ 0.302526] audit: type=2000 audit(1790250431.613:1): state=initialized audit_enabled=0 res=1956lb # [ 0.300591] audit: type=2000 audit(1790250431.613:1): state=initialized audit_enabled=0 res=1957client # [ 0.282373] ... max period: 00007fffffffffff958worker2 # [ 0.303655] thermal_sys: Registered thermal governor 'fair_share'959lb # [ 0.301884] thermal_sys: Registered thermal governor 'power_allocator'960client # [ 0.283188] ... global_ctrl mask: 000000000000003f961lb # [ 0.301897] cpuidle: using governor menu962worker2 # [ 0.303657] thermal_sys: Registered thermal governor 'bang_bang'963worker1 # [ 0.315703] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages964client # [ 0.283612] signal: max sigframe size: 3376965worker2 # [ 0.304524] thermal_sys: Registered thermal governor 'step_wise'966client # [ 0.284380] rcu: Hierarchical SRCU implementation.967worker1 # [ 0.316680] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page968lb # [ 0.303869] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5969client # [ 0.285065] rcu: Max phase no-delay instances is 400.970worker2 # [ 0.305463] thermal_sys: Registered thermal governor 'user_space'971worker1 # [ 0.317673] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages972lb # [ 0.304854] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]973worker2 # [ 0.306244] thermal_sys: Registered thermal governor 'power_allocator'974worker2 # [ 0.306544] cpuidle: using governor menu975lb # [ 0.305884] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry976lb # [ 0.306893] PCI: Using configuration type 1 for base access977worker2 # [ 0.308736] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5978lb # [ 0.307952] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.979worker2 # [ 0.309745] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]980client # [ 0.290111] smp: Bringing up secondary CPUs ...981worker2 # [ 0.310526] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry982client # [ 0.290517] smp: Brought up 1 node, 1 CPU983worker2 # [ 0.311542] PCI: Using configuration type 1 for base access984client # [ 0.291230] smpboot: Total of 1 processors activated (4800.02 BogoMIPS)985worker1 # [ 0.324680] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page986worker2 # [ 0.312635] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.987client # [ 0.291703] Memory: 1454600K/1572312K available (17247K kernel code, 2727K rwdata, 13616K rodata, 3652K init, 2976K bss, 109308K reserved, 0K cma-reserved)988client # [ 0.292815] devtmpfs: initialized989client # [ 0.293655] x86/mm: Memory block size: 128MB990lb # [ 0.315140] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages991lb # [ 0.315882] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page992worker1 # [ 0.331042] ACPI: Added _OSI(Module Device)993client # [ 0.295542] posixtimers hash table entries: 512 (order: 1, 8192 bytes, linear)994worker1 # [ 0.331682] ACPI: Added _OSI(Processor Device)995worker1 # [ 0.332408] ACPI: Added _OSI(Processor Aggregator Device)996client # [ 0.296530] futex hash table entries: 256 (16384 bytes on 1 NUMA nodes, total 16 KiB, linear).997worker2 # [ 0.319537] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages998client # [ 0.297597] pinctrl core: initialized pinctrl subsystem999worker2 # [ 0.320523] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1000client # [ 0.298675] PM: RTC time: 11:47:10, date: 2026-09-241001lb # [ 0.320883] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1002lb # [ 0.321882] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1003client # [ 0.302252] NET: Registered PF_NETLINK/PF_ROUTE protocol family1004worker2 # [ 0.325524] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1005client # [ 0.302984] DMA: preallocated 256 KiB GFP_KERNEL pool for atomic allocations1006worker2 # [ 0.326539] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1007worker1 # [ 0.340081] ACPI: 1 ACPI AML tables successfully acquired and loaded1008client # [ 0.303528] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA pool for atomic allocations1009client # [ 0.304791] DMA: preallocated 256 KiB GFP_KERNEL|GFP_DMA32 pool for atomic allocations1010client # [ 0.305513] audit: initializing netlink subsys (disabled)1011lb # [ 0.330473] ACPI: Added _OSI(Module Device)1012worker1 # [ 0.344080] ACPI: Interpreter enabled1013client # [ 0.306726] thermal_sys: Registered thermal governor 'fair_share'1014lb # [ 0.330883] ACPI: Added _OSI(Processor Device)1015worker1 # [ 0.344694] ACPI: PM: (supports S0 S3 S4 S5)1016client # [ 0.306728] thermal_sys: Registered thermal governor 'bang_bang'1017lb # [ 0.331612] ACPI: Added _OSI(Processor Aggregator Device)1018worker1 # [ 0.345433] ACPI: Using IOAPIC for interrupt routing1019client # [ 0.307505] thermal_sys: Registered thermal governor 'step_wise'1020client # [ 0.308432] thermal_sys: Registered thermal governor 'user_space'1021client # [ 0.309290] audit: type=2000 audit(1790250431.646:1): state=initialized audit_enabled=0 res=11022worker1 # [ 0.347733] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1023client # [ 0.311506] thermal_sys: Registered thermal governor 'power_allocator'1024worker1 # [ 0.348680] PCI: Using E820 reservations for host bridge windows1025client # [ 0.311529] cpuidle: using governor menu1026worker2 # [ 0.337818] ACPI: Added _OSI(Module Device)1027worker2 # [ 0.338499] ACPI: Added _OSI(Processor Device)1028client # [ 0.313715] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.51029worker2 # [ 0.339096] ACPI: Added _OSI(Processor Aggregator Device)1030lb # [ 0.338428] ACPI: 1 ACPI AML tables successfully acquired and loaded1031worker1 # [ 0.351796] ACPI: Enabled 2 GPEs in block 00 to 3F1032client # [ 0.314748] PCI: ECAM [mem 0xb0000000-0xbfffffff] (base 0xb0000000) for domain 0000 [bus 00-ff]1033client # [ 0.315506] PCI: ECAM [mem 0xb0000000-0xbfffffff] reserved as E820 entry1034client # [ 0.316517] PCI: Using configuration type 1 for base access1035lb # [ 0.341843] ACPI: Interpreter enabled1036lb # [ 0.342325] ACPI: PM: (supports S0 S3 S4 S5)1037client # [ 0.317655] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.1038lb # [ 0.342882] ACPI: Using IOAPIC for interrupt routing1039worker2 # [ 0.345067] ACPI: 1 ACPI AML tables successfully acquired and loaded1040lb # [ 0.343714] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1041worker1 # [ 0.359975] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1042client # [ 0.322836] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages1043worker2 # [ 0.348927] ACPI: Interpreter enabled1044worker1 # [ 0.360685] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1045worker2 # [ 0.349539] ACPI: PM: (supports S0 S3 S4 S5)1046client # [ 0.323505] HugeTLB: 16380 KiB vmemmap can be freed for a 1.00 GiB page1047worker1 # [ 0.361761] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1048lb # [ 0.348882] PCI: Using E820 reservations for host bridge windows1049worker2 # [ 0.350213] ACPI: Using IOAPIC for interrupt routing1050client # [ 0.324505] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages1051lb # [ 0.349948] ACPI: Enabled 2 GPEs in block 00 to 3F1052worker1 # [ 0.362803] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1053worker1 # [ 0.364062] PCI host bridge to bus 0000:001054worker1 # [ 0.364684] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1055worker2 # [ 0.352598] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1056worker1 # [ 0.365680] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1057worker2 # [ 0.353524] PCI: Using E820 reservations for host bridge windows1058worker1 # [ 0.366679] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1059worker1 # [ 0.367680] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1060client # [ 0.331505] HugeTLB: 28 KiB vmemmap can be freed for a 2.00 MiB page1061worker2 # [ 0.356663] ACPI: Enabled 2 GPEs in block 00 to 3F1062worker1 # [ 0.368680] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1063worker1 # [ 0.369679] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1064lb # [ 0.358622] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1065worker1 # [ 0.370680] pci_bus 0000:00: root bus resource [bus 00-ff]1066lb # [ 0.359204] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1067worker1 # [ 0.371734] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1068lb # [ 0.359959] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1069worker1 # [ 0.373160] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1070lb # [ 0.361003] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1071lb # [ 0.362224] PCI host bridge to bus 0000:001072worker2 # [ 0.364111] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1073client # [ 0.339889] ACPI: Added _OSI(Module Device)1074lb # [ 0.362886] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1075client # [ 0.340506] ACPI: Added _OSI(Processor Device)1076worker2 # [ 0.364530] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1077lb # [ 0.363881] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1078client # [ 0.341238] ACPI: Added _OSI(Processor Aggregator Device)1079worker2 # [ 0.365601] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1080lb # [ 0.364882] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1081worker2 # [ 0.366646] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1082lb # [ 0.365881] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1083worker2 # [ 0.367890] PCI host bridge to bus 0000:001084lb # [ 0.366881] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1085worker1 # [ 0.375766] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1086worker2 # [ 0.368528] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1087lb # [ 0.367882] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1088worker1 # [ 0.376692] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1089worker2 # [ 0.369524] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1090lb # [ 0.368882] pci_bus 0000:00: root bus resource [bus 00-ff]1091worker1 # [ 0.377672] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1092worker2 # [ 0.370523] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1093lb # [ 0.369867] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1094worker2 # [ 0.371524] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1095worker1 # [ 0.378959] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1096lb # [ 0.371326] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1097worker2 # [ 0.372523] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1098client # [ 0.350087] ACPI: 1 ACPI AML tables successfully acquired and loaded1099worker2 # [ 0.373524] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1100worker1 # [ 0.380386] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1101worker2 # [ 0.374524] pci_bus 0000:00: root bus resource [bus 00-ff]1102worker2 # [ 0.375573] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1103worker1 # [ 0.383719] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1104client # [ 0.353773] ACPI: Interpreter enabled1105client # [ 0.354402] ACPI: PM: (supports S0 S3 S4 S5)1106worker1 # [ 0.384623] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1107worker2 # [ 0.376974] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1108client # [ 0.355031] ACPI: Using IOAPIC for interrupt routing1109worker1 # [ 0.385446] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1110lb # [ 0.373946] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1111worker1 # [ 0.385686] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1112client # [ 0.355557] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug1113lb # [ 0.374894] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1114client # [ 0.356504] PCI: Using E820 reservations for host bridge windows1115lb # [ 0.375855] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1116worker1 # [ 0.387412] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1117lb # [ 0.376803] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1118client # [ 0.359643] ACPI: Enabled 2 GPEs in block 00 to 3F1119lb # [ 0.378617] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1120worker1 # [ 0.390653] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1121worker2 # [ 0.379572] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1122worker1 # [ 0.391342] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1123worker2 # [ 0.380536] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1124worker1 # [ 0.391703] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1125worker2 # [ 0.381545] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1126lb # [ 0.381925] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1127lb # [ 0.382783] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1128worker1 # [ 0.393231] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1129worker2 # [ 0.382691] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1130lb # [ 0.383616] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1131lb # [ 0.383888] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1132worker2 # [ 0.384286] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1133client # [ 0.368328] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])1134worker1 # [ 0.396021] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1135client # [ 0.369319] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI HPX-Type3]1136lb # [ 0.385560] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1137worker2 # [ 0.386562] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1138worker1 # [ 0.397342] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1139worker2 # [ 0.387440] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1140client # [ 0.371063] acpi PNP0A08:00: _OSC: platform does not support [PCIeHotplug LTR]1141worker1 # [ 0.397702] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1142worker2 # [ 0.388272] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1143client # [ 0.371635] acpi PNP0A08:00: _OSC: OS now controls [PME AER PCIeCapability]1144lb # [ 0.388923] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1145worker2 # [ 0.388529] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1146lb # [ 0.389792] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1147worker1 # [ 0.399233] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1148client # [ 0.372848] PCI host bridge to bus 0000:001149lb # [ 0.390579] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1150client # [ 0.373509] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]1151client # [ 0.374505] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]1152worker2 # [ 0.390231] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1153worker1 # [ 0.401681] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1154lb # [ 0.391468] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1155client # [ 0.375504] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]1156worker1 # [ 0.402529] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1157client # [ 0.376504] pci_bus 0000:00: root bus resource [mem 0x60000000-0xafffffff window]1158worker2 # [ 0.392567] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1159worker1 # [ 0.403432] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1160worker2 # [ 0.393447] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1161client # [ 0.377504] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff window]1162worker1 # [ 0.404541] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1163worker2 # [ 0.394308] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1164client # [ 0.378504] pci_bus 0000:00: root bus resource [mem 0x100000000-0x8ffffffff window]1165lb # [ 0.394141] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1166lb # [ 0.395485] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1167client # [ 0.379504] pci_bus 0000:00: root bus resource [bus 00-ff]1168worker1 # [ 0.406214] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1169worker2 # [ 0.395929] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1170lb # [ 0.395903] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1171client # [ 0.380475] pci 0000:00:00.0: [8086:29c0] type 00 class 0x060000 conventional PCI endpoint1172lb # [ 0.397457] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1173worker1 # [ 0.407729] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1174client # [ 0.381955] pci 0000:00:01.0: [1234:1111] type 00 class 0x030000 conventional PCI endpoint1175worker1 # [ 0.408702] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1176worker2 # [ 0.398798] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1177lb # [ 0.399692] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1178worker1 # [ 0.410297] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1179worker2 # [ 0.400242] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1180lb # [ 0.400525] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1181worker2 # [ 0.400545] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1182lb # [ 0.400904] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1183lb # [ 0.401887] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1184worker1 # [ 0.411693] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1185worker2 # [ 0.402063] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1186worker1 # [ 0.412703] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1187lb # [ 0.403420] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1188client # [ 0.384595] pci 0000:00:01.0: BAR 0 [mem 0xfd000000-0xfdffffff pref]1189worker2 # [ 0.403531] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1190client # [ 0.385588] pci 0000:00:01.0: BAR 2 [mem 0xfebd0000-0xfebd0fff]1191worker1 # [ 0.414294] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1192worker2 # [ 0.404379] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1193client # [ 0.386568] pci 0000:00:01.0: ROM [mem 0xfebc0000-0xfebcffff pref]1194lb # [ 0.404930] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1195worker2 # [ 0.405297] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1196lb # [ 0.405881] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1197worker1 # [ 0.415723] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1198client # [ 0.387862] pci 0000:00:01.0: Video device with shadowed ROM at [mem 0x000c0000-0x000dffff]1199worker2 # [ 0.406391] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1200worker1 # [ 0.416703] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1201lb # [ 0.407454] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1202client # [ 0.390219] pci 0000:00:02.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1203worker2 # [ 0.408092] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1204worker1 # [ 0.418214] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1205lb # [ 0.408894] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1206worker2 # [ 0.409543] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1207lb # [ 0.409852] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1208client # [ 0.393556] pci 0000:00:02.0: BAR 0 [io 0xc100-0xc11f]1209worker2 # [ 0.410545] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1210worker1 # [ 0.419724] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1211client # [ 0.394366] pci 0000:00:02.0: BAR 1 [mem 0xfebd1000-0xfebd1fff]1212worker1 # [ 0.421423] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1213lb # [ 0.411517] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1214client # [ 0.395231] pci 0000:00:02.0: BAR 4 [mem 0xfe000000-0xfe003fff 64bit pref]1215worker2 # [ 0.412138] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1216client # [ 0.395539] pci 0000:00:02.0: ROM [mem 0xfeb40000-0xfeb7ffff pref]1217worker1 # [ 0.422246] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1218lb # [ 0.412894] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1219worker2 # [ 0.413536] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1220lb # [ 0.413860] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1221client # [ 0.397043] pci 0000:00:03.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1222worker1 # [ 0.424469] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1223worker2 # [ 0.414498] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1224lb # [ 0.415277] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1225worker1 # [ 0.425324] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1226worker2 # [ 0.416070] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1227worker1 # [ 0.425701] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1228client # [ 0.400545] pci 0000:00:03.0: BAR 0 [io 0xc120-0xc13f]1229lb # [ 0.416894] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1230client # [ 0.401438] pci 0000:00:03.0: BAR 1 [mem 0xfebd2000-0xfebd2fff]1231worker2 # [ 0.417536] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1232client # [ 0.402245] pci 0000:00:03.0: BAR 4 [mem 0xfe004000-0xfe007fff 64bit pref]1233worker1 # [ 0.427263] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1234lb # [ 0.418414] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1235worker2 # [ 0.418523] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1236client # [ 0.403090] pci 0000:00:04.0: [1af4:1001] type 00 class 0x010000 conventional PCI endpoint1237lb # [ 0.419431] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1238worker2 # [ 0.420067] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1239worker1 # [ 0.429415] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1240worker1 # [ 0.429686] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1241worker2 # [ 0.421538] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1242worker1 # [ 0.430701] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1243lb # [ 0.421766] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1244lb # [ 0.422534] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1245worker2 # [ 0.422545] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1246client # [ 0.405513] pci 0000:00:04.0: BAR 0 [io 0xc000-0xc07f]1247worker1 # [ 0.432250] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1248lb # [ 0.422904] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1249client # [ 0.407154] pci 0000:00:04.0: BAR 1 [mem 0xfebd3000-0xfebd3fff]1250worker2 # [ 0.424065] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1251worker1 # [ 0.433477] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1252lb # [ 0.424424] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1253client # [ 0.407527] pci 0000:00:04.0: BAR 4 [mem 0xfe008000-0xfe00bfff 64bit pref]1254worker2 # [ 0.425544] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1255client # [ 0.409107] pci 0000:00:05.0: [1af4:1000] type 00 class 0x020000 conventional PCI endpoint1256worker1 # [ 0.434584] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1257worker2 # [ 0.426386] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1258lb # [ 0.426921] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1259worker1 # [ 0.436480] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1260worker2 # [ 0.427342] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1261lb # [ 0.427807] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1262client # [ 0.411364] pci 0000:00:05.0: BAR 0 [io 0xc140-0xc15f]1263worker1 # [ 0.437514] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1264lb # [ 0.428619] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1265client # [ 0.412186] pci 0000:00:05.0: BAR 1 [mem 0xfebd4000-0xfebd4fff]1266worker2 # [ 0.428979] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1267worker1 # [ 0.439322] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1268client # [ 0.412553] pci 0000:00:05.0: BAR 4 [mem 0xfe00c000-0xfe00ffff 64bit pref]1269lb # [ 0.429485] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1270lb # [ 0.430649] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1271client # [ 0.413510] pci 0000:00:05.0: ROM [mem 0xfeb80000-0xfebbffff pref]1272worker2 # [ 0.430531] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1273worker2 # [ 0.432191] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1274lb # [ 0.431113] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1275client # [ 0.415126] pci 0000:00:06.0: [1af4:1052] type 00 class 0x090000 conventional PCI endpoint1276worker2 # [ 0.432545] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1277lb # [ 0.432626] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1278worker2 # [ 0.434081] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1279client # [ 0.416517] pci 0000:00:06.0: BAR 1 [mem 0xfebd5000-0xfebd5fff]1280lb # [ 0.433076] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1281worker2 # [ 0.435266] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1282client # [ 0.417526] pci 0000:00:06.0: BAR 4 [mem 0xfe010000-0xfe013fff 64bit pref]1283lb # [ 0.434702] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1284worker2 # [ 0.435720] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1285client # [ 0.419099] pci 0000:00:07.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1286worker2 # [ 0.437189] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1287worker1 # [ 0.439916] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1288worker2 # [ 0.437742] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1289client # [ 0.420517] pci 0000:00:07.0: BAR 1 [mem 0xfebd6000-0xfebd6fff]1290worker1 # [ 0.442456] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1291worker2 # [ 0.439149] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1292client # [ 0.422295] pci 0000:00:07.0: BAR 4 [mem 0xfe014000-0xfe017fff 64bit pref]1293worker1 # [ 0.442971] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1294client # [ 0.423890] pci 0000:00:08.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1295worker1 # [ 0.443986] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1296client # [ 0.425540] pci 0000:00:08.0: BAR 1 [mem 0xfebd7000-0xfebd7fff]1297worker1 # [ 0.444932] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1298client # [ 0.426526] pci 0000:00:08.0: BAR 4 [mem 0xfe018000-0xfe01bfff 64bit pref]1299lb # [ 0.435715] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1300worker1 # [ 0.446720] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1301client # [ 0.428091] pci 0000:00:09.0: [1af4:105a] type 00 class 0x018000 conventional PCI endpoint1302worker1 # [ 0.447605] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1303lb # [ 0.437564] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1304lb # [ 0.438182] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1305worker1 # [ 0.448783] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1306client # [ 0.429516] pci 0000:00:09.0: BAR 1 [mem 0xfebd8000-0xfebd8fff]1307lb # [ 0.439176] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1308client # [ 0.431298] pci 0000:00:09.0: BAR 4 [mem 0xfe01c000-0xfe01ffff 64bit pref]1309worker1 # [ 0.450431] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1310worker2 # [ 0.439890] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1311lb # [ 0.440127] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1312client # [ 0.432903] pci 0000:00:0a.0: [1af4:1003] type 00 class 0x078000 conventional PCI endpoint1313worker2 # [ 0.442181] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1314lb # [ 0.441909] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1315lb # [ 0.442736] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1316worker2 # [ 0.442783] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1317worker1 # [ 0.453733] ACPI: PCI: Interrupt link LNKA configured for IRQ 101318client # [ 0.435294] pci 0000:00:0a.0: BAR 0 [io 0xc080-0xc0bf]1319client # [ 0.436144] pci 0000:00:0a.0: BAR 1 [mem 0xfebd9000-0xfebd9fff]1320worker1 # [ 0.454786] ACPI: PCI: Interrupt link LNKB configured for IRQ 101321lb # [ 0.443908] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1322worker2 # [ 0.443846] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1323client # [ 0.436526] pci 0000:00:0a.0: BAR 4 [mem 0xfe020000-0xfe023fff 64bit pref]1324worker1 # [ 0.455777] ACPI: PCI: Interrupt link LNKC configured for IRQ 111325lb # [ 0.445519] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1326worker1 # [ 0.456777] ACPI: PCI: Interrupt link LNKD configured for IRQ 111327worker2 # [ 0.444901] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1328worker1 # [ 0.457779] ACPI: PCI: Interrupt link LNKE configured for IRQ 101329client # [ 0.438052] pci 0000:00:0b.0: [1af4:1005] type 00 class 0x00ff00 conventional PCI endpoint1330worker2 # [ 0.446540] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1331worker1 # [ 0.458777] ACPI: PCI: Interrupt link LNKF configured for IRQ 101332worker2 # [ 0.447438] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1333worker1 # [ 0.459780] ACPI: PCI: Interrupt link LNKG configured for IRQ 111334lb # [ 0.449412] ACPI: PCI: Interrupt link LNKA configured for IRQ 101335client # [ 0.439511] pci 0000:00:0b.0: BAR 0 [io 0xc160-0xc17f]1336worker2 # [ 0.448625] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1337worker1 # [ 0.460784] ACPI: PCI: Interrupt link LNKH configured for IRQ 111338lb # [ 0.449987] ACPI: PCI: Interrupt link LNKB configured for IRQ 101339client # [ 0.440353] pci 0000:00:0b.0: BAR 1 [mem 0xfebda000-0xfebdafff]1340worker1 # [ 0.461722] ACPI: PCI: Interrupt link GSIA configured for IRQ 161341worker2 # [ 0.450163] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1342lb # [ 0.450984] ACPI: PCI: Interrupt link LNKC configured for IRQ 111343client # [ 0.441256] pci 0000:00:0b.0: BAR 4 [mem 0xfe024000-0xfe027fff 64bit pref]1344worker1 # [ 0.462696] ACPI: PCI: Interrupt link GSIB configured for IRQ 171345lb # [ 0.451979] ACPI: PCI: Interrupt link LNKD configured for IRQ 111346worker1 # [ 0.463693] ACPI: PCI: Interrupt link GSIC configured for IRQ 181347client # [ 0.442083] pci 0000:00:1d.0: [8086:2934] type 00 class 0x0c0300 conventional PCI endpoint1348lb # [ 0.452981] ACPI: PCI: Interrupt link LNKE configured for IRQ 101349worker1 # [ 0.464692] ACPI: PCI: Interrupt link GSID configured for IRQ 191350lb # [ 0.453980] ACPI: PCI: Interrupt link LNKF configured for IRQ 101351client # [ 0.443297] pci 0000:00:1d.0: BAR 4 [io 0xc180-0xc19f]1352worker1 # [ 0.465694] ACPI: PCI: Interrupt link GSIE configured for IRQ 201353lb # [ 0.454980] ACPI: PCI: Interrupt link LNKG configured for IRQ 111354worker2 # [ 0.453649] ACPI: PCI: Interrupt link LNKA configured for IRQ 101355worker1 # [ 0.466659] ACPI: PCI: Interrupt link GSIF configured for IRQ 211356lb # [ 0.455977] ACPI: PCI: Interrupt link LNKH configured for IRQ 111357worker2 # [ 0.454633] ACPI: PCI: Interrupt link LNKB configured for IRQ 101358worker1 # [ 0.467419] ACPI: PCI: Interrupt link GSIG configured for IRQ 221359client # [ 0.444362] pci 0000:00:1d.1: [8086:2935] type 00 class 0x0c0300 conventional PCI endpoint1360lb # [ 0.456918] ACPI: PCI: Interrupt link GSIA configured for IRQ 161361worker2 # [ 0.455621] ACPI: PCI: Interrupt link LNKC configured for IRQ 111362worker1 # [ 0.467695] ACPI: PCI: Interrupt link GSIH configured for IRQ 231363worker2 # [ 0.456623] ACPI: PCI: Interrupt link LNKD configured for IRQ 111364client # [ 0.446365] pci 0000:00:1d.1: BAR 4 [io 0xc1a0-0xc1bf]1365lb # [ 0.457842] ACPI: PCI: Interrupt link GSIB configured for IRQ 171366worker2 # [ 0.457619] ACPI: PCI: Interrupt link LNKE configured for IRQ 101367worker1 # [ 0.469707] iommu: Default domain type: Translated1368lb # [ 0.458716] ACPI: PCI: Interrupt link GSIC configured for IRQ 181369client # [ 0.447357] pci 0000:00:1d.2: [8086:2936] type 00 class 0x0c0300 conventional PCI endpoint1370worker2 # [ 0.458619] ACPI: PCI: Interrupt link LNKF configured for IRQ 101371worker1 # [ 0.470523] iommu: DMA domain TLB invalidation policy: lazy mode1372lb # [ 0.459617] ACPI: PCI: Interrupt link GSID configured for IRQ 191373worker2 # [ 0.459603] ACPI: PCI: Interrupt link LNKG configured for IRQ 111374worker1 # [ 0.471685] ACPI: bus type USB registered1375client # [ 0.449219] pci 0000:00:1d.2: BAR 4 [io 0xc1c0-0xc1df]1376lb # [ 0.459896] ACPI: PCI: Interrupt link GSIE configured for IRQ 201377worker1 # [ 0.472439] usbcore: registered new interface driver usbfs1378worker2 # [ 0.460617] ACPI: PCI: Interrupt link LNKH configured for IRQ 111379lb # [ 0.460817] ACPI: PCI: Interrupt link GSIF configured for IRQ 211380worker1 # [ 0.472692] usbcore: registered new interface driver hub1381worker2 # [ 0.461559] ACPI: PCI: Interrupt link GSIA configured for IRQ 161382lb # [ 0.461598] ACPI: PCI: Interrupt link GSIG configured for IRQ 221383worker1 # [ 0.473525] usbcore: registered new device driver usb1384lb # [ 0.461893] ACPI: PCI: Interrupt link GSIH configured for IRQ 231385worker2 # [ 0.462512] ACPI: PCI: Interrupt link GSIB configured for IRQ 171386worker2 # [ 0.463496] ACPI: PCI: Interrupt link GSIC configured for IRQ 181387worker1 # [ 0.475253] NetLabel: Initializing1388lb # [ 0.463817] iommu: Default domain type: Translated1389worker1 # [ 0.475680] NetLabel: domain hash size = 1281390worker2 # [ 0.464304] ACPI: PCI: Interrupt link GSID configured for IRQ 191391lb # [ 0.464443] iommu: DMA domain TLB invalidation policy: lazy mode1392worker1 # [ 0.476425] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1393worker2 # [ 0.465266] ACPI: PCI: Interrupt link GSIE configured for IRQ 201394lb # [ 0.465153] ACPI: bus type USB registered1395worker1 # [ 0.476720] NetLabel: unlabeled traffic allowed by default1396worker2 # [ 0.465535] ACPI: PCI: Interrupt link GSIF configured for IRQ 211397worker1 # [ 0.477609] PCI: Using ACPI for IRQ routing1398lb # [ 0.465864] usbcore: registered new interface driver usbfs1399worker2 # [ 0.466539] ACPI: PCI: Interrupt link GSIG configured for IRQ 221400lb # [ 0.466539] usbcore: registered new interface driver hub1401worker2 # [ 0.467538] ACPI: PCI: Interrupt link GSIH configured for IRQ 231402lb # [ 0.466890] usbcore: registered new device driver usb1403lb # [ 0.468586] NetLabel: Initializing1404worker2 # [ 0.469540] iommu: Default domain type: Translated1405lb # [ 0.468882] NetLabel: domain hash size = 1281406worker2 # [ 0.470345] iommu: DMA domain TLB invalidation policy: lazy mode1407lb # [ 0.469594] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1408worker2 # [ 0.471559] ACPI: bus type USB registered1409lb # [ 0.469930] NetLabel: unlabeled traffic allowed by default1410client # [ 0.449758] pci 0000:00:1d.7: [8086:293a] type 00 class 0x0c0320 conventional PCI endpoint1411lb # [ 0.470800] PCI: Using ACPI for IRQ routing1412worker2 # [ 0.472287] usbcore: registered new interface driver usbfs1413worker2 # [ 0.473184] usbcore: registered new interface driver hub1414client # [ 0.452283] pci 0000:00:1d.7: BAR 0 [mem 0xfebdb000-0xfebdbfff]1415worker2 # [ 0.473535] usbcore: registered new device driver usb1416client # [ 0.453525] pci 0000:00:1f.0: [8086:2918] type 00 class 0x060100 conventional PCI endpoint1417worker2 # [ 0.475218] NetLabel: Initializing1418worker2 # [ 0.475530] NetLabel: domain hash size = 1281419client # [ 0.454790] pci 0000:00:1f.0: quirk: [io 0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO1420worker2 # [ 0.476310] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1421worker2 # [ 0.476563] NetLabel: unlabeled traffic allowed by default1422client # [ 0.455781] pci 0000:00:1f.2: [8086:2922] type 00 class 0x010601 conventional PCI endpoint1423worker2 # [ 0.477499] PCI: Using ACPI for IRQ routing1424client # [ 0.457531] pci 0000:00:1f.2: BAR 4 [io 0xc1e0-0xc1ff]1425client # [ 0.458371] pci 0000:00:1f.2: BAR 5 [mem 0xfebdc000-0xfebdcfff]1426client # [ 0.459539] pci 0000:00:1f.3: [8086:2930] type 00 class 0x0c0500 conventional PCI endpoint1427client # [ 0.461247] pci 0000:00:1f.3: BAR 4 [io 0x0700-0x073f]1428client # [ 0.465248] ACPI: PCI: Interrupt link LNKA configured for IRQ 101429client # [ 0.465608] ACPI: PCI: Interrupt link LNKB configured for IRQ 101430client # [ 0.466607] ACPI: PCI: Interrupt link LNKC configured for IRQ 111431client # [ 0.467603] ACPI: PCI: Interrupt link LNKD configured for IRQ 111432client # [ 0.468603] ACPI: PCI: Interrupt link LNKE configured for IRQ 101433client # [ 0.469600] ACPI: PCI: Interrupt link LNKF configured for IRQ 101434client # [ 0.470610] ACPI: PCI: Interrupt link LNKG configured for IRQ 111435client # [ 0.471600] ACPI: PCI: Interrupt link LNKH configured for IRQ 111436client # [ 0.472551] ACPI: PCI: Interrupt link GSIA configured for IRQ 161437client # [ 0.473520] ACPI: PCI: Interrupt link GSIB configured for IRQ 171438client # [ 0.474475] ACPI: PCI: Interrupt link GSIC configured for IRQ 181439client # [ 0.475231] ACPI: PCI: Interrupt link GSID configured for IRQ 191440client # [ 0.475517] ACPI: PCI: Interrupt link GSIE configured for IRQ 201441client # [ 0.476488] ACPI: PCI: Interrupt link GSIF configured for IRQ 211442client # [ 0.477211] ACPI: PCI: Interrupt link GSIG configured for IRQ 221443client # [ 0.477516] ACPI: PCI: Interrupt link GSIH configured for IRQ 231444client # [ 0.479449] iommu: Default domain type: Translated1445client # [ 0.480093] iommu: DMA domain TLB invalidation policy: lazy mode1446client # [ 0.480793] ACPI: bus type USB registered1447client # [ 0.481532] usbcore: registered new interface driver usbfs1448client # [ 0.482398] usbcore: registered new interface driver hub1449client # [ 0.483134] usbcore: registered new device driver usb1450worker1 # [ 0.522288] pci 0000:00:01.0: vgaarb: setting as boot VGA device1451client # [ 0.484439] NetLabel: Initializing1452worker1 # [ 0.522676] pci 0000:00:01.0: vgaarb: bridge control possible1453client # [ 0.484911] NetLabel: domain hash size = 1281454worker1 # [ 0.522676] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1455worker1 # [ 0.522685] vgaarb: loaded1456client # [ 0.485505] NetLabel: protocols = UNLABELED CIPSOv4 CALIPSO1457client # [ 0.486437] NetLabel: unlabeled traffic allowed by default1458worker1 # [ 0.523458] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01459client # [ 0.487178] PCI: Using ACPI for IRQ routing1460worker1 # [ 0.523680] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1461lb # [ 0.514877] pci 0000:00:01.0: vgaarb: setting as boot VGA device1462lb # [ 0.514877] pci 0000:00:01.0: vgaarb: bridge control possible1463lb # [ 0.514877] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1464lb # [ 0.514887] vgaarb: loaded1465lb # [ 0.515624] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01466lb # [ 0.515882] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1467worker1 # [ 0.527765] clocksource: Switched to clocksource kvm-clock1468worker2 # [ 0.521241] pci 0000:00:01.0: vgaarb: setting as boot VGA device1469worker2 # [ 0.521520] pci 0000:00:01.0: vgaarb: bridge control possible1470worker1 # [ 0.531216] VFS: Disk quotas dquot_6.6.01471lb # [ 0.518965] clocksource: Switched to clocksource kvm-clock1472worker2 # [ 0.521520] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1473worker1 # [ 0.531955] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1474worker2 # [ 0.521529] vgaarb: loaded1475worker1 # [ 0.533341] pnp: PnP ACPI init1476worker2 # [ 0.522273] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01477worker1 # [ 0.534136] ACPI: IRQ 4 override to edge(!), high(!)1478worker2 # [ 0.522523] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1479lb # [ 0.522396] VFS: Disk quotas dquot_6.6.01480worker1 # [ 0.535063] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1481lb # [ 0.523092] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1482worker1 # [ 0.536638] pnp: PnP ACPI: found 5 devices1483lb # [ 0.524385] pnp: PnP ACPI init1484lb # [ 0.525167] ACPI: IRQ 4 override to edge(!), high(!)1485worker2 # [ 0.525604] clocksource: Switched to clocksource kvm-clock1486lb # [ 0.526090] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1487lb # [ 0.527569] pnp: PnP ACPI: found 5 devices1488worker2 # [ 0.528987] VFS: Disk quotas dquot_6.6.01489worker2 # [ 0.529751] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1490worker2 # [ 0.531145] pnp: PnP ACPI init1491worker1 # [ 0.544261] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1492worker2 # [ 0.531962] ACPI: IRQ 4 override to edge(!), high(!)1493worker1 # [ 0.545744] clocksource: Switched to clocksource acpi_pm1494worker2 # [ 0.532944] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1495worker1 # [ 0.546775] NET: Registered PF_INET protocol family1496worker2 # [ 0.534519] pnp: PnP ACPI: found 5 devices1497worker1 # [ 0.547965] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1498lb # [ 0.535196] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1499lb # [ 0.536586] clocksource: Switched to clocksource acpi_pm1500lb # [ 0.537509] NET: Registered PF_INET protocol family1501lb # [ 0.538656] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1502worker2 # [ 0.542064] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1503worker2 # [ 0.543575] clocksource: Switched to clocksource acpi_pm1504worker2 # [ 0.544557] NET: Registered PF_INET protocol family1505worker2 # [ 0.545728] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1506worker1 # [ 0.564522] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1507client # [ 0.529770] pci 0000:00:01.0: vgaarb: setting as boot VGA device1508client # [ 0.530500] pci 0000:00:01.0: vgaarb: bridge control possible1509worker1 # [ 0.566132] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1510worker1 # [ 0.567483] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1511client # [ 0.530500] pci 0000:00:01.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none1512lb # [ 0.554942] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1513client # [ 0.530508] vgaarb: loaded1514worker1 # [ 0.568798] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1515client # [ 0.531271] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 01516lb # [ 0.556424] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1517worker1 # [ 0.570057] TCP: Hash tables configured (established 16384 bind 16384)1518client # [ 0.531506] hpet0: 3 comparators, 64-bit 100.000000 MHz counter1519lb # [ 0.557707] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1520worker1 # [ 0.571195] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1521lb # [ 0.559020] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1522worker1 # [ 0.572494] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1523lb # [ 0.560269] TCP: Hash tables configured (established 16384 bind 16384)1524worker1 # [ 0.573621] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1525client # [ 0.534594] clocksource: Switched to clocksource kvm-clock1526lb # [ 0.561405] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1527worker1 # [ 0.574853] NET: Registered PF_UNIX/PF_LOCAL protocol family1528lb # [ 0.562609] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1529worker2 # [ 0.562048] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1530worker1 # [ 0.575875] NET: Registered PF_XDP protocol family1531lb # [ 0.563686] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1532worker1 # [ 0.576751] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1533worker2 # [ 0.563598] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1534lb # [ 0.564874] NET: Registered PF_UNIX/PF_LOCAL protocol family1535client # [ 0.538191] VFS: Disk quotas dquot_6.6.01536worker1 # [ 0.577763] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1537worker2 # [ 0.564900] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1538lb # [ 0.565814] NET: Registered PF_XDP protocol family1539worker1 # [ 0.578761] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1540client # [ 0.538867] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)1541lb # [ 0.566593] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1542worker2 # [ 0.566179] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1543client # [ 0.540151] pnp: PnP ACPI init1544worker1 # [ 0.579813] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1545lb # [ 0.567548] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1546worker2 # [ 0.567388] TCP: Hash tables configured (established 16384 bind 16384)1547client # [ 0.540952] ACPI: IRQ 4 override to edge(!), high(!)1548worker1 # [ 0.580891] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1549lb # [ 0.568515] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1550worker2 # [ 0.568448] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1551client # [ 0.541870] system 00:04: [mem 0xb0000000-0xbfffffff window] has been reserved1552worker1 # [ 0.581953] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1553lb # [ 0.569566] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1554worker2 # [ 0.569659] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1555client # [ 0.543343] pnp: PnP ACPI: found 5 devices1556worker1 # [ 0.583654] ACPI: \_SB_.GSIA: Enabled at IRQ 161557lb # [ 0.570645] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1558worker2 # [ 0.570740] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1559lb # [ 0.571714] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1560worker2 # [ 0.571944] NET: Registered PF_UNIX/PF_LOCAL protocol family1561worker2 # [ 0.572918] NET: Registered PF_XDP protocol family1562worker1 # [ 0.586212] ACPI: \_SB_.GSIB: Enabled at IRQ 171563lb # [ 0.573406] ACPI: \_SB_.GSIA: Enabled at IRQ 161564worker2 # [ 0.573758] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1565worker1 # [ 0.588137] ACPI: \_SB_.GSIC: Enabled at IRQ 181566worker2 # [ 0.574803] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1567lb # [ 0.575890] ACPI: \_SB_.GSIB: Enabled at IRQ 171568worker2 # [ 0.575838] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1569worker1 # [ 0.589999] ACPI: \_SB_.GSID: Enabled at IRQ 191570worker2 # [ 0.576957] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1571lb # [ 0.577779] ACPI: \_SB_.GSIC: Enabled at IRQ 181572client # [ 0.550876] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns1573worker2 # [ 0.578061] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1574worker1 # [ 0.591683] PCI: CLS 0 bytes, default 641575client # [ 0.552259] clocksource: Switched to clocksource acpi_pm1576lb # [ 0.579660] ACPI: \_SB_.GSID: Enabled at IRQ 191577worker2 # [ 0.579166] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1578client # [ 0.553179] NET: Registered PF_INET protocol family1579worker1 # [ 0.592559] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1580worker2 # [ 0.580817] ACPI: \_SB_.GSIA: Enabled at IRQ 161581lb # [ 0.581245] PCI: CLS 0 bytes, default 641582worker1 # [ 0.594249] Trying to unpack rootfs image as initramfs...1583client # [ 0.554307] IP idents hash table entries: 32768 (order: 6, 262144 bytes, linear)1584lb # [ 0.582109] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1585worker2 # [ 0.583280] ACPI: \_SB_.GSIB: Enabled at IRQ 171586lb # [ 0.583867] Trying to unpack rootfs image as initramfs...1587worker2 # [ 0.585160] ACPI: \_SB_.GSIC: Enabled at IRQ 181588worker2 # [ 0.587109] ACPI: \_SB_.GSID: Enabled at IRQ 191589worker2 # [ 0.588773] PCI: CLS 0 bytes, default 641590worker2 # [ 0.589665] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1591worker2 # [ 0.591438] Trying to unpack rootfs image as initramfs...1592client # [ 0.570589] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)1593client # [ 0.572088] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)1594client # [ 0.573326] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)1595client # [ 0.574651] TCP bind hash table entries: 16384 (order: 7, 524288 bytes, linear)1596client # [ 0.575851] TCP: Hash tables configured (established 16384 bind 16384)1597client # [ 0.576928] MPTCP token hash table entries: 2048 (order: 4, 49152 bytes, linear)1598client # [ 0.578173] UDP hash table entries: 1024 (order: 4, 65536 bytes, linear)1599client # [ 0.579209] UDP-Lite hash table entries: 1024 (order: 4, 65536 bytes, linear)1600client # [ 0.580347] NET: Registered PF_UNIX/PF_LOCAL protocol family1601client # [ 0.581260] NET: Registered PF_XDP protocol family1602client # [ 0.582065] pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7 window]1603client # [ 0.583025] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff window]1604client # [ 0.583997] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff window]1605client # [ 0.585053] pci_bus 0000:00: resource 7 [mem 0x60000000-0xafffffff window]1606client # [ 0.586109] pci_bus 0000:00: resource 8 [mem 0xc0000000-0xfebfffff window]1607client # [ 0.587163] pci_bus 0000:00: resource 9 [mem 0x100000000-0x8ffffffff window]1608client # [ 0.588785] ACPI: \_SB_.GSIA: Enabled at IRQ 161609client # [ 0.591441] ACPI: \_SB_.GSIB: Enabled at IRQ 171610client # [ 0.593296] ACPI: \_SB_.GSIC: Enabled at IRQ 181611client # [ 0.595214] ACPI: \_SB_.GSID: Enabled at IRQ 191612client # [ 0.596801] PCI: CLS 0 bytes, default 641613client # [ 0.597669] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x229842dd0bd, max_idle_ns: 440795319571 ns1614client # [ 0.599315] Trying to unpack rootfs image as initramfs...1615worker1 # [ 0.639514] Initialise system trusted keyrings1616lb # [ 0.627539] Initialise system trusted keyrings1617worker1 # [ 0.643153] workingset: timestamp_bits=40 max_order=19 bucket_order=01618lb # [ 0.630695] workingset: timestamp_bits=40 max_order=19 bucket_order=01619worker2 # [ 0.636209] Initialise system trusted keyrings1620worker2 # [ 0.639312] workingset: timestamp_bits=40 max_order=19 bucket_order=01621worker1 # [ 0.662501] Key type asymmetric registered1622lb # [ 0.649857] Key type asymmetric registered1623lb # [ 0.650548] Asymmetric key parser 'x509' registered1624lb # [ 0.653684] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1625worker1 # [ 0.667125] Asymmetric key parser 'x509' registered1626worker1 # [ 0.667974] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1627lb # [ 0.656688] io scheduler mq-deadline registered1628lb # [ 0.657470] io scheduler kyber registered1629worker1 # [ 0.671251] io scheduler mq-deadline registered1630worker2 # [ 0.658509] Key type asymmetric registered1631worker1 # [ 0.672006] io scheduler kyber registered1632worker2 # [ 0.659211] Asymmetric key parser 'x509' registered1633lb # [ 0.660194] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1634lb # [ 0.661514] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1635worker1 # [ 0.674664] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1636worker2 # [ 0.662306] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1637worker1 # [ 0.675981] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1638worker2 # [ 0.665316] io scheduler mq-deadline registered1639worker2 # [ 0.666135] io scheduler kyber registered1640lb # [ 0.667449] Linux agpgart interface v0.1031641lb # [ 0.668249] ACPI: bus type drm_connector registered1642worker1 # [ 0.681897] Linux agpgart interface v0.1031643worker2 # [ 0.668871] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1644worker1 # [ 0.682720] ACPI: bus type drm_connector registered1645client # [ 0.643026] Initialise system trusted keyrings1646worker2 # [ 0.670164] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1647lb # [ 0.672775] usbcore: registered new interface driver usbserial_generic1648client # [ 0.646029] workingset: timestamp_bits=40 max_order=19 bucket_order=01649lb # [ 0.673917] usbserial: USB Serial support registered for generic1650worker1 # [ 0.687547] usbcore: registered new interface driver usbserial_generic1651worker1 # [ 0.688703] usbserial: USB Serial support registered for generic1652worker2 # [ 0.676028] Linux agpgart interface v0.1031653worker2 # [ 0.676798] ACPI: bus type drm_connector registered1654lb # [ 0.676653] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1655lb # [ 0.676653] Please enable it if your BIOS has the CPPC option.1656worker1 # [ 0.691122] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1657lb # [ 0.678993] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1658worker1 # [ 0.691122] Please enable it if your BIOS has the CPPC option.1659worker2 # [ 0.679709] usbcore: registered new interface driver usbserial_generic1660worker1 # [ 0.693444] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1661worker2 # [ 0.680807] usbserial: USB Serial support registered for generic1662worker1 # [ 0.696252] drop_monitor: Initializing network drop monitor service1663lb # [ 0.683794] drop_monitor: Initializing network drop monitor service1664worker2 # [ 0.683283] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1665worker1 # [ 0.697499] NET: Registered PF_INET6 protocol family1666worker2 # [ 0.683283] Please enable it if your BIOS has the CPPC option.1667lb # [ 0.684957] NET: Registered PF_INET6 protocol family1668worker2 # [ 0.685469] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1669lb # [ 0.688095] Segment Routing with IPv61670lb # [ 0.688843] In-situ OAM (IOAM) with IPv61671worker1 # [ 0.702577] Segment Routing with IPv61672worker2 # [ 0.689389] drop_monitor: Initializing network drop monitor service1673worker2 # [ 0.690541] NET: Registered PF_INET6 protocol family1674lb # [ 0.691939] IPI shorthand broadcast: enabled1675worker1 # [ 0.706122] In-situ OAM (IOAM) with IPv61676worker2 # [ 0.693698] Segment Routing with IPv61677worker1 # [ 0.707186] IPI shorthand broadcast: enabled1678client # [ 0.669008] Key type asymmetric registered1679worker2 # [ 0.696327] In-situ OAM (IOAM) with IPv61680client # [ 0.669751] Asymmetric key parser 'x509' registered1681worker2 # [ 0.697430] IPI shorthand broadcast: enabled1682client # [ 0.670612] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 248)1683lb # [ 0.700221] sched_clock: Marking stable (569014277, 130616052)->(773821831, -74191502)1684client # [ 0.674130] io scheduler mq-deadline registered1685client # [ 0.674894] io scheduler kyber registered1686worker1 # [ 0.715272] sched_clock: Marking stable (579015392, 136085070)->(790385278, -75284816)1687lb # [ 0.703823] registered taskstats version 11688lb # [ 0.704851] Loading compiled-in X.509 certificates1689client # [ 0.677582] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled1690worker1 # [ 0.718200] registered taskstats version 11691client # [ 0.678848] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A1692worker1 # [ 0.719195] Loading compiled-in X.509 certificates1693worker2 # [ 0.705716] sched_clock: Marking stable (571015118, 134242788)->(780131319, -74873413)1694worker2 # [ 0.710357] registered taskstats version 11695client # [ 0.684130] Linux agpgart interface v0.1031696worker2 # [ 0.711343] Loading compiled-in X.509 certificates1697client # [ 0.684864] ACPI: bus type drm_connector registered1698client # [ 0.688147] usbcore: registered new interface driver usbserial_generic1699client # [ 0.689179] usbserial: USB Serial support registered for generic1700client # [ 0.691998] amd_pstate: The CPPC feature is supported but currently disabled by the BIOS.1701client # [ 0.691998] Please enable it if your BIOS has the CPPC option.1702client # [ 0.694127] amd_pstate: the _CPC object is not present in SBIOS or ACPI disabled1703lb # [ 0.723649] Demotion targets for Node 0: null1704lb # [ 0.724569] Key type .fscrypt registered1705worker1 # [ 0.738113] Demotion targets for Node 0: null1706client # [ 0.698186] drop_monitor: Initializing network drop monitor service1707worker1 # [ 0.738996] Key type .fscrypt registered1708client # [ 0.699296] NET: Registered PF_INET6 protocol family1709lb # [ 0.727637] Key type fscrypt-provisioning registered1710worker1 # [ 0.741116] Key type fscrypt-provisioning registered1711lb # [ 0.728593] ima: No TPM chip found, activating TPM-bypass!1712client # [ 0.702470] Segment Routing with IPv61713worker1 # [ 0.742019] ima: No TPM chip found, activating TPM-bypass!1714lb # [ 0.729499] ima: Allocated hash algorithm: sha11715worker2 # [ 0.729270] Demotion targets for Node 0: null1716client # [ 0.703111] In-situ OAM (IOAM) with IPv61717worker1 # [ 0.742925] ima: Allocated hash algorithm: sha11718worker2 # [ 0.730107] Key type .fscrypt registered1719lb # [ 0.731662] ima: No architecture policies found1720client # [ 0.705277] IPI shorthand broadcast: enabled1721lb # [ 0.732658] PM: Magic number: 10:225:7831722worker2 # [ 0.733263] Key type fscrypt-provisioning registered1723worker1 # [ 0.747159] ima: No architecture policies found1724lb # [ 0.734263] RAS: Correctable Errors collector initialized.1725worker2 # [ 0.734214] ima: No TPM chip found, activating TPM-bypass!1726worker1 # [ 0.748072] PM: Magic number: 10:225:7831727worker2 # [ 0.735115] ima: Allocated hash algorithm: sha11728worker2 # [ 0.735943] ima: No architecture policies found1729worker2 # [ 0.738274] PM: Magic number: 10:225:7831730worker1 # [ 0.751708] RAS: Correctable Errors collector initialized.1731worker2 # [ 0.739858] RAS: Correctable Errors collector initialized.1732client # [ 0.713629] sched_clock: Marking stable (579015663, 133957140)->(786847606, -73874803)1733lb # [ 0.744298] clk: Disabling unused clocks1734client # [ 0.718151] registered taskstats version 11735client # [ 0.719125] Loading compiled-in X.509 certificates1736lb # [ 0.746651] PM: genpd: Disabling unused power domains1737worker1 # [ 0.760484] clk: Disabling unused clocks1738worker2 # [ 0.748905] clk: Disabling unused clocks1739worker1 # [ 0.763111] PM: genpd: Disabling unused power domains1740worker2 # [ 0.752277] PM: genpd: Disabling unused power domains1741client # [ 0.736988] Demotion targets for Node 0: null1742client # [ 0.740019] Key type .fscrypt registered1743client # [ 0.740686] Key type fscrypt-provisioning registered1744client # [ 0.741634] ima: No TPM chip found, activating TPM-bypass!1745client # [ 0.743990] ima: Allocated hash algorithm: sha11746client # [ 0.744760] ima: No architecture policies found1747client # [ 0.746986] PM: Magic number: 10:225:7831748client # [ 0.748615] RAS: Correctable Errors collector initialized.1749client # [ 0.757695] clk: Disabling unused clocks1750client # [ 0.760990] PM: genpd: Disabling unused power domains1751worker1 # [ 0.916428] Freeing initrd memory: 29056K1752lb # [ 0.905742] Freeing initrd memory: 29064K1753worker1 # [ 0.919595] Freeing unused decrypted memory: 2028K1754lb # [ 0.908893] Freeing unused decrypted memory: 2028K1755worker1 # [ 0.922297] Freeing unused kernel image (initmem) memory: 3652K1756worker1 # [ 0.923410] Write protecting the kernel read-only data: 32768k1757lb # [ 0.911499] Freeing unused kernel image (initmem) memory: 3652K1758lb # [ 0.912592] Write protecting the kernel read-only data: 32768k1759worker1 # [ 0.925331] Freeing unused kernel image (text/rodata gap) memory: 1184K1760worker1 # [ 0.926836] Freeing unused kernel image (rodata/data gap) memory: 720K1761worker2 # [ 0.914290] Freeing initrd memory: 29060K1762lb # [ 0.914433] Freeing unused kernel image (text/rodata gap) memory: 1184K1763lb # [ 0.915896] Freeing unused kernel image (rodata/data gap) memory: 720K1764worker2 # [ 0.917414] Freeing unused decrypted memory: 2028K1765worker2 # [ 0.919960] Freeing unused kernel image (initmem) memory: 3652K1766worker2 # [ 0.921179] Write protecting the kernel read-only data: 32768k1767worker2 # [ 0.923010] Freeing unused kernel image (text/rodata gap) memory: 1184K1768worker2 # [ 0.924568] Freeing unused kernel image (rodata/data gap) memory: 720K1769client # [ 0.926661] Freeing initrd memory: 29056K1770client # [ 0.929848] Freeing unused decrypted memory: 2028K1771client # [ 0.932483] Freeing unused kernel image (initmem) memory: 3652K1772client # [ 0.933589] Write protecting the kernel read-only data: 32768k1773client # [ 0.935449] Freeing unused kernel image (text/rodata gap) memory: 1184K1774client # [ 0.936914] Freeing unused kernel image (rodata/data gap) memory: 720K1775worker1 # [ 0.977882] x86/mm: Checked W+X mappings: passed, no W+X pages found.1776worker1 # [ 0.979012] Run /init as init process1777lb # [ 0.966938] x86/mm: Checked W+X mappings: passed, no W+X pages found.1778lb # [ 0.968065] Run /init as init process1779worker2 # [ 0.975505] x86/mm: Checked W+X mappings: passed, no W+X pages found.1780worker2 # [ 0.976762] Run /init as init process1781worker1 # [ 0.989680] systemd[1]: Inserted module 'autofs4'1782lb # [ 0.978590] systemd[1]: Inserted module 'autofs4'1783worker2 # [ 0.987090] systemd[1]: Inserted module 'autofs4'1784worker1 # [ 1.009690] fuse: init (API version 7.45)1785lb # [ 0.998293] fuse: init (API version 7.45)1786worker1 # [ 1.017128] ACPI: \_SB_.GSIG: Enabled at IRQ 221787lb # [ 1.005688] ACPI: \_SB_.GSIG: Enabled at IRQ 221788worker1 # [ 1.019771] ACPI: \_SB_.GSIH: Enabled at IRQ 231789worker2 # [ 1.006746] fuse: init (API version 7.45)1790lb # [ 1.009079] ACPI: \_SB_.GSIH: Enabled at IRQ 231791worker1 # [ 1.023330] ACPI: \_SB_.GSIE: Enabled at IRQ 201792lb # [ 1.012420] ACPI: \_SB_.GSIE: Enabled at IRQ 201793worker1 # [ 1.025589] ACPI: \_SB_.GSIF: Enabled at IRQ 211794worker2 # [ 1.014050] ACPI: \_SB_.GSIG: Enabled at IRQ 221795lb # [ 1.014713] ACPI: \_SB_.GSIF: Enabled at IRQ 211796client # [ 0.987660] x86/mm: Checked W+X mappings: passed, no W+X pages found.1797client # [ 0.988724] Run /init as init process1798worker2 # [ 1.016595] ACPI: \_SB_.GSIH: Enabled at IRQ 231799worker1 # [ 1.032549] virtiofs virtio5: discovered new tag: nix-store1800worker2 # [ 1.019953] ACPI: \_SB_.GSIE: Enabled at IRQ 201801lb # [ 1.020201] virtiofs virtio5: discovered new tag: nix-store1802worker1 # [ 1.034119] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1803lb # [ 1.021872] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1804worker2 # [ 1.022111] ACPI: \_SB_.GSIF: Enabled at IRQ 211805client # [ 0.999502] systemd[1]: Inserted module 'autofs4'1806worker2 # [ 1.028173] virtiofs virtio5: discovered new tag: nix-store1807worker1 # [ 1.041896] virtiofs virtio6: discovered new tag: shared1808lb # [ 1.029371] virtiofs virtio6: discovered new tag: shared1809worker2 # [ 1.029772] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1810worker1 # [ 1.043564] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1811lb # [ 1.030951] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1812worker1 # [ 1.047110] virtiofs virtio7: discovered new tag: xchg1813lb # [ 1.034362] virtiofs virtio7: discovered new tag: xchg1814worker1 # [ 1.048564] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1815lb # [ 1.035901] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1816worker2 # [ 1.036872] virtiofs virtio6: discovered new tag: shared1817worker2 # [ 1.038419] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1818worker2 # [ 1.041798] virtiofs virtio7: discovered new tag: xchg1819worker2 # [ 1.043302] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1820client # [ 1.019247] fuse: init (API version 7.45)1821client # [ 1.026633] ACPI: \_SB_.GSIG: Enabled at IRQ 221822client # [ 1.029069] ACPI: \_SB_.GSIH: Enabled at IRQ 231823lb # [ 1.057731] systemd[1]: Successfully made /usr/ read-only.1824worker1 # [ 1.070846] systemd[1]: Successfully made /usr/ read-only.1825client # [ 1.032628] ACPI: \_SB_.GSIE: Enabled at IRQ 201826client # [ 1.035104] ACPI: \_SB_.GSIF: Enabled at IRQ 211827worker2 # [ 1.062958] systemd[1]: Successfully made /usr/ read-only.1828client # [ 1.041001] virtiofs virtio5: discovered new tag: nix-store1829client # [ 1.042769] virtiofs virtio5: virtio_fs_setup_dax: No cache capability1830client # [ 1.049817] virtiofs virtio6: discovered new tag: shared1831client # [ 1.051425] virtiofs virtio6: virtio_fs_setup_dax: No cache capability1832client # [ 1.055019] virtiofs virtio7: discovered new tag: xchg1833client # [ 1.056559] virtiofs virtio7: virtio_fs_setup_dax: No cache capability1834client # [ 1.078053] systemd[1]: Successfully made /usr/ read-only.1835worker1 # [ 1.406328] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1836worker1 # [ 1.411523] systemd[1]: Detected virtualization kvm.1837worker1 # [ 1.412406] systemd[1]: Detected architecture x86-64.1838worker1 # [ 1.413278] systemd[1]: Running in initrd.1839worker1 # [ 1.414318] systemd[1]: Initializing machine ID from random generator.1840worker2 # [ 1.398149] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1841worker1 # [ 1.415509] systemd[1]: Hostname set to <worker1>.1842worker2 # [ 1.403261] systemd[1]: Detected virtualization kvm.1843worker2 # [ 1.404101] systemd[1]: Detected architecture x86-64.1844worker2 # [ 1.404996] systemd[1]: Running in initrd.1845lb # [ 1.394500] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1846worker2 # [ 1.406087] systemd[1]: Initializing machine ID from random generator.1847worker2 # [ 1.407345] systemd[1]: Hostname set to <worker2>.1848lb # [ 1.406594] systemd[1]: Detected virtualization kvm.1849lb # [ 1.408800] systemd[1]: Detected architecture x86-64.1850lb # [ 1.410949] systemd[1]: Running in initrd.1851lb # [ 1.413461] systemd[1]: Initializing machine ID from random generator.1852lb # [ 1.416367] systemd[1]: Hostname set to <lb>.1853client # [ 1.414714] systemd[1]: systemd 261.2 running in system mode (+PAM +AUDIT -SELINUX +APPARMOR +IMA +IPE +SMACK +SECCOMP +GCRYPT -GNUTLS +OPENSSL +ACL +BLKID +CURL +ELFUTILS +FIDO2 +IDN2 +KMOD +LIBCRYPTSETUP +LIBCRYPTSETUP_PLUGINS +LIBFDISK +PCRE2 +PWQUALITY +P11KIT +QRENCODE +TPM2 +BZIP2 +LZ4 +XZ +ZLIB +ZSTD +BPF_FRAMEWORK -BTF -XKBCOMMON +UTMP +LIBARCHIVE)1854client # [ 1.426853] systemd[1]: Detected virtualization kvm.1855client # [ 1.429079] systemd[1]: Detected architecture x86-64.1856client # [ 1.431178] systemd[1]: Running in initrd.1857client # [ 1.433668] systemd[1]: Initializing machine ID from random generator.1858client # [ 1.436553] systemd[1]: Hostname set to <client>.1859worker1 # [ 1.605271] systemd[1]: bpf-restrict-fs: LSM BPF program attached1860worker2 # [ 1.594462] systemd[1]: bpf-restrict-fs: LSM BPF program attached1861worker1 # [ 1.642501] systemd[1]: Queued start job for default target Initrd Default Target.1862worker2 # [ 1.631717] systemd[1]: Queued start job for default target Initrd Default Target.1863worker1 # [ 1.646353] systemd[1]: Created slice Slice /system/modprobe.1864worker1 # [ 1.647552] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1865worker2 # [ 1.635524] systemd[1]: Created slice Slice /system/modprobe.1866worker1 # [ 1.648993] systemd[1]: Expecting device /dev/disk/by-label/nixos...1867worker1 # [ 1.650120] systemd[1]: Reached target Path Units.1868worker2 # [ 1.636698] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1869worker1 # [ 1.650956] systemd[1]: Reached target Slice Units.1870worker2 # [ 1.638100] systemd[1]: Expecting device /dev/disk/by-label/nixos...1871lb # [ 1.637132] systemd[1]: bpf-restrict-fs: LSM BPF program attached1872worker1 # [ 1.651853] systemd[1]: Reached target Swaps.1873worker2 # [ 1.639180] systemd[1]: Reached target Path Units.1874worker1 # [ 1.652629] systemd[1]: Reached target Timer Units.1875worker2 # [ 1.640032] systemd[1]: Reached target Slice Units.1876worker1 # [ 1.653607] systemd[1]: Listening on D-Bus System Message Bus Socket.1877worker2 # [ 1.640926] systemd[1]: Reached target Swaps.1878worker2 # [ 1.641746] systemd[1]: Reached target Timer Units.1879worker1 # [ 1.654822] systemd[1]: Listening on Journal Socket (/dev/log).1880worker1 # [ 1.656001] systemd[1]: Listening on Journal Sockets.1881worker2 # [ 1.642711] systemd[1]: Listening on D-Bus System Message Bus Socket.1882worker1 # [ 1.657011] systemd[1]: Listening on udev Control Socket.1883worker2 # [ 1.643944] systemd[1]: Listening on Journal Socket (/dev/log).1884worker1 # [ 1.658029] systemd[1]: Listening on udev Kernel Socket.1885worker2 # [ 1.645161] systemd[1]: Listening on Journal Sockets.1886worker1 # [ 1.658961] systemd[1]: Reached target Socket Units.1887worker2 # [ 1.646155] systemd[1]: Listening on udev Control Socket.1888worker2 # [ 1.647198] systemd[1]: Listening on udev Kernel Socket.1889worker1 # [ 1.660703] systemd[1]: Starting Create List of Static Device Nodes...1890worker2 # [ 1.648177] systemd[1]: Reached target Socket Units.1891worker2 # [ 1.649931] systemd[1]: Starting Create List of Static Device Nodes...1892worker1 # [ 1.665178] systemd[1]: Starting Load Kernel Module configfs...1893worker2 # [ 1.654325] systemd[1]: Starting Load Kernel Module configfs...1894worker1 # [ 1.675222] systemd[1]: Starting Journal Service...1895worker2 # [ 1.663810] systemd[1]: Starting Journal Service...1896worker1 # [ 1.698155] systemd[1]: Starting Load Kernel Modules...1897client # [ 1.658508] systemd[1]: bpf-restrict-fs: LSM BPF program attached1898worker2 # [ 1.687338] systemd[1]: Starting Load Kernel Modules...1899worker1 # [ 1.702275] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1900lb # [ 1.691337] systemd[1]: Queued start job for default target Initrd Default Target.1901worker2 # [ 1.691407] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1902lb # [ 1.696020] systemd[1]: Created slice Slice /system/modprobe.1903worker1 # [ 1.710296] systemd[1]: Starting Coldplug All udev Devices...1904lb # [ 1.697338] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1905lb # [ 1.698783] systemd[1]: Expecting device /dev/disk/by-label/nixos...1906lb # [ 1.699954] systemd[1]: Reached target Path Units.1907lb # [ 1.700917] systemd[1]: Reached target Slice Units.1908worker2 # [ 1.700587] systemd[1]: Starting Coldplug All udev Devices...1909lb # [ 1.701814] systemd[1]: Reached target Swaps.1910lb # [ 1.702634] systemd[1]: Reached target Timer Units.1911lb # [ 1.703647] systemd[1]: Listening on D-Bus System Message Bus Socket.1912worker1 # [ 1.716759] systemd-journald[66]: Collecting audit messages is disabled.1913lb # [ 1.704908] systemd[1]: Listening on Journal Socket (/dev/log).1914lb # [ 1.706121] systemd[1]: Listening on Journal Sockets.1915lb # [ 1.707190] systemd[1]: Listening on udev Control Socket.1916lb # [ 1.708316] systemd[1]: Listening on udev Kernel Socket.1917lb # [ 1.709318] systemd[1]: Reached target Socket Units.1918worker2 # [ 1.708838] systemd-journald[66]: Collecting audit messages is disabled.1919lb # [ 1.711271] systemd[1]: Starting Create List of Static Device Nodes...1920worker1 # [ 1.724226] systemd[1]: Finished Create List of Static Device Nodes.1921worker2 # [ 1.712675] systemd[1]: Finished Create List of Static Device Nodes.1922worker1 # [ 1.728736] systemd[1]: modprobe@configfs.service: Deactivated successfully.1923lb # [ 1.716727] systemd[1]: Starting Load Kernel Module configfs...1924worker2 # [ 1.720854] systemd[1]: modprobe@configfs.service: Deactivated successfully.1925worker1 # [ 1.736519] systemd[1]: Finished Load Kernel Module configfs.1926worker2 # [ 1.726642] systemd[1]: Finished Load Kernel Module configfs.1927worker1 # [ 1.740828] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1928worker2 # [ 1.733659] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1929lb # [ 1.725724] systemd[1]: Starting Journal Service...1930client # [ 1.712633] systemd[1]: Queued start job for default target Initrd Default Target.1931worker1 # [ 1.754173] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1932client # [ 1.717335] systemd[1]: Created slice Slice /system/modprobe.1933client # [ 1.718533] systemd[1]: Started Dispatch Password Requests to Console Directory Watch.1934worker2 # [ 1.745346] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1935client # [ 1.720035] systemd[1]: Expecting device /dev/disk/by-label/nixos...1936client # [ 1.721173] systemd[1]: Reached target Path Units.1937client # [ 1.722090] systemd[1]: Reached target Slice Units.1938client # [ 1.723005] systemd[1]: Reached target Swaps.1939worker2 # [ 1.749106] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1940worker1 # [ 1.762485] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1941client # [ 1.723787] systemd[1]: Reached target Timer Units.1942lb # [ 1.750873] systemd[1]: Starting Load Kernel Modules...1943client # [ 1.724758] systemd[1]: Listening on D-Bus System Message Bus Socket.1944client # [ 1.726043] systemd[1]: Listening on Journal Socket (/dev/log).1945client # [ 1.727265] systemd[1]: Listening on Journal Sockets.1946client # [ 1.728267] systemd[1]: Listening on udev Control Socket.1947lb # [ 1.754736] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1948client # [ 1.729240] systemd[1]: Listening on udev Kernel Socket.1949client # [ 1.730205] systemd[1]: Reached target Socket Units.1950worker1 # [ 1.771292] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1951client # [ 1.732038] systemd[1]: Starting Create List of Static Device Nodes...1952lb # [ 1.761818] systemd[1]: Starting Coldplug All udev Devices...1953worker2 # [ 1.761437] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1954client # [ 1.735834] systemd[1]: Starting Load Kernel Module configfs...1955lb # [ 1.775158] systemd-journald[66]: Collecting audit messages is disabled.1956worker1 # [ 1.790648] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1957lb # [ 1.778692] systemd[1]: Finished Create List of Static Device Nodes.1958lb # [ 1.780521] systemd[1]: modprobe@configfs.service: Deactivated successfully.1959client # [ 1.745168] systemd[1]: Starting Journal Service...1960worker2 # [ 1.785826] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1961worker1 # [ 1.799306] systemd[1]: Starting Create Static Device Nodes in /dev...1962lb # [ 1.789048] systemd[1]: Finished Load Kernel Module configfs.1963worker2 # [ 1.795406] systemd[1]: Starting Create Static Device Nodes in /dev...1964lb # [ 1.795133] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1965client # [ 1.772159] systemd[1]: Starting Load Kernel Modules...1966worker1 # [ 1.812193] systemd[1]: Finished Load Kernel Modules.1967client # [ 1.776077] systemd[1]: Early TPM SRK Setup skipped, unmet condition check ConditionSecurity=measured-os1968worker2 # [ 1.804788] systemd[1]: Finished Load Kernel Modules.1969worker1 # [ 1.819490] systemd[1]: Starting Apply Kernel Variables...1970lb # [ 1.806757] systemd[1]: Starting Create Static Device Nodes in /dev gracefully...1971client # [ 1.785121] systemd[1]: Starting Coldplug All udev Devices...1972worker2 # [ 1.813446] systemd[1]: Starting Apply Kernel Variables...1973lb # [ 1.813414] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.1974client # [ 1.797887] systemd-journald[66]: Collecting audit messages is disabled.1975worker1 # [ 1.838214] systemd[1]: Finished Create Static Device Nodes in /dev.1976client # [ 1.799160] systemd[1]: Finished Create List of Static Device Nodes.1977lb # [ 1.826845] device-mapper: ioctl: 4.50.0-ioctl (2025-04-28) initialised: dm-devel@lists.linux.dev1978worker1 # [ 1.704556] systemd-modules-load[67]: Inserted module 'dm_mod'1979worker1 # [ 1.707288] systemd-modules-load[67]: Inserted module 'virtio_balloon'1980worker1 # [ 1.708300] systemd-modules-load[67]: Inserted module 'virtio_gpu'1981worker2 # [ 1.831388] systemd[1]: Started Journal Service.1982client # [ 1.805697] systemd[1]: modprobe@configfs.service: Deactivated successfully.1983worker2 # [ 1.698475] systemd-modules-load[67]: Inserted module 'dm_mod'1984worker1 # [ 1.847008] systemd[1]: Started Journal Service.1985worker2 # [ 1.700412] systemd-modules-load[67]: Inserted module 'virtio_balloon'1986client # [ 1.811827] systemd[1]: Finished Load Kernel Module configfs.1987worker2 # [ 1.705314] systemd-modules-load[67]: Inserted module 'virtio_gpu'1988worker2 # [ 1.706322] systemd[1]: Finished Create Static Device Nodes in /dev.1989worker2 # [ 1.707380] systemd[1]: Reached target Preparation for Local File Systems.1990worker2 # [ 1.709486] systemd[1]: Reached target Local File Systems.1991worker1 # [ 1.723111] systemd[1]: Finished Apply Kernel Variables.1992client # [ 1.818431] systemd[1]: Kernel Configuration File System skipped, unmet condition check ConditionPathExists=/sys/kernel/config1993worker1 # [ 1.723950] systemd[1]: Reached target Preparation for Local File Systems.1994worker1 # [ 1.724968] systemd[1]: Reached target Local File Systems.1995worker2 # [ 1.714070] systemd[1]: Starting Create System Files and Directories...1996lb # [ 1.848288] systemd[1]: Finished Create Static Device Nodes in /dev gracefully.1997worker1 # [ 1.728067] systemd[1]: Starting Create System Files and Directories...1998worker2 # [ 1.720142] systemd[1]: Starting Rule-based Manager for Device Events and Files...1999lb # [ 1.857827] systemd[1]: Starting Create Static Device Nodes in /dev...2000worker1 # [ 1.735070] systemd[1]: Starting Rule-based Manager for Device Events and Files...
loading 2,587 hidden lines…
4588worker2 # [ 49.806465] nix-grpc-daemon[870]: time=2026-09-24T11:48:00.044Z level=INFO msg="Upload complete. (189ms)"4589lb # [ 49.814772] niks3-server[974]: 2026/09/24 11:48:00 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.ls4590lb # [ 49.817942] niks3-server[974]: 2026/09/24 11:48:00 INFO Registered completed upload object_key=v688vnrkfcqiwaygym5ifmnr3yawh4xd.narinfo4591worker2 # [ 49.811924] nix-grpc-daemon[799]: ts=2026-09-24T11:48:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=1 drv=y2wm8aizyyg449ia2pcscrjc2pklsrz4-farm-top.drv assign_id=4 outputs=14592worker2 # [ 49.826164] nix-daemon[857]: reaped child process 991, status = succeeded4593client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b >&2, in 1.71 seconds)4594client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath4595client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag t1 --argstr top t1b outPath, in 0.12 seconds)4596lb: must succeed: curl -sf localhost:9901/clusters4597lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)4598lb: must succeed: curl -sf localhost:9901/clusters4599lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)4600worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true4601worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep -c 'event=want ' || true, in 0.04 seconds)4602(finished: subtest: inputs already in the cache are not sent to the scheduler, in 2.03 seconds)4603subtest: outputs read back through any worker carry the cache signature4604client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&24605client # don't know how to build these paths:4606client # /nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4607worker1 # [ 50.222237] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565464608worker1 # [ 50.227661] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565464609worker1 # [ 50.245646] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=debug event=want client=4 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv cached=04610worker1 # [ 50.251619] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=debug event=assigned drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv worker=192.168.1.3:50051 clients=14611worker1 # [ 50.256244] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464612worker1 # [ 50.267746] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565464613client # copying 1 paths to 'lb:50051'...4614client # copying path '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' to 'lb:50051'...4615worker1 # [ 50.279300] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565464616worker1 # [ 50.290845] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4617worker1 # [ 50.310873] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=info event=niks3_push_started pid=9174618worker1 # [ 50.437851] nix-grpc-daemon[917]: time=2026-09-24T11:48:00.729Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""4619worker1 # [ 50.530929] nix-daemon[867]: accepted connection from pid 930, user nix-grpc-daemon (trusted)4620worker1 # [ 50.544378] nix-daemon[867]: reaped child process 933, status = succeeded4621lb # [ 50.545132] niks3-server[974]: 2026/09/24 11:48:00 INFO Received push request method=POST path=/api/pushes4622worker1 # [ 50.560447] nix-grpc-daemon[917]: time=2026-09-24T11:48:00.852Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4623worker1 # [ 50.561943] nix-grpc-daemon[917]: time=2026-09-24T11:48:00.853Z level=INFO msg="Uploading lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv (504B)"4624lb # [ 50.584656] niks3-server[974]: 2026/09/24 11:48:01 INFO Received sign narinfos request method=POST path=/api/pushes/9/sign4625worker1 # [ 50.598813] nix-grpc-daemon[917]: time=2026-09-24T11:48:00.890Z level=INFO msg="Uploading 1 narinfos"4626lb # [ 50.591403] niks3-server[974]: 2026/09/24 11:48:01 INFO Registered completed upload object_key=nar/0bddczcnpndyzwmafx9yqfqrkrp1jg9rr5hsixz6rpy99kg84g1g.nar.zst4627lb # [ 50.595738] niks3-server[974]: 2026/09/24 11:48:01 INFO Signed narinfos id=9 count=14628lb # [ 50.599962] niks3-server[974]: 2026/09/24 11:48:01 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.ls4629lb # [ 50.611132] niks3-server[974]: 2026/09/24 11:48:01 INFO Received complete push request method=POST path=/api/pushes/9/complete4630worker1 # [ 50.626389] nix-grpc-daemon[917]: time=2026-09-24T11:48:00.918Z level=INFO msg="Upload complete. (183ms)"4631lb # [ 50.620202] niks3-server[974]: 2026/09/24 11:48:01 INFO Registered completed upload object_key=lqi1rvcqjj8q9l279yzgrgaghp394zs5.narinfo4632worker1 # [ 50.628605] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_in=5044633worker1 # [ 50.633201] nix-grpc-daemon[524]: ts=2026-09-24T11:48:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464634worker1 # [ 50.639544] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4635worker1 # [ 50.640819] nix-daemon[867]: reaped child process 915, status = succeeded4636worker1 # [ 50.645869] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4637client # building '/nix/store/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv' on lb:50051...4638client # farm-a> node-a: building lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4639client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds4640client # farm-a> LOG-farm-a-sig4641worker1 # [ 51.259394] nix-daemon[867]: reaped child process 937, status = succeeded4642worker1 # [ 51.334401] nix-daemon[867]: accepted connection from pid 953, user nix-grpc-daemon (trusted)4643worker1 # [ 51.347730] nix-daemon[867]: reaped child process 956, status = succeeded4644lb # [ 51.346491] niks3-server[974]: 2026/09/24 11:48:01 INFO Received push request method=POST path=/api/pushes4645worker1 # [ 51.362196] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.654Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4646worker1 # [ 51.363667] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.655Z level=INFO msg="Uploading qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a (128B)"4647lb # [ 51.379891] niks3-server[974]: 2026/09/24 11:48:01 INFO Registered completed upload object_key=log/lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv4648lb # [ 51.395496] niks3-server[974]: 2026/09/24 11:48:01 INFO Received sign narinfos request method=POST path=/api/pushes/10/sign4649lb # [ 51.399907] niks3-server[974]: 2026/09/24 11:48:01 INFO Signed narinfos id=10 count=14650worker1 # [ 51.411171] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.702Z level=INFO msg="Uploading 1 narinfos"4651lb # [ 51.407925] niks3-server[974]: 2026/09/24 11:48:01 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.ls4652lb # [ 51.413697] niks3-server[974]: 2026/09/24 11:48:01 INFO Registered completed upload object_key=nar/05mvm4xykg9y2d7fm3g502ld2ajkd821jwng7v5g8vd73ynsbnah.nar.zst4653lb # [ 51.423179] niks3-server[974]: 2026/09/24 11:48:01 INFO Received complete push request method=POST path=/api/pushes/10/complete4654worker1 # [ 51.435720] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.727Z level=INFO msg="Upload complete. (183ms)"4655lb # [ 51.431234] niks3-server[974]: 2026/09/24 11:48:01 INFO Registered completed upload object_key=qi07z82gylcaha6pl0vfp8s3j6f2gl91.narinfo4656worker1 # [ 51.441422] nix-grpc-daemon[524]: ts=2026-09-24T11:48:01Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 drv=lqi1rvcqjj8q9l279yzgrgaghp394zs5-farm-a.drv assign_id=5 outputs=14657worker1 # [ 51.450973] nix-grpc-daemon[524]: ts=2026-09-24T11:48:01Z level=debug event=want client=4 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv cached=04658worker1 # [ 51.455449] nix-grpc-daemon[524]: ts=2026-09-24T11:48:01Z level=debug event=assigned drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv worker=192.168.1.3:50051 clients=14659worker1 # [ 51.458828] nix-grpc-daemon[524]: ts=2026-09-24T11:48:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464660worker1 # [ 51.467048] nix-daemon[867]: reaped child process 935, status = succeeded4661worker1 # [ 51.468656] nix-grpc-daemon[524]: ts=2026-09-24T11:48:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565464662worker1 # [ 51.474346] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.766Z level=INFO msg="All 1 paths already cached"4663client # copying 1 paths to 'lb:50051'...4664worker1 # [ 51.479655] nix-grpc-daemon[524]: ts=2026-09-24T11:48:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565464665client # copying path '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' to 'lb:50051'...4666worker1 # [ 51.485576] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4667worker1 # [ 51.570037] nix-daemon[867]: accepted connection from pid 962, user nix-grpc-daemon (trusted)4668worker1 # [ 51.583817] nix-daemon[867]: reaped child process 965, status = succeeded4669lb # [ 51.584106] niks3-server[974]: 2026/09/24 11:48:02 INFO Received push request method=POST path=/api/pushes4670worker1 # [ 51.609892] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.901Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4671worker1 # [ 51.611757] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.903Z level=INFO msg="Uploading 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv (624B)"4672lb # [ 51.623790] niks3-server[974]: 2026/09/24 11:48:02 INFO Received sign narinfos request method=POST path=/api/pushes/11/sign4673worker1 # [ 51.636678] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.928Z level=INFO msg="Uploading 1 narinfos"4674lb # [ 51.629140] niks3-server[974]: 2026/09/24 11:48:02 INFO Signed narinfos id=11 count=14675lb # [ 51.636761] niks3-server[974]: 2026/09/24 11:48:02 INFO Registered completed upload object_key=nar/1m1mvw8bmb91glp26qj699lxqv98q2iz8wm4ik3miz0xj6vi3mw6.nar.zst4676lb # [ 51.645882] niks3-server[974]: 2026/09/24 11:48:02 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.ls4677lb # [ 51.652417] niks3-server[974]: 2026/09/24 11:48:02 INFO Received complete push request method=POST path=/api/pushes/11/complete4678lb # [ 51.661313] niks3-server[974]: 2026/09/24 11:48:02 INFO Registered completed upload object_key=08wvq73c5i2kpl6g578glyb1rcqhv2iw.narinfo4679worker1 # [ 51.671377] nix-grpc-daemon[917]: time=2026-09-24T11:48:01.963Z level=INFO msg="Upload complete. (180ms)"4680worker1 # [ 51.675944] nix-grpc-daemon[524]: ts=2026-09-24T11:48:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_in=6244681worker1 # [ 51.682314] nix-grpc-daemon[524]: ts=2026-09-24T11:48:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464682worker1 # [ 51.691250] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4683worker1 # [ 51.700340] nix-daemon[867]: reaped child process 960, status = succeeded4684worker1 # [ 51.706084] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4685client # building '/nix/store/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv' on lb:50051...4686client # farm-b> node-a: building 08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4687client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds4688client # farm-b> LOG-farm-b-sig4689worker1 # [ 52.519518] nix-daemon[867]: reaped child process 969, status = succeeded4690worker1 # [ 52.594888] nix-daemon[867]: accepted connection from pid 985, user nix-grpc-daemon (trusted)4691worker1 # [ 52.608692] nix-daemon[867]: reaped child process 988, status = succeeded4692lb # [ 52.607373] niks3-server[974]: 2026/09/24 11:48:03 INFO Received push request method=POST path=/api/pushes4693worker1 # [ 52.622716] nix-grpc-daemon[917]: time=2026-09-24T11:48:02.914Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"4694worker1 # [ 52.624222] nix-grpc-daemon[917]: time=2026-09-24T11:48:02.916Z level=INFO msg="Uploading 2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b (176B)"4695lb # [ 52.647865] niks3-server[974]: 2026/09/24 11:48:03 INFO Received sign narinfos request method=POST path=/api/pushes/12/sign4696lb # [ 52.656436] niks3-server[974]: 2026/09/24 11:48:03 INFO Signed narinfos id=12 count=14697worker1 # [ 52.666863] nix-grpc-daemon[917]: time=2026-09-24T11:48:02.958Z level=INFO msg="Uploading 1 narinfos"4698lb # [ 52.664596] niks3-server[974]: 2026/09/24 11:48:03 INFO Registered completed upload object_key=nar/0y3xlf15z3z540fgrc9sgnn2rl0aj3w6x9mjqd50hhqgmaxbiivd.nar.zst4699lb # [ 52.672058] niks3-server[974]: 2026/09/24 11:48:03 INFO Registered completed upload object_key=log/08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv4700lb # [ 52.676155] niks3-server[974]: 2026/09/24 11:48:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.ls4701lb # [ 52.684961] niks3-server[974]: 2026/09/24 11:48:03 INFO Received complete push request method=POST path=/api/pushes/12/complete4702lb # [ 52.694106] niks3-server[974]: 2026/09/24 11:48:03 INFO Registered completed upload object_key=2f96mrg6ljwjqld1y3yaibmb07jkz3by.narinfo4703worker1 # [ 52.700764] nix-grpc-daemon[917]: time=2026-09-24T11:48:02.992Z level=INFO msg="Upload complete. (189ms)"4704worker1 # [ 52.710800] nix-grpc-daemon[524]: ts=2026-09-24T11:48:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=1 drv=08wvq73c5i2kpl6g578glyb1rcqhv2iw-farm-b.drv assign_id=6 outputs=14705worker1 # [ 52.719885] nix-grpc-daemon[524]: ts=2026-09-24T11:48:03Z level=debug event=want client=4 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv cached=04706worker1 # [ 52.725950] nix-grpc-daemon[524]: ts=2026-09-24T11:48:03Z level=debug event=assigned drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv worker=192.168.1.3:50051 clients=14707worker1 # [ 52.731416] nix-grpc-daemon[524]: ts=2026-09-24T11:48:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464708worker1 # [ 52.740556] nix-grpc-daemon[524]: ts=2026-09-24T11:48:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565464709worker1 # [ 52.746349] nix-daemon[867]: reaped child process 967, status = succeeded4710worker1 # [ 52.751590] nix-grpc-daemon[917]: time=2026-09-24T11:48:03.043Z level=INFO msg="All 2 paths already cached"4711client # copying 1 paths to 'lb:50051'...4712worker1 # [ 52.757229] nix-grpc-daemon[524]: ts=2026-09-24T11:48:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565464713client # copying path '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' to 'lb:50051'...4714worker1 # [ 52.762399] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4715worker1 # [ 52.847295] nix-daemon[867]: accepted connection from pid 994, user nix-grpc-daemon (trusted)4716worker1 # [ 52.861599] nix-daemon[867]: reaped child process 997, status = succeeded4717lb # [ 52.860320] niks3-server[974]: 2026/09/24 11:48:03 INFO Received push request method=POST path=/api/pushes4718worker1 # [ 52.876325] nix-grpc-daemon[917]: time=2026-09-24T11:48:03.168Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4719worker1 # [ 52.878046] nix-grpc-daemon[917]: time=2026-09-24T11:48:03.169Z level=INFO msg="Uploading cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv (752B)"4720lb # [ 52.891719] niks3-server[974]: 2026/09/24 11:48:03 INFO Received sign narinfos request method=POST path=/api/pushes/13/sign4721lb # [ 52.899863] niks3-server[974]: 2026/09/24 11:48:03 INFO Signed narinfos id=13 count=14722worker1 # [ 52.910989] nix-grpc-daemon[917]: time=2026-09-24T11:48:03.202Z level=INFO msg="Uploading 1 narinfos"4723lb # [ 52.907669] niks3-server[974]: 2026/09/24 11:48:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.ls4724lb # [ 52.913843] niks3-server[974]: 2026/09/24 11:48:03 INFO Registered completed upload object_key=nar/0w21abkvlzg18nqkl2x4zxs7d0376kbld1n0fkr68vcpqpq1g5zg.nar.zst4725lb # [ 52.928299] niks3-server[974]: 2026/09/24 11:48:03 INFO Received complete push request method=POST path=/api/pushes/13/complete4726lb # [ 52.933558] niks3-server[974]: 2026/09/24 11:48:03 INFO Registered completed upload object_key=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx.narinfo4727worker1 # [ 52.944493] nix-grpc-daemon[917]: time=2026-09-24T11:48:03.236Z level=INFO msg="Upload complete. (176ms)"4728worker1 # [ 52.948600] nix-grpc-daemon[524]: ts=2026-09-24T11:48:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_in=7524729worker1 # [ 52.955493] nix-grpc-daemon[524]: ts=2026-09-24T11:48:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464730worker1 # [ 52.964530] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4731worker1 # [ 52.975668] nix-daemon[867]: reaped child process 992, status = succeeded4732worker1 # [ 52.979946] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4733client # building '/nix/store/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv' on lb:50051...4734client # farm-top> node-a: building cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4735client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds4736client # farm-top> LOG-farm-top-sig4737worker1 # [ 54.416413] nix-daemon[867]: reaped child process 1001, status = succeeded4738worker1 # [ 54.495748] nix-daemon[867]: accepted connection from pid 1017, user nix-grpc-daemon (trusted)4739worker1 # [ 54.508633] nix-daemon[867]: reaped child process 1020, status = succeeded4740lb # [ 54.509838] niks3-server[974]: 2026/09/24 11:48:04 INFO Received push request method=POST path=/api/pushes4741worker1 # [ 54.526279] nix-grpc-daemon[917]: time=2026-09-24T11:48:04.817Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"4742worker1 # [ 54.528152] nix-grpc-daemon[917]: time=2026-09-24T11:48:04.819Z level=INFO msg="Uploading dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top (232B)"4743lb # [ 54.553063] niks3-server[974]: 2026/09/24 11:48:04 INFO Received sign narinfos request method=POST path=/api/pushes/14/sign4744lb # [ 54.558424] niks3-server[974]: 2026/09/24 11:48:05 INFO Signed narinfos id=14 count=14745worker1 # [ 54.571311] nix-grpc-daemon[917]: time=2026-09-24T11:48:04.863Z level=INFO msg="Uploading 1 narinfos"4746lb # [ 54.569669] niks3-server[974]: 2026/09/24 11:48:05 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.ls4747lb # [ 54.573935] niks3-server[974]: 2026/09/24 11:48:05 INFO Registered completed upload object_key=nar/0h4zvhqjxsvdqli2l6i5ajh5sh40hz57gzchdirnvbisq2aks0k8.nar.zst4748lb # [ 54.584220] niks3-server[974]: 2026/09/24 11:48:05 INFO Registered completed upload object_key=log/cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv4749lb # [ 54.589145] niks3-server[974]: 2026/09/24 11:48:05 INFO Received complete push request method=POST path=/api/pushes/14/complete4750lb # [ 54.596816] niks3-server[974]: 2026/09/24 11:48:05 INFO Registered completed upload object_key=dp6wdd8xfalk9qg1xqalcnk58hz7fnr5.narinfo4751worker1 # [ 54.603293] nix-grpc-daemon[917]: time=2026-09-24T11:48:04.895Z level=INFO msg="Upload complete. (215ms)"4752worker1 # [ 54.610248] nix-grpc-daemon[524]: ts=2026-09-24T11:48:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=1 drv=cs7l7i2nvvh5pcyf2vglzb4mlwzjklnx-farm-top.drv assign_id=7 outputs=14753worker1 # [ 54.625429] nix-daemon[867]: reaped child process 999, status = succeeded4754client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig >&2, in 4.56 seconds)4755client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath4756client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag sig outPath, in 0.11 seconds)4757client: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4758worker1 # [ 54.866695] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:422224759worker1 # [ 54.872831] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:422224760client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker1:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.13 seconds)4761client: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4762worker2 # [ 54.984405] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:576564763client # these 3 paths will be fetched (0.0 KiB download, 0.5 KiB unpacked):4764client # /nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a4765client # /nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b4766client # /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4767worker2 # [ 55.018917] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv6:%5B2001:db8:1::1%5D:576564768worker2 # [ 55.025875] nix-grpc-daemon[799]: copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'http://lb:5751'...4769worker2 # [ 55.039659] nix-grpc-daemon[799]: copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'http://lb:5751'...4770worker2 # [ 55.051636] nix-grpc-daemon[799]: copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'http://lb:5751'...4771client: (finished: must succeed: nix path-info --sigs --store 'grpc://worker2:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&ca-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ca.pem' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.21 seconds)4772client: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top4773client # finding garbage collector roots...4774client # removing stale temporary roots file "/nix/var/nix/temproots/827"4775client # removing stale temporary roots file "/nix/var/nix/temproots/812"4776client # removing stale temporary roots file "/nix/var/nix/temproots/860"4777client # removing stale temporary roots file "/nix/var/nix/temproots/848"4778client # removing stale temporary roots file "/nix/var/nix/temproots/868"4779client # removing stale temporary roots file "/nix/var/nix/temproots/881"4780client # removing stale temporary roots file "/nix/var/nix/temproots/889"4781client # removing stale temporary roots file "/nix/var/nix/temproots/904"4782client # deleting specified paths...4783client # deleting '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top'4784client # deleting unused links...4785client # note: hard linking is currently saving 0.0 KiB4786worker2 # [ 55.331753] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428944787worker2 # [ 55.338978] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428944788worker2 # [ 55.348910] nix-grpc-daemon[870]: time=2026-09-24T11:48:05.586Z level=INFO msg="All 1 paths already cached"4789worker1 # [ 55.367404] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:565464790worker2 # [ 55.363366] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:428944791client # copying 3 paths...4792worker2 # [ 55.373369] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:428944793client # copying path '/nix/store/qi07z82gylcaha6pl0vfp8s3j6f2gl91-farm-a' from 'grpc://lb:50051'...4794worker1 # [ 55.399957] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565464795worker1 # [ 55.405557] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_out=2324796worker1 # [ 55.412073] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565464797worker2 # [ 55.402904] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:428944798worker1 # [ 55.416302] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_out=1764799worker2 # [ 55.408153] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=1 nar_bytes_out=1284800lb # [ 55.413155] envoy[913]: conn peer=[2001:db8:1::1]:37154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9464 tx=4329 ms=46444801client # copying path '/nix/store/2f96mrg6ljwjqld1y3yaibmb07jkz3by-farm-b' from 'grpc://lb:50051'...4802lb # [ 55.416861] envoy[913]: conn peer=[2001:db8:1::1]:35406 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=991 tx=480 ms=474803client # copying path '/nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top' from 'grpc://lb:50051'...4804lb # [ 55.420519] envoy[913]: conn peer=[2001:db8:1::1]:35408 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4135 tx=1862 ms=15434805lb # [ 55.424282] envoy[913]: conn peer=[2001:db8:1::1]:35414 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9481 tx=4331 ms=44054806client: (finished: must succeed: nix-store --delete /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top && nix copy --from 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --option trusted-public-keys 'farm-test-1:RkClDwvfixdOwourBI4UD9hudE3xfU5EBQcMFUVuRV8=' /nix/store/dp6wdd8xfalk9qg1xqalcnk58hz7fnr5-farm-top, in 0.35 seconds)4807(finished: subtest: outputs read back through any worker carry the cache signature, in 5.36 seconds)4808subtest: two clients wanting the same drv build it once4809client: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4810client # [ 55.437359] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4811client # Running as unit: dedup1.service; invocation ID: 43c18f6dd92f4211a30cafbf5a322fd34812client: (finished: must succeed: systemd-run --unit dedup1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 0.03 seconds)4813client: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup4814client # [ 55.569549] nix[957]: don't know how to build these paths:4815client # [ 55.570458] nix[957]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4816worker2 # [ 55.594565] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428944817worker1 # [ 55.614851] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=debug event=want client=5 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04818worker1 # [ 55.619739] nix-grpc-daemon[524]: ts=2026-09-24T11:48:05Z level=debug event=assigned drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv worker=192.168.1.4:50051 clients=14819worker2 # [ 55.610959] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428944820worker2 # [ 55.622873] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428944821client # [ 55.606111] nix[957]: copying 1 paths to 'lb:50051'...4822client # [ 55.606907] nix[957]: copying path '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' to 'lb:50051'...4823worker2 # [ 55.633593] nix-grpc-daemon[799]: ts=2026-09-24T11:48:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428944824worker2 # [ 55.644327] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)4825worker2 # [ 55.749471] nix-daemon[857]: accepted connection from pid 1019, user nix-grpc-daemon (trusted)4826worker2 # [ 55.763230] nix-daemon[857]: reaped child process 1022, status = succeeded4827lb # [ 55.774706] niks3-server[974]: 2026/09/24 11:48:06 INFO Received push request method=POST path=/api/pushes4828worker2 # [ 55.779562] nix-grpc-daemon[870]: time=2026-09-24T11:48:06.017Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4829worker2 # [ 55.781037] nix-grpc-daemon[870]: time=2026-09-24T11:48:06.018Z level=INFO msg="Uploading k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv (432B)"4830lb # [ 55.805658] niks3-server[974]: 2026/09/24 11:48:06 INFO Received sign narinfos request method=POST path=/api/pushes/15/sign4831worker2 # [ 55.811792] nix-grpc-daemon[870]: time=2026-09-24T11:48:06.049Z level=INFO msg="Uploading 1 narinfos"4832lb # [ 55.815278] niks3-server[974]: 2026/09/24 11:48:06 INFO Signed narinfos id=15 count=14833lb # [ 55.822060] niks3-server[974]: 2026/09/24 11:48:06 INFO Registered completed upload object_key=nar/1rprin71wk1rn18ghpyxh5nm1akxfkjivwxfnlzqzmri5xdvl2l0.nar.zst4834lb # [ 55.827643] niks3-server[974]: 2026/09/24 11:48:06 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.ls4835lb # [ 55.840861] niks3-server[974]: 2026/09/24 11:48:06 INFO Received complete push request method=POST path=/api/pushes/15/complete4836lb # [ 55.849661] niks3-server[974]: 2026/09/24 11:48:06 INFO Registered completed upload object_key=k75hvg0drf5n5cgm1w4hms82xb1l5nmz.narinfo4837worker2 # [ 55.849419] nix-grpc-daemon[870]: time=2026-09-24T11:48:06.087Z level=INFO msg="Upload complete. (193ms)"4838worker2 # [ 55.854220] nix-grpc-daemon[799]: ts=2026-09-24T11:48:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=1 nar_bytes_in=4324839worker2 # [ 55.859625] nix-grpc-daemon[799]: ts=2026-09-24T11:48:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428944840worker2 # [ 55.872115] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)4841worker2 # [ 55.879187] nix-daemon[857]: reaped child process 1017, status = succeeded4842worker2 # [ 55.886690] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)4843client # [ 55.860270] nix[957]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4844client # [ 56.491914] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup.4845client # Running as unit: dedup2.service; invocation ID: 2509be9f014c4f18a8c5ce2cd52f3b644846client: (finished: must succeed: sleep 1; systemd-run --unit dedup2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag dedup, in 1.07 seconds)4847worker1 # [ 56.688802] nix-grpc-daemon[524]: ts=2026-09-24T11:48:06Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565464848client # [ 56.658387] nix[973]: don't know how to build these paths:4849client # [ 56.659240] nix[973]: /nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4850worker1 # [ 56.705651] nix-grpc-daemon[524]: ts=2026-09-24T11:48:06Z level=debug event=want client=6 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv cached=04851worker2 # [ 56.700293] nix-grpc-daemon[799]: ts=2026-09-24T11:48:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428944852client # [ 56.678924] nix[973]: building '/nix/store/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv' on lb:50051...4853client: must succeed: sleep 24854client: (finished: must succeed: sleep 2, in 2.04 seconds)4855client: waiting for success: ! systemctl is-active dedup1 dedup24856worker2 # [ 59.904831] nix-daemon[857]: reaped child process 1026, status = succeeded4857worker2 # [ 59.980749] nix-daemon[857]: accepted connection from pid 1056, user nix-grpc-daemon (trusted)4858worker2 # [ 59.993958] nix-daemon[857]: reaped child process 1059, status = succeeded4859lb # [ 60.006371] niks3-server[974]: 2026/09/24 11:48:10 INFO Received push request method=POST path=/api/pushes4860worker2 # [ 60.007807] nix-grpc-daemon[870]: time=2026-09-24T11:48:10.245Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4861worker2 # [ 60.010057] nix-grpc-daemon[870]: time=2026-09-24T11:48:10.247Z level=INFO msg="Uploading pihyvyr6gd3qnm368ii42wvcpgsfv1di-slow-dedup (120B)"4862lb # [ 60.029815] niks3-server[974]: 2026/09/24 11:48:10 INFO Received sign narinfos request method=POST path=/api/pushes/16/sign4863lb # [ 60.032669] niks3-server[974]: 2026/09/24 11:48:10 INFO Signed narinfos id=16 count=14864worker2 # [ 60.028880] nix-grpc-daemon[870]: time=2026-09-24T11:48:10.266Z level=INFO msg="Uploading 1 narinfos"4865lb # [ 60.035883] niks3-server[974]: 2026/09/24 11:48:10 INFO Registered completed upload object_key=nar/02xhr0h0sj1ji8m0s8995xzbbb8mhbqdpdl1ggx94yqqaykihvz7.nar.zst4866lb # [ 60.040201] niks3-server[974]: 2026/09/24 11:48:10 INFO Registered completed upload object_key=log/k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv4867lb # [ 60.042956] niks3-server[974]: 2026/09/24 11:48:10 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.ls4868lb # [ 60.050545] niks3-server[974]: 2026/09/24 11:48:10 INFO Received complete push request method=POST path=/api/pushes/16/complete4869worker2 # [ 60.051480] nix-grpc-daemon[870]: time=2026-09-24T11:48:10.289Z level=INFO msg="Upload complete. (155ms)"4870worker2 # [ 60.054402] nix-grpc-daemon[799]: ts=2026-09-24T11:48:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=3 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14871lb # [ 60.058897] niks3-server[974]: 2026/09/24 11:48:10 INFO Registered completed upload object_key=pihyvyr6gd3qnm368ii42wvcpgsfv1di.narinfo4872worker2 # [ 60.056900] nix-grpc-daemon[799]: ts=2026-09-24T11:48:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=4 drv=k75hvg0drf5n5cgm1w4hms82xb1l5nmz-slow-dedup.drv assign_id=8 outputs=14873worker2 # [ 60.063871] nix-daemon[857]: reaped child process 1024, status = succeeded4874client # [ 60.075280] systemd[1]: dedup2.service: Deactivated successfully.4875client # [ 60.080200] systemd[1]: dedup2.service: Consumed 140ms CPU time over 3.586s wall clock time, 35.4M memory peak, 5.9K incoming IP traffic, 5.6K outgoing IP traffic.4876client # [ 60.087266] systemd[1]: dedup1.service: Deactivated successfully.4877client # [ 60.090145] systemd[1]: dedup1.service: Consumed 117ms CPU time over 4.652s wall clock time, 43.5M memory peak, 7.1K incoming IP traffic, 8.2K outgoing IP traffic.4878client: (finished: waiting for success: ! systemctl is-active dedup1 dedup2, in 1.09 seconds)4879client: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }4880client: (finished: must succeed: systemctl show -p Result dedup1 dedup2 | grep -c success | grep -qx 2 || { journalctl -u dedup1 -u dedup2 >&2; false; }, in 0.04 seconds)4881worker1: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4882worker1: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4883worker2: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true4884worker2: (finished: must succeed: journalctl -u nix-grpc-daemon -o cat | grep 'method=BuildDerivation.*slow-dedup' | grep -o 'assign_id=[0-9]*' || true, in 0.05 seconds)4885worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4886worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4887worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4888worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4889(finished: subtest: two clients wanting the same drv build it once, in 5.74 seconds)4890subtest: --timeout and --max-silent-time reach the builder4891worker2 # [ 61.329703] nix-grpc-daemon[799]: ts=2026-09-24T11:48:11Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428944892worker1 # [ 61.354872] nix-grpc-daemon[524]: ts=2026-09-24T11:48:11Z level=debug event=want client=7 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv cached=04893worker1 # [ 61.360342] nix-grpc-daemon[524]: ts=2026-09-24T11:48:11Z level=debug event=assigned drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv worker=192.168.1.3:50051 clients=14894worker1 # [ 61.367554] nix-grpc-daemon[524]: ts=2026-09-24T11:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464895worker1 # [ 61.375438] nix-grpc-daemon[524]: ts=2026-09-24T11:48:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565464896worker1 # [ 61.386858] nix-grpc-daemon[524]: ts=2026-09-24T11:48:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565464897worker1 # [ 61.397671] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4898worker1 # [ 61.497502] nix-daemon[867]: accepted connection from pid 1051, user nix-grpc-daemon (trusted)4899worker1 # [ 61.510539] nix-daemon[867]: reaped child process 1054, status = succeeded4900lb # [ 61.509910] niks3-server[974]: 2026/09/24 11:48:11 INFO Received push request method=POST path=/api/pushes4901worker1 # [ 61.525326] nix-grpc-daemon[917]: time=2026-09-24T11:48:11.817Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4902worker1 # [ 61.527250] nix-grpc-daemon[917]: time=2026-09-24T11:48:11.818Z level=INFO msg="Uploading qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv (424B)"4903lb # [ 61.537873] niks3-server[974]: 2026/09/24 11:48:11 INFO Received sign narinfos request method=POST path=/api/pushes/17/sign4904lb # [ 61.544379] niks3-server[974]: 2026/09/24 11:48:11 INFO Signed narinfos id=17 count=14905worker1 # [ 61.554980] nix-grpc-daemon[917]: time=2026-09-24T11:48:11.846Z level=INFO msg="Uploading 1 narinfos"4906lb # [ 61.554202] niks3-server[974]: 2026/09/24 11:48:11 INFO Registered completed upload object_key=nar/1s4sciw269ksj1s7ymwz5cy4dw9n7bxv3k8wqw69h694spdh4ha2.nar.zst4907lb # [ 61.562122] niks3-server[974]: 2026/09/24 11:48:12 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.ls4908lb # [ 61.571594] niks3-server[974]: 2026/09/24 11:48:12 INFO Received complete push request method=POST path=/api/pushes/17/complete4909worker1 # [ 61.586671] nix-grpc-daemon[917]: time=2026-09-24T11:48:11.878Z level=INFO msg="Upload complete. (178ms)"4910lb # [ 61.580407] niks3-server[974]: 2026/09/24 11:48:12 INFO Registered completed upload object_key=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k.narinfo4911worker1 # [ 61.591448] nix-grpc-daemon[524]: ts=2026-09-24T11:48:11Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_in=4244912worker1 # [ 61.596870] nix-grpc-daemon[524]: ts=2026-09-24T11:48:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464913worker1 # [ 61.609132] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4914worker1 # [ 61.612122] nix-daemon[867]: reaped child process 1049, status = succeeded4915worker1 # [ 61.616812] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4916lb # [ 65.413495] envoy[913]: conn peer=[2001:db8:1::1]:47516 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1034 tx=2870 ms=1004917lb # [ 65.417294] envoy[913]: conn peer=[2001:db8:1::1]:47538 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=329 ms=294918lb # [ 65.420710] envoy[913]: conn peer=[2001:db8:1::1]:47524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=382 tx=376 ms=364919lb # [ 65.424352] envoy[913]: conn peer=[2001:db8:1::1]:47520 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=384 tx=410 ms=434920lb # [ 65.427824] envoy[913]: conn peer=[2001:db8:1::1]:47548 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2981 tx=1521 ms=44744921lb # [ 65.431916] envoy[913]: conn peer=[2001:db8:1::1]:47552 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1440 tx=1194 ms=33924922worker1 # [ 67.628637] nix-grpc-daemon[524]: ts=2026-09-24T11:48:17Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=6 drv=qkzy7qnn48bqc3ll9lb2m9zsrx1dr87k-slow-tmot.drv assign_id=9 outputs=04923worker1 # [ 67.640491] nix-daemon[867]: reaped child process 1056, status = succeeded4924worker1 # [ 67.641528] nix-daemon[867]: reaped child process 1058, status = succeeded4925worker1 # [ 67.792316] nix-grpc-daemon[524]: ts=2026-09-24T11:48:18Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565464926worker1 # [ 67.803305] nix-grpc-daemon[524]: ts=2026-09-24T11:48:18Z level=debug event=want client=8 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv cached=04927worker1 # [ 67.807689] nix-grpc-daemon[524]: ts=2026-09-24T11:48:18Z level=debug event=assigned drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv worker=192.168.1.4:50051 clients=14928worker2 # [ 67.795872] nix-grpc-daemon[799]: ts=2026-09-24T11:48:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428944929worker2 # [ 67.809859] nix-grpc-daemon[799]: ts=2026-09-24T11:48:18Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428944930worker2 # [ 67.821762] nix-grpc-daemon[799]: ts=2026-09-24T11:48:18Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428944931worker2 # [ 67.832976] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)4932worker2 # [ 67.939180] nix-daemon[857]: accepted connection from pid 1072, user nix-grpc-daemon (trusted)4933worker2 # [ 67.951957] nix-daemon[857]: reaped child process 1075, status = succeeded4934lb # [ 67.964070] niks3-server[974]: 2026/09/24 11:48:18 INFO Received push request method=POST path=/api/pushes4935worker2 # [ 67.964918] nix-grpc-daemon[870]: time=2026-09-24T11:48:18.202Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4936worker2 # [ 67.966391] nix-grpc-daemon[870]: time=2026-09-24T11:48:18.204Z level=INFO msg="Uploading fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv (424B)"4937lb # [ 67.978880] niks3-server[974]: 2026/09/24 11:48:18 INFO Received sign narinfos request method=POST path=/api/pushes/18/sign4938lb # [ 67.980902] niks3-server[974]: 2026/09/24 11:48:18 INFO Signed narinfos id=18 count=14939worker2 # [ 67.978601] nix-grpc-daemon[870]: time=2026-09-24T11:48:18.216Z level=INFO msg="Uploading 1 narinfos"4940lb # [ 67.987323] niks3-server[974]: 2026/09/24 11:48:18 INFO Registered completed upload object_key=nar/013y5sy2r6v0qdwv0dzpgvdfy08bds4mfh2qpvbgjazzrg2v71hp.nar.zst4941lb # [ 67.989599] niks3-server[974]: 2026/09/24 11:48:18 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.ls4942lb # [ 67.992481] niks3-server[974]: 2026/09/24 11:48:18 INFO Received complete push request method=POST path=/api/pushes/18/complete4943worker2 # [ 67.993921] nix-grpc-daemon[870]: time=2026-09-24T11:48:18.231Z level=INFO msg="Upload complete. (148ms)"4944worker2 # [ 67.995878] nix-grpc-daemon[799]: ts=2026-09-24T11:48:18Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=1 nar_bytes_in=4244945lb # [ 68.000511] niks3-server[974]: 2026/09/24 11:48:18 INFO Registered completed upload object_key=fvgcnys8vngcgklsp349d5pjxr4c66sg.narinfo4946worker2 # [ 67.999090] nix-grpc-daemon[799]: ts=2026-09-24T11:48:18Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428944947worker2 # [ 68.006408] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)4948worker2 # [ 68.008314] nix-daemon[857]: reaped child process 1070, status = succeeded4949worker2 # [ 68.012784] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)4950worker2 # [ 74.225358] nix-grpc-daemon[799]: ts=2026-09-24T11:48:24Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=6 drv=fvgcnys8vngcgklsp349d5pjxr4c66sg-slow-tmom.drv assign_id=10 outputs=04951worker2 # [ 74.237691] nix-daemon[857]: reaped child process 1077, status = succeeded4952worker2 # [ 74.238789] nix-daemon[857]: reaped child process 1079, status = succeeded4953worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4954worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4955worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4956worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4957(finished: subtest: --timeout and --max-silent-time reach the builder, in 13.26 seconds)4958subtest: a failing build is counted with its own reason4959worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4960worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4961worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4962worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4963worker2 # [ 74.673696] nix-grpc-daemon[799]: ts=2026-09-24T11:48:24Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:428944964worker1 # [ 74.695255] nix-grpc-daemon[524]: ts=2026-09-24T11:48:24Z level=debug event=want client=9 drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv cached=04965worker1 # [ 74.701189] nix-grpc-daemon[524]: ts=2026-09-24T11:48:24Z level=debug event=assigned drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv worker=192.168.1.3:50051 clients=14966worker1 # [ 74.705535] nix-grpc-daemon[524]: ts=2026-09-24T11:48:24Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464967worker1 # [ 74.717483] nix-grpc-daemon[524]: ts=2026-09-24T11:48:25Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565464968worker1 # [ 74.729146] nix-grpc-daemon[524]: ts=2026-09-24T11:48:25Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565464969worker1 # [ 74.739480] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4970worker1 # [ 74.836493] nix-daemon[867]: accepted connection from pid 1091, user nix-grpc-daemon (trusted)4971worker1 # [ 74.851621] nix-daemon[867]: reaped child process 1094, status = succeeded4972lb # [ 74.847685] niks3-server[974]: 2026/09/24 11:48:25 INFO Received push request method=POST path=/api/pushes4973worker1 # [ 74.863530] nix-grpc-daemon[917]: time=2026-09-24T11:48:25.154Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"4974worker1 # [ 74.864975] nix-grpc-daemon[917]: time=2026-09-24T11:48:25.154Z level=INFO msg="Uploading j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv (400B)"4975lb # [ 74.875375] niks3-server[974]: 2026/09/24 11:48:25 INFO Received sign narinfos request method=POST path=/api/pushes/19/sign4976worker1 # [ 74.888720] nix-grpc-daemon[917]: time=2026-09-24T11:48:25.180Z level=INFO msg="Uploading 1 narinfos"4977lb # [ 74.881921] niks3-server[974]: 2026/09/24 11:48:25 INFO Signed narinfos id=19 count=14978lb # [ 74.905207] niks3-server[974]: 2026/09/24 11:48:25 INFO Received complete push request method=POST path=/api/pushes/19/complete4979lb # [ 74.907472] niks3-server[974]: 2026/09/24 11:48:25 INFO Registered completed upload object_key=nar/1awa0khyixxmb53bhxv3vk1qy0fd3xzskbp3b8g2fzdmcz3cvv97.nar.zst4980lb # [ 74.912111] niks3-server[974]: 2026/09/24 11:48:25 INFO Registered completed upload object_key=j5sh943zaa0qjagii0xkivjfb4c2c5p0.ls4981worker1 # [ 74.922208] nix-grpc-daemon[917]: time=2026-09-24T11:48:25.214Z level=INFO msg="Upload complete. (171ms)"4982lb # [ 74.916337] niks3-server[974]: 2026/09/24 11:48:25 INFO Registered completed upload object_key=j5sh943zaa0qjagii0xkivjfb4c2c5p0.narinfo4983worker1 # [ 74.926476] nix-grpc-daemon[524]: ts=2026-09-24T11:48:25Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_in=4004984worker1 # [ 74.931386] nix-grpc-daemon[524]: ts=2026-09-24T11:48:25Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565464985worker1 # [ 74.940358] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4986worker1 # [ 74.949193] nix-daemon[867]: reaped child process 1089, status = succeeded4987worker1 # [ 74.952527] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)4988lb # [ 75.414679] envoy[913]: conn peer=[2001:db8:1::1]:47556 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2966 tx=933 ms=63044989lb # [ 75.418488] envoy[913]: conn peer=[2001:db8:1::1]:50214 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=960 ms=64574990worker1 # [ 76.162506] nix-grpc-daemon[524]: ts=2026-09-24T11:48:26Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=1 drv=j5sh943zaa0qjagii0xkivjfb4c2c5p0-fail-boom.drv assign_id=11 outputs=04991worker1 # [ 76.186471] nix-daemon[867]: reaped child process 1096, status = succeeded4992worker1 # [ 76.188433] nix-daemon[867]: reaped child process 1098, status = succeeded4993worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4994worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4995worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics4996worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)4997worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics4998worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)4999worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5000worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5001worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5002worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5003worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5004worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5005worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5006worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.03 seconds)5007worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5008worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5009(finished: subtest: a failing build is counted with its own reason, in 2.08 seconds)5010subtest: the first client leaving does not fail the second5011client: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5012client # [ 76.541890] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5013client # Running as unit: share1.service; invocation ID: b717931866a94f139f06708b172872e05014client: (finished: must succeed: systemd-run --unit share1 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 0.06 seconds)5015client: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share5016worker1 # [ 76.725399] nix-grpc-daemon[524]: ts=2026-09-24T11:48:27Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565465017client # [ 76.693979] nix[1062]: don't know how to build these paths:5018client # [ 76.694928] nix[1062]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5019worker1 # [ 76.738164] nix-grpc-daemon[524]: ts=2026-09-24T11:48:27Z level=debug event=want client=10 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05020worker1 # [ 76.742411] nix-grpc-daemon[524]: ts=2026-09-24T11:48:27Z level=debug event=assigned drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv worker=192.168.1.4:50051 clients=15021worker2 # [ 76.740760] nix-grpc-daemon[799]: ts=2026-09-24T11:48:26Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428945022worker2 # [ 76.751771] nix-grpc-daemon[799]: ts=2026-09-24T11:48:26Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428945023client # [ 76.735083] nix[1062]: copying 1 paths to 'lb:50051'...5024client # [ 76.735874] nix[1062]: copying path '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' to 'lb:50051'...5025worker2 # [ 76.762636] nix-grpc-daemon[799]: ts=2026-09-24T11:48:27Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428945026worker2 # [ 76.771672] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5027worker2 # [ 76.876200] nix-daemon[857]: accepted connection from pid 1120, user nix-grpc-daemon (trusted)5028worker2 # [ 76.890346] nix-daemon[857]: reaped child process 1123, status = succeeded5029lb # [ 76.901845] niks3-server[974]: 2026/09/24 11:48:27 INFO Received push request method=POST path=/api/pushes5030worker2 # [ 76.905115] nix-grpc-daemon[870]: time=2026-09-24T11:48:27.142Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5031worker2 # [ 76.907203] nix-grpc-daemon[870]: time=2026-09-24T11:48:27.144Z level=INFO msg="Uploading j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv (432B)"5032lb # [ 76.932997] niks3-server[974]: 2026/09/24 11:48:27 INFO Received sign narinfos request method=POST path=/api/pushes/20/sign5033lb # [ 76.938122] niks3-server[974]: 2026/09/24 11:48:27 INFO Signed narinfos id=20 count=15034worker2 # [ 76.937710] nix-grpc-daemon[870]: time=2026-09-24T11:48:27.175Z level=INFO msg="Uploading 1 narinfos"5035lb # [ 76.947052] niks3-server[974]: 2026/09/24 11:48:27 INFO Registered completed upload object_key=nar/0s0b0gfhn36p0aylk1qa64cgw01bi7a8ca93iv4zg2q7wqcjcqhs.nar.zst5036lb # [ 76.959966] niks3-server[974]: 2026/09/24 11:48:27 INFO Received complete push request method=POST path=/api/pushes/20/complete5037worker2 # [ 76.966080] nix-grpc-daemon[870]: time=2026-09-24T11:48:27.203Z level=INFO msg="Upload complete. (180ms)"5038lb # [ 76.971141] niks3-server[974]: 2026/09/24 11:48:27 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.ls5039worker2 # [ 76.969464] nix-grpc-daemon[799]: ts=2026-09-24T11:48:27Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=1 nar_bytes_in=4325040lb # [ 76.975949] niks3-server[974]: 2026/09/24 11:48:27 INFO Registered completed upload object_key=j2xs6vfpc2dkbabf87x31z8q4j8chjfw.narinfo5041worker2 # [ 76.977133] nix-grpc-daemon[799]: ts=2026-09-24T11:48:27Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428945042worker2 # [ 76.984996] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5043worker2 # [ 76.992279] nix-daemon[857]: reaped child process 1118, status = succeeded5044worker2 # [ 76.997584] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5045client # [ 76.970329] nix[1062]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5046client # [ 77.612660] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share.5047client # Running as unit: share2.service; invocation ID: 47f371b9fb514b42bc4ccd4098cb3bdf5048client: (finished: must succeed: sleep 1; systemd-run --unit share2 nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag share, in 1.07 seconds)5049worker1 # [ 77.793263] nix-grpc-daemon[524]: ts=2026-09-24T11:48:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565465050client # [ 77.761533] nix[1078]: don't know how to build these paths:5051client # [ 77.762491] nix[1078]: /nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5052worker1 # [ 77.806204] nix-grpc-daemon[524]: ts=2026-09-24T11:48:28Z level=debug event=want client=11 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv cached=05053worker2 # [ 77.798120] nix-grpc-daemon[799]: ts=2026-09-24T11:48:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428945054client # [ 77.776862] nix[1078]: building '/nix/store/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv' on lb:50051...5055client: must succeed: sleep 2; systemctl kill -s INT share15056client # [ 80.871644] systemd[1]: share1.service: Sent signal SIGINT to main process 1062 (nix) on client request.5057client # [ 80.877875] nix[1062]: shutting downj2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)5058client # [ 80.883989] nix[1062]: error: interrupted by the user5059client: (finished: must succeed: sleep 2; systemctl kill -s INT share1, in 2.06 seconds)5060client: waiting for success: ! systemctl is-active share15061client # [ 80.926479] systemd[1]: share1.service: Main process exited, code=exited, status=1/FAILURE5062client # [ 80.928055] systemd[1]: share1.service: Failed with result 'exit-code'.5063client # [ 80.930058] systemd[1]: share1.service: Consumed 119ms CPU time over 4.387s wall clock time, 41.5M memory peak, 6K incoming IP traffic, 8.1K outgoing IP traffic.5064client: (finished: waiting for success: ! systemctl is-active share1, in 1.08 seconds)5065client: waiting for success: ! systemctl is-active share25066worker2 # [ 82.214543] nix-daemon[857]: reaped child process 1127, status = succeeded5067worker2 # [ 82.291591] nix-daemon[857]: accepted connection from pid 1161, user nix-grpc-daemon (trusted)5068worker2 # [ 82.303317] nix-daemon[857]: reaped child process 1164, status = succeeded5069lb # [ 82.316865] niks3-server[974]: 2026/09/24 11:48:32 INFO Received push request method=POST path=/api/pushes5070worker2 # [ 82.321213] nix-grpc-daemon[870]: time=2026-09-24T11:48:32.558Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5071worker2 # [ 82.322871] nix-grpc-daemon[870]: time=2026-09-24T11:48:32.560Z level=INFO msg="Uploading b9lg6znd0rinl36gy6ixq1dcj6b8bhhw-slow-share (120B)"5072lb # [ 82.358679] niks3-server[974]: 2026/09/24 11:48:32 INFO Registered completed upload object_key=nar/172qq270pqbv9ipsik19pmld0m7fj8r5nyyxq3slfx9n7zcaa9s0.nar.zst5073lb # [ 82.370654] niks3-server[974]: 2026/09/24 11:48:32 INFO Received sign narinfos request method=POST path=/api/pushes/21/sign5074worker2 # [ 82.374064] nix-grpc-daemon[870]: time=2026-09-24T11:48:32.611Z level=INFO msg="Uploading 1 narinfos"5075lb # [ 82.378142] niks3-server[974]: 2026/09/24 11:48:32 INFO Signed narinfos id=21 count=15076lb # [ 82.382893] niks3-server[974]: 2026/09/24 11:48:32 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.ls5077lb # [ 82.388886] niks3-server[974]: 2026/09/24 11:48:32 INFO Registered completed upload object_key=log/j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv5078lb # [ 82.399578] niks3-server[974]: 2026/09/24 11:48:32 INFO Received complete push request method=POST path=/api/pushes/21/complete5079worker2 # [ 82.400745] nix-grpc-daemon[870]: time=2026-09-24T11:48:32.638Z level=INFO msg="Upload complete. (192ms)"5080lb # [ 82.406516] niks3-server[974]: 2026/09/24 11:48:32 INFO Registered completed upload object_key=b9lg6znd0rinl36gy6ixq1dcj6b8bhhw.narinfo5081worker2 # [ 82.407338] nix-grpc-daemon[799]: ts=2026-09-24T11:48:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=4 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=12 outputs=15082worker2 # [ 82.413774] nix-grpc-daemon[799]: ts=2026-09-24T11:48:32Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=5 drv=j2xs6vfpc2dkbabf87x31z8q4j8chjfw-slow-share.drv assign_id=12 outputs=15083worker2 # [ 82.427216] nix-daemon[857]: reaped child process 1125, status = succeeded5084client # [ 82.413071] systemd[1]: share2.service: Deactivated successfully.5085client # [ 82.416829] systemd[1]: share2.service: Consumed 120ms CPU time over 4.803s wall clock time, 35.4M memory peak, 6K incoming IP traffic, 5.7K outgoing IP traffic.5086client: (finished: waiting for success: ! systemctl is-active share2, in 1.08 seconds)5087client: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }5088client: (finished: must succeed: systemctl show -p Result share2 | grep -q success || { journalctl -u share2 >&2; false; }, in 0.03 seconds)5089(finished: subtest: the first client leaving does not fail the second, in 6.76 seconds)5090subtest: build hook: nix-daemon with builders = grpc://lb5091client: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log5092client # [ 83.349338] systemd[1]: Started Nix Daemon.5093client # [ 83.436869] nix-daemon[1129]: accepted connection from pid 1126, user root (trusted)5094worker1 # [ 83.588722] nix-grpc-daemon[524]: ts=2026-09-24T11:48:33Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:565465095worker1 # [ 83.594475] nix-grpc-daemon[524]: ts=2026-09-24T11:48:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565465096worker1 # [ 83.604244] nix-grpc-daemon[524]: ts=2026-09-24T11:48:33Z level=debug event=want client=12 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv cached=05097worker1 # [ 83.608205] nix-grpc-daemon[524]: ts=2026-09-24T11:48:33Z level=debug event=assigned drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv worker=192.168.1.3:50051 clients=15098worker1 # [ 83.615479] nix-grpc-daemon[524]: ts=2026-09-24T11:48:33Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565465099worker1 # [ 83.627171] nix-grpc-daemon[524]: ts=2026-09-24T11:48:33Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565465100worker1 # [ 83.638354] nix-grpc-daemon[524]: ts=2026-09-24T11:48:33Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565465101worker1 # [ 83.649230] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5102worker1 # [ 83.743209] nix-daemon[867]: accepted connection from pid 1150, user nix-grpc-daemon (trusted)5103worker1 # [ 83.758745] nix-daemon[867]: reaped child process 1153, status = succeeded5104lb # [ 83.751201] niks3-server[974]: 2026/09/24 11:48:34 INFO Received push request method=POST path=/api/pushes5105worker1 # [ 83.766794] nix-grpc-daemon[917]: time=2026-09-24T11:48:34.058Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5106worker1 # [ 83.768753] nix-grpc-daemon[917]: time=2026-09-24T11:48:34.060Z level=INFO msg="Uploading pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv (512B)"5107lb # [ 83.780997] niks3-server[974]: 2026/09/24 11:48:34 INFO Received sign narinfos request method=POST path=/api/pushes/22/sign5108lb # [ 83.790248] niks3-server[974]: 2026/09/24 11:48:34 INFO Signed narinfos id=22 count=15109worker1 # [ 83.800851] nix-grpc-daemon[917]: time=2026-09-24T11:48:34.092Z level=INFO msg="Uploading 1 narinfos"5110lb # [ 83.795914] niks3-server[974]: 2026/09/24 11:48:34 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.ls5111lb # [ 83.802097] niks3-server[974]: 2026/09/24 11:48:34 INFO Registered completed upload object_key=nar/08mhnmxkz6k9gck9zisdh6jb3dqb53xqy0jak6cm9lpasdvr4sk7.nar.zst5112lb # [ 83.814498] niks3-server[974]: 2026/09/24 11:48:34 INFO Received complete push request method=POST path=/api/pushes/22/complete5113worker1 # [ 83.828666] nix-grpc-daemon[917]: time=2026-09-24T11:48:34.120Z level=INFO msg="Upload complete. (169ms)"5114lb # [ 83.823580] niks3-server[974]: 2026/09/24 11:48:34 INFO Registered completed upload object_key=pd51i21khb2xrd9paajf80pb96sw8zn0.narinfo5115worker1 # [ 83.831843] nix-grpc-daemon[524]: ts=2026-09-24T11:48:34Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_in=5125116worker1 # [ 83.837759] nix-grpc-daemon[524]: ts=2026-09-24T11:48:34Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565465117worker1 # [ 83.846758] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5118worker1 # [ 83.852794] nix-daemon[867]: reaped child process 1148, status = succeeded5119worker1 # [ 83.856805] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5120worker1 # [ 84.671769] nix-daemon[867]: reaped child process 1157, status = succeeded5121worker1 # [ 84.745945] nix-daemon[867]: accepted connection from pid 1173, user nix-grpc-daemon (trusted)5122worker1 # [ 84.757609] nix-daemon[867]: reaped child process 1176, status = succeeded5123lb # [ 84.756354] niks3-server[974]: 2026/09/24 11:48:35 INFO Received push request method=POST path=/api/pushes5124worker1 # [ 84.773916] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.065Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5125worker1 # [ 84.775965] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.067Z level=INFO msg="Uploading 6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a (128B)"5126lb # [ 84.798495] niks3-server[974]: 2026/09/24 11:48:35 INFO Registered completed upload object_key=nar/0pbbxl4kxy7zl7x4lp43ywljwn82mxzna2wgpb1bz1z765nwp9ki.nar.zst5127lb # [ 84.807830] niks3-server[974]: 2026/09/24 11:48:35 INFO Received sign narinfos request method=POST path=/api/pushes/23/sign5128lb # [ 84.812315] niks3-server[974]: 2026/09/24 11:48:35 INFO Signed narinfos id=23 count=15129worker1 # [ 84.822960] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.114Z level=INFO msg="Uploading 1 narinfos"5130lb # [ 84.820842] niks3-server[974]: 2026/09/24 11:48:35 INFO Registered completed upload object_key=log/pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv5131lb # [ 84.827907] niks3-server[974]: 2026/09/24 11:48:35 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.ls5132lb # [ 84.837755] niks3-server[974]: 2026/09/24 11:48:35 INFO Received complete push request method=POST path=/api/pushes/23/complete5133lb # [ 84.842154] niks3-server[974]: 2026/09/24 11:48:35 INFO Registered completed upload object_key=6zgzdy628dcdg0dbc5qi7nisk08n4kp2.narinfo5134worker1 # [ 84.852741] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.144Z level=INFO msg="Upload complete. (189ms)"5135worker1 # [ 84.860276] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=1 drv=pd51i21khb2xrd9paajf80pb96sw8zn0-farm-a.drv assign_id=13 outputs=15136worker2 # [ 84.853695] nix-grpc-daemon[799]: ts=2026-09-24T11:48:35Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:428945137worker1 # [ 84.874200] nix-daemon[867]: reaped child process 1155, status = succeeded5138worker2 # [ 84.863153] nix-grpc-daemon[799]: copying path '/nix/store/6zgzdy628dcdg0dbc5qi7nisk08n4kp2-farm-a' from 'http://lb:5751'...5139worker1 # [ 84.898193] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565465140worker1 # [ 84.903771] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_out=1285141worker2 # [ 85.065588] nix-grpc-daemon[799]: ts=2026-09-24T11:48:35Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:428945142worker2 # [ 85.072647] nix-grpc-daemon[799]: ts=2026-09-24T11:48:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428945143worker2 # [ 85.082646] nix-grpc-daemon[870]: time=2026-09-24T11:48:35.319Z level=INFO msg="All 1 paths already cached"5144worker1 # [ 85.108617] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=debug event=want client=13 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv cached=05145worker1 # [ 85.114921] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=debug event=assigned drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv worker=192.168.1.3:50051 clients=15146worker1 # [ 85.119865] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565465147worker1 # [ 85.131189] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565465148worker1 # [ 85.143887] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.435Z level=INFO msg="All 1 paths already cached"5149worker1 # [ 85.151454] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565465150worker1 # [ 85.159822] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5151worker1 # [ 85.265397] nix-daemon[867]: accepted connection from pid 1182, user nix-grpc-daemon (trusted)5152worker1 # [ 85.280187] nix-daemon[867]: reaped child process 1185, status = succeeded5153lb # [ 85.277077] niks3-server[974]: 2026/09/24 11:48:35 INFO Received push request method=POST path=/api/pushes5154worker1 # [ 85.292743] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.584Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5155worker1 # [ 85.294230] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.586Z level=INFO msg="Uploading vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv (624B)"5156lb # [ 85.311194] niks3-server[974]: 2026/09/24 11:48:35 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.ls5157lb # [ 85.314681] niks3-server[974]: 2026/09/24 11:48:35 INFO Received sign narinfos request method=POST path=/api/pushes/24/sign5158lb # [ 85.320305] niks3-server[974]: 2026/09/24 11:48:35 INFO Signed narinfos id=24 count=15159worker1 # [ 85.330854] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.622Z level=INFO msg="Uploading 1 narinfos"5160lb # [ 85.333858] niks3-server[974]: 2026/09/24 11:48:35 INFO Registered completed upload object_key=nar/1190412lxhybpj2b0886g97h0cpm7xx8r4virf8q4n7i91409ivg.nar.zst5161lb # [ 85.342699] niks3-server[974]: 2026/09/24 11:48:35 INFO Received complete push request method=POST path=/api/pushes/24/complete5162lb # [ 85.349983] niks3-server[974]: 2026/09/24 11:48:35 INFO Registered completed upload object_key=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8.narinfo5163worker1 # [ 85.362184] nix-grpc-daemon[917]: time=2026-09-24T11:48:35.653Z level=INFO msg="Upload complete. (188ms)"5164worker1 # [ 85.366826] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_in=6245165worker1 # [ 85.373316] nix-grpc-daemon[524]: ts=2026-09-24T11:48:35Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565465166worker1 # [ 85.385117] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5167worker1 # [ 85.392528] nix-daemon[867]: reaped child process 1180, status = succeeded5168worker1 # [ 85.400506] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5169lb # [ 85.414379] envoy[913]: conn peer=[2001:db8:1::1]:51380 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2866 tx=1247 ms=14955170lb # [ 85.418096] envoy[913]: conn peer=[2001:db8:1::1]:51390 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2965 tx=795 ms=42095171lb # [ 85.421593] envoy[913]: conn peer=[2001:db8:1::1]:51392 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1427 tx=1194 ms=46375172lb # [ 85.425103] envoy[913]: conn peer=[2001:db8:1::1]:56272 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3380 tx=1923 ms=13375173lb # [ 85.428769] envoy[913]: conn peer=[2001:db8:1::1]:56284 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=409 tx=333 ms=275174worker1 # [ 86.237199] nix-daemon[867]: reaped child process 1189, status = succeeded5175worker1 # [ 86.326086] nix-daemon[867]: accepted connection from pid 1205, user nix-grpc-daemon (trusted)5176worker1 # [ 86.338941] nix-daemon[867]: reaped child process 1208, status = succeeded5177lb # [ 86.336077] niks3-server[974]: 2026/09/24 11:48:36 INFO Received push request method=POST path=/api/pushes5178worker1 # [ 86.351778] nix-grpc-daemon[917]: time=2026-09-24T11:48:36.643Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5179worker1 # [ 86.354057] nix-grpc-daemon[917]: time=2026-09-24T11:48:36.645Z level=INFO msg="Uploading mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b (176B)"5180lb # [ 86.375353] niks3-server[974]: 2026/09/24 11:48:36 INFO Registered completed upload object_key=nar/1mgm44m4ndgi5l3yi2qv3v3sx48190m8d4ycw3r4pfzy4z1wq2s6.nar.zst5181lb # [ 86.386310] niks3-server[974]: 2026/09/24 11:48:36 INFO Received sign narinfos request method=POST path=/api/pushes/25/sign5182lb # [ 86.396127] niks3-server[974]: 2026/09/24 11:48:36 INFO Signed narinfos id=25 count=15183worker1 # [ 86.406205] nix-grpc-daemon[917]: time=2026-09-24T11:48:36.698Z level=INFO msg="Uploading 1 narinfos"5184lb # [ 86.401929] niks3-server[974]: 2026/09/24 11:48:36 INFO Registered completed upload object_key=log/vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv5185lb # [ 86.406765] niks3-server[974]: 2026/09/24 11:48:36 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.ls5186lb # [ 86.418365] niks3-server[974]: 2026/09/24 11:48:36 INFO Received complete push request method=POST path=/api/pushes/25/complete5187lb # [ 86.421448] niks3-server[974]: 2026/09/24 11:48:36 INFO Registered completed upload object_key=mrrcl9jb4hh6rly2df2vfym5sq1235hf.narinfo5188worker1 # [ 86.433450] nix-grpc-daemon[917]: time=2026-09-24T11:48:36.725Z level=INFO msg="Upload complete. (226ms)"5189worker1 # [ 86.439658] nix-grpc-daemon[524]: ts=2026-09-24T11:48:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=1 drv=vwnd00v6q8nvv6nb52vdjqfl5vdlzdd8-farm-b.drv assign_id=14 outputs=15190worker1 # [ 86.448840] nix-grpc-daemon[524]: ts=2026-09-24T11:48:36Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:565465191worker1 # [ 86.459629] nix-daemon[867]: reaped child process 1187, status = succeeded5192worker1 # [ 86.465241] nix-grpc-daemon[524]: ts=2026-09-24T11:48:36Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565465193worker1 # [ 86.469159] nix-grpc-daemon[524]: ts=2026-09-24T11:48:36Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_out=1765194worker2 # [ 86.624208] nix-grpc-daemon[799]: ts=2026-09-24T11:48:36Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:428945195worker2 # [ 86.630258] nix-grpc-daemon[799]: ts=2026-09-24T11:48:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428945196worker2 # [ 86.644902] nix-grpc-daemon[870]: time=2026-09-24T11:48:36.882Z level=INFO msg="All 1 paths already cached"5197worker1 # [ 86.670855] nix-grpc-daemon[524]: ts=2026-09-24T11:48:36Z level=debug event=want client=14 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv cached=05198worker1 # [ 86.677460] nix-grpc-daemon[524]: ts=2026-09-24T11:48:36Z level=debug event=assigned drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv worker=192.168.1.3:50051 clients=15199worker1 # [ 86.682282] nix-grpc-daemon[524]: ts=2026-09-24T11:48:36Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565465200worker1 # [ 86.693838] nix-grpc-daemon[524]: ts=2026-09-24T11:48:36Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565465201worker1 # [ 86.706641] nix-grpc-daemon[917]: time=2026-09-24T11:48:36.998Z level=INFO msg="All 2 paths already cached"5202worker1 # [ 86.714768] nix-grpc-daemon[524]: ts=2026-09-24T11:48:37Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565465203worker1 # [ 86.722694] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5204worker1 # [ 86.829354] nix-daemon[867]: accepted connection from pid 1214, user nix-grpc-daemon (trusted)5205worker1 # [ 86.844872] nix-daemon[867]: reaped child process 1217, status = succeeded5206lb # [ 86.841492] niks3-server[974]: 2026/09/24 11:48:37 INFO Received push request method=POST path=/api/pushes5207worker1 # [ 86.856487] nix-grpc-daemon[917]: time=2026-09-24T11:48:37.148Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5208worker1 # [ 86.858003] nix-grpc-daemon[917]: time=2026-09-24T11:48:37.149Z level=INFO msg="Uploading bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv (752B)"5209lb # [ 86.872960] niks3-server[974]: 2026/09/24 11:48:37 INFO Received sign narinfos request method=POST path=/api/pushes/26/sign5210lb # [ 86.881636] niks3-server[974]: 2026/09/24 11:48:37 INFO Signed narinfos id=26 count=15211worker1 # [ 86.893482] nix-grpc-daemon[917]: time=2026-09-24T11:48:37.185Z level=INFO msg="Uploading 1 narinfos"5212lb # [ 86.889189] niks3-server[974]: 2026/09/24 11:48:37 INFO Registered completed upload object_key=nar/0d4r016fc570pq02p1j53gahxygvvyr62937d84sa58ri3rzk000.nar.zst5213lb # [ 86.895596] niks3-server[974]: 2026/09/24 11:48:37 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.ls5214lb # [ 86.907214] niks3-server[974]: 2026/09/24 11:48:37 INFO Received complete push request method=POST path=/api/pushes/26/complete5215lb # [ 86.916848] niks3-server[974]: 2026/09/24 11:48:37 INFO Registered completed upload object_key=bshr7sr3z72cf1fa1a1b8fifr45jgpsw.narinfo5216worker1 # [ 86.925801] nix-grpc-daemon[917]: time=2026-09-24T11:48:37.215Z level=INFO msg="Upload complete. (187ms)"5217worker1 # [ 86.929996] nix-grpc-daemon[524]: ts=2026-09-24T11:48:37Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_in=7525218worker1 # [ 86.935281] nix-grpc-daemon[524]: ts=2026-09-24T11:48:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565465219worker1 # [ 86.946117] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5220worker1 # [ 86.954389] nix-daemon[867]: reaped child process 1212, status = succeeded5221worker1 # [ 86.960506] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5222worker1 # [ 87.998415] nix-daemon[867]: reaped child process 1221, status = succeeded5223worker1 # [ 88.077709] nix-daemon[867]: accepted connection from pid 1237, user nix-grpc-daemon (trusted)5224worker1 # [ 88.091339] nix-daemon[867]: reaped child process 1240, status = succeeded5225lb # [ 88.088209] niks3-server[974]: 2026/09/24 11:48:38 INFO Received push request method=POST path=/api/pushes5226worker1 # [ 88.105698] nix-grpc-daemon[917]: time=2026-09-24T11:48:38.397Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5227worker1 # [ 88.107786] nix-grpc-daemon[917]: time=2026-09-24T11:48:38.399Z level=INFO msg="Uploading aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top (232B)"5228lb # [ 88.130723] niks3-server[974]: 2026/09/24 11:48:38 INFO Received sign narinfos request method=POST path=/api/pushes/27/sign5229worker1 # [ 88.143368] nix-grpc-daemon[917]: time=2026-09-24T11:48:38.435Z level=INFO msg="Uploading 1 narinfos"5230lb # [ 88.137380] niks3-server[974]: 2026/09/24 11:48:38 INFO Signed narinfos id=27 count=15231lb # [ 88.142299] niks3-server[974]: 2026/09/24 11:48:38 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.ls5232lb # [ 88.145872] niks3-server[974]: 2026/09/24 11:48:38 INFO Registered completed upload object_key=nar/1r64sjcvz2vpjvwv3psvwf0bfifdzwiafvy1m89v5is1mm18ds87.nar.zst5233lb # [ 88.151421] niks3-server[974]: 2026/09/24 11:48:38 INFO Registered completed upload object_key=log/bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv5234lb # [ 88.156179] niks3-server[974]: 2026/09/24 11:48:38 INFO Received complete push request method=POST path=/api/pushes/27/complete5235lb # [ 88.161575] niks3-server[974]: 2026/09/24 11:48:38 INFO Registered completed upload object_key=aayfavl0hksy8qs9qc49znrwnfc5d2sr.narinfo5236worker1 # [ 88.168455] nix-grpc-daemon[917]: time=2026-09-24T11:48:38.460Z level=INFO msg="Upload complete. (198ms)"5237worker1 # [ 88.173854] nix-grpc-daemon[524]: ts=2026-09-24T11:48:38Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=1 drv=bshr7sr3z72cf1fa1a1b8fifr45jgpsw-farm-top.drv assign_id=15 outputs=15238worker1 # [ 88.182131] nix-grpc-daemon[524]: ts=2026-09-24T11:48:38Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:565465239worker1 # [ 88.190138] nix-daemon[867]: reaped child process 1219, status = succeeded5240worker2 # [ 88.188141] nix-grpc-daemon[799]: ts=2026-09-24T11:48:38Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:428945241worker2 # [ 88.197498] nix-grpc-daemon[799]: copying path '/nix/store/mrrcl9jb4hh6rly2df2vfym5sq1235hf-farm-b' from 'http://lb:5751'...5242worker2 # [ 88.207587] nix-grpc-daemon[799]: copying path '/nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top' from 'http://lb:5751'...5243worker2 # [ 88.219784] nix-grpc-daemon[799]: ts=2026-09-24T11:48:38Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=1 nar_bytes_out=2325244client # [ 88.270103] nix-daemon[1129]: reaped child process 1131, status = succeeded5245client: (finished: must succeed: NIX_REMOTE=daemon nix build --log-format internal-json --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --print-out-paths --no-link -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag hook 2>/tmp/hook.log, in 5.02 seconds)5246client: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top5247client: (finished: must succeed: grep farm-top-hook /nix/store/aayfavl0hksy8qs9qc49znrwnfc5d2sr-farm-top, in 0.02 seconds)5248client: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }5249client: (finished: must succeed: grep -F 'LOG-farm-top-hook' /tmp/hook.log | grep -qF '"type":101' && grep -F 'farmPhase' /tmp/hook.log | grep -qF '"type":104' || { cat /tmp/hook.log >&2; false; }, in 0.02 seconds)5250client: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }5251client: (finished: must succeed: grep -F '"type":105' /tmp/hook.log | grep -F lb:50051 | grep -qF '"fields":["/nix/store/' || { grep -F '"type":105' /tmp/hook.log >&2; false; }, in 0.02 seconds)5252(finished: subtest: build hook: nix-daemon with builders = grpc://lb, in 5.08 seconds)5253subtest: an input only one worker has still reaches the builder5254client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only5255client # this derivation will be built:5256client # /nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv5257client # building '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv'...5258client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag w1only, in 0.24 seconds)5259client: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure5260client: (finished: must succeed: nix-store --export /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only > /tmp/shared/inp.closure, in 0.10 seconds)5261worker1: must succeed: nix-store --import < /tmp/shared/inp.closure5262worker1: (finished: must succeed: nix-store --import < /tmp/shared/inp.closure, in 0.13 seconds)5263worker2: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only5264worker2: (finished: must fail: test -e /nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only, in 0.02 seconds)5265client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&25266client # [ 88.933088] nix-daemon[1129]: accepted connection from pid 1211, user root (trusted)5267client # this derivation will be built:5268client # /nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5269client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5270worker1 # [ 89.086663] nix-grpc-daemon[524]: ts=2026-09-24T11:48:39Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:565465271worker1 # [ 89.092959] nix-grpc-daemon[524]: ts=2026-09-24T11:48:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565465272worker1 # [ 89.199842] nix-daemon[867]: accepted connection from pid 1247, user nix-grpc-daemon (trusted)5273worker1 # [ 89.212429] nix-daemon[867]: reaped child process 1250, status = succeeded5274lb # [ 89.210453] niks3-server[974]: 2026/09/24 11:48:39 INFO Received push request method=POST path=/api/pushes5275worker1 # [ 89.226313] nix-grpc-daemon[917]: time=2026-09-24T11:48:39.518Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5276worker1 # [ 89.228243] nix-grpc-daemon[917]: time=2026-09-24T11:48:39.519Z level=INFO msg="Uploading gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only (120B)"5277lb # [ 89.241667] niks3-server[974]: 2026/09/24 11:48:39 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.ls5278lb # [ 89.247838] niks3-server[974]: 2026/09/24 11:48:39 INFO Received sign narinfos request method=POST path=/api/pushes/28/sign5279lb # [ 89.252700] niks3-server[974]: 2026/09/24 11:48:39 INFO Signed narinfos id=28 count=15280worker1 # [ 89.263675] nix-grpc-daemon[917]: time=2026-09-24T11:48:39.555Z level=INFO msg="Uploading 1 narinfos"5281lb # [ 89.267147] niks3-server[974]: 2026/09/24 11:48:39 INFO Registered completed upload object_key=nar/08ghl87f009pwqz85nbqv9hm0ln6gai6ggzilfmw3qnkaalfi6ss.nar.zst5282lb # [ 89.275597] niks3-server[974]: 2026/09/24 11:48:39 INFO Received complete push request method=POST path=/api/pushes/28/complete5283lb # [ 89.283358] niks3-server[974]: 2026/09/24 11:48:39 INFO Registered completed upload object_key=gycxa7jajm34fqsmi3xn0zjr3pvaya0x.narinfo5284worker1 # [ 89.296059] nix-grpc-daemon[917]: time=2026-09-24T11:48:39.587Z level=INFO msg="Upload complete. (204ms)"5285worker1 # [ 89.315358] nix-grpc-daemon[524]: ts=2026-09-24T11:48:39Z level=debug event=want client=15 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv cached=05286worker2 # [ 89.308550] nix-grpc-daemon[799]: ts=2026-09-24T11:48:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428945287worker1 # [ 89.320104] nix-grpc-daemon[524]: ts=2026-09-24T11:48:39Z level=debug event=assigned drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv worker=192.168.1.4:50051 clients=15288worker2 # [ 89.318653] nix-grpc-daemon[799]: ts=2026-09-24T11:48:39Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428945289client # copying 2 paths to 'lb:50051'...5290client # copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' to 'lb:50051'...5291worker2 # [ 89.333384] nix-grpc-daemon[799]: ts=2026-09-24T11:48:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428945292client # copying path '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' to 'lb:50051'...5293worker2 # [ 89.347283] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5294worker2 # [ 89.451228] nix-daemon[857]: accepted connection from pid 1174, user nix-grpc-daemon (trusted)5295worker2 # [ 89.463594] nix-daemon[857]: reaped child process 1177, status = succeeded5296lb # [ 89.475278] niks3-server[974]: 2026/09/24 11:48:39 INFO Received push request method=POST path=/api/pushes5297worker2 # [ 89.480076] nix-grpc-daemon[870]: time=2026-09-24T11:48:39.717Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5298worker2 # [ 89.483234] nix-grpc-daemon[870]: time=2026-09-24T11:48:39.719Z level=INFO msg="Uploading k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv (560B)"5299worker2 # [ 89.484902] nix-grpc-daemon[870]: time=2026-09-24T11:48:39.719Z level=INFO msg="Uploading s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv (416B)"5300lb # [ 89.520660] niks3-server[974]: 2026/09/24 11:48:39 INFO Received sign narinfos request method=POST path=/api/pushes/29/sign5301worker2 # [ 89.524130] nix-grpc-daemon[870]: time=2026-09-24T11:48:39.761Z level=INFO msg="Uploading 2 narinfos"5302lb # [ 89.528765] niks3-server[974]: 2026/09/24 11:48:39 INFO Signed narinfos id=29 count=25303lb # [ 89.537920] niks3-server[974]: 2026/09/24 11:48:39 INFO Registered completed upload object_key=nar/136ms4k64sfvy4m17vvd9w9yncq4zydv4cjbhchdcg4j7ccbx3b9.nar.zst5304lb # [ 89.544190] niks3-server[974]: 2026/09/24 11:48:39 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.ls5305lb # [ 89.546242] niks3-server[974]: 2026/09/24 11:48:39 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.ls5306lb # [ 89.553481] niks3-server[974]: 2026/09/24 11:48:39 INFO Registered completed upload object_key=nar/0s7awvvc5phwxzwwzhl47kzq38pdgwj6jp3kwxf5w2xddnxnkl6h.nar.zst5307lb # [ 89.556460] niks3-server[974]: 2026/09/24 11:48:40 INFO Received complete push request method=POST path=/api/pushes/29/complete5308worker2 # [ 89.559749] nix-grpc-daemon[870]: time=2026-09-24T11:48:39.797Z level=INFO msg="Upload complete. (199ms)"5309lb # [ 89.565145] niks3-server[974]: 2026/09/24 11:48:40 INFO Registered completed upload object_key=k9va03aiirjmgih65p90fpa9mr396d2b.narinfo5310lb # [ 89.567165] niks3-server[974]: 2026/09/24 11:48:40 INFO Registered completed upload object_key=s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr.narinfo5311worker2 # [ 89.564453] nix-grpc-daemon[799]: ts=2026-09-24T11:48:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=2 nar_bytes_in=9765312worker2 # [ 89.571217] nix-grpc-daemon[799]: ts=2026-09-24T11:48:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428945313worker2 # [ 89.580823] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5314worker2 # [ 89.592123] nix-daemon[857]: reaped child process 1172, status = succeeded5315worker2 # [ 89.595885] nix-grpc-daemon[799]: copying path '/nix/store/gycxa7jajm34fqsmi3xn0zjr3pvaya0x-local-input-w1only' from 'http://lb:5751'...5316worker2 # [ 89.602443] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5317client # building '/nix/store/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv' on lb:50051...5318client # uses-input-w1onlya> worker2: building k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5319client # uses-input-w1onlya> disabling binary cache 'https://cache.nixos.org' for 60 seconds5320worker2 # [ 90.417445] nix-daemon[857]: reaped child process 1182, status = succeeded5321worker2 # [ 90.493307] nix-daemon[857]: accepted connection from pid 1198, user nix-grpc-daemon (trusted)5322worker2 # [ 90.505173] nix-daemon[857]: reaped child process 1201, status = succeeded5323lb # [ 90.520432] niks3-server[974]: 2026/09/24 11:48:40 INFO Received push request method=POST path=/api/pushes5324worker2 # [ 90.529562] nix-grpc-daemon[870]: time=2026-09-24T11:48:40.767Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5325lb # [ 90.562510] niks3-server[974]: 2026/09/24 11:48:41 INFO Received sign narinfos request method=POST path=/api/pushes/30/sign5326lb # [ 90.567228] niks3-server[974]: 2026/09/24 11:48:41 INFO Signed narinfos id=30 count=15327worker2 # [ 90.565787] nix-grpc-daemon[870]: time=2026-09-24T11:48:40.803Z level=INFO msg="Uploading 1 narinfos"5328lb # [ 90.575334] niks3-server[974]: 2026/09/24 11:48:41 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.ls5329lb # [ 90.582270] niks3-server[974]: 2026/09/24 11:48:41 INFO Received complete push request method=POST path=/api/pushes/30/complete5330lb # [ 90.587414] niks3-server[974]: 2026/09/24 11:48:41 INFO Registered completed upload object_key=7fihpafywah8yif4mhbmz6i2i6adiqwj.narinfo5331lb # [ 90.591109] niks3-server[974]: 2026/09/24 11:48:41 INFO Registered completed upload object_key=log/k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv5332worker2 # [ 90.591083] nix-grpc-daemon[870]: time=2026-09-24T11:48:40.828Z level=INFO msg="Upload complete. (182ms)"5333client # copying 1 paths...5334worker2 # [ 90.596484] nix-grpc-daemon[799]: ts=2026-09-24T11:48:40Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=1 drv=k9va03aiirjmgih65p90fpa9mr396d2b-uses-input-w1onlya.drv assign_id=16 outputs=15335worker1 # [ 90.612847] nix-grpc-daemon[524]: ts=2026-09-24T11:48:40Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:565465336worker2 # [ 90.609195] nix-daemon[857]: reaped child process 1179, status = succeeded5337worker1 # [ 90.621507] nix-grpc-daemon[524]: copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'http://lb:5751'...5338client # copying path '/nix/store/7fihpafywah8yif4mhbmz6i2i6adiqwj-uses-input-w1onlya' from 'grpc://lb:50051'...5339worker1 # [ 90.648433] nix-grpc-daemon[524]: ts=2026-09-24T11:48:40Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:565465340worker1 # [ 90.653787] nix-grpc-daemon[524]: ts=2026-09-24T11:48:40Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=0 paths=1 nar_bytes_out=1205341client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt a >&2, in 1.87 seconds)5342client # [ 90.689554] nix-daemon[1129]: reaped child process 1214, status = succeeded5343client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&25344client # [ 90.782919] nix-daemon[1129]: accepted connection from pid 1233, user root (trusted)5345client # this derivation will be built:5346client # /nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5347worker2 # [ 90.924499] nix-grpc-daemon[799]: ts=2026-09-24T11:48:41Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:428945348client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5349worker2 # [ 90.930693] nix-grpc-daemon[799]: ts=2026-09-24T11:48:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428945350worker2 # [ 90.935274] nix-grpc-daemon[870]: time=2026-09-24T11:48:41.172Z level=INFO msg="All 1 paths already cached"5351worker1 # [ 90.960706] nix-grpc-daemon[524]: ts=2026-09-24T11:48:41Z level=debug event=want client=16 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv cached=05352worker1 # [ 90.967208] nix-grpc-daemon[524]: ts=2026-09-24T11:48:41Z level=debug event=assigned drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv worker=192.168.1.3:50051 clients=15353worker1 # [ 90.972272] nix-grpc-daemon[524]: ts=2026-09-24T11:48:41Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565465354worker1 # [ 90.982974] nix-grpc-daemon[524]: ts=2026-09-24T11:48:41Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565465355client # copying 1 paths to 'lb:50051'...5356client # copying path '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' to 'lb:50051'...5357worker1 # [ 90.999471] nix-grpc-daemon[524]: ts=2026-09-24T11:48:41Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:565465358worker1 # [ 91.010156] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5359worker1 # [ 91.025874] nix-grpc-daemon[524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 45 ms (attempt 1/5)5360worker1 # [ 91.077227] nix-grpc-daemon[524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 79 ms (attempt 2/5)5361worker1 # [ 91.157950] nix-grpc-daemon[524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 354 ms (attempt 3/5)5362worker1 # [ 91.513974] nix-grpc-daemon[524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 513 ms (attempt 4/5)5363worker1 # [ 92.029812] nix-grpc-daemon[524]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5364worker1 # [ 92.033690] nix-grpc-daemon[524]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5365worker1 # [ 92.038726] nix-grpc-daemon[524]: copying path '/nix/store/s56ppgyv7qz2bls7ls2jsv2ni3g7fhhr-local-input-w1only.drv' from 'http://lb:5751'...5366worker1 # [ 92.149694] nix-daemon[867]: accepted connection from pid 1264, user nix-grpc-daemon (trusted)5367worker1 # [ 92.166491] nix-daemon[867]: reaped child process 1267, status = succeeded5368lb # [ 92.165060] niks3-server[974]: 2026/09/24 11:48:42 INFO Received push request method=POST path=/api/pushes5369worker1 # [ 92.180508] nix-grpc-daemon[917]: time=2026-09-24T11:48:42.472Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5370worker1 # [ 92.182499] nix-grpc-daemon[917]: time=2026-09-24T11:48:42.473Z level=INFO msg="Uploading 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv (560B)"5371lb # [ 92.196132] niks3-server[974]: 2026/09/24 11:48:42 INFO Registered completed upload object_key=nar/0bp8pfsa12j91bab6f50rs7hh6sphjrig1zqlr7v80mn285qxax2.nar.zst5372lb # [ 92.200274] niks3-server[974]: 2026/09/24 11:48:42 INFO Received sign narinfos request method=POST path=/api/pushes/31/sign5373lb # [ 92.205590] niks3-server[974]: 2026/09/24 11:48:42 INFO Signed narinfos id=31 count=15374worker1 # [ 92.216375] nix-grpc-daemon[917]: time=2026-09-24T11:48:42.508Z level=INFO msg="Uploading 1 narinfos"5375lb # [ 92.216569] niks3-server[974]: 2026/09/24 11:48:42 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.ls5376lb # [ 92.226104] niks3-server[974]: 2026/09/24 11:48:42 INFO Received complete push request method=POST path=/api/pushes/31/complete5377worker1 # [ 92.241781] nix-grpc-daemon[917]: time=2026-09-24T11:48:42.533Z level=INFO msg="Upload complete. (185ms)"5378lb # [ 92.236223] niks3-server[974]: 2026/09/24 11:48:42 INFO Registered completed upload object_key=2ign0w9nlrywl4qfkq5i314zinl1cxqs.narinfo5379worker1 # [ 92.245811] nix-grpc-daemon[524]: ts=2026-09-24T11:48:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=1 paths=1 nar_bytes_in=5605380worker1 # [ 92.251435] nix-grpc-daemon[524]: ts=2026-09-24T11:48:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:565465381worker1 # [ 92.260120] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5382worker1 # [ 92.270884] nix-daemon[867]: reaped child process 1257, status = succeeded5383worker1 # [ 92.273660] nix-daemon[867]: accepted connection from pid 524, user nix-grpc-daemon (trusted)5384client # building '/nix/store/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv' on lb:50051...5385client # uses-input-w1onlyb> node-a: building 2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5386client # uses-input-w1onlyb> disabling binary cache 'https://cache.nixos.org' for 60 seconds5387worker1 # [ 93.319377] nix-daemon[867]: reaped child process 1271, status = succeeded5388worker1 # [ 93.393814] nix-daemon[867]: accepted connection from pid 1287, user nix-grpc-daemon (trusted)5389worker1 # [ 93.405413] nix-daemon[867]: reaped child process 1290, status = succeeded5390lb # [ 93.403632] niks3-server[974]: 2026/09/24 11:48:43 INFO Received push request method=POST path=/api/pushes5391worker1 # [ 93.420962] nix-grpc-daemon[917]: time=2026-09-24T11:48:43.712Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5392lb # [ 93.435495] niks3-server[974]: 2026/09/24 11:48:43 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.ls5393lb # [ 93.442076] niks3-server[974]: 2026/09/24 11:48:43 INFO Received sign narinfos request method=POST path=/api/pushes/32/sign5394lb # [ 93.447643] niks3-server[974]: 2026/09/24 11:48:43 INFO Signed narinfos id=32 count=15395worker1 # [ 93.458373] nix-grpc-daemon[917]: time=2026-09-24T11:48:43.750Z level=INFO msg="Uploading 1 narinfos"5396lb # [ 93.457706] niks3-server[974]: 2026/09/24 11:48:43 INFO Registered completed upload object_key=log/2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv5397lb # [ 93.469087] niks3-server[974]: 2026/09/24 11:48:43 INFO Received complete push request method=POST path=/api/pushes/32/complete5398lb # [ 93.479274] niks3-server[974]: 2026/09/24 11:48:43 INFO Registered completed upload object_key=rs93p5yqksm4zlblqlsfa8kxl8lkhrbg.narinfo5399worker1 # [ 93.490168] nix-grpc-daemon[917]: time=2026-09-24T11:48:43.782Z level=INFO msg="Upload complete. (207ms)"5400worker1 # [ 93.496613] nix-grpc-daemon[524]: ts=2026-09-24T11:48:43Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:56546 duration_s=1 drv=2ign0w9nlrywl4qfkq5i314zinl1cxqs-uses-input-w1onlyb.drv assign_id=17 outputs=15401client # copying 1 paths...5402worker1 # [ 93.506348] nix-grpc-daemon[524]: ts=2026-09-24T11:48:43Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:565465403client # copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'grpc://lb:50051'...5404worker1 # [ 93.515157] nix-daemon[867]: reaped child process 1269, status = succeeded5405worker2 # [ 93.516646] nix-grpc-daemon[799]: ts=2026-09-24T11:48:43Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:428945406worker2 # [ 93.527732] nix-grpc-daemon[799]: copying path '/nix/store/rs93p5yqksm4zlblqlsfa8kxl8lkhrbg-uses-input-w1onlyb' from 'http://lb:5751'...5407worker2 # [ 93.541188] nix-grpc-daemon[799]: ts=2026-09-24T11:48:43Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=1 nar_bytes_out=1205408client # [ 93.585953] nix-daemon[1129]: reaped child process 1236, status = succeeded5409client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt b >&2, in 2.90 seconds)5410client: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&25411client # [ 93.687109] nix-daemon[1129]: accepted connection from pid 1255, user root (trusted)5412client # this derivation will be built:5413client # /nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5414worker1 # [ 93.841688] nix-grpc-daemon[524]: ts=2026-09-24T11:48:44Z level=debug event=rpc_start method=StoreInfo cn=ci-1 peer=ipv4:192.168.1.2:565465415client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux'...5416worker1 # [ 93.847733] nix-grpc-daemon[524]: ts=2026-09-24T11:48:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:565465417worker1 # [ 93.853073] nix-grpc-daemon[917]: time=2026-09-24T11:48:44.143Z level=INFO msg="All 1 paths already cached"5418worker1 # [ 93.862227] nix-grpc-daemon[524]: ts=2026-09-24T11:48:44Z level=debug event=want client=17 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv cached=05419worker1 # [ 93.866969] nix-grpc-daemon[524]: ts=2026-09-24T11:48:44Z level=debug event=assigned drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv worker=192.168.1.4:50051 clients=15420worker2 # [ 93.858330] nix-grpc-daemon[799]: ts=2026-09-24T11:48:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428945421worker2 # [ 93.870469] nix-grpc-daemon[799]: ts=2026-09-24T11:48:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:428945422worker2 # [ 93.884388] nix-grpc-daemon[870]: time=2026-09-24T11:48:44.121Z level=INFO msg="All 1 paths already cached"5423client # copying 1 paths to 'lb:50051'...5424client # copying path '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' to 'lb:50051'...5425worker2 # [ 93.892357] nix-grpc-daemon[799]: ts=2026-09-24T11:48:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:428945426worker2 # [ 93.903351] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5427worker2 # [ 94.009948] nix-daemon[857]: accepted connection from pid 1206, user nix-grpc-daemon (trusted)5428worker2 # [ 94.023432] nix-daemon[857]: reaped child process 1209, status = succeeded5429lb # [ 94.035431] niks3-server[974]: 2026/09/24 11:48:44 INFO Received push request method=POST path=/api/pushes5430worker2 # [ 94.039550] nix-grpc-daemon[870]: time=2026-09-24T11:48:44.277Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5431worker2 # [ 94.041464] nix-grpc-daemon[870]: time=2026-09-24T11:48:44.278Z level=INFO msg="Uploading ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv (560B)"5432lb # [ 94.063991] niks3-server[974]: 2026/09/24 11:48:44 INFO Received sign narinfos request method=POST path=/api/pushes/33/sign5433lb # [ 94.072122] niks3-server[974]: 2026/09/24 11:48:44 INFO Signed narinfos id=33 count=15434worker2 # [ 94.071592] nix-grpc-daemon[870]: time=2026-09-24T11:48:44.309Z level=INFO msg="Uploading 1 narinfos"5435lb # [ 94.078675] niks3-server[974]: 2026/09/24 11:48:44 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.ls5436lb # [ 94.091208] niks3-server[974]: 2026/09/24 11:48:44 INFO Registered completed upload object_key=nar/1iq9qm6vw4b3p68mlkiw8nf0phmz21m8g79izsy14sydxmd4gmqa.nar.zst5437lb # [ 94.095142] niks3-server[974]: 2026/09/24 11:48:44 INFO Received complete push request method=POST path=/api/pushes/33/complete5438worker2 # [ 94.099039] nix-grpc-daemon[870]: time=2026-09-24T11:48:44.336Z level=INFO msg="Upload complete. (183ms)"5439worker2 # [ 94.102694] nix-grpc-daemon[799]: ts=2026-09-24T11:48:44Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=1 nar_bytes_in=5605440lb # [ 94.107814] niks3-server[974]: 2026/09/24 11:48:44 INFO Registered completed upload object_key=ych53lrh5qfmdxlnx672apx2v80x3aan.narinfo5441worker2 # [ 94.108652] nix-grpc-daemon[799]: ts=2026-09-24T11:48:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:428945442worker2 # [ 94.119133] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5443worker2 # [ 94.124102] nix-daemon[857]: reaped child process 1204, status = succeeded5444worker2 # [ 94.127308] nix-daemon[857]: accepted connection from pid 799, user nix-grpc-daemon (trusted)5445client # building '/nix/store/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv' on lb:50051...5446client # uses-input-w1onlyc> worker2: building ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5447client # uses-input-w1onlyc> disabling binary cache 'https://cache.nixos.org' for 60 seconds5448worker2 # [ 95.160488] nix-daemon[857]: reaped child process 1213, status = succeeded5449worker2 # [ 95.249084] nix-daemon[857]: accepted connection from pid 1229, user nix-grpc-daemon (trusted)5450worker2 # [ 95.261862] nix-daemon[857]: reaped child process 1232, status = succeeded5451lb # [ 95.273735] niks3-server[974]: 2026/09/24 11:48:45 INFO Received push request method=POST path=/api/pushes5452worker2 # [ 95.278091] nix-grpc-daemon[870]: time=2026-09-24T11:48:45.515Z level=INFO msg="Uploading 0 paths to lb (1 already cached)"5453lb # [ 95.304059] niks3-server[974]: 2026/09/24 11:48:45 INFO Received sign narinfos request method=POST path=/api/pushes/34/sign5454lb # [ 95.308906] niks3-server[974]: 2026/09/24 11:48:45 INFO Signed narinfos id=34 count=15455worker2 # [ 95.308149] nix-grpc-daemon[870]: time=2026-09-24T11:48:45.545Z level=INFO msg="Uploading 1 narinfos"5456lb # [ 95.316957] niks3-server[974]: 2026/09/24 11:48:45 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.ls5457lb # [ 95.322198] niks3-server[974]: 2026/09/24 11:48:45 INFO Registered completed upload object_key=log/ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv5458lb # [ 95.334883] niks3-server[974]: 2026/09/24 11:48:45 INFO Received complete push request method=POST path=/api/pushes/34/complete5459worker2 # [ 95.339503] nix-grpc-daemon[870]: time=2026-09-24T11:48:45.577Z level=INFO msg="Upload complete. (202ms)"5460lb # [ 95.345590] niks3-server[974]: 2026/09/24 11:48:45 INFO Registered completed upload object_key=af2xkgzqdrcsfmk8dgmypxmxww9cx3rk.narinfo5461worker2 # [ 95.343745] nix-grpc-daemon[799]: ts=2026-09-24T11:48:45Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=1 drv=ych53lrh5qfmdxlnx672apx2v80x3aan-uses-input-w1onlyc.drv assign_id=18 outputs=15462client # copying 1 paths...5463worker2 # [ 95.352854] nix-grpc-daemon[799]: ts=2026-09-24T11:48:45Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:428945464client # copying path '/nix/store/af2xkgzqdrcsfmk8dgmypxmxww9cx3rk-uses-input-w1onlyc' from 'grpc://lb:50051'...5465worker2 # [ 95.360286] nix-daemon[857]: reaped child process 1211, status = succeeded5466worker2 # [ 95.372041] nix-grpc-daemon[799]: ts=2026-09-24T11:48:45Z level=debug event=rpc_start method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:428945467worker2 # [ 95.374163] nix-grpc-daemon[799]: ts=2026-09-24T11:48:45Z level=info event=rpc method=FetchNars cn=ci-1 peer=ipv4:192.168.1.2:42894 duration_s=0 paths=1 nar_bytes_out=1205468lb # [ 95.415481] envoy[913]: conn peer=[2001:db8:1::1]:56286 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3876 tx=2137 ms=14035469lb # [ 95.419100] envoy[913]: conn peer=[2001:db8:1::1]:56296 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=448 tx=357 ms=155470lb # [ 95.422441] envoy[913]: conn peer=[2001:db8:1::1]:56318 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=437 tx=411 ms=505471lb # [ 95.425757] envoy[913]: conn peer=[2001:db8:1::1]:56310 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=4324 tx=2367 ms=16145472lb # [ 95.429201] envoy[913]: conn peer=[2001:db8:1::1]:56320 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3845 tx=2024 ms=15775473lb # [ 95.432807] envoy[913]: conn peer=[2001:db8:1::1]:56336 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="TLS_error:|33554464:system_library:OPENSSL_internal:Broken_pipe:TLS_error_end" rx=421 tx=328 ms=185474lb # [ 95.438369] envoy[913]: conn peer=[2001:db8:1::1]:56342 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3741 tx=2124 ms=26285475lb # [ 95.441803] envoy[913]: conn peer=[2001:db8:1::1]:44524 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=329 ms=385476lb # [ 95.445270] envoy[913]: conn peer=[2001:db8:1::1]:44530 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3725 tx=2107 ms=15555477client # [ 95.417460] nix-daemon[1129]: reaped child process 1258, status = succeeded5478lb # [ 95.448567] envoy[913]: conn peer=[2001:db8:1::1]:44546 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=421 tx=328 ms=135479client: (finished: must succeed: NIX_REMOTE=daemon nix build -L --max-jobs 0 --builders 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&system=x86_64-linux x86_64-linux - 4' --no-link -f /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix job --argstr tag w1only --argstr salt c >&2, in 1.83 seconds)5480(finished: subtest: an input only one worker has still reaches the builder, in 7.09 seconds)5481subtest: upload whose reference the worker lacks is completed from the cache5482client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache5483client # this derivation will be built:5484client # /nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv5485client # building '/nix/store/zpv7g9hl0xcjhjz0g0hd92sv1sf580zh-local-input-viacache.drv'...5486client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A input --argstr tag viacache, in 0.24 seconds)5487client: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache5488client # this derivation will be built:5489client # /nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv5490client # building '/nix/store/8higgmxp1c6ag4yj6sk9g6z66n43x70l-refers-to-input-viacache.drv'...5491client: (finished: must succeed: nix-build --no-out-link /nix/store/f31x0nfj3flb882ga1as2gddqp8fka8w-dep.nix -A referrer --argstr tag viacache, in 0.24 seconds)5492client: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure5493client: (finished: must succeed: nix-store --export /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache > /tmp/shared/ref.closure, in 0.10 seconds)5494worker1: must succeed: nix-store --import < /tmp/shared/ref.closure5495worker1: (finished: must succeed: nix-store --import < /tmp/shared/ref.closure, in 0.13 seconds)5496worker2 # [ 96.196982] systemd[1]: Stopping Nix worker-protocol over gRPC...5497worker2 # [ 96.791193] nix-grpc-daemon[799]: ts=2026-09-24T11:48:47Z level=info event=signal_exit5498worker2 # [ 98.776826] nix-grpc-daemon[799]: ts=2026-09-24T11:48:49Z level=info event=scheduler_restarting5499worker1 # [ 98.798362] nix-grpc-daemon[524]: ts=2026-09-24T11:48:49Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:498265500worker1 # [ 98.805371] nix-grpc-daemon[524]: ts=2026-09-24T11:48:49Z level=info event=worker_gone addr=192.168.1.4:500515501worker2 # [ 98.801348] nix-grpc-daemon[799]: ts=2026-09-24T11:48:49Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5502worker2 # [ 98.821588] nix-daemon[857]: reaped child process 859, status = succeeded5503worker2 # [ 100.576507] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5504worker2 # [ 100.580332] systemd[1]: Stopped Nix worker-protocol over gRPC.5505worker2 # [ 100.583830] systemd[1]: nix-grpc-daemon.service: Consumed 2.046s CPU time over 1min 9.955s wall clock time, 64.2M memory peak, 172.9K incoming IP traffic, 294.7K outgoing IP traffic.5506lb: waiting until exactly ['worker1'] are healthy members of x86_64-linux5507lb: must succeed: curl -sf localhost:9901/clusters5508lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5509lb: (finished: waiting until exactly ['worker1'] are healthy members of x86_64-linux, in 0.05 seconds)5510client: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&25511worker1 # [ 100.799703] nix-grpc-daemon[524]: ts=2026-09-24T11:48:51Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:565465512worker1 # [ 100.806636] nix-grpc-daemon[524]: ts=2026-09-24T11:48:51Z level=debug event=rpc_start method=QueryPathInfos cn=ci-1 peer=ipv4:192.168.1.2:565465513client # /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5514client: (finished: must succeed: nix path-info --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache >&2, in 0.17 seconds)5515worker2: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache5516worker2: (finished: must fail: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache, in 0.02 seconds)5517worker2 # [ 100.879643] systemd[1]: Starting scheduler-token.service...5518worker2 # [ 100.916151] systemd[1]: scheduler-token.service: Deactivated successfully.5519worker2 # [ 100.917704] systemd[1]: Finished scheduler-token.service.5520worker2 # [ 100.920148] systemd[1]: scheduler-token.service: Consumed 16ms CPU time over 37ms wall clock time, 2.3M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.5521worker2 # [ 100.923289] systemd[1]: Starting Nix worker-protocol over gRPC...5522worker2 # [ 101.003062] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:51Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5523worker2 # [ 101.005126] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:51Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5524worker2 # [ 101.008502] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:51Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=15525worker2 # [ 101.012751] nix-grpc-daemon[1256]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5526worker2 # [ 101.014138] nix-grpc-daemon[1256]: E0000 00:00:1790250531.250334 1256 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5527worker2 # [ 101.018147] nix-grpc-daemon[1256]: E0000 00:00:1790250531.255823 1256 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5528worker2 # [ 101.022591] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:51Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5529worker2 # [ 101.025132] systemd[1]: Started Nix worker-protocol over gRPC.5530worker2 # [ 101.031110] nix-grpc-daemon[1256]: E0000 00:00:1790250531.267918 1299 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5531worker2 # [ 101.043087] nix-grpc-daemon[1256]: E0000 00:00:1790250531.280868 1299 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5532worker2 # [ 101.047056] nix-grpc-daemon[1256]: E0000 00:00:1790250531.283112 1299 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5533worker2 # [ 101.057040] nix-grpc-daemon[1256]: E0000 00:00:1790250531.294752 1299 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5534worker2 # [ 101.066459] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:51Z level=info event=scheduler_connected addr=lb:500515535worker1 # [ 101.083565] nix-grpc-daemon[524]: ts=2026-09-24T11:48:51Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:498265536worker1 # [ 101.090501] nix-grpc-daemon[524]: ts=2026-09-24T11:48:51Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05537worker1 # [ 101.098774] systemd[1]: Stopping Nix worker-protocol over gRPC...5538worker1 # [ 101.168291] nix-grpc-daemon[524]: ts=2026-09-24T11:48:51Z level=info event=signal_exit5539lb # [ 101.316855] niks3-server[974]: 2026/09/24 11:48:51 INFO lead: released remote=192.168.1.3:426885540worker1 # [ 102.323956] nix-grpc-daemon[524]: ts=2026-09-24T11:48:52Z level=info event=scheduler_restarting5541lb # [ 102.315720] envoy[913]: [2026-09-24 11:48:52.758][913][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5542worker1 # [ 102.330533] nix-grpc-daemon[524]: ts=2026-09-24T11:48:52Z level=info event=scheduler_restarting addr=lb:500515543worker2 # [ 102.320848] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:52Z level=info event=scheduler_restarting addr=lb:500515544worker1 # [ 102.337778] nix-grpc-daemon[524]: ts=2026-09-24T11:48:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:498265545worker1 # [ 102.341571] nix-grpc-daemon[524]: ts=2026-09-24T11:48:52Z level=info event=worker_gone addr=192.168.1.3:500515546worker1 # [ 102.345120] nix-grpc-daemon[524]: ts=2026-09-24T11:48:52Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:498265547worker1 # [ 102.348407] nix-grpc-daemon[524]: ts=2026-09-24T11:48:52Z level=info event=worker_gone addr=192.168.1.4:500515548worker1 # [ 102.370671] nix-daemon[867]: reaped child process 869, status = succeeded5549worker2 # [ 102.521176] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:52Z level=info event=scheduler_connected addr=lb:500515550worker2 # [ 102.525223] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5551worker1 # [ 102.545340] nix-grpc-daemon[524]: ts=2026-09-24T11:48:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5552worker2 # [ 102.726337] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:52Z level=info event=scheduler_connected addr=lb:500515553worker2 # [ 102.730274] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:52Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5554worker2 # [ 102.931345] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_connected addr=lb:500515555worker2 # [ 102.935278] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5556worker2 # [ 103.136307] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_connected addr=lb:500515557worker2 # [ 103.140337] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5558worker2 # [ 103.342221] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_connected addr=lb:500515559worker2 # [ 103.346153] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5560worker2 # [ 103.547746] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_connected addr=lb:500515561worker2 # [ 103.551763] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5562worker2 # [ 103.752910] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_connected addr=lb:500515563worker2 # [ 103.756904] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:53Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5564worker2 # [ 103.958151] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=info event=scheduler_connected addr=lb:500515565worker2 # [ 103.962197] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5566worker2 # [ 104.163133] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=info event=scheduler_connected addr=lb:500515567worker2 # [ 104.167159] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5568worker2 # [ 104.368934] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=info event=scheduler_connected addr=lb:500515569worker2 # [ 104.372848] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5570worker1 # [ 104.553652] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5571worker1 # [ 104.557969] systemd[1]: Stopped Nix worker-protocol over gRPC.5572worker1 # [ 104.560542] systemd[1]: nix-grpc-daemon.service: Consumed 2.353s CPU time over 1min 36.655s wall clock time, 66.9M memory peak, 440.5K incoming IP traffic, 387.1K outgoing IP traffic.5573lb: waiting until exactly ['worker2'] are healthy members of x86_64-linux5574lb: must succeed: curl -sf localhost:9901/clusters5575worker2 # [ 104.574257] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=info event=scheduler_connected addr=lb:500515576worker2 # [ 104.578154] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5577lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5578lb: (finished: waiting until exactly ['worker2'] are healthy members of x86_64-linux, in 0.05 seconds)5579client: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&25580worker2 # [ 104.758702] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575145581worker2 # [ 104.770396] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5582worker2 # [ 104.778867] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_connected addr=lb:500515583worker2 # [ 104.782582] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5584worker2 # [ 104.794758] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 47 ms (attempt 1/5)5585worker2 # [ 104.849369] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 36 ms (attempt 2/5)5586worker2 # [ 104.887124] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 282 ms (attempt 3/5)5587worker2 # [ 104.983822] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_connected addr=lb:500515588worker2 # [ 104.987797] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5589worker2 # [ 105.171279] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 78 ms (attempt 4/5)5590worker2 # [ 105.188718] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_connected addr=lb:500515591worker2 # [ 105.192583] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5592worker2 # [ 105.251580] nix-grpc-daemon[1256]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5593worker2 # [ 105.254765] nix-grpc-daemon[1256]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5594client # copying 2 paths...5595client # copying path '/nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache' to 'grpc://lb:50051'...5596worker2 # [ 105.276762] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575145597client # copying path '/nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache' to 'grpc://lb:50051'...5598worker2 # [ 105.289169] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5599worker2 # [ 105.311903] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=niks3_push_started pid=13165600worker2 # [ 105.393983] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_connected addr=lb:500515601worker2 # [ 105.398359] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5602lb # [ 105.415653] envoy[913]: conn peer=[2001:db8:1::4]:57146 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3000 tx=1021 ms=679785603lb # [ 105.419514] envoy[913]: conn peer=[2001:db8:1::1]:44560 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=591 tx=522 ms=215604lb # [ 105.422968] envoy[913]: conn peer=192.168.1.3:44248 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=2327 tx=1039 ms=693485605worker2 # [ 105.420278] nix-grpc-daemon[1316]: time=2026-09-24T11:48:55.657Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""5606worker2 # [ 105.509115] nix-daemon[857]: accepted connection from pid 1329, user nix-grpc-daemon (trusted)5607worker2 # [ 105.521929] nix-daemon[857]: reaped child process 1332, status = succeeded5608lb # [ 105.533601] niks3-server[974]: 2026/09/24 11:48:55 INFO Received push request method=POST path=/api/pushes5609worker2 # [ 105.541152] nix-grpc-daemon[1316]: time=2026-09-24T11:48:55.778Z level=INFO msg="Uploading 2 paths to lb (0 already cached)"5610worker2 # [ 105.542725] nix-grpc-daemon[1316]: time=2026-09-24T11:48:55.780Z level=INFO msg="Uploading 6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache (184B)"5611worker2 # [ 105.545046] nix-grpc-daemon[1316]: time=2026-09-24T11:48:55.782Z level=INFO msg="Uploading cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache (128B)"5612lb # [ 105.595296] niks3-server[974]: 2026/09/24 11:48:56 INFO Registered completed upload object_key=nar/15y5akaw5jxrf5q28c25cfgfdz5zw27ihvpagi1mmvpnaw9na2a5.nar.zst5613lb # [ 105.598251] niks3-server[974]: 2026/09/24 11:48:56 INFO Registered completed upload object_key=nar/1kmnh5f2l7jjcgaq9b9988bjix6z8zjf6v350xllq5hlzgnb1f75.nar.zst5614worker2 # [ 105.596928] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_connected addr=lb:500515615worker2 # [ 105.598649] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5616lb # [ 105.605288] niks3-server[974]: 2026/09/24 11:48:56 INFO Received sign narinfos request method=POST path=/api/pushes/35/sign5617lb # [ 105.607379] niks3-server[974]: 2026/09/24 11:48:56 INFO Signed narinfos id=35 count=25618worker2 # [ 105.604378] nix-grpc-daemon[1316]: time=2026-09-24T11:48:55.841Z level=INFO msg="Uploading 2 narinfos"5619lb # [ 105.611099] niks3-server[974]: 2026/09/24 11:48:56 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.ls5620lb # [ 105.619713] niks3-server[974]: 2026/09/24 11:48:56 INFO Received complete push request method=POST path=/api/pushes/35/complete5621lb # [ 105.623468] niks3-server[974]: 2026/09/24 11:48:56 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.ls5622worker2 # [ 105.625342] nix-grpc-daemon[1316]: time=2026-09-24T11:48:55.863Z level=INFO msg="Upload complete. (201ms)"5623lb # [ 105.630672] niks3-server[974]: 2026/09/24 11:48:56 INFO Registered completed upload object_key=cmsrl1s0jpx66sskcbsv9jgl7ryk8101.narinfo5624worker2 # [ 105.627700] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=0 paths=2 nar_bytes_in=3125625lb # [ 105.633410] niks3-server[974]: 2026/09/24 11:48:56 INFO Registered completed upload object_key=6zp5fijxpjrd2vv7855sb0z7aag6mm8d.narinfo5626worker2 # [ 105.634153] nix-daemon[857]: reaped child process 1314, status = succeeded5627client: (finished: must succeed: nix copy --no-check-sigs --to 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache >&2, in 1.05 seconds)5628worker2: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache5629worker2: (finished: must succeed: test -e /nix/store/cmsrl1s0jpx66sskcbsv9jgl7ryk8101-local-input-viacache && test -e /nix/store/6zp5fijxpjrd2vv7855sb0z7aag6mm8d-refers-to-input-viacache, in 0.02 seconds)5630worker1 # [ 105.735688] systemd[1]: Starting Nix worker-protocol over gRPC...5631worker2 # [ 105.800130] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515632worker2 # [ 105.804163] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5633worker1 # [ 105.820123] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"5634worker1 # [ 105.827590] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc5635worker1 # [ 105.831104] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=15636worker1 # [ 105.835844] nix-grpc-daemon[1315]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR5637worker1 # [ 105.837912] nix-grpc-daemon[1315]: E0000 00:00:1790250536.117522 1315 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5638lb # [ 105.831487] niks3-server[974]: 2026/09/24 11:48:56 INFO lead: acquired remote=192.168.1.3:455745639worker1 # [ 105.842066] nix-grpc-daemon[1315]: E0000 00:00:1790250536.119469 1315 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5640worker1 # [ 105.848131] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket5641worker1 # [ 105.850110] nix-grpc-daemon[1315]: E0000 00:00:1790250536.133024 1358 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5642worker1 # [ 105.852314] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_take_over5643worker1 # [ 105.857137] systemd[1]: Started Nix worker-protocol over gRPC.5644worker1 # [ 105.860574] nix-grpc-daemon[1315]: E0000 00:00:1790250536.152526 1358 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5645lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux5646lb: must succeed: curl -sf localhost:9901/clusters5647worker1 # [ 105.865307] nix-grpc-daemon[1315]: E0000 00:00:1790250536.154933 1358 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.5648worker1 # [ 105.874945] nix-grpc-daemon[1315]: E0000 00:00:1790250536.166930 1358 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.5649worker1 # [ 105.887772] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515650worker1 # [ 105.889355] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5651lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5652worker2 # [ 106.005319] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515653worker2 # [ 106.009165] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5654worker1 # [ 106.090359] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515655worker1 # [ 106.094416] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5656worker2 # [ 106.210326] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515657worker2 # [ 106.214249] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5658worker1 # [ 106.295324] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515659worker1 # [ 106.299306] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5660worker2 # [ 106.416267] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515661worker2 # [ 106.419797] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5662worker1 # [ 106.500265] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515663worker1 # [ 106.504267] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5664worker2 # [ 106.620944] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515665worker2 # [ 106.625090] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:56Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5666worker1 # [ 106.704898] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:56Z level=info event=scheduler_connected addr=lb:500515667worker1 # [ 106.712132] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:540545668worker1 # [ 106.715709] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05669worker2 # [ 106.826146] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:57Z level=info event=scheduler_connected addr=lb:500515670worker1 # [ 106.845735] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=info event=oidc_jwks provider=mock keys=15671worker1 # [ 106.849957] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:540545672worker1 # [ 106.853994] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05673lb: must succeed: curl -sf localhost:9901/clusters5674lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5675lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 1.10 seconds)5676(finished: subtest: upload whose reference the worker lacks is completed from the cache, in 11.50 seconds)5677subtest: a niks3 restart does not cost the leader its role5678lb: must succeed: curl -sf localhost:9901/clusters5679lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)5680lb: must succeed: curl -sf localhost:9901/clusters5681lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5682lb # [ 107.023268] systemd[1]: Stopping niks3 server...5683lb # [ 107.026156] niks3-server[974]: 2026/09/24 11:48:57 INFO Shutdown signal received, draining in-flight requests timeout=10s5684worker2 # [ 107.022566] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:57Z level=info event=scheduler_lock_lost status=2005685lb # [ 107.030653] niks3-server[974]: 2026/09/24 11:48:57 INFO lead: released remote=192.168.1.3:455745686worker1 # [ 107.038359] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=info event=scheduler_lock_lost status=2005687lb # [ 107.039275] systemd[1]: niks3.service: Deactivated successfully.5688lb # [ 107.041235] systemd[1]: Stopped niks3 server.5689lb # [ 107.041897] systemd[1]: niks3.service: Consumed 750ms CPU time over 1min 27.960s wall clock time, 28.6M memory peak, 98K incoming IP traffic, 141.3K outgoing IP traffic.5690lb # [ 107.048704] systemd[1]: Starting niks3 server...5691lb # [ 107.096894] niks3-server[1150]: 2026/09/24 11:48:57 goose: no migrations to run. current version: 202609231200005692lb # [ 107.103278] niks3-server[1150]: 2026/09/24 11:48:57 OK 1_commit_pending_closure.sql (4.37ms)5693lb # [ 107.106809] niks3-server[1150]: 2026/09/24 11:48:57 OK 2_object_stats_trigger.sql (3.5ms)5694lb # [ 107.110107] niks3-server[1150]: 2026/09/24 11:48:57 OK 3_commit_push.sql (3.36ms)5695lb # [ 107.111267] niks3-server[1150]: 2026/09/24 11:48:57 goose: up to current file version: 35696lb # [ 107.113063] niks3-server[1150]: 2026/09/24 11:48:57 INFO Loaded signing key name=farm-test-1 path=/nix/store/hk0nhypl37jvkn3gr36v3y7qx593p39i-key5697lb # [ 107.123280] niks3-server[1150]: 2026/09/24 11:48:57 INFO Read proxy enabled — serving cache objects from S35698lb # [ 107.125497] niks3-server[1150]: 2026/09/24 11:48:57 INFO Using socket-activated listener address=0.0.0.0:57515699lb # [ 107.128238] niks3-server[1150]: 2026/09/24 11:48:57 INFO systemd watchdog enabled interval=15s5700lb # [ 107.129560] niks3-server[1150]: 2026/09/24 11:48:57 INFO Starting HTTP server address=0.0.0.0:57515701lb # [ 107.130961] systemd[1]: Started niks3 server.5702lb: waiting for unit niks3.service5703lb: (finished: waiting for unit niks3.service, in 0.02 seconds)5704lb: must succeed: curl -sf localhost:9901/clusters5705lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5706worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics5707worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)5708lb: must succeed: curl -sf localhost:9901/clusters5709lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5710lb: must succeed: curl -sf localhost:9901/clusters5711lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5712worker1: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield5713worker1: (finished: must fail: journalctl -u nix-grpc-daemon | grep -q event=scheduler_yield, in 0.04 seconds)5714client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&25715worker2 # [ 107.443691] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:57Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575145716client # don't know how to build these paths:5717client # /nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5718worker2 # [ 107.452553] nix-grpc-daemon[1256]: ts=2026-09-24T11:48:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575145719worker1 # [ 107.488618] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=debug event=want client=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv cached=05720worker1 # [ 107.494295] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=debug event=assigned drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv worker=192.168.1.3:50051 clients=15721worker1 # [ 107.502829] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540585722worker1 # [ 107.514125] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5723worker1 # [ 107.529782] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 80 ms (attempt 1/5)5724worker1 # [ 107.616445] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 85 ms (attempt 2/5)5725worker1 # [ 107.703353] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 368 ms (attempt 3/5)5726lb # [ 108.035879] niks3-server[1150]: 2026/09/24 11:48:58 INFO lead: acquired remote=192.168.1.3:455925727worker1 # [ 108.073694] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 11 ms (attempt 4/5)5728worker1 # [ 108.085957] nix-grpc-daemon[1315]: disabling binary cache 'https://cache.nixos.org' for 60 seconds5729worker1 # [ 108.089225] nix-grpc-daemon[1315]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org5730worker1 # [ 108.105959] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:58Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540585731client # copying 1 paths to 'lb:50051'...5732client # copying path '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' to 'lb:50051'...5733worker1 # [ 108.119846] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:58Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540585734worker1 # [ 108.130073] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5735worker1 # [ 108.150600] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:58Z level=info event=niks3_push_started pid=13875736worker1 # [ 108.242810] nix-grpc-daemon[1387]: time=2026-09-24T11:48:58.534Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""5737worker1 # [ 108.332385] nix-daemon[867]: accepted connection from pid 1399, user nix-grpc-daemon (trusted)5738worker1 # [ 108.346170] nix-daemon[867]: reaped child process 1402, status = succeeded5739lb # [ 108.343564] niks3-server[1150]: 2026/09/24 11:48:58 INFO Received push request method=POST path=/api/pushes5740worker1 # [ 108.364025] nix-grpc-daemon[1387]: time=2026-09-24T11:48:58.655Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5741worker1 # [ 108.365943] nix-grpc-daemon[1387]: time=2026-09-24T11:48:58.657Z level=INFO msg="Uploading 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv (536B)"5742lb # [ 108.384181] niks3-server[1150]: 2026/09/24 11:48:58 INFO Received sign narinfos request method=POST path=/api/pushes/36/sign5743lb # [ 108.409028] niks3-server[1150]: 2026/09/24 11:48:58 INFO Signed narinfos id=36 count=15744worker1 # [ 108.419035] nix-grpc-daemon[1387]: time=2026-09-24T11:48:58.710Z level=INFO msg="Uploading 1 narinfos"5745lb # [ 108.418247] niks3-server[1150]: 2026/09/24 11:48:58 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.ls5746lb # [ 108.421901] niks3-server[1150]: 2026/09/24 11:48:58 INFO Received complete push request method=POST path=/api/pushes/36/complete5747lb # [ 108.423831] niks3-server[1150]: 2026/09/24 11:48:58 INFO Registered completed upload object_key=nar/0wjrljh500qj2dh74gryz4ik45bn1iv9w72490z0f2w0l2x4c87h.nar.zst5748lb # [ 108.432614] niks3-server[1150]: 2026/09/24 11:48:58 INFO Registered completed upload object_key=6finahhb2q4vla8rjb6gi4z4cvf71qmc.narinfo5749worker1 # [ 108.443712] nix-grpc-daemon[1387]: time=2026-09-24T11:48:58.735Z level=INFO msg="Upload complete. (197ms)"5750worker1 # [ 108.448527] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:58Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54058 duration_s=0 paths=1 nar_bytes_in=5365751worker1 # [ 108.454353] nix-grpc-daemon[1315]: ts=2026-09-24T11:48:58Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540585752worker1 # [ 108.467826] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5753worker1 # [ 108.475198] nix-daemon[867]: reaped child process 1385, status = succeeded5754worker1 # [ 108.478668] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5755client # building '/nix/store/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv' on lb:50051...5756client # farm-a> node-a: building 6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5757client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5758client # farm-a> LOG-farm-a-after-niks3-restart5759worker1 # [ 109.694417] nix-daemon[867]: reaped child process 1407, status = succeeded5760worker1 # [ 109.770886] nix-daemon[867]: accepted connection from pid 1423, user nix-grpc-daemon (trusted)5761worker1 # [ 109.782383] nix-daemon[867]: reaped child process 1426, status = succeeded5762lb # [ 109.783076] niks3-server[1150]: 2026/09/24 11:49:00 INFO Received push request method=POST path=/api/pushes5763worker1 # [ 109.801218] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.092Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5764worker1 # [ 109.802991] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.094Z level=INFO msg="Uploading vllq1krpcvis69h349hy98vfz55ld1wv-farm-a (144B)"5765lb # [ 109.823139] niks3-server[1150]: 2026/09/24 11:49:00 INFO Registered completed upload object_key=nar/1diw08qzlsh0zkrx4kvj1mvbc57g7r65j1f459ankpg9yc5rgcpx.nar.zst5766lb # [ 109.833692] niks3-server[1150]: 2026/09/24 11:49:00 INFO Registered completed upload object_key=log/6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv5767lb # [ 109.841826] niks3-server[1150]: 2026/09/24 11:49:00 INFO Received sign narinfos request method=POST path=/api/pushes/37/sign5768worker1 # [ 109.855852] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.147Z level=INFO msg="Uploading 1 narinfos"5769lb # [ 109.849125] niks3-server[1150]: 2026/09/24 11:49:00 INFO Signed narinfos id=37 count=15770lb # [ 109.855792] niks3-server[1150]: 2026/09/24 11:49:00 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.ls5771lb # [ 109.865148] niks3-server[1150]: 2026/09/24 11:49:00 INFO Received complete push request method=POST path=/api/pushes/37/complete5772worker1 # [ 109.876861] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.168Z level=INFO msg="Upload complete. (191ms)"5773lb # [ 109.870477] niks3-server[1150]: 2026/09/24 11:49:00 INFO Registered completed upload object_key=vllq1krpcvis69h349hy98vfz55ld1wv.narinfo5774worker1 # [ 109.879696] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54058 duration_s=1 drv=6finahhb2q4vla8rjb6gi4z4cvf71qmc-farm-a.drv assign_id=1 outputs=15775worker1 # [ 109.888137] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=want client=1 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv cached=05776worker1 # [ 109.890965] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=assigned drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv worker=192.168.1.3:50051 clients=15777worker1 # [ 109.892902] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540585778worker1 # [ 109.896987] nix-daemon[867]: reaped child process 1405, status = succeeded5779worker1 # [ 109.900565] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540585780worker1 # [ 109.906963] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.198Z level=INFO msg="All 1 paths already cached"5781client # copying 1 paths to 'lb:50051'...5782worker1 # [ 109.912097] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540585783client # copying path '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' to 'lb:50051'...5784worker1 # [ 109.918150] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5785worker1 # [ 110.003217] nix-daemon[867]: accepted connection from pid 1432, user nix-grpc-daemon (trusted)5786worker1 # [ 110.016907] nix-daemon[867]: reaped child process 1435, status = succeeded5787worker1 # [ 110.021679] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:540645788lb # [ 110.019908] niks3-server[1150]: 2026/09/24 11:49:00 INFO Received push request method=POST path=/api/pushes5789worker1 # [ 110.038098] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.329Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5790worker1 # [ 110.039512] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.329Z level=INFO msg="Uploading 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv (656B)"5791lb # [ 110.053418] niks3-server[1150]: 2026/09/24 11:49:00 INFO Received sign narinfos request method=POST path=/api/pushes/38/sign5792lb # [ 110.059383] niks3-server[1150]: 2026/09/24 11:49:00 INFO Signed narinfos id=38 count=15793worker1 # [ 110.070071] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.361Z level=INFO msg="Uploading 1 narinfos"5794lb # [ 110.065090] niks3-server[1150]: 2026/09/24 11:49:00 INFO Registered completed upload object_key=nar/0sjnwmbwn7i8g7g7czwn9r2pzaaa1j1yx3dfa4gw29ynhmpz9v3j.nar.zst5795lb # [ 110.076123] niks3-server[1150]: 2026/09/24 11:49:00 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.ls5796lb # [ 110.079535] niks3-server[1150]: 2026/09/24 11:49:00 INFO Received complete push request method=POST path=/api/pushes/38/complete5797lb # [ 110.086457] niks3-server[1150]: 2026/09/24 11:49:00 INFO Registered completed upload object_key=5g2krgx4inyydhy0jrhhylv4xhhlm7j7.narinfo5798worker1 # [ 110.097356] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.389Z level=INFO msg="Upload complete. (174ms)"5799worker1 # [ 110.099385] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54058 duration_s=0 paths=1 nar_bytes_in=6565800worker1 # [ 110.102346] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540585801worker1 # [ 110.107318] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5802worker1 # [ 110.112171] nix-daemon[867]: reaped child process 1430, status = succeeded5803worker1 # [ 110.116558] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5804client # building '/nix/store/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv' on lb:50051...5805client # farm-b> node-a: building 5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5806client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds5807client # farm-b> LOG-farm-b-after-niks3-restart5808worker1 # [ 110.526638] nix-daemon[867]: reaped child process 1439, status = succeeded5809worker1 # [ 110.605595] nix-daemon[867]: accepted connection from pid 1455, user nix-grpc-daemon (trusted)5810worker1 # [ 110.618600] nix-daemon[867]: reaped child process 1458, status = succeeded5811lb # [ 110.617817] niks3-server[1150]: 2026/09/24 11:49:01 INFO Received push request method=POST path=/api/pushes5812worker1 # [ 110.632890] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.924Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"5813worker1 # [ 110.634351] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.924Z level=INFO msg="Uploading vkqg7lcax8h1mxs0g750ficqiii1falw-farm-b (192B)"5814lb # [ 110.643300] niks3-server[1150]: 2026/09/24 11:49:01 INFO Received sign narinfos request method=POST path=/api/pushes/39/sign5815lb # [ 110.645518] niks3-server[1150]: 2026/09/24 11:49:01 INFO Signed narinfos id=39 count=15816worker1 # [ 110.654402] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.946Z level=INFO msg="Uploading 1 narinfos"5817lb # [ 110.649129] niks3-server[1150]: 2026/09/24 11:49:01 INFO Registered completed upload object_key=log/5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv5818lb # [ 110.656428] niks3-server[1150]: 2026/09/24 11:49:01 INFO Registered completed upload object_key=nar/12zl6l8xhwx5hdivnvf1vw18yzvz3764mhdd9n03x81phzqblxj0.nar.zst5819lb # [ 110.660545] niks3-server[1150]: 2026/09/24 11:49:01 INFO Received complete push request method=POST path=/api/pushes/39/complete5820worker1 # [ 110.672233] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.964Z level=INFO msg="Upload complete. (149ms)"5821lb # [ 110.665819] niks3-server[1150]: 2026/09/24 11:49:01 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.ls5822worker1 # [ 110.674378] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54058 duration_s=0 drv=5g2krgx4inyydhy0jrhhylv4xhhlm7j7-farm-b.drv assign_id=2 outputs=15823lb # [ 110.669901] niks3-server[1150]: 2026/09/24 11:49:01 INFO Registered completed upload object_key=vkqg7lcax8h1mxs0g750ficqiii1falw.narinfo5824worker1 # [ 110.680315] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=want client=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv cached=05825worker1 # [ 110.683129] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=assigned drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv worker=192.168.1.3:50051 clients=15826worker1 # [ 110.685463] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540585827worker1 # [ 110.691460] nix-daemon[867]: reaped child process 1437, status = succeeded5828worker1 # [ 110.692750] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:540585829worker1 # [ 110.698936] nix-grpc-daemon[1387]: time=2026-09-24T11:49:00.990Z level=INFO msg="All 2 paths already cached"5830client # copying 1 paths to 'lb:50051'...5831worker1 # [ 110.703927] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:00Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:540585832client # copying path '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' to 'lb:50051'...5833worker1 # [ 110.710196] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5834worker1 # [ 110.794396] nix-daemon[867]: accepted connection from pid 1464, user nix-grpc-daemon (trusted)5835worker1 # [ 110.807649] nix-daemon[867]: reaped child process 1467, status = succeeded5836lb # [ 110.807767] niks3-server[1150]: 2026/09/24 11:49:01 INFO Received push request method=POST path=/api/pushes5837worker1 # [ 110.824702] nix-grpc-daemon[1387]: time=2026-09-24T11:49:01.116Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5838worker1 # [ 110.826288] nix-grpc-daemon[1387]: time=2026-09-24T11:49:01.116Z level=INFO msg="Uploading v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv (784B)"5839lb # [ 110.838199] niks3-server[1150]: 2026/09/24 11:49:01 INFO Registered completed upload object_key=nar/0ppnnsmr641nka691g6vrp1crps1bbn31zm7nc5za9pl2y2iwb0f.nar.zst5840lb # [ 110.845472] niks3-server[1150]: 2026/09/24 11:49:01 INFO Received sign narinfos request method=POST path=/api/pushes/40/sign5841lb # [ 110.849927] niks3-server[1150]: 2026/09/24 11:49:01 INFO Signed narinfos id=40 count=15842worker1 # [ 110.860834] nix-grpc-daemon[1387]: time=2026-09-24T11:49:01.152Z level=INFO msg="Uploading 1 narinfos"5843lb # [ 110.859296] niks3-server[1150]: 2026/09/24 11:49:01 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.ls5844lb # [ 110.872084] niks3-server[1150]: 2026/09/24 11:49:01 INFO Received complete push request method=POST path=/api/pushes/40/complete5845worker1 # [ 110.882485] nix-grpc-daemon[1387]: time=2026-09-24T11:49:01.174Z level=INFO msg="Upload complete. (167ms)"5846lb # [ 110.876665] niks3-server[1150]: 2026/09/24 11:49:01 INFO Registered completed upload object_key=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c.narinfo5847worker1 # [ 110.885739] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:54058 duration_s=0 paths=1 nar_bytes_in=7845848worker1 # [ 110.891282] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:540585849worker1 # [ 110.900877] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5850worker1 # [ 110.904949] nix-daemon[867]: reaped child process 1462, status = succeeded5851worker1 # [ 110.909374] nix-daemon[867]: accepted connection from pid 1315, user nix-grpc-daemon (trusted)5852client # building '/nix/store/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv' on lb:50051...5853client # farm-top> node-a: building v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5854client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds5855client # farm-top> LOG-farm-top-after-niks3-restart5856worker1 # [ 111.738434] nix-daemon[867]: reaped child process 1471, status = succeeded5857worker1 # [ 111.806171] nix-daemon[867]: accepted connection from pid 1487, user nix-grpc-daemon (trusted)5858worker1 # [ 111.818883] nix-daemon[867]: reaped child process 1490, status = succeeded5859lb # [ 111.818271] niks3-server[1150]: 2026/09/24 11:49:02 INFO Received push request method=POST path=/api/pushes5860worker1 # [ 111.835263] nix-grpc-daemon[1387]: time=2026-09-24T11:49:02.126Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"5861worker1 # [ 111.838060] nix-grpc-daemon[1387]: time=2026-09-24T11:49:02.128Z level=INFO msg="Uploading lyi0sw9pvmhznkzgfl39callq9c0n87r-farm-top (248B)"5862lb # [ 111.852398] niks3-server[1150]: 2026/09/24 11:49:02 INFO Registered completed upload object_key=log/v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv5863lb # [ 111.861646] niks3-server[1150]: 2026/09/24 11:49:02 INFO Received sign narinfos request method=POST path=/api/pushes/41/sign5864worker1 # [ 111.874788] nix-grpc-daemon[1387]: time=2026-09-24T11:49:02.166Z level=INFO msg="Uploading 1 narinfos"5865lb # [ 111.867103] niks3-server[1150]: 2026/09/24 11:49:02 INFO Signed narinfos id=41 count=15866lb # [ 111.874431] niks3-server[1150]: 2026/09/24 11:49:02 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.ls5867lb # [ 111.878816] niks3-server[1150]: 2026/09/24 11:49:02 INFO Received complete push request method=POST path=/api/pushes/41/complete5868lb # [ 111.882904] niks3-server[1150]: 2026/09/24 11:49:02 INFO Registered completed upload object_key=nar/141220x0digv3f58fbh9c79avwg0fwgsk5jyac9ymkxmr8zs2prw.nar.zst5869lb # [ 111.886509] niks3-server[1150]: 2026/09/24 11:49:02 INFO Registered completed upload object_key=lyi0sw9pvmhznkzgfl39callq9c0n87r.narinfo5870worker1 # [ 111.897094] nix-grpc-daemon[1387]: time=2026-09-24T11:49:02.189Z level=INFO msg="Upload complete. (181ms)"5871worker1 # [ 111.902870] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:02Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:54058 duration_s=1 drv=v0ppd6w8hc0jlggdq33p3mhzyjib8r6c-farm-top.drv assign_id=3 outputs=15872worker1 # [ 111.915484] nix-daemon[867]: reaped child process 1469, status = succeeded5873client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart >&2, in 4.63 seconds)5874client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath5875client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag after-niks3-restart outPath, in 0.12 seconds)5876(finished: subtest: a niks3 restart does not cost the leader its role, in 5.10 seconds)5877subtest: scheduler down: the other node takes the lock and keeps it5878lb: must succeed: curl -sf localhost:9901/clusters5879lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)5880lb: must succeed: curl -sf localhost:9901/clusters5881lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5882lb: must succeed: curl -sf localhost:9901/clusters5883lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5884lb: must succeed: curl -sf localhost:9901/clusters5885lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)5886worker1 # [ 112.217734] systemd[1]: Stopping Nix worker-protocol over gRPC...5887worker1 # [ 112.832110] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:03Z level=info event=signal_exit5888lb # [ 113.038872] niks3-server[1150]: 2026/09/24 11:49:03 INFO lead: released remote=192.168.1.3:455925889worker1 # [ 114.047740] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:04Z level=info event=scheduler_restarting5890lb # [ 114.039252] envoy[913]: [2026-09-24 11:49:04.482][913][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}5891worker1 # [ 114.055304] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:04Z level=info event=scheduler_restarting addr=lb:500515892worker2 # [ 114.044171] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:04Z level=info event=scheduler_restarting addr=lb:500515893worker1 # [ 114.061630] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540545894worker1 # [ 114.065435] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:04Z level=info event=worker_gone addr=192.168.1.4:500515895worker1 # [ 114.068509] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:04Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:540545896worker1 # [ 114.072463] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:04Z level=info event=worker_gone addr=192.168.1.3:500515897worker1 # [ 114.091711] nix-daemon[867]: reaped child process 1377, status = succeeded5898worker2 # [ 114.244230] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:04Z level=info event=scheduler_connected addr=lb:500515899worker2 # [ 114.248400] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5900worker1 # [ 114.268227] nix-grpc-daemon[1315]: ts=2026-09-24T11:49:04Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"5901worker2 # [ 114.449552] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:04Z level=info event=scheduler_connected addr=lb:500515902worker2 # [ 114.453556] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5903worker2 # [ 114.654484] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:04Z level=info event=scheduler_connected addr=lb:500515904worker2 # [ 114.658356] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:04Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5905worker2 # [ 114.859436] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_connected addr=lb:500515906worker2 # [ 114.863424] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5907worker2 # [ 115.065830] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_connected addr=lb:500515908worker2 # [ 115.069103] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5909worker1 # [ 115.254850] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.5910worker1 # [ 115.257912] systemd[1]: Stopped Nix worker-protocol over gRPC.5911worker1 # [ 115.261975] systemd[1]: nix-grpc-daemon.service: Consumed 739ms CPU time over 9.520s wall clock time, 36.9M memory peak, 184.8K incoming IP traffic, 87.3K outgoing IP traffic.5912worker2: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over5913worker2 # [ 115.267346] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_connected addr=lb:500515914worker2 # [ 115.272405] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5915lb # [ 115.416596] envoy[913]: conn peer=[2001:db8:1::1]:46902 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=1005 tx=282 ms=8835916lb # [ 115.420560] envoy[913]: conn peer=[2001:db8:1::1]:46916 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9623 tx=4382 ms=44645917lb # [ 115.424120] envoy[913]: conn peer=[2001:db8:1::3]:52264 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=1925 tx=620 ms=83785918worker2 # [ 115.473499] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_connected addr=lb:500515919worker2 # [ 115.477446] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5920worker2 # [ 115.678436] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_connected addr=lb:500515921worker2 # [ 115.682363] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:05Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5922worker2 # [ 115.883372] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_connected addr=lb:500515923worker2 # [ 115.887303] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5924worker2 # [ 116.088954] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_connected addr=lb:500515925worker2 # [ 116.092814] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5926worker2 # [ 116.293773] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_connected addr=lb:500515927worker2 # [ 116.297677] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5928worker2 # [ 116.498790] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_connected addr=lb:500515929worker2 # [ 116.502725] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5930worker2 # [ 116.703857] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_connected addr=lb:500515931worker2 # [ 116.707784] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:06Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5932worker2 # [ 116.908890] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_connected addr=lb:500515933worker2 # [ 116.912929] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5934worker2 # [ 117.114707] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_connected addr=lb:500515935worker2 # [ 117.118070] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5936worker2 # [ 117.315901] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_connected addr=lb:500515937worker2 # [ 117.319867] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5938worker2 # [ 117.520967] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_connected addr=lb:500515939worker2 # [ 117.525118] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5940worker2 # [ 117.725981] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_connected addr=lb:500515941worker2 # [ 117.730135] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:07Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5942worker2 # [ 117.931422] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_connected addr=lb:500515943worker2 # [ 117.935409] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5944lb # [ 118.034265] niks3-server[1150]: 2026/09/24 11:49:08 INFO lead: acquired remote=192.168.1.4:542265945worker2 # [ 118.030696] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_take_over5946worker2 # [ 118.137485] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_connected addr=lb:500515947worker2 # [ 118.140951] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5948worker2 # [ 118.342269] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_connected addr=lb:500515949worker2 # [ 118.346276] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5950worker2: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q event=scheduler_take_over, in 3.21 seconds)5951worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5952worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.02 seconds)5953worker2 # [ 118.546668] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_connected addr=lb:500515954worker2 # [ 118.548444] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"5955worker2 # [ 118.749748] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:08Z level=info event=scheduler_connected addr=lb:500515956worker2 # [ 118.768158] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=info event=oidc_jwks provider=mock keys=15957worker2 # [ 118.771229] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:590805958worker2 # [ 118.775645] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=05959worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics5960worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)5961client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&25962worker2 # [ 119.685503] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:575145963client # don't know how to build these paths:5964client # /nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv5965worker2 # [ 119.694184] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575145966worker2 # [ 119.716249] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=debug event=want client=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv cached=05967worker2 # [ 119.721174] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=debug event=assigned drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv worker=192.168.1.4:50051 clients=15968worker2 # [ 119.727522] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575145969worker2 # [ 119.738558] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575145970client # copying 1 paths to 'lb:50051'...5971client # copying path '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' to 'lb:50051'...5972worker2 # [ 119.750105] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575145973worker2 # [ 119.761185] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5974worker2 # [ 119.859768] nix-daemon[857]: accepted connection from pid 1373, user nix-grpc-daemon (trusted)5975worker2 # [ 119.871330] nix-daemon[857]: reaped child process 1376, status = succeeded5976lb # [ 119.885070] niks3-server[1150]: 2026/09/24 11:49:10 INFO Received push request method=POST path=/api/pushes5977worker2 # [ 119.889472] nix-grpc-daemon[1316]: time=2026-09-24T11:49:10.127Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"5978worker2 # [ 119.891670] nix-grpc-daemon[1316]: time=2026-09-24T11:49:10.128Z level=INFO msg="Uploading 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv (520B)"5979lb # [ 119.914864] niks3-server[1150]: 2026/09/24 11:49:10 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.ls5980lb # [ 119.920153] niks3-server[1150]: 2026/09/24 11:49:10 INFO Received sign narinfos request method=POST path=/api/pushes/42/sign5981worker2 # [ 119.925845] nix-grpc-daemon[1316]: time=2026-09-24T11:49:10.163Z level=INFO msg="Uploading 1 narinfos"5982lb # [ 119.929122] niks3-server[1150]: 2026/09/24 11:49:10 INFO Signed narinfos id=42 count=15983lb # [ 119.938728] niks3-server[1150]: 2026/09/24 11:49:10 INFO Registered completed upload object_key=nar/013ynk63lqgbvdpr2qqb6pflhzfcrl005f3zkvnag3x7hqxjhibs.nar.zst5984lb # [ 119.950720] niks3-server[1150]: 2026/09/24 11:49:10 INFO Received complete push request method=POST path=/api/pushes/42/complete5985lb # [ 119.958922] niks3-server[1150]: 2026/09/24 11:49:10 INFO Registered completed upload object_key=0i80y1x9s0d06s86lhapa650pgsxgi0g.narinfo5986worker2 # [ 119.960517] nix-grpc-daemon[1316]: time=2026-09-24T11:49:10.198Z level=INFO msg="Upload complete. (188ms)"5987worker2 # [ 119.965246] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=0 paths=1 nar_bytes_in=5205988worker2 # [ 119.970878] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575145989worker2 # [ 119.982105] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5990worker2 # [ 119.990239] nix-daemon[857]: reaped child process 1371, status = succeeded5991worker2 # [ 119.996921] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)5992client # building '/nix/store/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv' on lb:50051...5993client # farm-a> worker2: building 0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv5994worker2 # [ 120.551519] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:10Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:590845995client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds5996client # farm-a> LOG-farm-a-on-standby5997worker2 # [ 121.422936] nix-daemon[857]: reaped child process 1380, status = succeeded5998worker2 # [ 121.502343] nix-daemon[857]: accepted connection from pid 1396, user nix-grpc-daemon (trusted)5999worker2 # [ 121.514771] nix-daemon[857]: reaped child process 1399, status = succeeded6000lb # [ 121.527796] niks3-server[1150]: 2026/09/24 11:49:11 INFO Received push request method=POST path=/api/pushes6001worker2 # [ 121.535493] nix-grpc-daemon[1316]: time=2026-09-24T11:49:11.773Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6002worker2 # [ 121.537101] nix-grpc-daemon[1316]: time=2026-09-24T11:49:11.774Z level=INFO msg="Uploading fp54697jryq5rvfg8ml51wzkr9xjvhrg-farm-a (136B)"6003lb # [ 121.571258] niks3-server[1150]: 2026/09/24 11:49:12 INFO Registered completed upload object_key=log/0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv6004lb # [ 121.578626] niks3-server[1150]: 2026/09/24 11:49:12 INFO Received sign narinfos request method=POST path=/api/pushes/43/sign6005lb # [ 121.588442] niks3-server[1150]: 2026/09/24 11:49:12 INFO Signed narinfos id=43 count=16006worker2 # [ 121.587405] nix-grpc-daemon[1316]: time=2026-09-24T11:49:11.825Z level=INFO msg="Uploading 1 narinfos"6007lb # [ 121.592285] niks3-server[1150]: 2026/09/24 11:49:12 INFO Registered completed upload object_key=nar/0x76wzgdd1nk8rm930z8l69v9961jpc8qhzmgganv0nhagq8lv9c.nar.zst6008lb # [ 121.608822] niks3-server[1150]: 2026/09/24 11:49:12 INFO Received complete push request method=POST path=/api/pushes/43/complete6009worker2 # [ 121.611244] nix-grpc-daemon[1316]: time=2026-09-24T11:49:11.848Z level=INFO msg="Upload complete. (205ms)"6010lb # [ 121.617244] niks3-server[1150]: 2026/09/24 11:49:12 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.ls6011lb # [ 121.619912] niks3-server[1150]: 2026/09/24 11:49:12 INFO Registered completed upload object_key=fp54697jryq5rvfg8ml51wzkr9xjvhrg.narinfo6012worker2 # [ 121.617607] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:11Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=1 drv=0i80y1x9s0d06s86lhapa650pgsxgi0g-farm-a.drv assign_id=1 outputs=16013worker2 # [ 121.632764] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:11Z level=debug event=want client=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv cached=06014worker2 # [ 121.636770] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:11Z level=debug event=assigned drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv worker=192.168.1.4:50051 clients=16015worker2 # [ 121.642607] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:11Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575146016worker2 # [ 121.651859] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:11Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575146017worker2 # [ 121.654124] nix-daemon[857]: reaped child process 1378, status = succeeded6018worker2 # [ 121.658279] nix-grpc-daemon[1316]: time=2026-09-24T11:49:11.896Z level=INFO msg="All 1 paths already cached"6019client # copying 1 paths to 'lb:50051'...6020worker2 # [ 121.663769] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:11Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575146021client # copying path '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' to 'lb:50051'...6022worker2 # [ 121.668872] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6023worker2 # [ 121.755124] nix-daemon[857]: accepted connection from pid 1405, user nix-grpc-daemon (trusted)6024worker2 # [ 121.768943] nix-daemon[857]: reaped child process 1408, status = succeeded6025lb # [ 121.778723] niks3-server[1150]: 2026/09/24 11:49:12 INFO Received push request method=POST path=/api/pushes6026worker2 # [ 121.785426] nix-grpc-daemon[1316]: time=2026-09-24T11:49:12.023Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6027worker2 # [ 121.787414] nix-grpc-daemon[1316]: time=2026-09-24T11:49:12.024Z level=INFO msg="Uploading 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv (640B)"6028lb # [ 121.809890] niks3-server[1150]: 2026/09/24 11:49:12 INFO Received sign narinfos request method=POST path=/api/pushes/44/sign6029worker2 # [ 121.815504] nix-grpc-daemon[1316]: time=2026-09-24T11:49:12.053Z level=INFO msg="Uploading 1 narinfos"6030lb # [ 121.819379] niks3-server[1150]: 2026/09/24 11:49:12 INFO Signed narinfos id=44 count=16031lb # [ 121.825849] niks3-server[1150]: 2026/09/24 11:49:12 INFO Registered completed upload object_key=nar/12kaxaz59hi3nqzgsxphwdpbajdwrfjqknsv96cnrhpibnnzjlg0.nar.zst6032lb # [ 121.831941] niks3-server[1150]: 2026/09/24 11:49:12 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.ls6033lb # [ 121.842432] niks3-server[1150]: 2026/09/24 11:49:12 INFO Received complete push request method=POST path=/api/pushes/44/complete6034lb # [ 121.850304] niks3-server[1150]: 2026/09/24 11:49:12 INFO Registered completed upload object_key=6532q0zszdl4cnys1gb39vpngyzd7y8j.narinfo6035worker2 # [ 121.850198] nix-grpc-daemon[1316]: time=2026-09-24T11:49:12.087Z level=INFO msg="Upload complete. (175ms)"6036worker2 # [ 121.854727] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=0 paths=1 nar_bytes_in=6406037worker2 # [ 121.860434] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575146038worker2 # [ 121.870253] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6039worker2 # [ 121.879264] nix-daemon[857]: reaped child process 1403, status = succeeded6040worker2 # [ 121.883643] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6041client # building '/nix/store/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv' on lb:50051...6042client # farm-b> worker2: building 6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6043client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6044client # farm-b> LOG-farm-b-on-standby6045worker2 # [ 122.919528] nix-daemon[857]: reaped child process 1412, status = succeeded6046worker2 # [ 123.007244] nix-daemon[857]: accepted connection from pid 1428, user nix-grpc-daemon (trusted)6047worker2 # [ 123.018392] nix-daemon[857]: reaped child process 1431, status = succeeded6048lb # [ 123.030911] niks3-server[1150]: 2026/09/24 11:49:13 INFO Received push request method=POST path=/api/pushes6049worker2 # [ 123.038234] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.275Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6050worker2 # [ 123.040534] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.277Z level=INFO msg="Uploading 593ccq0xcckfsdii9xb4x5h0bapmahkp-farm-b (184B)"6051lb # [ 123.067428] niks3-server[1150]: 2026/09/24 11:49:13 INFO Received sign narinfos request method=POST path=/api/pushes/45/sign6052lb # [ 123.074833] niks3-server[1150]: 2026/09/24 11:49:13 INFO Signed narinfos id=45 count=16053worker2 # [ 123.073917] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.311Z level=INFO msg="Uploading 1 narinfos"6054lb # [ 123.082430] niks3-server[1150]: 2026/09/24 11:49:13 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.ls6055lb # [ 123.088592] niks3-server[1150]: 2026/09/24 11:49:13 INFO Registered completed upload object_key=log/6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv6056lb # [ 123.091885] niks3-server[1150]: 2026/09/24 11:49:13 INFO Registered completed upload object_key=nar/0k21bb46wfcxlq6p91pdvlzw7kkc20p87xpxr7zq5c5zr17k6rda.nar.zst6057lb # [ 123.096458] niks3-server[1150]: 2026/09/24 11:49:13 INFO Received complete push request method=POST path=/api/pushes/45/complete6058lb # [ 123.102431] niks3-server[1150]: 2026/09/24 11:49:13 INFO Registered completed upload object_key=593ccq0xcckfsdii9xb4x5h0bapmahkp.narinfo6059worker2 # [ 123.101814] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.339Z level=INFO msg="Upload complete. (211ms)"6060worker2 # [ 123.107572] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=1 drv=6532q0zszdl4cnys1gb39vpngyzd7y8j-farm-b.drv assign_id=2 outputs=16061worker2 # [ 123.116817] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:13Z level=debug event=want client=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv cached=06062worker2 # [ 123.122253] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:13Z level=debug event=assigned drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv worker=192.168.1.4:50051 clients=16063worker2 # [ 123.125772] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575146064worker2 # [ 123.132075] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575146065worker2 # [ 123.134040] nix-daemon[857]: reaped child process 1410, status = succeeded6066worker2 # [ 123.139388] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.377Z level=INFO msg="All 2 paths already cached"6067client # copying 1 paths to 'lb:50051'...6068worker2 # [ 123.144629] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575146069client # copying path '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' to 'lb:50051'...6070worker2 # [ 123.150575] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6071worker2 # [ 123.236299] nix-daemon[857]: accepted connection from pid 1437, user nix-grpc-daemon (trusted)6072worker2 # [ 123.248726] nix-daemon[857]: reaped child process 1440, status = succeeded6073lb # [ 123.260704] niks3-server[1150]: 2026/09/24 11:49:13 INFO Received push request method=POST path=/api/pushes6074worker2 # [ 123.266978] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.504Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6075worker2 # [ 123.268987] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.506Z level=INFO msg="Uploading bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv (768B)"6076lb # [ 123.292855] niks3-server[1150]: 2026/09/24 11:49:13 INFO Received sign narinfos request method=POST path=/api/pushes/46/sign6077worker2 # [ 123.293368] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.530Z level=INFO msg="Uploading 1 narinfos"6078lb # [ 123.297170] niks3-server[1150]: 2026/09/24 11:49:13 INFO Signed narinfos id=46 count=16079lb # [ 123.303686] niks3-server[1150]: 2026/09/24 11:49:13 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.ls6080lb # [ 123.308055] niks3-server[1150]: 2026/09/24 11:49:13 INFO Registered completed upload object_key=nar/0y5h6gmqp28k8bkzgrmc35pj8c8qhdi5bc0l17s14j217c1rzscm.nar.zst6081lb # [ 123.317333] niks3-server[1150]: 2026/09/24 11:49:13 INFO Received complete push request method=POST path=/api/pushes/46/complete6082worker2 # [ 123.320726] nix-grpc-daemon[1316]: time=2026-09-24T11:49:13.558Z level=INFO msg="Upload complete. (165ms)"6083lb # [ 123.325870] niks3-server[1150]: 2026/09/24 11:49:13 INFO Registered completed upload object_key=bgavyqwkalx49wvzslr2znbhxfj7h1mb.narinfo6084worker2 # [ 123.323059] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=0 paths=1 nar_bytes_in=7686085worker2 # [ 123.326446] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575146086worker2 # [ 123.332788] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6087worker2 # [ 123.335064] nix-daemon[857]: reaped child process 1435, status = succeeded6088worker2 # [ 123.340432] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6089client # building '/nix/store/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv' on lb:50051...6090client # farm-top> worker2: building bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6091client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6092client # farm-top> LOG-farm-top-on-standby6093worker2 # [ 124.185211] nix-daemon[857]: reaped child process 1444, status = succeeded6094worker2 # [ 124.264103] nix-daemon[857]: accepted connection from pid 1460, user nix-grpc-daemon (trusted)6095worker2 # [ 124.277196] nix-daemon[857]: reaped child process 1463, status = succeeded6096lb # [ 124.288942] niks3-server[1150]: 2026/09/24 11:49:14 INFO Received push request method=POST path=/api/pushes6097worker2 # [ 124.293563] nix-grpc-daemon[1316]: time=2026-09-24T11:49:14.531Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6098worker2 # [ 124.295120] nix-grpc-daemon[1316]: time=2026-09-24T11:49:14.532Z level=INFO msg="Uploading bna0adh7g5jsp1sgpmaqnl7brapllfgc-farm-top (240B)"6099lb # [ 124.329856] niks3-server[1150]: 2026/09/24 11:49:14 INFO Registered completed upload object_key=log/bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv6100lb # [ 124.336130] niks3-server[1150]: 2026/09/24 11:49:14 INFO Registered completed upload object_key=nar/0bglib3l3asq436b8p5hj4yyk8c77b2qg09pcdiw1w2dfpvnd026.nar.zst6101lb # [ 124.345117] niks3-server[1150]: 2026/09/24 11:49:14 INFO Received sign narinfos request method=POST path=/api/pushes/47/sign6102worker2 # [ 124.346919] nix-grpc-daemon[1316]: time=2026-09-24T11:49:14.584Z level=INFO msg="Uploading 1 narinfos"6103lb # [ 124.350182] niks3-server[1150]: 2026/09/24 11:49:14 INFO Signed narinfos id=47 count=16104lb # [ 124.356359] niks3-server[1150]: 2026/09/24 11:49:14 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.ls6105lb # [ 124.367671] niks3-server[1150]: 2026/09/24 11:49:14 INFO Received complete push request method=POST path=/api/pushes/47/complete6106lb # [ 124.373887] niks3-server[1150]: 2026/09/24 11:49:14 INFO Registered completed upload object_key=bna0adh7g5jsp1sgpmaqnl7brapllfgc.narinfo6107worker2 # [ 124.373785] nix-grpc-daemon[1316]: time=2026-09-24T11:49:14.611Z level=INFO msg="Upload complete. (226ms)"6108worker2 # [ 124.381457] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=1 drv=bgavyqwkalx49wvzslr2znbhxfj7h1mb-farm-top.drv assign_id=3 outputs=16109worker2 # [ 124.397829] nix-daemon[857]: reaped child process 1442, status = succeeded6110client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby >&2, in 4.88 seconds)6111client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath6112client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag on-standby outPath, in 0.12 seconds)6113worker1 # [ 124.604533] systemd[1]: Starting Nix worker-protocol over gRPC...6114worker1 # [ 124.690899] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:14Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6115worker1 # [ 124.692935] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:14Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6116worker1 # [ 124.696215] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:14Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16117worker1 # [ 124.700462] nix-grpc-daemon[1508]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6118worker1 # [ 124.701809] nix-grpc-daemon[1508]: E0000 00:00:1790250554.992334 1508 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6119worker1 # [ 124.705821] nix-grpc-daemon[1508]: E0000 00:00:1790250554.997711 1508 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6120worker1 # [ 124.709558] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:15Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6121worker1 # [ 124.713470] systemd[1]: Started Nix worker-protocol over gRPC.6122worker1 # [ 124.717956] nix-grpc-daemon[1508]: E0000 00:00:1790250555.009223 1550 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6123worker1 # [ 124.730830] nix-grpc-daemon[1508]: E0000 00:00:1790250555.022695 1550 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6124worker1 # [ 124.736061] nix-grpc-daemon[1508]: E0000 00:00:1790250555.025107 1550 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6125lb: must succeed: curl -sf localhost:9901/clusters6126worker1 # [ 124.746147] nix-grpc-daemon[1508]: E0000 00:00:1790250555.038075 1550 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6127worker1 # [ 124.757312] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:15Z level=info event=scheduler_connected addr=lb:500516128worker2 # [ 124.748179] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:14Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:590806129worker2 # [ 124.752892] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:14Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06130lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6131worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6132worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6133lb: must succeed: curl -sf localhost:9901/clusters6134lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6135lb: must succeed: curl -sf localhost:9901/clusters6136lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6137worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics6138worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6139(finished: subtest: scheduler down: the other node takes the lock and keeps it, in 12.90 seconds)6140subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise6141client: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs6142client # [ 124.967412] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs.6143client # Running as unit: rs.service; invocation ID: e1c8fcb3d5c04ad28bf4ae8413b34a056144client: (finished: must succeed: systemd-run --unit rs nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag rs, in 0.06 seconds)6145worker1 # [ 125.147757] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:354286146worker1 # [ 125.159523] nix-daemon[867]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)6147client # [ 125.134344] nix[1379]: don't know how to build these paths:6148client # [ 125.135285] nix[1379]: /nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6149worker2 # [ 125.168624] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:15Z level=debug event=want client=2 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv cached=06150worker2 # [ 125.174281] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:15Z level=debug event=assigned drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv worker=192.168.1.4:50051 clients=16151worker2 # [ 125.180533] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575146152worker2 # [ 125.193204] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:575146153client # [ 125.176304] nix[1379]: copying 1 paths to 'lb:50051'...6154client # [ 125.180183] nix[1379]: copying path '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' to 'lb:50051'...6155worker2 # [ 125.207762] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:15Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:575146156worker2 # [ 125.218969] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6157worker2 # [ 125.319886] nix-daemon[857]: accepted connection from pid 1476, user nix-grpc-daemon (trusted)6158worker2 # [ 125.334787] nix-daemon[857]: reaped child process 1479, status = succeeded6159lb # [ 125.345440] niks3-server[1150]: 2026/09/24 11:49:15 INFO Received push request method=POST path=/api/pushes6160worker2 # [ 125.350942] nix-grpc-daemon[1316]: time=2026-09-24T11:49:15.588Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6161worker2 # [ 125.353105] nix-grpc-daemon[1316]: time=2026-09-24T11:49:15.590Z level=INFO msg="Uploading 2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv (416B)"6162lb # [ 125.377815] niks3-server[1150]: 2026/09/24 11:49:15 INFO Received sign narinfos request method=POST path=/api/pushes/48/sign6163lb # [ 125.387808] niks3-server[1150]: 2026/09/24 11:49:15 INFO Signed narinfos id=48 count=16164worker2 # [ 125.387506] nix-grpc-daemon[1316]: time=2026-09-24T11:49:15.625Z level=INFO msg="Uploading 1 narinfos"6165lb # [ 125.394142] niks3-server[1150]: 2026/09/24 11:49:15 INFO Registered completed upload object_key=nar/1p73bzh7zqz68fm48xgidr9p1hfd08x3vp8znnaaycsmagbr8hl6.nar.zst6166lb # [ 125.400721] niks3-server[1150]: 2026/09/24 11:49:15 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.ls6167lb # [ 125.411127] niks3-server[1150]: 2026/09/24 11:49:15 INFO Received complete push request method=POST path=/api/pushes/48/complete6168lb # [ 125.416494] envoy[913]: conn peer=[2001:db8:1::1]:58054 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9569 tx=4333 ms=47156169worker2 # [ 125.419073] nix-grpc-daemon[1316]: time=2026-09-24T11:49:15.656Z level=INFO msg="Upload complete. (189ms)"6170lb # [ 125.424358] niks3-server[1150]: 2026/09/24 11:49:15 INFO Registered completed upload object_key=2765168rrfa4dbs7pfx66j88710jx2fm.narinfo6171worker2 # [ 125.425470] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:15Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=0 paths=1 nar_bytes_in=4166172worker2 # [ 125.431717] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:15Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:575146173worker2 # [ 125.442485] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6174worker2 # [ 125.454109] nix-daemon[857]: reaped child process 1474, status = succeeded6175worker2 # [ 125.457267] nix-daemon[857]: accepted connection from pid 1256, user nix-grpc-daemon (trusted)6176client # [ 125.430855] nix[1379]: building '/nix/store/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv' on lb:50051...6177lb: must succeed: curl -sf localhost:9901/clusters6178lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6179lb: must succeed: curl -sf localhost:9901/clusters6180lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6181lb: must succeed: curl -sf localhost:9901/clusters6182lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6183lb: must succeed: curl -sf localhost:9901/clusters6184lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6185worker1: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true6186worker1 # [ 127.702972] systemd-journald[283]: Received client request to rotate journal, rotating.6187worker1 # [ 127.723128] systemd-journald[283]: Vacuuming done, freed 0B of archived journals from /var/log/journal/e9d64d16249d4a07866faaf99960649c.6188worker1: (finished: must succeed: journalctl --rotate --vacuum-time=1s -u nix-grpc-daemon >/dev/null 2>&1 || true, in 0.06 seconds)6189worker2: must succeed: systemctl restart --no-block nix-grpc-daemon.service6190worker2 # [ 127.622852] systemd[1]: Stopping Nix worker-protocol over gRPC...6191worker2: (finished: must succeed: systemctl restart --no-block nix-grpc-daemon.service, in 0.03 seconds)6192client: must succeed: sleep 5.06193worker2 # [ 128.035448] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:18Z level=info event=draining6194client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6195worker1: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr='6196worker2 # [ 132.877979] nix-daemon[857]: reaped child process 1483, status = succeeded6197worker2 # [ 132.956424] nix-daemon[857]: accepted connection from pid 1524, user nix-grpc-daemon (trusted)6198worker2 # [ 132.968803] nix-daemon[857]: reaped child process 1527, status = succeeded6199lb # [ 132.984229] niks3-server[1150]: 2026/09/24 11:49:23 INFO Received push request method=POST path=/api/pushes6200worker2 # [ 132.988466] nix-grpc-daemon[1316]: time=2026-09-24T11:49:23.226Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6201worker2 # [ 132.989973] nix-grpc-daemon[1316]: time=2026-09-24T11:49:23.227Z level=INFO msg="Uploading 6qby8qgv9vwglq2yryw4zqyixrizh5d2-slow-rs (120B)"6202lb # [ 133.022571] niks3-server[1150]: 2026/09/24 11:49:23 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.ls6203lb # [ 133.034735] niks3-server[1150]: 2026/09/24 11:49:23 INFO Registered completed upload object_key=nar/19z3qdigvc4gggg31dgc0wi5w85pgaq9dk7bvngh0yhckpy55jgq.nar.zst6204lb # [ 133.043169] niks3-server[1150]: 2026/09/24 11:49:23 INFO Received sign narinfos request method=POST path=/api/pushes/49/sign6205worker2 # [ 133.043003] nix-grpc-daemon[1316]: time=2026-09-24T11:49:23.280Z level=INFO msg="Uploading 1 narinfos"6206lb # [ 133.047187] niks3-server[1150]: 2026/09/24 11:49:23 INFO Signed narinfos id=49 count=16207lb # [ 133.052113] niks3-server[1150]: 2026/09/24 11:49:23 INFO Registered completed upload object_key=log/2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv6208lb # [ 133.061804] niks3-server[1150]: 2026/09/24 11:49:23 INFO Received complete push request method=POST path=/api/pushes/49/complete6209lb # [ 133.068702] niks3-server[1150]: 2026/09/24 11:49:23 INFO Registered completed upload object_key=6qby8qgv9vwglq2yryw4zqyixrizh5d2.narinfo6210worker2 # [ 133.063924] nix-grpc-daemon[1316]: time=2026-09-24T11:49:23.301Z level=INFO msg="Upload complete. (191ms)"6211worker2 # [ 133.072447] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:23Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:57514 duration_s=7 drv=2765168rrfa4dbs7pfx66j88710jx2fm-slow-rs.drv assign_id=4 outputs=16212worker2 # [ 133.085793] nix-daemon[857]: reaped child process 1481, status = succeeded6213client # [ 133.074502] systemd[1]: rs.service: Deactivated successfully.6214client # [ 133.078185] systemd[1]: rs.service: Consumed 126ms CPU time over 8.109s wall clock time, 39.4M memory peak, 7.1K incoming IP traffic, 8.2K outgoing IP traffic.6215worker2 # [ 134.036231] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:24Z level=info event=signal_exit6216lb # [ 135.039628] niks3-server[1150]: 2026/09/24 11:49:25 INFO lead: released remote=192.168.1.4:542266217lb # [ 135.416916] envoy[913]: conn peer=[2001:db8:1::1]:33898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2956 tx=1500 ms=79496218worker2 # [ 136.035947] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:26Z level=info event=scheduler_restarting6219lb # [ 136.038594] envoy[913]: [2026-09-24 11:49:26.482][913][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 1, Cancelled on the server side{via_upstream}6220worker2 # [ 136.042167] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:26Z level=info event=scheduler_restarting addr=lb:500516221worker1 # [ 136.055422] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:26Z level=info event=scheduler_restarting addr=lb:500516222worker2 # [ 136.048616] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590806223worker2 # [ 136.053229] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:26Z level=info event=worker_gone addr=192.168.1.4:500516224worker2 # [ 136.056307] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:26Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:590806225worker2 # [ 136.060359] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:26Z level=info event=worker_gone addr=192.168.1.3:500516226worker2 # [ 136.081971] nix-daemon[857]: reaped child process 1308, status = succeeded6227worker1 # [ 136.255058] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:26Z level=info event=scheduler_connected addr=lb:500516228worker1 # [ 136.259615] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6229worker2 # [ 136.255339] nix-grpc-daemon[1256]: ts=2026-09-24T11:49:26Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6230worker1 # [ 136.460191] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:26Z level=info event=scheduler_connected addr=lb:500516231worker1 # [ 136.464277] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6232worker1 # [ 136.664869] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:26Z level=info event=scheduler_connected addr=lb:500516233worker1 # [ 136.668928] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:26Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6234worker1 # [ 136.869616] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516235worker1 # [ 136.873622] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6236worker2 # [ 136.863745] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6237worker2 # [ 136.866771] systemd[1]: Stopped Nix worker-protocol over gRPC.6238worker2 # [ 136.869334] systemd[1]: nix-grpc-daemon.service: Consumed 1.251s CPU time over 35.939s wall clock time, 39.1M memory peak, 349.1K incoming IP traffic, 211.6K outgoing IP traffic.6239worker2 # [ 136.884944] systemd[1]: Starting scheduler-token.service...6240worker2 # [ 136.929277] systemd[1]: scheduler-token.service: Deactivated successfully.6241worker2 # [ 136.930777] systemd[1]: Finished scheduler-token.service.6242worker2 # [ 136.933113] systemd[1]: scheduler-token.service: Consumed 23ms CPU time over 46ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6243worker2 # [ 136.935845] systemd[1]: Starting Nix worker-protocol over gRPC...6244worker2 # [ 137.015754] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6245worker2 # [ 137.017743] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6246worker2 # [ 137.020946] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16247worker2 # [ 137.026050] nix-grpc-daemon[1538]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6248worker2 # [ 137.028160] nix-grpc-daemon[1538]: E0000 00:00:1790250567.263753 1538 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6249worker2 # [ 137.031117] nix-grpc-daemon[1538]: E0000 00:00:1790250567.268734 1538 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6250worker2 # [ 137.035513] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6251worker2 # [ 137.038760] systemd[1]: Started Nix worker-protocol over gRPC.6252worker2 # [ 137.042071] nix-grpc-daemon[1538]: E0000 00:00:1790250567.279049 1581 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6253lb # [ 137.047334] niks3-server[1150]: 2026/09/24 11:49:27 INFO lead: acquired remote=192.168.1.4:470986254worker2 # [ 137.046320] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=scheduler_take_over6255worker2 # [ 137.054776] nix-grpc-daemon[1538]: E0000 00:00:1790250567.292496 1581 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6256worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon | grep -q 'event=scheduler_restarting addr=', in 4.21 seconds)6257client: waiting for success: ! systemctl is-active rs6258worker2 # [ 137.059054] nix-grpc-daemon[1538]: E0000 00:00:1790250567.294787 1581 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6259worker1 # [ 137.074680] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516260worker1 # [ 137.078314] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6261worker2 # [ 137.069605] nix-grpc-daemon[1538]: E0000 00:00:1790250567.307332 1581 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6262worker2 # [ 137.078989] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516263worker2 # [ 137.080569] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6264client: (finished: waiting for success: ! systemctl is-active rs, in 0.04 seconds)6265client: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }6266client: (finished: must succeed: systemctl show -p Result --value rs | grep -qx success || { journalctl -u rs >&2; false; }, in 0.03 seconds)6267client: must fail: journalctl -u rs | grep -q 'reconnecting'6268client: (finished: must fail: journalctl -u rs | grep -q 'reconnecting', in 0.01 seconds)6269lb: must succeed: curl -sf localhost:9901/clusters6270lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.04 seconds)6271worker1 # [ 137.278695] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516272worker1 # [ 137.280595] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6273worker2 # [ 137.280766] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516274worker2 # [ 137.284739] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6275worker1 # [ 137.481180] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516276worker1 # [ 137.483112] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6277worker2 # [ 137.485572] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516278worker2 # [ 137.489489] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6279worker1 # [ 137.684120] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516280worker1 # [ 137.688606] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:27Z level=info event=scheduler_disconnected addr=lb:50051 error="no healthy upstream"6281worker2 # [ 137.690488] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=scheduler_connected addr=lb:500516282worker2 # [ 137.703131] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=oidc_jwks provider=mock keys=16283worker2 # [ 137.706227] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:489006284worker2 # [ 137.710623] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:27Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06285worker1 # [ 137.889491] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:28Z level=info event=scheduler_connected addr=lb:500516286worker2 # [ 137.881298] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:28Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:489006287worker2 # [ 137.885094] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:28Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06288lb: must succeed: curl -sf localhost:9901/clusters6289lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6290worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6291worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6292(finished: subtest: clean scheduler restart: peers are told, running builds survive, no reconnect noise, in 13.33 seconds)6293subtest: scheduler restart mid-build: build finishes, no double build6294client: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid6295client # [ 138.298597] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid.6296client # Running as unit: mid.service; invocation ID: 2161d4aaf14d4b99af09cca51ab357e46297client: (finished: must succeed: systemd-run --unit mid nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag mid, in 0.06 seconds)6298worker1 # [ 138.471225] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:28Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:354286299client # [ 138.440320] nix[1415]: don't know how to build these paths:6300client # [ 138.441262] nix[1415]: /nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6301worker2 # [ 138.475787] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:28Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06302worker2 # [ 138.481498] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:28Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.4:50051 clients=16303worker2 # [ 138.489288] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:28Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489026304worker2 # [ 138.496630] nix-daemon[857]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)6305worker2 # [ 138.512940] nix-grpc-daemon[1538]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 53 ms (attempt 1/5)6306worker2 # [ 138.572310] nix-grpc-daemon[1538]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 42 ms (attempt 2/5)6307worker2 # [ 138.616208] nix-grpc-daemon[1538]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 390 ms (attempt 3/5)6308worker2 # [ 139.008660] nix-grpc-daemon[1538]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 22 ms (attempt 4/5)6309worker2 # [ 139.038367] nix-grpc-daemon[1538]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6310worker2 # [ 139.041527] nix-grpc-daemon[1538]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6311worker2 # [ 139.059211] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:29Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:489026312client # [ 139.043363] nix[1415]: copying 1 paths to 'lb:50051'...6313client # [ 139.047166] nix[1415]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' to 'lb:50051'...6314worker2 # [ 139.074173] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:29Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:489026315worker2 # [ 139.084162] nix-daemon[857]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)6316worker2 # [ 139.107320] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:29Z level=info event=niks3_push_started pid=16076317worker2 # [ 139.213585] nix-grpc-daemon[1607]: time=2026-09-24T11:49:29.450Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6318worker2 # [ 139.302532] nix-daemon[857]: accepted connection from pid 1620, user nix-grpc-daemon (trusted)6319worker2 # [ 139.313442] nix-daemon[857]: reaped child process 1623, status = succeeded6320lb # [ 139.327588] niks3-server[1150]: 2026/09/24 11:49:29 INFO Received push request method=POST path=/api/pushes6321worker2 # [ 139.332295] nix-grpc-daemon[1607]: time=2026-09-24T11:49:29.570Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6322worker2 # [ 139.333829] nix-grpc-daemon[1607]: time=2026-09-24T11:49:29.571Z level=INFO msg="Uploading n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv (424B)"6323lb # [ 139.366725] niks3-server[1150]: 2026/09/24 11:49:29 INFO Received sign narinfos request method=POST path=/api/pushes/50/sign6324lb # [ 139.374110] niks3-server[1150]: 2026/09/24 11:49:29 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.ls6325lb # [ 139.385480] niks3-server[1150]: 2026/09/24 11:49:29 INFO Registered completed upload object_key=nar/0pdyhjlvyi83rdx6rr15l7kj7a7hlcbqgllxm29xw3hvr3akprgq.nar.zst6326worker2 # [ 139.393443] nix-grpc-daemon[1607]: time=2026-09-24T11:49:29.630Z level=INFO msg="Uploading 1 narinfos"6327lb # [ 139.396431] niks3-server[1150]: 2026/09/24 11:49:29 INFO Signed narinfos id=50 count=16328lb # [ 139.408284] niks3-server[1150]: 2026/09/24 11:49:29 INFO Received complete push request method=POST path=/api/pushes/50/complete6329worker2 # [ 139.413520] nix-grpc-daemon[1607]: time=2026-09-24T11:49:29.651Z level=INFO msg="Upload complete. (196ms)"6330lb # [ 139.419907] niks3-server[1150]: 2026/09/24 11:49:29 INFO Registered completed upload object_key=n23kf6h4k511p0qpzsydmqrbq2qn9g5w.narinfo6331worker2 # [ 139.417677] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:29Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:48902 duration_s=0 paths=1 nar_bytes_in=4246332worker2 # [ 139.423262] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:29Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:489026333worker2 # [ 139.431748] nix-daemon[857]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)6334worker2 # [ 139.439996] nix-daemon[857]: reaped child process 1605, status = succeeded6335worker2 # [ 139.444849] nix-daemon[857]: accepted connection from pid 1538, user nix-grpc-daemon (trusted)6336client # [ 139.418460] nix[1415]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6337lb: must succeed: curl -sf localhost:9901/clusters6338worker2 # [ 140.710488] nix-grpc-daemon[1538]: ts=2026-09-24T11:49:30Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:489046339lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6340lb: must succeed: curl -sf localhost:9901/clusters6341lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6342worker2: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service6343worker2 # [ 140.807825] systemd[1]: nix-grpc-daemon.service: Sent signal SIGKILL to main process 1538 (nix-grpc-daemon) on client request.6344worker2 # [ 140.812177] systemd[1]: nix-grpc-daemon.service: Killed unit cgroup '/system.slice/nix-grpc-daemon.service' with SIGKILL on client request.6345lb # [ 140.824823] niks3-server[1150]: 2026/09/24 11:49:31 INFO lead: released remote=192.168.1.4:470986346worker2 # [ 140.829232] systemd[1]: Starting scheduler-token.service...6347lb # [ 140.830479] envoy[913]: [2026-09-24 11:49:31.273][913][warning][config] [./source/extensions/config_subscription/grpc/grpc_stream.h:189] StreamEndpoints gRPC config stream to sched closed: 13, upstream_reset_after_response_started{connection_termination}6348client # [ 140.800770] nix[1415]: n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), asking the scheduler again6349worker1 # [ 140.837954] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:31Z level=info event=scheduler_disconnected addr=lb:50051 error="upstream connect error or disconnect/reset before headers. reset reason: connection termination"6350worker2 # [ 140.831414] systemd[1]: nix-grpc-daemon.service: Main process exited, code=killed, status=9/KILL6351client # [ 140.805636] nix[1415]: scheduler at lb:50051: Stream removed (RST_STREAM (Received RST_STREAM with error code 2)), reconnecting6352worker2 # [ 140.835994] systemd[1]: nix-grpc-daemon.service: Failed with result 'signal'.6353worker2 # [ 140.837446] systemd[1]: nix-grpc-daemon.service: Consumed 313ms CPU time over 3.894s wall clock time, 35.3M memory peak, 149.7K incoming IP traffic, 39.6K outgoing IP traffic.6354worker2 # [ 140.858903] nix-daemon[857]: reaped child process 1625, status = succeeded6355worker2 # [ 140.861144] nix-daemon[857]: reaped child process 1599, status = succeeded6356worker2 # [ 140.864755] nix-daemon[1627]: unexpected Nix daemon error: error: interrupted by the user6357worker2 # [ 140.870051] nix-daemon[857]: reaped child process 1627, status = failed with exit code 16358worker2 # [ 140.893576] systemd[1]: scheduler-token.service: Deactivated successfully.6359worker2 # [ 140.895106] systemd[1]: Finished scheduler-token.service.6360worker2 # [ 140.896705] systemd[1]: scheduler-token.service: Consumed 17ms CPU time over 65ms wall clock time, 2.2M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6361worker2 # [ 140.899156] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6362worker2 # [ 140.902881] systemd[1]: Starting scheduler-token.service...6363worker2 # [ 140.936829] systemd[1]: scheduler-token.service: Deactivated successfully.6364worker2 # [ 140.938440] systemd[1]: Finished scheduler-token.service.6365worker2 # [ 140.940091] systemd[1]: scheduler-token.service: Consumed 15ms CPU time over 35ms wall clock time, 2.1M memory peak, 925B incoming IP traffic, 452B outgoing IP traffic.6366worker2 # [ 140.944245] systemd[1]: Starting Nix worker-protocol over gRPC...6367worker2 # [ 141.020369] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6368worker2 # [ 141.022371] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6369worker2 # [ 141.025516] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.4:50051 niks3=http://lb:5751 max_jobs=16370worker1 # [ 141.037953] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:31Z level=info event=scheduler_connected addr=lb:500516371worker2 # [ 141.030030] nix-grpc-daemon[1670]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6372worker2 # [ 141.033171] nix-grpc-daemon[1670]: E0000 00:00:1790250571.267693 1670 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6373worker2 # [ 141.035423] nix-grpc-daemon[1670]: E0000 00:00:1790250571.270756 1670 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6374worker2 # [ 141.038066] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6375worker2 # [ 141.042136] systemd[1]: Started Nix worker-protocol over gRPC.6376lb # [ 141.051196] niks3-server[1150]: 2026/09/24 11:49:31 INFO lead: acquired remote=192.168.1.4:471646377worker2 # [ 141.048110] nix-grpc-daemon[1670]: E0000 00:00:1790250571.284800 1713 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6378worker2: (finished: must succeed: systemctl kill -s KILL nix-grpc-daemon.service; systemctl start nix-grpc-daemon.service, in 0.27 seconds)6379worker2 # [ 141.055150] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=scheduler_take_over6380client: must succeed: sleep 5.06381worker2 # [ 141.058348] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:489326382worker2 # [ 141.062133] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06383worker2 # [ 141.068615] nix-grpc-daemon[1670]: E0000 00:00:1790250571.306340 1713 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6384worker2 # [ 141.073052] nix-grpc-daemon[1670]: E0000 00:00:1790250571.308610 1713 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6385worker2 # [ 141.082766] nix-grpc-daemon[1670]: E0000 00:00:1790250571.320500 1713 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6386worker2 # [ 141.092066] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=scheduler_connected addr=lb:500516387worker2 # [ 141.097061] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=oidc_jwks provider=mock keys=16388worker2 # [ 141.098571] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=worker_session_open cn=oidc:mock:node:worker2 peer=ipv4:192.168.1.2:489326389worker2 # [ 141.100147] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=worker_hello addr=192.168.1.4:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06390worker2 # [ 141.242537] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=info event=eds_stream_open cn=lb-1 peer=ipv4:192.168.1.2:489486391worker2 # [ 141.341310] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=debug event=want client=1 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv cached=06392worker1 # [ 141.356695] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:31Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354286393worker2 # [ 141.346089] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:31Z level=debug event=assigned drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv worker=192.168.1.3:50051 clients=16394worker1 # [ 141.368459] nix-grpc-daemon[1508]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 41 ms (attempt 1/5)6395worker1 # [ 141.416962] nix-grpc-daemon[1508]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 75 ms (attempt 2/5)6396worker1 # [ 141.493990] nix-grpc-daemon[1508]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 221 ms (attempt 3/5)6397worker1 # [ 141.717610] nix-grpc-daemon[1508]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 147 ms (attempt 4/5)6398worker1 # [ 141.874942] nix-grpc-daemon[1508]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6399worker1 # [ 141.878509] nix-grpc-daemon[1508]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6400worker1 # [ 141.894399] nix-grpc-daemon[1508]: copying path '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' from 'http://lb:5751'...6401worker1 # [ 141.920112] nix-daemon[867]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)6402client # [ 141.895964] nix[1415]: building '/nix/store/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv' on lb:50051...6403worker1 # [ 141.934526] nix-daemon[867]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)6404lb # [ 145.417521] envoy[913]: conn peer=[2001:db8:1::4]:45694 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=39281 tx=2128 ms=351846405lb # [ 145.421511] envoy[913]: conn peer=[2001:db8:1::4]:36018 tls=TLSv1.3 subject="CN=worker-2" sni=- flags=- tls_fail="-" rx=3640 tx=383 ms=37516406client: (finished: must succeed: sleep 5.0, in 5.03 seconds)6407client: waiting for success: ! systemctl is-active mid6408worker1 # [ 146.354539] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:36Z level=info event=niks3_push_started pid=16726409worker1 # [ 146.387939] nix-daemon[867]: reaped child process 1647, status = succeeded6410worker1 # [ 146.461651] nix-grpc-daemon[1672]: time=2026-09-24T11:49:36.753Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6411worker1 # [ 146.547794] nix-daemon[867]: accepted connection from pid 1685, user nix-grpc-daemon (trusted)6412worker1 # [ 146.558994] nix-daemon[867]: reaped child process 1688, status = succeeded6413lb # [ 146.560521] niks3-server[1150]: 2026/09/24 11:49:37 INFO Received push request method=POST path=/api/pushes6414worker1 # [ 146.576061] nix-grpc-daemon[1672]: time=2026-09-24T11:49:36.868Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6415worker1 # [ 146.577680] nix-grpc-daemon[1672]: time=2026-09-24T11:49:36.869Z level=INFO msg="Uploading cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj-slow-mid (120B)"6416lb # [ 146.611787] niks3-server[1150]: 2026/09/24 11:49:37 INFO Received sign narinfos request method=POST path=/api/pushes/51/sign6417lb # [ 146.616815] niks3-server[1150]: 2026/09/24 11:49:37 INFO Signed narinfos id=51 count=16418worker1 # [ 146.626597] nix-grpc-daemon[1672]: time=2026-09-24T11:49:36.918Z level=INFO msg="Uploading 1 narinfos"6419lb # [ 146.629089] niks3-server[1150]: 2026/09/24 11:49:37 INFO Registered completed upload object_key=nar/0462jb7zwy2d73k3q6v09xnk3bx318v8ik5ckmp32w2hzvwj4k2d.nar.zst6420lb # [ 146.641681] niks3-server[1150]: 2026/09/24 11:49:37 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.ls6421lb # [ 146.647543] niks3-server[1150]: 2026/09/24 11:49:37 INFO Received complete push request method=POST path=/api/pushes/51/complete6422lb # [ 146.650706] niks3-server[1150]: 2026/09/24 11:49:37 INFO Registered completed upload object_key=log/n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv6423lb # [ 146.656927] niks3-server[1150]: 2026/09/24 11:49:37 INFO Registered completed upload object_key=cz6wmnnj5y0qn84jrv9qb9sj0dv1iylj.narinfo6424worker1 # [ 146.663475] nix-grpc-daemon[1672]: time=2026-09-24T11:49:36.955Z level=INFO msg="Upload complete. (198ms)"6425worker1 # [ 146.669364] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:36Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35428 duration_s=5 drv=n23kf6h4k511p0qpzsydmqrbq2qn9g5w-slow-mid.drv assign_id=1 outputs=16426worker1 # [ 146.683679] nix-daemon[867]: reaped child process 1645, status = succeeded6427client # [ 146.664352] systemd[1]: mid.service: Deactivated successfully.6428client # [ 146.667952] systemd[1]: mid.service: Consumed 141ms CPU time over 8.369s wall clock time, 41.4M memory peak, 8.6K incoming IP traffic, 10.5K outgoing IP traffic.6429client: (finished: waiting for success: ! systemctl is-active mid, in 1.08 seconds)6430client: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }6431client: (finished: must succeed: systemctl show -p Result --value mid | grep -qx success || { journalctl -u mid >&2; false; }, in 0.03 seconds)6432(finished: subtest: scheduler restart mid-build: build finishes, no double build, in 9.09 seconds)6433subtest: interrupting the client stops the build on the worker6434client: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr6435client # [ 147.367928] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr.6436client # Running as unit: intr.service; invocation ID: b7391762d6d249a88f313806f03dd8bf6437client: (finished: must succeed: systemd-run --unit intr nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag intr, in 0.03 seconds)6438worker2 # [ 147.514245] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:37Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582626439worker2 # [ 147.524901] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6440client # [ 147.508624] nix[1451]: don't know how to build these paths:6441client # [ 147.509559] nix[1451]: /nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6442worker2 # [ 147.542466] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:37Z level=debug event=want client=2 drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv cached=06443worker2 # [ 147.546480] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:37Z level=debug event=assigned drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv worker=192.168.1.4:50051 clients=16444worker2 # [ 147.551649] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:37Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626445worker2 # [ 147.559852] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 70 ms (attempt 1/5)6446worker2 # [ 147.634322] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 166 ms (attempt 2/5)6447worker2 # [ 147.802331] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 131 ms (attempt 3/5)6448worker2 # [ 147.935232] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 795 ms (attempt 4/5)6449worker2 # [ 148.733829] nix-grpc-daemon[1670]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6450worker2 # [ 148.737475] nix-grpc-daemon[1670]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6451worker2 # [ 148.754248] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:38Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582626452client # [ 148.737921] nix[1451]: copying 1 paths to 'lb:50051'...6453client # [ 148.741529] nix[1451]: copying path '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' to 'lb:50051'...6454worker2 # [ 148.768496] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:39Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582626455worker2 # [ 148.779219] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6456worker2 # [ 148.797475] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:39Z level=info event=niks3_push_started pid=17506457worker2 # [ 148.897993] nix-grpc-daemon[1750]: time=2026-09-24T11:49:39.135Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker2.key ca=""6458worker2 # [ 148.982343] nix-daemon[857]: accepted connection from pid 1763, user nix-grpc-daemon (trusted)6459worker2 # [ 148.994639] nix-daemon[857]: reaped child process 1766, status = succeeded6460lb # [ 149.007689] niks3-server[1150]: 2026/09/24 11:49:39 INFO Received push request method=POST path=/api/pushes6461worker2 # [ 149.014967] nix-grpc-daemon[1750]: time=2026-09-24T11:49:39.252Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6462worker2 # [ 149.016523] nix-grpc-daemon[1750]: time=2026-09-24T11:49:39.254Z level=INFO msg="Uploading x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv (424B)"6463lb # [ 149.050188] niks3-server[1150]: 2026/09/24 11:49:39 INFO Received sign narinfos request method=POST path=/api/pushes/52/sign6464lb # [ 149.055454] niks3-server[1150]: 2026/09/24 11:49:39 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.ls6465worker2 # [ 149.058534] nix-grpc-daemon[1750]: time=2026-09-24T11:49:39.296Z level=INFO msg="Uploading 1 narinfos"6466lb # [ 149.062311] niks3-server[1150]: 2026/09/24 11:49:39 INFO Signed narinfos id=52 count=16467lb # [ 149.069567] niks3-server[1150]: 2026/09/24 11:49:39 INFO Registered completed upload object_key=nar/0dw6lfdicxarqmcw4m9j94irjhgkdvc1nww51zwj454yj03v5iri.nar.zst6468lb # [ 149.085787] niks3-server[1150]: 2026/09/24 11:49:39 INFO Received complete push request method=POST path=/api/pushes/52/complete6469lb # [ 149.090543] niks3-server[1150]: 2026/09/24 11:49:39 INFO Registered completed upload object_key=x43jxvh618x0k4fbhc2wyxhr1lymdssr.narinfo6470worker2 # [ 149.089937] nix-grpc-daemon[1750]: time=2026-09-24T11:49:39.327Z level=INFO msg="Upload complete. (188ms)"6471worker2 # [ 149.094402] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:39Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=4246472worker2 # [ 149.100462] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:39Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626473worker2 # [ 149.112186] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6474worker2 # [ 149.117137] nix-daemon[857]: reaped child process 1748, status = succeeded6475worker2 # [ 149.120988] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6476client # [ 149.094664] nix[1451]: building '/nix/store/x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv' on lb:50051...6477client: must succeed: systemctl kill -s INT intr6478client # [ 150.786082] systemd[1]: intr.service: Sent signal SIGINT to main process 1451 (nix) on client request.6479worker2 # [ 150.819305] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:41Z level=info event=revoked_running drv=x43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv6480client # [ 150.792890] nix[1451]: shutting downx43jxvh618x0k4fbhc2wyxhr1lymdssr-slow-intr.drv: error: gRPC BuildDerivation on 'lb:50051' failed: Stream removed (CANCELLED)6481client # [ 150.798963] nix[1451]: error: interrupted by the user6482client: (finished: must succeed: systemctl kill -s INT intr, in 0.05 seconds)6483client # [ 150.828740] systemd[1]: intr.service: Main process exited, code=exited, status=1/FAILURE6484client # [ 150.831489] systemd[1]: intr.service: Failed with result 'exit-code'.6485client # [ 150.835635] systemd[1]: intr.service: Consumed 119ms CPU time over 3.466s wall clock time, 41.4M memory peak, 6.4K incoming IP traffic, 8.3K outgoing IP traffic.6486worker2 # [ 150.937969] nix-daemon[857]: reaped child process 1768, status = succeeded6487worker2 # [ 150.940293] nix-daemon[1770]: unexpected Nix daemon error: error: interrupted by the user6488worker2 # [ 150.944665] nix-daemon[857]: reaped child process 1770, status = failed with exit code 16489(finished: subtest: interrupting the client stops the build on the worker, in 4.62 seconds)6490subtest: a deploy mid-build drains: the build finishes, the new generation takes over6491client: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw6492client # [ 152.014505] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw.6493client # Running as unit: sw.service; invocation ID: 6368a7c1a51840e5b2c28e659c4f12376494client: (finished: must succeed: systemd-run --unit sw nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix --argstr tag sw, in 0.06 seconds)6495client # [ 152.151311] nix[1471]: don't know how to build these paths:6496client # [ 152.152637] nix[1471]: /nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6497worker2 # [ 152.177412] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:42Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582626498worker2 # [ 152.188173] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:42Z level=debug event=want client=3 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv cached=06499worker2 # [ 152.192689] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:42Z level=debug event=assigned drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv worker=192.168.1.3:50051 clients=16500worker1 # [ 152.204571] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354286501worker1 # [ 152.216593] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:42Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354286502client # [ 152.188753] nix[1471]: copying 1 paths to 'lb:50051'...6503client # [ 152.189637] nix[1471]: copying path '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' to 'lb:50051'...6504worker1 # [ 152.227369] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:42Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:354286505worker1 # [ 152.238473] nix-daemon[867]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)6506worker1 # [ 152.336465] nix-daemon[867]: accepted connection from pid 1720, user nix-grpc-daemon (trusted)6507worker1 # [ 152.349786] nix-daemon[867]: reaped child process 1723, status = succeeded6508lb # [ 152.350623] niks3-server[1150]: 2026/09/24 11:49:42 INFO Received push request method=POST path=/api/pushes6509worker1 # [ 152.366283] nix-grpc-daemon[1672]: time=2026-09-24T11:49:42.658Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6510worker1 # [ 152.368659] nix-grpc-daemon[1672]: time=2026-09-24T11:49:42.659Z level=INFO msg="Uploading q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv (416B)"6511lb # [ 152.380268] niks3-server[1150]: 2026/09/24 11:49:42 INFO Received sign narinfos request method=POST path=/api/pushes/53/sign6512worker1 # [ 152.398801] nix-grpc-daemon[1672]: time=2026-09-24T11:49:42.690Z level=INFO msg="Uploading 1 narinfos"6513lb # [ 152.390778] niks3-server[1150]: 2026/09/24 11:49:42 INFO Signed narinfos id=53 count=16514lb # [ 152.396861] niks3-server[1150]: 2026/09/24 11:49:42 INFO Registered completed upload object_key=nar/1rby34xp94wipnqa4abnh2771jnqp34zv82lymaabrhj07vmwiix.nar.zst6515lb # [ 152.404176] niks3-server[1150]: 2026/09/24 11:49:42 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.ls6516lb # [ 152.412184] niks3-server[1150]: 2026/09/24 11:49:42 INFO Received complete push request method=POST path=/api/pushes/53/complete6517worker1 # [ 152.431490] nix-grpc-daemon[1672]: time=2026-09-24T11:49:42.723Z level=INFO msg="Upload complete. (181ms)"6518lb # [ 152.425461] niks3-server[1150]: 2026/09/24 11:49:42 INFO Registered completed upload object_key=q9pl41yvj7mi4fqnyiarzy5py0107h7b.narinfo6519worker1 # [ 152.436502] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:42Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:35428 duration_s=0 paths=1 nar_bytes_in=4166520worker1 # [ 152.442301] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:42Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:354286521worker1 # [ 152.454187] nix-daemon[867]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)6522worker1 # [ 152.461718] nix-daemon[867]: reaped child process 1718, status = succeeded6523worker1 # [ 152.467052] nix-daemon[867]: accepted connection from pid 1508, user nix-grpc-daemon (trusted)6524client # [ 152.429203] nix[1471]: building '/nix/store/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv' on lb:50051...6525worker1: must succeed: systemctl reload nix-grpc-daemon.service6526worker1 # [ 154.463463] systemd[1]: Reloading Nix worker-protocol over gRPC...6527worker1 # [ 154.487112] systemd[1]: Reloaded Nix worker-protocol over gRPC.6528worker1: (finished: must succeed: systemctl reload nix-grpc-daemon.service, in 0.06 seconds)6529worker1: must succeed: pgrep -f 'read -t [9]0 x'6530worker1: (finished: must succeed: pgrep -f 'read -t [9]0 x', in 0.03 seconds)6531client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&26532worker1 # [ 154.671581] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:44Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:354286533client # don't know how to build these paths:6534client # /nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6535worker1 # [ 154.677231] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:354286536worker2 # [ 154.687655] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:44Z level=debug event=want client=4 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv cached=06537worker2 # [ 154.691719] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:44Z level=debug event=assigned drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv worker=192.168.1.4:50051 clients=16538worker2 # [ 154.698341] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:44Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626539worker1 # [ 154.721131] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:45Z level=info event=draining6540worker2 # [ 154.710723] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:44Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582626541client # copying 1 paths to 'lb:50051'...6542worker2 # [ 154.725774] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:44Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582626543client # copying path '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' to 'lb:50051'...6544worker2 # [ 154.737974] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6545worker2 # [ 154.826449] nix-daemon[857]: accepted connection from pid 1825, user nix-grpc-daemon (trusted)6546worker2 # [ 154.838435] nix-daemon[857]: reaped child process 1828, status = succeeded6547lb # [ 154.851882] niks3-server[1150]: 2026/09/24 11:49:45 INFO Received push request method=POST path=/api/pushes6548worker2 # [ 154.857489] nix-grpc-daemon[1750]: time=2026-09-24T11:49:45.095Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6549worker2 # [ 154.859714] nix-grpc-daemon[1750]: time=2026-09-24T11:49:45.096Z level=INFO msg="Uploading s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv (528B)"6550lb # [ 154.891878] niks3-server[1150]: 2026/09/24 11:49:45 INFO Received sign narinfos request method=POST path=/api/pushes/54/sign6551worker2 # [ 154.892918] nix-grpc-daemon[1750]: time=2026-09-24T11:49:45.130Z level=INFO msg="Uploading 1 narinfos"6552lb # [ 154.897296] niks3-server[1150]: 2026/09/24 11:49:45 INFO Signed narinfos id=54 count=16553lb # [ 154.911768] niks3-server[1150]: 2026/09/24 11:49:45 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.ls6554lb # [ 154.917829] niks3-server[1150]: 2026/09/24 11:49:45 INFO Registered completed upload object_key=nar/0n1m04akn0svzkx0szkj4q7kvaw06fa9yx6k2jibkbkkv85mm55n.nar.zst6555lb # [ 154.923605] niks3-server[1150]: 2026/09/24 11:49:45 INFO Received complete push request method=POST path=/api/pushes/54/complete6556worker2 # [ 154.926780] nix-grpc-daemon[1750]: time=2026-09-24T11:49:45.164Z level=INFO msg="Upload complete. (182ms)"6557lb # [ 154.931270] niks3-server[1150]: 2026/09/24 11:49:45 INFO Registered completed upload object_key=s7lncxagqa332zn7mhwmk3ry9d83sbvq.narinfo6558worker2 # [ 154.929070] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:45Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=5286559worker2 # [ 154.931778] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:45Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626560worker2 # [ 154.939343] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6561worker2 # [ 154.941674] nix-daemon[857]: reaped child process 1823, status = succeeded6562worker2 # [ 154.946850] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6563client # building '/nix/store/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv' on lb:50051...6564client # farm-a> worker2: building s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6565lb # [ 155.418894] envoy[913]: conn peer=[2001:db8:1::1]:34026 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=3934 tx=1942 ms=82116566lb # [ 155.422925] envoy[913]: conn peer=[2001:db8:1::1]:32850 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2980 tx=852 ms=33266567client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds6568client # farm-a> LOG-farm-a-during-drain6569worker2 # [ 155.761127] nix-daemon[857]: reaped child process 1832, status = succeeded6570worker2 # [ 155.837791] nix-daemon[857]: accepted connection from pid 1848, user nix-grpc-daemon (trusted)6571worker2 # [ 155.851515] nix-daemon[857]: reaped child process 1851, status = succeeded6572lb # [ 155.861910] niks3-server[1150]: 2026/09/24 11:49:46 INFO Received push request method=POST path=/api/pushes6573worker2 # [ 155.869632] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.107Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6574worker2 # [ 155.871333] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.108Z level=INFO msg="Uploading zhl4afvdy7vs34m6ngc3z1xk04qgpcfs-farm-a (136B)"6575lb # [ 155.903296] niks3-server[1150]: 2026/09/24 11:49:46 INFO Registered completed upload object_key=log/s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv6576lb # [ 155.914654] niks3-server[1150]: 2026/09/24 11:49:46 INFO Received sign narinfos request method=POST path=/api/pushes/55/sign6577lb # [ 155.918092] niks3-server[1150]: 2026/09/24 11:49:46 INFO Signed narinfos id=55 count=16578worker2 # [ 155.916938] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.154Z level=INFO msg="Uploading 1 narinfos"6579lb # [ 155.925934] niks3-server[1150]: 2026/09/24 11:49:46 INFO Registered completed upload object_key=nar/0r0px9vw070lid3q4jhnmka11sk2y8kj26ysyvjy9dplavd7mdz4.nar.zst6580lb # [ 155.932381] niks3-server[1150]: 2026/09/24 11:49:46 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.ls6581lb # [ 155.935208] niks3-server[1150]: 2026/09/24 11:49:46 INFO Received complete push request method=POST path=/api/pushes/55/complete6582lb # [ 155.941262] niks3-server[1150]: 2026/09/24 11:49:46 INFO Registered completed upload object_key=zhl4afvdy7vs34m6ngc3z1xk04qgpcfs.narinfo6583worker2 # [ 155.941784] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.179Z level=INFO msg="Upload complete. (189ms)"6584worker2 # [ 155.944440] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:46Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=1 drv=s7lncxagqa332zn7mhwmk3ry9d83sbvq-farm-a.drv assign_id=4 outputs=16585worker2 # [ 155.950330] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:46Z level=debug event=want client=4 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv cached=06586worker2 # [ 155.952893] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:46Z level=debug event=assigned drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv worker=192.168.1.4:50051 clients=16587worker2 # [ 155.957440] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626588worker2 # [ 155.960372] nix-daemon[857]: reaped child process 1830, status = succeeded6589worker2 # [ 155.964096] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:46Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582626590worker2 # [ 155.970603] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.208Z level=INFO msg="All 1 paths already cached"6591client # copying 1 paths to 'lb:50051'...6592worker2 # [ 155.975711] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:46Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582626593client # copying path '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' to 'lb:50051'...6594worker2 # [ 155.991309] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6595worker2 # [ 156.078480] nix-daemon[857]: accepted connection from pid 1857, user nix-grpc-daemon (trusted)6596worker2 # [ 156.091169] nix-daemon[857]: reaped child process 1860, status = succeeded6597lb # [ 156.103607] niks3-server[1150]: 2026/09/24 11:49:46 INFO Received push request method=POST path=/api/pushes6598worker2 # [ 156.110067] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.347Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6599worker2 # [ 156.111488] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.347Z level=INFO msg="Uploading pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv (640B)"6600lb # [ 156.134830] niks3-server[1150]: 2026/09/24 11:49:46 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.ls6601lb # [ 156.143099] niks3-server[1150]: 2026/09/24 11:49:46 INFO Received sign narinfos request method=POST path=/api/pushes/56/sign6602worker2 # [ 156.144100] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.381Z level=INFO msg="Uploading 1 narinfos"6603lb # [ 156.149137] niks3-server[1150]: 2026/09/24 11:49:46 INFO Signed narinfos id=56 count=16604lb # [ 156.155527] niks3-server[1150]: 2026/09/24 11:49:46 INFO Registered completed upload object_key=nar/111ql9hf3mg62sp5fwspqwm8cl88198mlcc1ij8flm7l424sz63r.nar.zst6605lb # [ 156.160575] niks3-server[1150]: 2026/09/24 11:49:46 INFO Received complete push request method=POST path=/api/pushes/56/complete6606lb # [ 156.167670] niks3-server[1150]: 2026/09/24 11:49:46 INFO Registered completed upload object_key=pmcfhwqsl1hiyc8gjd850vixdadvskwn.narinfo6607worker2 # [ 156.163100] nix-grpc-daemon[1750]: time=2026-09-24T11:49:46.400Z level=INFO msg="Upload complete. (166ms)"6608worker2 # [ 156.167118] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:46Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=6406609worker2 # [ 156.171816] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:46Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626610worker2 # [ 156.179541] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6611worker2 # [ 156.185496] nix-daemon[857]: reaped child process 1855, status = succeeded6612worker2 # [ 156.189537] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6613client # building '/nix/store/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv' on lb:50051...6614client # farm-b> worker2: building pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6615client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds6616client # farm-b> LOG-farm-b-during-drain6617worker2 # [ 156.821401] nix-daemon[857]: reaped child process 1864, status = succeeded6618worker2 # [ 156.903330] nix-daemon[857]: accepted connection from pid 1880, user nix-grpc-daemon (trusted)6619worker2 # [ 156.915061] nix-daemon[857]: reaped child process 1883, status = succeeded6620lb # [ 156.927639] niks3-server[1150]: 2026/09/24 11:49:47 INFO Received push request method=POST path=/api/pushes6621worker2 # [ 156.936573] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.174Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"6622worker2 # [ 156.938352] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.175Z level=INFO msg="Uploading l458nfm9f1f1zyghsy09ijx02qwk7v65-farm-b (184B)"6623lb # [ 156.974792] niks3-server[1150]: 2026/09/24 11:49:47 INFO Received sign narinfos request method=POST path=/api/pushes/57/sign6624worker2 # [ 156.980020] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.217Z level=INFO msg="Uploading 1 narinfos"6625lb # [ 156.983579] niks3-server[1150]: 2026/09/24 11:49:47 INFO Signed narinfos id=57 count=16626lb # [ 156.994468] niks3-server[1150]: 2026/09/24 11:49:47 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.ls6627lb # [ 156.998117] niks3-server[1150]: 2026/09/24 11:49:47 INFO Registered completed upload object_key=nar/0m50ra4cd4i97vcbcrdsw7xn8f0b3zc5pdyr2ab8kmp3hscd8j80.nar.zst6628lb # [ 157.004629] niks3-server[1150]: 2026/09/24 11:49:47 INFO Registered completed upload object_key=log/pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv6629lb # [ 157.013539] niks3-server[1150]: 2026/09/24 11:49:47 INFO Received complete push request method=POST path=/api/pushes/57/complete6630worker2 # [ 157.018054] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.255Z level=INFO msg="Upload complete. (222ms)"6631lb # [ 157.023623] niks3-server[1150]: 2026/09/24 11:49:47 INFO Registered completed upload object_key=l458nfm9f1f1zyghsy09ijx02qwk7v65.narinfo6632worker2 # [ 157.028403] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:47Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 drv=pmcfhwqsl1hiyc8gjd850vixdadvskwn-farm-b.drv assign_id=5 outputs=16633worker2 # [ 157.038073] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:47Z level=debug event=want client=4 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv cached=06634worker2 # [ 157.042198] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:47Z level=debug event=assigned drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv worker=192.168.1.4:50051 clients=16635worker2 # [ 157.046519] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626636worker2 # [ 157.057498] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582626637worker2 # [ 157.061210] nix-daemon[857]: reaped child process 1862, status = succeeded6638worker2 # [ 157.066335] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.304Z level=INFO msg="All 2 paths already cached"6639client # copying 1 paths to 'lb:50051'...6640worker2 # [ 157.072232] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582626641client # copying path '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' to 'lb:50051'...6642worker2 # [ 157.083936] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6643worker2 # [ 157.170875] nix-daemon[857]: accepted connection from pid 1889, user nix-grpc-daemon (trusted)6644worker2 # [ 157.184595] nix-daemon[857]: reaped child process 1892, status = succeeded6645lb # [ 157.195101] niks3-server[1150]: 2026/09/24 11:49:47 INFO Received push request method=POST path=/api/pushes6646worker2 # [ 157.200507] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.438Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6647worker2 # [ 157.202188] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.439Z level=INFO msg="Uploading w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv (768B)"6648lb # [ 157.232960] niks3-server[1150]: 2026/09/24 11:49:47 INFO Registered completed upload object_key=nar/1crrgw2l7pdmf28ls634k8y85m0pc1v13pxxp4rd6yg5mk3kr5fs.nar.zst6649lb # [ 157.240124] niks3-server[1150]: 2026/09/24 11:49:47 INFO Received sign narinfos request method=POST path=/api/pushes/58/sign6650worker2 # [ 157.242069] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.479Z level=INFO msg="Uploading 1 narinfos"6651lb # [ 157.247147] niks3-server[1150]: 2026/09/24 11:49:47 INFO Signed narinfos id=58 count=16652lb # [ 157.259441] niks3-server[1150]: 2026/09/24 11:49:47 INFO Received complete push request method=POST path=/api/pushes/58/complete6653lb # [ 157.269597] niks3-server[1150]: 2026/09/24 11:49:47 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.ls6654lb # [ 157.273485] niks3-server[1150]: 2026/09/24 11:49:47 INFO Registered completed upload object_key=w4kjbckcz25f2jivly0r4lbq18qf474f.narinfo6655worker2 # [ 157.268662] nix-grpc-daemon[1750]: time=2026-09-24T11:49:47.506Z level=INFO msg="Upload complete. (180ms)"6656worker2 # [ 157.273604] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:47Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=7686657worker2 # [ 157.279462] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626658worker2 # [ 157.289666] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6659worker2 # [ 157.299910] nix-daemon[857]: reaped child process 1887, status = succeeded6660worker2 # [ 157.303871] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6661client # building '/nix/store/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv' on lb:50051...6662client # farm-top> worker2: building w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6663client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds6664client # farm-top> LOG-farm-top-during-drain6665worker2 # [ 158.336519] nix-daemon[857]: reaped child process 1896, status = succeeded6666worker2 # [ 158.412716] nix-daemon[857]: accepted connection from pid 1912, user nix-grpc-daemon (trusted)6667worker2 # [ 158.425068] nix-daemon[857]: reaped child process 1915, status = succeeded6668lb # [ 158.438766] niks3-server[1150]: 2026/09/24 11:49:48 INFO Received push request method=POST path=/api/pushes6669worker2 # [ 158.445500] nix-grpc-daemon[1750]: time=2026-09-24T11:49:48.683Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"6670worker2 # [ 158.447089] nix-grpc-daemon[1750]: time=2026-09-24T11:49:48.684Z level=INFO msg="Uploading 08ap9ki0d5gixmyr2sac4hj06j9jsqhg-farm-top (240B)"6671lb # [ 158.484143] niks3-server[1150]: 2026/09/24 11:49:48 INFO Registered completed upload object_key=log/w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv6672lb # [ 158.490936] niks3-server[1150]: 2026/09/24 11:49:48 INFO Received sign narinfos request method=POST path=/api/pushes/59/sign6673lb # [ 158.496778] niks3-server[1150]: 2026/09/24 11:49:48 INFO Registered completed upload object_key=nar/01f0abadm30zaav33qck3x7ypr9nn6rqh2vbrvih4gyb8c26x9in.nar.zst6674lb # [ 158.503302] niks3-server[1150]: 2026/09/24 11:49:48 INFO Signed narinfos id=59 count=16675worker2 # [ 158.502523] nix-grpc-daemon[1750]: time=2026-09-24T11:49:48.739Z level=INFO msg="Uploading 1 narinfos"6676lb # [ 158.506844] niks3-server[1150]: 2026/09/24 11:49:48 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.ls6677lb # [ 158.518844] niks3-server[1150]: 2026/09/24 11:49:48 INFO Received complete push request method=POST path=/api/pushes/59/complete6678lb # [ 158.524450] niks3-server[1150]: 2026/09/24 11:49:48 INFO Registered completed upload object_key=08ap9ki0d5gixmyr2sac4hj06j9jsqhg.narinfo6679worker2 # [ 158.519259] nix-grpc-daemon[1750]: time=2026-09-24T11:49:48.756Z level=INFO msg="Upload complete. (208ms)"6680worker2 # [ 158.525961] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:48Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=1 drv=w4kjbckcz25f2jivly0r4lbq18qf474f-farm-top.drv assign_id=6 outputs=16681worker2 # [ 158.540549] nix-daemon[857]: reaped child process 1894, status = succeeded6682client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain >&2, in 4.06 seconds)6683client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath6684client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag during-drain outPath, in 0.11 seconds)6685worker1: must succeed: pkill -f 'read -t [9]0 x'6686worker1: (finished: must succeed: pkill -f 'read -t [9]0 x', in 0.04 seconds)6687client: waiting for success: ! systemctl is-active sw6688worker1 # [ 158.918488] nix-daemon[867]: reaped child process 1727, status = succeeded6689worker1 # [ 158.989083] nix-daemon[867]: accepted connection from pid 1769, user nix-grpc-daemon (trusted)6690worker1 # [ 159.002610] nix-daemon[867]: reaped child process 1772, status = succeeded6691lb # [ 158.997994] niks3-server[1150]: 2026/09/24 11:49:49 INFO Received push request method=POST path=/api/pushes6692worker1 # [ 159.015726] nix-grpc-daemon[1672]: time=2026-09-24T11:49:49.307Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6693worker1 # [ 159.018251] nix-grpc-daemon[1672]: time=2026-09-24T11:49:49.309Z level=INFO msg="Uploading mlk55snqrikr02pwsxjamphvl61rvl0r-slow-sw (120B)"6694lb # [ 159.037197] niks3-server[1150]: 2026/09/24 11:49:49 INFO Registered completed upload object_key=nar/0vi6cx31wc05wns04pr5vr2bak9iib1idndvy6gik6c2rz9q8c19.nar.zst6695lb # [ 159.050558] niks3-server[1150]: 2026/09/24 11:49:49 INFO Received sign narinfos request method=POST path=/api/pushes/60/sign6696lb # [ 159.061535] niks3-server[1150]: 2026/09/24 11:49:49 INFO Signed narinfos id=60 count=16697worker1 # [ 159.072608] nix-grpc-daemon[1672]: time=2026-09-24T11:49:49.364Z level=INFO msg="Uploading 1 narinfos"6698lb # [ 159.067496] niks3-server[1150]: 2026/09/24 11:49:49 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.ls6699lb # [ 159.071176] niks3-server[1150]: 2026/09/24 11:49:49 INFO Registered completed upload object_key=log/q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv6700lb # [ 159.080064] niks3-server[1150]: 2026/09/24 11:49:49 INFO Received complete push request method=POST path=/api/pushes/60/complete6701lb # [ 159.086062] niks3-server[1150]: 2026/09/24 11:49:49 INFO Registered completed upload object_key=mlk55snqrikr02pwsxjamphvl61rvl0r.narinfo6702worker1 # [ 159.092006] nix-grpc-daemon[1672]: time=2026-09-24T11:49:49.383Z level=INFO msg="Upload complete. (211ms)"6703worker1 # [ 159.099081] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:35428 duration_s=6 drv=q9pl41yvj7mi4fqnyiarzy5py0107h7b-slow-sw.drv assign_id=3 outputs=16704worker1 # [ 159.113725] nix-daemon[867]: reaped child process 1725, status = succeeded6705client # [ 159.091811] systemd[1]: sw.service: Deactivated successfully.6706client # [ 159.096112] systemd[1]: sw.service: Consumed 124ms CPU time over 7.079s wall clock time, 41.4M memory peak, 7.3K incoming IP traffic, 8.4K outgoing IP traffic.6707worker1 # [ 159.722853] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:50Z level=info event=signal_exit6708client: (finished: waiting for success: ! systemctl is-active sw, in 1.06 seconds)6709client: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }6710client: (finished: must succeed: systemctl show -p Result --value sw | grep -qx success || { journalctl -u sw >&2; false; }, in 0.04 seconds)6711worker1: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service6712worker2 # [ 161.716601] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:51Z level=info event=worker_session_closed peer=ipv4:192.168.1.2:489326713worker1 # [ 161.732326] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:52Z level=info event=scheduler_restarting6714worker2 # [ 161.723997] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:51Z level=info event=worker_gone addr=192.168.1.3:500516715worker1 # [ 161.742543] nix-grpc-daemon[1508]: ts=2026-09-24T11:49:52Z level=info event=niks3_push_exited status=succeeded why="niks3 push exited"6716worker1 # [ 161.761818] nix-daemon[867]: reaped child process 1567, status = succeeded6717worker1 # [ 163.022910] systemd[1]: nix-grpc-daemon.service: Deactivated successfully.6718worker1 # [ 163.026644] systemd[1]: nix-grpc-daemon.service: Consumed 673ms CPU time over 38.421s wall clock time, 34.8M memory peak, 58.5K incoming IP traffic, 68.1K outgoing IP traffic.6719worker1 # [ 163.038950] systemd[1]: nix-grpc-daemon.service: Scheduled restart job immediately on client request, restart counter is at 1.6720worker1 # [ 163.049888] systemd[1]: Starting Nix worker-protocol over gRPC...6721worker1 # [ 163.136303] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:53Z level=info event=warning msg="--niks3 http:// sends the bearer token in clear"6722worker1 # [ 163.138273] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:53Z level=info event=oidc_provider name=mock issuer=http://192.168.1.2:8080/oidc6723worker1 # [ 163.143891] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:53Z level=info event=roles builder=1 scheduler=1 scheduler_addr=lb:50051 advertise=192.168.1.3:50051 niks3=http://lb:5751 max_jobs=16724worker1 # [ 163.148348] nix-grpc-daemon[1787]: WARNING: All log messages before absl::InitializeLog() is called are written to STDERR6725worker1 # [ 163.151707] nix-grpc-daemon[1787]: E0000 00:00:1790250593.438172 1787 tls_security_connector.cc:771] TlsServerCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6726worker1 # [ 163.156106] nix-grpc-daemon[1787]: E0000 00:00:1790250593.439475 1787 tls_security_connector.cc:767] TlsServerCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6727worker1 # [ 163.159602] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:53Z level=info event=startup listen=[::]:50051 proxy_socket=/nix/var/nix/daemon-socket/socket6728worker1 # [ 163.161646] systemd[1]: Started Nix worker-protocol over gRPC.6729worker1 # [ 163.164939] nix-grpc-daemon[1787]: E0000 00:00:1790250593.453868 1829 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6730worker1 # [ 163.176217] nix-grpc-daemon[1787]: E0000 00:00:1790250593.468089 1829 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6731worker1 # [ 163.180139] nix-grpc-daemon[1787]: E0000 00:00:1790250593.470416 1829 tls_security_connector.cc:493] TlsChannelCertificateWatcher getting identity_cert_error: UNKNOWN:Unable to get latest identity certificates.6732worker1: (finished: waiting for success: systemctl is-active nix-grpc-daemon.service || systemctl start nix-grpc-daemon.service, in 3.36 seconds)6733lb: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux6734lb: must succeed: curl -sf localhost:9901/clusters6735worker1 # [ 163.191139] nix-grpc-daemon[1787]: E0000 00:00:1790250593.483066 1829 tls_security_connector.cc:489] TlsChannelCertificateWatcher getting root_cert_error: UNKNOWN:Unable to get latest root certificates.6736worker1 # [ 163.204200] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:53Z level=info event=scheduler_connected addr=lb:500516737worker2 # [ 163.196279] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=info event=worker_session_open cn=worker-1 peer=ipv4:192.168.1.2:489326738worker2 # [ 163.200681] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=info event=worker_hello addr=192.168.1.3:50051 systems=x86_64-linux,i686-linux,x86_64-v1-linux,x86_64-v2-linux,x86_64-v3-linux,x86_64-v4-linux max_jobs=1 running=06739lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6740lb: (finished: waiting until exactly ['worker1', 'worker2'] are healthy members of x86_64-linux, in 0.06 seconds)6741lb: must succeed: curl -sf localhost:9901/clusters6742lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6743worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6744worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6745(finished: subtest: a deploy mid-build drains: the build finishes, the new generation takes over, in 11.31 seconds)6746subtest: queue gauges show a burst while every build blocks6747lb: must succeed: curl -sf localhost:9901/clusters6748lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6749lb: must succeed: curl -sf localhost:9901/clusters6750lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6751worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6752worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6753client: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]'6754client # [ 163.411966] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key" --eval-store auto --impure --expr "map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ \"q0\" \"q1\" \"q2\" \"q3\" ]".6755client # Running as unit: burst.service; invocation ID: db0f297a5abd4c529f282fd5ac270af46756client: (finished: must succeed: systemd-run --unit burst nix build --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --impure --expr 'map (tag: import /nix/store/xm9d525p8sv28c6nivsdjrjpb8kgp3fa-slow.nix { inherit tag; }) [ "q0" "q1" "q2" "q3" ]', in 0.06 seconds)6757lb: must succeed: curl -sf localhost:9901/clusters6758lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6759lb: must succeed: curl -sf localhost:9901/clusters6760lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6761worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6762client # [ 163.549962] nix[1526]: don't know how to build these paths:6763client # [ 163.550905] nix[1526]: /nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6764client # [ 163.551922] nix[1526]: /nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6765worker2 # [ 163.574769] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582626766client # [ 163.553159] nix[1526]: /nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6767client # [ 163.554204] nix[1526]: /nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6768worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)6769worker2 # [ 163.584801] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=want client=5 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv cached=06770worker2 # [ 163.589229] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=assigned drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv worker=192.168.1.4:50051 clients=16771worker1 # [ 163.603172] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555186772worker2 # [ 163.593982] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=want client=5 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv cached=06773worker2 # [ 163.597572] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=assigned drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv worker=192.168.1.3:50051 clients=16774worker2 # [ 163.601103] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=want client=5 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv cached=06775worker1 # [ 163.614632] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6776worker2 # [ 163.604270] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=want client=5 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv cached=06777worker2 # [ 163.606551] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626778worker2 # [ 163.614899] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582626779client # [ 163.595066] nix[1526]: copying 1 paths to 'lb:50051'...6780worker2 # [ 163.621860] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:53Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582626781client # [ 163.595952] nix[1526]: copying path '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' to 'lb:50051'...6782worker1 # [ 163.630849] nix-grpc-daemon[1787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 50 ms (attempt 1/5)6783worker2 # [ 163.625686] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6784worker1 # [ 163.688906] nix-grpc-daemon[1787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 114 ms (attempt 2/5)6785worker2 # [ 163.713228] nix-daemon[857]: accepted connection from pid 1930, user nix-grpc-daemon (trusted)6786worker2 # [ 163.725388] nix-daemon[857]: reaped child process 1933, status = succeeded6787lb # [ 163.738938] niks3-server[1150]: 2026/09/24 11:49:54 INFO Received push request method=POST path=/api/pushes6788worker2 # [ 163.744214] nix-grpc-daemon[1750]: time=2026-09-24T11:49:53.981Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6789worker2 # [ 163.746214] nix-grpc-daemon[1750]: time=2026-09-24T11:49:53.983Z level=INFO msg="Uploading 2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv (416B)"6790lb # [ 163.779815] niks3-server[1150]: 2026/09/24 11:49:54 INFO Received sign narinfos request method=POST path=/api/pushes/61/sign6791lb # [ 163.783888] niks3-server[1150]: 2026/09/24 11:49:54 INFO Registered completed upload object_key=nar/0raymp7y3ngiv7lj2qq6nkp63sms2510yhhlnr7hnz7ldnl7b3qc.nar.zst6792worker2 # [ 163.784935] nix-grpc-daemon[1750]: time=2026-09-24T11:49:54.022Z level=INFO msg="Uploading 1 narinfos"6793lb # [ 163.788996] niks3-server[1150]: 2026/09/24 11:49:54 INFO Signed narinfos id=61 count=16794lb # [ 163.800262] niks3-server[1150]: 2026/09/24 11:49:54 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.ls6795worker1 # [ 163.804879] nix-grpc-daemon[1787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 143 ms (attempt 3/5)6796lb # [ 163.809759] niks3-server[1150]: 2026/09/24 11:49:54 INFO Received complete push request method=POST path=/api/pushes/61/complete6797worker2 # [ 163.809706] nix-grpc-daemon[1750]: time=2026-09-24T11:49:54.047Z level=INFO msg="Upload complete. (178ms)"6798lb # [ 163.816195] niks3-server[1150]: 2026/09/24 11:49:54 INFO Registered completed upload object_key=2s9psgqm7awsyhlfyaclma7yr7qmrk0g.narinfo6799worker2 # [ 163.813554] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:54Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=4166800worker2 # [ 163.818673] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:54Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626801worker2 # [ 163.829173] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6802worker2 # [ 163.837039] nix-daemon[857]: reaped child process 1928, status = succeeded6803worker2 # [ 163.841581] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6804client # [ 163.814916] nix[1526]: building '/nix/store/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv' on lb:50051...6805worker1 # [ 163.949865] nix-grpc-daemon[1787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org; retrying in 507 ms (attempt 4/5)6806worker1 # [ 164.487875] nix-grpc-daemon[1787]: disabling binary cache 'https://cache.nixos.org' for 60 seconds6807worker1 # [ 164.491496] nix-grpc-daemon[1787]: warning: unable to download 'https://cache.nixos.org/nix-cache-info': Could not resolve hostname (6) Could not resolve host: cache.nixos.org6808worker1 # [ 164.509492] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:54Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555186809client # [ 164.482313] nix[1526]: copying 1 paths to 'lb:50051'...6810client # [ 164.486895] nix[1526]: copying path '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' to 'lb:50051'...6811worker1 # [ 164.524693] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:54Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555186812worker1 # [ 164.532975] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6813worker1 # [ 164.556214] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:54Z level=info event=niks3_push_started pid=18476814lb: must succeed: curl -sf localhost:9901/clusters6815lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.06 seconds)6816lb: must succeed: curl -sf localhost:9901/clusters6817worker1 # [ 164.660578] nix-grpc-daemon[1847]: time=2026-09-24T11:49:54.952Z level=INFO msg="Configuring client TLS" cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.pem key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/worker1.key ca=""6818lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)6819worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6820worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6821worker1 # [ 164.753084] nix-daemon[867]: accepted connection from pid 1860, user nix-grpc-daemon (trusted)6822worker1 # [ 164.772919] nix-daemon[867]: reaped child process 1866, status = succeeded6823lb # [ 164.766340] niks3-server[1150]: 2026/09/24 11:49:55 INFO Received push request method=POST path=/api/pushes6824worker1 # [ 164.784989] nix-grpc-daemon[1847]: time=2026-09-24T11:49:55.076Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6825worker1 # [ 164.788054] nix-grpc-daemon[1847]: time=2026-09-24T11:49:55.076Z level=INFO msg="Uploading 885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv (416B)"6826lb # [ 164.805924] niks3-server[1150]: 2026/09/24 11:49:55 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.ls6827lb # [ 164.808653] niks3-server[1150]: 2026/09/24 11:49:55 INFO Received sign narinfos request method=POST path=/api/pushes/62/sign6828worker1 # [ 164.819232] nix-grpc-daemon[1847]: time=2026-09-24T11:49:55.111Z level=INFO msg="Uploading 1 narinfos"6829lb # [ 164.812579] niks3-server[1150]: 2026/09/24 11:49:55 INFO Signed narinfos id=62 count=16830lb # [ 164.819070] niks3-server[1150]: 2026/09/24 11:49:55 INFO Registered completed upload object_key=nar/0mmcg3cvna05ngbx6v5vspkmc5gj8lna2zc48rllhxr0g8i9mg6y.nar.zst6831lb # [ 164.824712] niks3-server[1150]: 2026/09/24 11:49:55 INFO Received complete push request method=POST path=/api/pushes/62/complete6832worker1 # [ 164.836251] nix-grpc-daemon[1847]: time=2026-09-24T11:49:55.128Z level=INFO msg="Upload complete. (172ms)"6833worker1 # [ 164.838494] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:55Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=4166834lb # [ 164.831571] niks3-server[1150]: 2026/09/24 11:49:55 INFO Registered completed upload object_key=885n55y9srkrjs4p487lr255wvhla7hq.narinfo6835worker1 # [ 164.841818] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:55Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555186836worker1 # [ 164.848546] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6837worker1 # [ 164.850844] nix-daemon[867]: reaped child process 1845, status = succeeded6838worker1 # [ 164.855927] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6839client # [ 164.819879] nix[1526]: building '/nix/store/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv' on lb:50051...6840lb # [ 165.418932] envoy[913]: conn peer=[2001:db8:1::1]:48154 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9538 tx=4345 ms=38856841lb # [ 165.423140] envoy[913]: conn peer=[2001:db8:1::1]:32854 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=2975 tx=1515 ms=69216842lb # [ 165.426922] envoy[913]: conn peer=[2001:db8:1::3]:51276 tls=TLSv1.3 subject="CN=worker-1" sni=- flags=- tls_fail="-" rx=3038 tx=857 ms=369766843worker2 # [ 166.070871] nix-daemon[857]: reaped child process 1937, status = succeeded6844worker2 # [ 166.146714] nix-daemon[857]: accepted connection from pid 1965, user nix-grpc-daemon (trusted)6845worker2 # [ 166.159072] nix-daemon[857]: reaped child process 1968, status = succeeded6846lb # [ 166.173721] niks3-server[1150]: 2026/09/24 11:49:56 INFO Received push request method=POST path=/api/pushes6847worker2 # [ 166.177914] nix-grpc-daemon[1750]: time=2026-09-24T11:49:56.415Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6848worker2 # [ 166.179502] nix-grpc-daemon[1750]: time=2026-09-24T11:49:56.417Z level=INFO msg="Uploading vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq-slow-q3 (120B)"6849lb # [ 166.221781] niks3-server[1150]: 2026/09/24 11:49:56 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.ls6850lb # [ 166.231512] niks3-server[1150]: 2026/09/24 11:49:56 INFO Received sign narinfos request method=POST path=/api/pushes/63/sign6851lb # [ 166.234789] niks3-server[1150]: 2026/09/24 11:49:56 INFO Registered completed upload object_key=nar/1v1grzkvw3yj09x0dkdm2avihvqvnri156qzp768hk85zbw7lkn9.nar.zst6852lb # [ 166.240555] niks3-server[1150]: 2026/09/24 11:49:56 INFO Signed narinfos id=63 count=16853worker2 # [ 166.239526] nix-grpc-daemon[1750]: time=2026-09-24T11:49:56.477Z level=INFO msg="Uploading 1 narinfos"6854lb # [ 166.247250] niks3-server[1150]: 2026/09/24 11:49:56 INFO Registered completed upload object_key=log/2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv6855lb # [ 166.258285] niks3-server[1150]: 2026/09/24 11:49:56 INFO Received complete push request method=POST path=/api/pushes/63/complete6856lb # [ 166.264432] niks3-server[1150]: 2026/09/24 11:49:56 INFO Registered completed upload object_key=vn3s5fgnxq5d2l4d4nn96l2q9wfdg4mq.narinfo6857worker2 # [ 166.259215] nix-grpc-daemon[1750]: time=2026-09-24T11:49:56.496Z level=INFO msg="Upload complete. (209ms)"6858worker2 # [ 166.267737] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:56Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=2 drv=2s9psgqm7awsyhlfyaclma7yr7qmrk0g-slow-q3.drv assign_id=7 outputs=16859worker2 # [ 166.276242] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:56Z level=debug event=assigned drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv worker=192.168.1.4:50051 clients=16860worker2 # [ 166.280665] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626861worker2 # [ 166.290526] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:56Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582626862worker2 # [ 166.297360] nix-daemon[857]: reaped child process 1935, status = succeeded6863worker2 # [ 166.299434] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:56Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582626864client # [ 166.272993] nix[1526]: copying 1 paths to 'lb:50051'...6865client # [ 166.276231] nix[1526]: copying path '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' to 'lb:50051'...6866worker2 # [ 166.308630] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6867worker2 # [ 166.395288] nix-daemon[857]: accepted connection from pid 1972, user nix-grpc-daemon (trusted)6868worker2 # [ 166.407431] nix-daemon[857]: reaped child process 1975, status = succeeded6869lb # [ 166.419876] niks3-server[1150]: 2026/09/24 11:49:56 INFO Received push request method=POST path=/api/pushes6870worker2 # [ 166.424789] nix-grpc-daemon[1750]: time=2026-09-24T11:49:56.662Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6871worker2 # [ 166.426700] nix-grpc-daemon[1750]: time=2026-09-24T11:49:56.664Z level=INFO msg="Uploading 8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv (416B)"6872lb # [ 166.454061] niks3-server[1150]: 2026/09/24 11:49:56 INFO Received sign narinfos request method=POST path=/api/pushes/64/sign6873lb # [ 166.460908] niks3-server[1150]: 2026/09/24 11:49:56 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.ls6874lb # [ 166.467551] niks3-server[1150]: 2026/09/24 11:49:56 INFO Signed narinfos id=64 count=16875worker2 # [ 166.466923] nix-grpc-daemon[1750]: time=2026-09-24T11:49:56.704Z level=INFO msg="Uploading 1 narinfos"6876lb # [ 166.474667] niks3-server[1150]: 2026/09/24 11:49:56 INFO Registered completed upload object_key=nar/1xibay372y9by9a678y9h7bfzwahsxi0vq51acgywak31q6q2m87.nar.zst6877lb # [ 166.487081] niks3-server[1150]: 2026/09/24 11:49:56 INFO Received complete push request method=POST path=/api/pushes/64/complete6878lb # [ 166.493717] niks3-server[1150]: 2026/09/24 11:49:56 INFO Registered completed upload object_key=8v867csm7d9sip4xd4nlg9jwvrpqqwv9.narinfo6879worker2 # [ 166.488994] nix-grpc-daemon[1750]: time=2026-09-24T11:49:56.726Z level=INFO msg="Upload complete. (174ms)"6880worker2 # [ 166.493565] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:56Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=4166881worker2 # [ 166.500309] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:56Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626882worker2 # [ 166.508222] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6883worker2 # [ 166.518227] nix-daemon[857]: reaped child process 1970, status = succeeded6884worker2 # [ 166.522941] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6885client # [ 166.496425] nix[1526]: building '/nix/store/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv' on lb:50051...6886worker2 # [ 167.137879] nix-daemon[857]: reaped child process 1979, status = succeeded6887worker2 # [ 167.212887] nix-daemon[857]: accepted connection from pid 2000, user nix-grpc-daemon (trusted)6888worker2 # [ 167.224718] nix-daemon[857]: reaped child process 2003, status = succeeded6889lb # [ 167.238594] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received push request method=POST path=/api/pushes6890worker2 # [ 167.245482] nix-grpc-daemon[1750]: time=2026-09-24T11:49:57.483Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6891worker2 # [ 167.247616] nix-grpc-daemon[1750]: time=2026-09-24T11:49:57.484Z level=INFO msg="Uploading wmw8972gq2nf739xhg7is7lylxf8299y-slow-q0 (120B)"6892lb # [ 167.289344] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=nar/0xr809jj32p5n6mnc6i7lqsz48wrm5f9jp14m77k724041711iq7.nar.zst6893lb # [ 167.294859] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.ls6894lb # [ 167.299891] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received sign narinfos request method=POST path=/api/pushes/65/sign6895lb # [ 167.306520] niks3-server[1150]: 2026/09/24 11:49:57 INFO Signed narinfos id=65 count=16896worker1 # [ 167.309589] nix-daemon[867]: reaped child process 1871, status = succeeded6897worker2 # [ 167.306217] nix-grpc-daemon[1750]: time=2026-09-24T11:49:57.543Z level=INFO msg="Uploading 1 narinfos"6898lb # [ 167.312690] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=log/8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv6899lb # [ 167.323661] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received complete push request method=POST path=/api/pushes/65/complete6900lb # [ 167.329310] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=wmw8972gq2nf739xhg7is7lylxf8299y.narinfo6901worker2 # [ 167.324917] nix-grpc-daemon[1750]: time=2026-09-24T11:49:57.562Z level=INFO msg="Upload complete. (196ms)"6902worker2 # [ 167.333594] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 drv=8v867csm7d9sip4xd4nlg9jwvrpqqwv9-slow-q0.drv assign_id=9 outputs=16903worker2 # [ 167.340629] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:57Z level=debug event=assigned drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv worker=192.168.1.4:50051 clients=16904worker2 # [ 167.346232] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626905worker2 # [ 167.356510] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:57Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582626906worker1 # [ 167.371615] nix-daemon[867]: accepted connection from pid 1896, user nix-grpc-daemon (trusted)6907worker2 # [ 167.363184] nix-daemon[857]: reaped child process 1977, status = succeeded6908client # [ 167.338899] nix[1526]: copying 1 paths to 'lb:50051'...6909worker2 # [ 167.365867] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:57Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582626910client # [ 167.341481] nix[1526]: copying path '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' to 'lb:50051'...6911worker1 # [ 167.382582] nix-daemon[867]: reaped child process 1899, status = succeeded6912worker2 # [ 167.374299] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6913lb # [ 167.381647] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received push request method=POST path=/api/pushes6914worker1 # [ 167.397328] nix-grpc-daemon[1847]: time=2026-09-24T11:49:57.689Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6915worker1 # [ 167.400057] nix-grpc-daemon[1847]: time=2026-09-24T11:49:57.690Z level=INFO msg="Uploading nv0kp5b571hsjibnd9v06981i024iima-slow-q2 (120B)"6916lb # [ 167.411813] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=log/885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv6917lb # [ 167.414778] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received sign narinfos request method=POST path=/api/pushes/66/sign6918lb # [ 167.417587] niks3-server[1150]: 2026/09/24 11:49:57 INFO Signed narinfos id=66 count=16919worker1 # [ 167.425439] nix-grpc-daemon[1847]: time=2026-09-24T11:49:57.717Z level=INFO msg="Uploading 1 narinfos"6920lb # [ 167.419280] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=nar/15imdx7zyy6mzjjv6qvdzywh0d9w1n6ikc0rq4rq95aw9x6swfzj.nar.zst6921lb # [ 167.426091] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received complete push request method=POST path=/api/pushes/66/complete6922lb # [ 167.429102] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.ls6923worker1 # [ 167.440157] nix-grpc-daemon[1847]: time=2026-09-24T11:49:57.732Z level=INFO msg="Upload complete. (175ms)"6924lb # [ 167.433653] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=nv0kp5b571hsjibnd9v06981i024iima.narinfo6925worker1 # [ 167.442723] nix-grpc-daemon[1787]: ts=2026-09-24T11:49:57Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=2 drv=885n55y9srkrjs4p487lr255wvhla7hq-slow-q2.drv assign_id=8 outputs=16926worker1 # [ 167.449354] nix-daemon[867]: reaped child process 1869, status = succeeded6927worker2 # [ 167.462810] nix-daemon[857]: accepted connection from pid 2007, user nix-grpc-daemon (trusted)6928worker2 # [ 167.475329] nix-daemon[857]: reaped child process 2010, status = succeeded6929lb # [ 167.487204] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received push request method=POST path=/api/pushes6930worker2 # [ 167.494092] nix-grpc-daemon[1750]: time=2026-09-24T11:49:57.731Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6931worker2 # [ 167.495574] nix-grpc-daemon[1750]: time=2026-09-24T11:49:57.731Z level=INFO msg="Uploading qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv (416B)"6932lb # [ 167.515904] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.ls6933lb # [ 167.517976] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received sign narinfos request method=POST path=/api/pushes/67/sign6934lb # [ 167.520139] niks3-server[1150]: 2026/09/24 11:49:57 INFO Signed narinfos id=67 count=16935worker2 # [ 167.516265] nix-grpc-daemon[1750]: time=2026-09-24T11:49:57.753Z level=INFO msg="Uploading 1 narinfos"6936lb # [ 167.524547] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=nar/1spxkiwzvlj9sgbjal4jf4p68h5kbylxymfww73s5va157qxgafi.nar.zst6937lb # [ 167.528268] niks3-server[1150]: 2026/09/24 11:49:57 INFO Received complete push request method=POST path=/api/pushes/67/complete6938worker2 # [ 167.530242] nix-grpc-daemon[1750]: time=2026-09-24T11:49:57.768Z level=INFO msg="Upload complete. (150ms)"6939lb # [ 167.535834] niks3-server[1150]: 2026/09/24 11:49:57 INFO Registered completed upload object_key=qs0cgdr560r5cllf2djj9bb4gz0vrp4p.narinfo6940worker2 # [ 167.532390] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:57Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=4166941worker2 # [ 167.536654] nix-grpc-daemon[1670]: ts=2026-09-24T11:49:57Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582626942worker2 # [ 167.543054] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6943worker2 # [ 167.544212] nix-daemon[857]: reaped child process 2005, status = succeeded6944worker2 # [ 167.549328] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)6945client # [ 167.522619] nix[1526]: building '/nix/store/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv' on lb:50051...6946worker2 # [ 169.587559] nix-daemon[857]: reaped child process 2014, status = succeeded6947worker2 # [ 169.673250] nix-daemon[857]: accepted connection from pid 2039, user nix-grpc-daemon (trusted)6948worker2 # [ 169.686073] nix-daemon[857]: reaped child process 2042, status = succeeded6949lb # [ 169.698528] niks3-server[1150]: 2026/09/24 11:50:00 INFO Received push request method=POST path=/api/pushes6950worker2 # [ 169.703378] nix-grpc-daemon[1750]: time=2026-09-24T11:49:59.940Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6951worker2 # [ 169.705333] nix-grpc-daemon[1750]: time=2026-09-24T11:49:59.942Z level=INFO msg="Uploading 106i40yblbkazayvn9ppswrq6kqs26sj-slow-q1 (120B)"6952lb # [ 169.741947] niks3-server[1150]: 2026/09/24 11:50:00 INFO Received sign narinfos request method=POST path=/api/pushes/68/sign6953worker2 # [ 169.744294] nix-grpc-daemon[1750]: time=2026-09-24T11:49:59.981Z level=INFO msg="Uploading 1 narinfos"6954lb # [ 169.748485] niks3-server[1150]: 2026/09/24 11:50:00 INFO Signed narinfos id=68 count=16955lb # [ 169.756983] niks3-server[1150]: 2026/09/24 11:50:00 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.ls6956lb # [ 169.761400] niks3-server[1150]: 2026/09/24 11:50:00 INFO Registered completed upload object_key=log/qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv6957lb # [ 169.769050] niks3-server[1150]: 2026/09/24 11:50:00 INFO Registered completed upload object_key=nar/0c1c367r3msa4fz6imwp70zxi4fxvrkb86v3hv921wsrf13yd8v3.nar.zst6958lb # [ 169.781117] niks3-server[1150]: 2026/09/24 11:50:00 INFO Received complete push request method=POST path=/api/pushes/68/complete6959worker2 # [ 169.783878] nix-grpc-daemon[1750]: time=2026-09-24T11:50:00.021Z level=INFO msg="Upload complete. (224ms)"6960lb # [ 169.789393] niks3-server[1150]: 2026/09/24 11:50:00 INFO Registered completed upload object_key=106i40yblbkazayvn9ppswrq6kqs26sj.narinfo6961worker2 # [ 169.790751] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:00Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=2 drv=qs0cgdr560r5cllf2djj9bb4gz0vrp4p-slow-q1.drv assign_id=10 outputs=16962worker2 # [ 169.805937] nix-daemon[857]: reaped child process 2012, status = succeeded6963client # [ 169.800911] systemd[1]: burst.service: Deactivated successfully.6964client # [ 169.804790] systemd[1]: burst.service: Consumed 163ms CPU time over 6.393s wall clock time, 55.9M memory peak, 13.9K incoming IP traffic, 18.7K outgoing IP traffic.6965client: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }6966client: (finished: must succeed: systemctl show -p Result --value burst | grep -qx success || { journalctl -u burst >&2; false; }, in 0.04 seconds)6967lb: must succeed: curl -sf localhost:9901/clusters6968lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)6969lb: must succeed: curl -sf localhost:9901/clusters6970lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)6971worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics6972worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)6973(finished: subtest: queue gauges show a burst while every build blocks, in 7.39 seconds)6974subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does6975client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&16976worker1 # [ 170.846515] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:01Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:555186977worker2 # [ 170.843789] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582626978worker2 # [ 170.886131] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:01Z level=debug event=want client=6 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv cached=06979worker1 # [ 170.900652] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555186980worker2 # [ 170.890711] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:01Z level=debug event=assigned drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv worker=192.168.1.3:50051 clients=16981worker2 # [ 170.895196] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:01Z level=debug event=want client=6 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv cached=06982worker2 # [ 170.899380] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:01Z level=debug event=want client=6 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv cached=06983worker1 # [ 170.911346] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:01Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555186984worker1 # [ 170.921890] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:01Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555186985worker1 # [ 170.933111] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)6986worker1 # [ 171.035209] nix-daemon[867]: accepted connection from pid 1915, user nix-grpc-daemon (trusted)6987worker1 # [ 171.047655] nix-daemon[867]: reaped child process 1918, status = succeeded6988lb # [ 171.051088] niks3-server[1150]: 2026/09/24 11:50:01 INFO Received push request method=POST path=/api/pushes6989worker1 # [ 171.066543] nix-grpc-daemon[1847]: time=2026-09-24T11:50:01.358Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"6990worker1 # [ 171.068619] nix-grpc-daemon[1847]: time=2026-09-24T11:50:01.360Z level=INFO msg="Uploading cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv (504B)"6991lb # [ 171.079420] niks3-server[1150]: 2026/09/24 11:50:01 INFO Received sign narinfos request method=POST path=/api/pushes/69/sign6992worker1 # [ 171.095515] nix-grpc-daemon[1847]: time=2026-09-24T11:50:01.387Z level=INFO msg="Uploading 1 narinfos"6993lb # [ 171.087519] niks3-server[1150]: 2026/09/24 11:50:01 INFO Signed narinfos id=69 count=16994lb # [ 171.097131] niks3-server[1150]: 2026/09/24 11:50:01 INFO Registered completed upload object_key=nar/1ifhv0ll3mnj6a2hrp33fzwnm2yl2zjxwqfrqjriibhs5cbnyiwi.nar.zst6995lb # [ 171.101388] niks3-server[1150]: 2026/09/24 11:50:01 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.ls6996lb # [ 171.109052] niks3-server[1150]: 2026/09/24 11:50:01 INFO Received complete push request method=POST path=/api/pushes/69/complete6997lb # [ 171.115252] niks3-server[1150]: 2026/09/24 11:50:01 INFO Registered completed upload object_key=cv151lgjr4lv54sqzb49xk9w1ak9xsm9.narinfo6998worker1 # [ 171.120898] nix-grpc-daemon[1847]: time=2026-09-24T11:50:01.412Z level=INFO msg="Upload complete. (176ms)"6999worker1 # [ 171.124916] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:01Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=5047000worker1 # [ 171.129857] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:01Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187001worker1 # [ 171.139396] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7002worker1 # [ 171.144406] nix-daemon[867]: reaped child process 1913, status = succeeded7003worker1 # [ 171.148790] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7004worker1 # [ 172.585615] nix-daemon[867]: reaped child process 1922, status = succeeded7005worker1 # [ 172.669483] nix-daemon[867]: accepted connection from pid 1938, user nix-grpc-daemon (trusted)7006worker1 # [ 172.681469] nix-daemon[867]: reaped child process 1941, status = succeeded7007lb # [ 172.680904] niks3-server[1150]: 2026/09/24 11:50:03 INFO Received push request method=POST path=/api/pushes7008worker1 # [ 172.699452] nix-grpc-daemon[1847]: time=2026-09-24T11:50:02.991Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7009worker1 # [ 172.700955] nix-grpc-daemon[1847]: time=2026-09-24T11:50:02.992Z level=INFO msg="Uploading wfziwgf0gxmjhy17lyfmn47628s166p7-farm-a (128B)"7010lb # [ 172.724887] niks3-server[1150]: 2026/09/24 11:50:03 INFO Received sign narinfos request method=POST path=/api/pushes/70/sign7011worker1 # [ 172.738264] nix-grpc-daemon[1847]: time=2026-09-24T11:50:03.030Z level=INFO msg="Uploading 1 narinfos"7012lb # [ 172.730543] niks3-server[1150]: 2026/09/24 11:50:03 INFO Signed narinfos id=70 count=17013lb # [ 172.742137] niks3-server[1150]: 2026/09/24 11:50:03 INFO Registered completed upload object_key=log/cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv7014lb # [ 172.746822] niks3-server[1150]: 2026/09/24 11:50:03 INFO Registered completed upload object_key=nar/05i7ka1qa7vmhiys96ln3pp3nj3rfdlslnbydrxdqj40cmvmqcwf.nar.zst7015lb # [ 172.757720] niks3-server[1150]: 2026/09/24 11:50:03 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.ls7016lb # [ 172.761073] niks3-server[1150]: 2026/09/24 11:50:03 INFO Received complete push request method=POST path=/api/pushes/70/complete7017worker1 # [ 172.773762] nix-grpc-daemon[1847]: time=2026-09-24T11:50:03.065Z level=INFO msg="Upload complete. (217ms)"7018lb # [ 172.768254] niks3-server[1150]: 2026/09/24 11:50:03 INFO Registered completed upload object_key=wfziwgf0gxmjhy17lyfmn47628s166p7.narinfo7019worker1 # [ 172.779795] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:03Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=cv151lgjr4lv54sqzb49xk9w1ak9xsm9-farm-a.drv assign_id=11 outputs=17020worker2 # [ 172.769986] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:03Z level=debug event=assigned drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv worker=192.168.1.3:50051 clients=17021worker1 # [ 172.790343] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187022worker2 # [ 172.781170] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:03Z level=debug event=want client=6 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv cached=07023worker1 # [ 172.802540] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:03Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555187024worker1 # [ 172.808335] nix-daemon[867]: reaped child process 1920, status = succeeded7025worker1 # [ 172.813406] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:03Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555187026worker1 # [ 172.820517] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7027worker1 # [ 172.907438] nix-daemon[867]: accepted connection from pid 1945, user nix-grpc-daemon (trusted)7028worker1 # [ 172.919517] nix-daemon[867]: reaped child process 1948, status = succeeded7029lb # [ 172.920821] niks3-server[1150]: 2026/09/24 11:50:03 INFO Received push request method=POST path=/api/pushes7030worker1 # [ 172.936477] nix-grpc-daemon[1847]: time=2026-09-24T11:50:03.228Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7031worker1 # [ 172.938518] nix-grpc-daemon[1847]: time=2026-09-24T11:50:03.229Z level=INFO msg="Uploading fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv (504B)"7032lb # [ 172.952535] niks3-server[1150]: 2026/09/24 11:50:03 INFO Registered completed upload object_key=nar/170l67nd7hvasq2l36jibfpivx7r9ssdy0wlzifggpplhcgf2rk0.nar.zst7033lb # [ 172.959951] niks3-server[1150]: 2026/09/24 11:50:03 INFO Received sign narinfos request method=POST path=/api/pushes/71/sign7034lb # [ 172.964351] niks3-server[1150]: 2026/09/24 11:50:03 INFO Signed narinfos id=71 count=17035worker1 # [ 172.974894] nix-grpc-daemon[1847]: time=2026-09-24T11:50:03.266Z level=INFO msg="Uploading 1 narinfos"7036lb # [ 172.975663] niks3-server[1150]: 2026/09/24 11:50:03 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.ls7037lb # [ 172.986575] niks3-server[1150]: 2026/09/24 11:50:03 INFO Received complete push request method=POST path=/api/pushes/71/complete7038lb # [ 172.996965] niks3-server[1150]: 2026/09/24 11:50:03 INFO Registered completed upload object_key=fggw0ziag6jlrc830j6x1b6qw68kybhf.narinfo7039worker1 # [ 173.003130] nix-grpc-daemon[1847]: time=2026-09-24T11:50:03.294Z level=INFO msg="Upload complete. (176ms)"7040worker1 # [ 173.007549] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:03Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=5047041worker1 # [ 173.014178] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:03Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187042worker1 # [ 173.022108] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7043worker1 # [ 173.032146] nix-daemon[867]: reaped child process 1943, status = succeeded7044worker1 # [ 173.035957] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7045worker1 # [ 173.850267] nix-daemon[867]: reaped child process 1952, status = succeeded7046worker1 # [ 173.926683] nix-daemon[867]: accepted connection from pid 1968, user nix-grpc-daemon (trusted)7047worker1 # [ 173.938736] nix-daemon[867]: reaped child process 1971, status = succeeded7048lb # [ 173.938636] niks3-server[1150]: 2026/09/24 11:50:04 INFO Received push request method=POST path=/api/pushes7049worker1 # [ 173.955635] nix-grpc-daemon[1847]: time=2026-09-24T11:50:04.247Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7050worker1 # [ 173.957146] nix-grpc-daemon[1847]: time=2026-09-24T11:50:04.249Z level=INFO msg="Uploading 63zyaryk3r1ikmmzi1w30vadf20kdw3y-farm-a (128B)"7051lb # [ 173.982100] niks3-server[1150]: 2026/09/24 11:50:04 INFO Received sign narinfos request method=POST path=/api/pushes/72/sign7052lb # [ 173.987103] niks3-server[1150]: 2026/09/24 11:50:04 INFO Signed narinfos id=72 count=17053worker1 # [ 173.999391] nix-grpc-daemon[1847]: time=2026-09-24T11:50:04.291Z level=INFO msg="Uploading 1 narinfos"7054lb # [ 173.998653] niks3-server[1150]: 2026/09/24 11:50:04 INFO Registered completed upload object_key=nar/09na6djn66dpys8h08gqw39p0vw7vxqsdzbiw55rknyna8s239bm.nar.zst7055lb # [ 174.003471] niks3-server[1150]: 2026/09/24 11:50:04 INFO Registered completed upload object_key=log/fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv7056lb # [ 174.011705] niks3-server[1150]: 2026/09/24 11:50:04 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.ls7057lb # [ 174.017829] niks3-server[1150]: 2026/09/24 11:50:04 INFO Received complete push request method=POST path=/api/pushes/72/complete7058worker1 # [ 174.031399] nix-grpc-daemon[1847]: time=2026-09-24T11:50:04.323Z level=INFO msg="Upload complete. (189ms)"7059lb # [ 174.026063] niks3-server[1150]: 2026/09/24 11:50:04 INFO Registered completed upload object_key=63zyaryk3r1ikmmzi1w30vadf20kdw3y.narinfo7060worker2 # [ 174.027228] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:04Z level=debug event=assigned drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv worker=192.168.1.3:50051 clients=17061worker1 # [ 174.036988] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:04Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=fggw0ziag6jlrc830j6x1b6qw68kybhf-farm-a.drv assign_id=12 outputs=17062worker1 # [ 174.046150] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187063worker2 # [ 174.036989] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:04Z level=debug event=want client=6 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv cached=07064worker1 # [ 174.059818] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:04Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555187065worker1 # [ 174.063944] nix-daemon[867]: reaped child process 1950, status = succeeded7066worker1 # [ 174.069154] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:04Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555187067worker1 # [ 174.076155] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7068worker1 # [ 174.162185] nix-daemon[867]: accepted connection from pid 1975, user nix-grpc-daemon (trusted)7069worker1 # [ 174.174332] nix-daemon[867]: reaped child process 1978, status = succeeded7070lb # [ 174.175280] niks3-server[1150]: 2026/09/24 11:50:04 INFO Received push request method=POST path=/api/pushes7071worker1 # [ 174.190583] nix-grpc-daemon[1847]: time=2026-09-24T11:50:04.482Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7072worker1 # [ 174.192070] nix-grpc-daemon[1847]: time=2026-09-24T11:50:04.484Z level=INFO msg="Uploading j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv (504B)"7073lb # [ 174.206864] niks3-server[1150]: 2026/09/24 11:50:04 INFO Received sign narinfos request method=POST path=/api/pushes/73/sign7074worker1 # [ 174.220439] nix-grpc-daemon[1847]: time=2026-09-24T11:50:04.512Z level=INFO msg="Uploading 1 narinfos"7075lb # [ 174.212719] niks3-server[1150]: 2026/09/24 11:50:04 INFO Signed narinfos id=73 count=17076lb # [ 174.222535] niks3-server[1150]: 2026/09/24 11:50:04 INFO Registered completed upload object_key=nar/0qr7hdc916yrqpvqx2bwc5l0rwy3zm6rsdlcf8rk9iid53b80xsn.nar.zst7077lb # [ 174.232408] niks3-server[1150]: 2026/09/24 11:50:04 INFO Received complete push request method=POST path=/api/pushes/73/complete7078worker1 # [ 174.249702] nix-grpc-daemon[1847]: time=2026-09-24T11:50:04.541Z level=INFO msg="Upload complete. (167ms)"7079lb # [ 174.243987] niks3-server[1150]: 2026/09/24 11:50:04 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.ls7080lb # [ 174.248331] niks3-server[1150]: 2026/09/24 11:50:04 INFO Registered completed upload object_key=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz.narinfo7081worker1 # [ 174.254121] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:04Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=5047082worker1 # [ 174.259309] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:04Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187083worker1 # [ 174.267529] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7084worker1 # [ 174.273657] nix-daemon[867]: reaped child process 1973, status = succeeded7085worker1 # [ 174.277854] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7086worker1 # [ 175.311761] nix-daemon[867]: reaped child process 1982, status = succeeded7087worker1 # [ 175.398936] nix-daemon[867]: accepted connection from pid 1998, user nix-grpc-daemon (trusted)7088worker1 # [ 175.409910] nix-daemon[867]: reaped child process 2001, status = succeeded7089lb # [ 175.411655] niks3-server[1150]: 2026/09/24 11:50:05 INFO Received push request method=POST path=/api/pushes7090lb # [ 175.418940] envoy[913]: conn peer=[2001:db8:1::1]:53898 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=9621 tx=5188 ms=62317091worker1 # [ 175.431835] nix-grpc-daemon[1847]: time=2026-09-24T11:50:05.723Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7092worker1 # [ 175.433853] nix-grpc-daemon[1847]: time=2026-09-24T11:50:05.725Z level=INFO msg="Uploading pnim4ls33hwi0fl1wv96w4yj4afz9w4m-farm-a (128B)"7093lb # [ 175.455806] niks3-server[1150]: 2026/09/24 11:50:05 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.ls7094lb # [ 175.465178] niks3-server[1150]: 2026/09/24 11:50:05 INFO Received sign narinfos request method=POST path=/api/pushes/74/sign7095worker1 # [ 175.480299] nix-grpc-daemon[1847]: time=2026-09-24T11:50:05.772Z level=INFO msg="Uploading 1 narinfos"7096lb # [ 175.472779] niks3-server[1150]: 2026/09/24 11:50:05 INFO Signed narinfos id=74 count=17097lb # [ 175.475702] niks3-server[1150]: 2026/09/24 11:50:05 INFO Registered completed upload object_key=log/j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv7098lb # [ 175.482534] niks3-server[1150]: 2026/09/24 11:50:05 INFO Registered completed upload object_key=nar/1j2671k9mycq8lxyfx5k6a66d5narfqw2wqwlpwxgwzi93vnvbdn.nar.zst7099lb # [ 175.489068] niks3-server[1150]: 2026/09/24 11:50:05 INFO Received complete push request method=POST path=/api/pushes/74/complete7100lb # [ 175.495119] niks3-server[1150]: 2026/09/24 11:50:05 INFO Registered completed upload object_key=pnim4ls33hwi0fl1wv96w4yj4afz9w4m.narinfo7101worker1 # [ 175.501942] nix-grpc-daemon[1847]: time=2026-09-24T11:50:05.793Z level=INFO msg="Upload complete. (216ms)"7102worker1 # [ 175.508070] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:05Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=j7ssjgj9yiiadgfwc3qmhbma7i6ylnrz-farm-a.drv assign_id=13 outputs=17103worker2 # [ 175.501614] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:05Z level=debug event=assigned drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv worker=192.168.1.3:50051 clients=17104worker1 # [ 175.517306] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:05Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187105worker2 # [ 175.507404] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:05Z level=debug event=want client=6 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv cached=07106worker1 # [ 175.529510] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:05Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555187107worker1 # [ 175.536119] nix-daemon[867]: reaped child process 1980, status = succeeded7108worker1 # [ 175.538396] nix-grpc-daemon[1847]: time=2026-09-24T11:50:05.830Z level=INFO msg="All 1 paths already cached"7109worker1 # [ 175.544268] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:05Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555187110worker1 # [ 175.552041] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7111worker1 # [ 175.635409] nix-daemon[867]: accepted connection from pid 2005, user nix-grpc-daemon (trusted)7112worker1 # [ 175.648990] nix-daemon[867]: reaped child process 2008, status = succeeded7113lb # [ 175.647675] niks3-server[1150]: 2026/09/24 11:50:06 INFO Received push request method=POST path=/api/pushes7114worker1 # [ 175.663191] nix-grpc-daemon[1847]: time=2026-09-24T11:50:05.955Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7115worker1 # [ 175.665166] nix-grpc-daemon[1847]: time=2026-09-24T11:50:05.955Z level=INFO msg="Uploading 6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv (624B)"7116lb # [ 175.677665] niks3-server[1150]: 2026/09/24 11:50:06 INFO Registered completed upload object_key=nar/06n0qn8haxwg1vfa3qwa7h14mzz5fz7nizpv8bn995f4gddci6y6.nar.zst7117lb # [ 175.684959] niks3-server[1150]: 2026/09/24 11:50:06 INFO Received sign narinfos request method=POST path=/api/pushes/75/sign7118lb # [ 175.689258] niks3-server[1150]: 2026/09/24 11:50:06 INFO Signed narinfos id=75 count=17119worker1 # [ 175.699799] nix-grpc-daemon[1847]: time=2026-09-24T11:50:05.991Z level=INFO msg="Uploading 1 narinfos"7120lb # [ 175.700548] niks3-server[1150]: 2026/09/24 11:50:06 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.ls7121lb # [ 175.710590] niks3-server[1150]: 2026/09/24 11:50:06 INFO Received complete push request method=POST path=/api/pushes/75/complete7122worker1 # [ 175.726275] nix-grpc-daemon[1847]: time=2026-09-24T11:50:06.018Z level=INFO msg="Upload complete. (169ms)"7123lb # [ 175.720046] niks3-server[1150]: 2026/09/24 11:50:06 INFO Registered completed upload object_key=6cjjd3vbzygfms2ccxvnmabbgq5pv310.narinfo7124worker1 # [ 175.728484] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:06Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=6247125worker1 # [ 175.733685] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:06Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187126worker1 # [ 175.745144] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7127worker1 # [ 175.749142] nix-daemon[867]: reaped child process 2003, status = succeeded7128worker1 # [ 175.753918] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7129worker1 # [ 177.189980] nix-daemon[867]: reaped child process 2012, status = succeeded7130worker1 # [ 177.269084] nix-daemon[867]: accepted connection from pid 2028, user nix-grpc-daemon (trusted)7131worker1 # [ 177.281818] nix-daemon[867]: reaped child process 2031, status = succeeded7132lb # [ 177.280380] niks3-server[1150]: 2026/09/24 11:50:07 INFO Received push request method=POST path=/api/pushes7133worker1 # [ 177.297166] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.588Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7134worker1 # [ 177.299340] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.590Z level=INFO msg="Uploading 7gfg490kigdqainqjk27vmks1h3wpv5q-farm-b (176B)"7135lb # [ 177.320171] niks3-server[1150]: 2026/09/24 11:50:07 INFO Registered completed upload object_key=nar/07q9c11jk9djv06znvadbhgp1j3kwqf4z0z60kwx2sjqjkpssnf0.nar.zst7136lb # [ 177.333219] niks3-server[1150]: 2026/09/24 11:50:07 INFO Received sign narinfos request method=POST path=/api/pushes/76/sign7137lb # [ 177.343138] niks3-server[1150]: 2026/09/24 11:50:07 INFO Signed narinfos id=76 count=17138worker1 # [ 177.354881] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.646Z level=INFO msg="Uploading 1 narinfos"7139lb # [ 177.348226] niks3-server[1150]: 2026/09/24 11:50:07 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.ls7140lb # [ 177.352716] niks3-server[1150]: 2026/09/24 11:50:07 INFO Registered completed upload object_key=log/6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv7141lb # [ 177.364366] niks3-server[1150]: 2026/09/24 11:50:07 INFO Received complete push request method=POST path=/api/pushes/76/complete7142lb # [ 177.368984] niks3-server[1150]: 2026/09/24 11:50:07 INFO Registered completed upload object_key=7gfg490kigdqainqjk27vmks1h3wpv5q.narinfo7143worker1 # [ 177.380568] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.672Z level=INFO msg="Upload complete. (218ms)"7144worker1 # [ 177.386735] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:07Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=6cjjd3vbzygfms2ccxvnmabbgq5pv310-farm-b.drv assign_id=14 outputs=17145worker2 # [ 177.376714] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:07Z level=debug event=assigned drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv worker=192.168.1.3:50051 clients=17146worker1 # [ 177.395544] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187147worker2 # [ 177.385935] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:07Z level=debug event=want client=6 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv cached=07148worker1 # [ 177.407173] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:07Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555187149worker1 # [ 177.416340] nix-daemon[867]: reaped child process 2010, status = succeeded7150worker1 # [ 177.419538] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.710Z level=INFO msg="All 1 paths already cached"7151worker1 # [ 177.423581] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:07Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555187152worker1 # [ 177.430138] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7153worker1 # [ 177.514597] nix-daemon[867]: accepted connection from pid 2035, user nix-grpc-daemon (trusted)7154worker1 # [ 177.527071] nix-daemon[867]: reaped child process 2038, status = succeeded7155lb # [ 177.527357] niks3-server[1150]: 2026/09/24 11:50:07 INFO Received push request method=POST path=/api/pushes7156worker1 # [ 177.543043] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.835Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7157worker1 # [ 177.544697] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.836Z level=INFO msg="Uploading 6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv (624B)"7158lb # [ 177.560901] niks3-server[1150]: 2026/09/24 11:50:08 INFO Registered completed upload object_key=nar/0w4dmpczwycj1p7ngvcwdynpk5hh6r5ivyz44k3lx2gh5vpk4l0i.nar.zst7159lb # [ 177.567601] niks3-server[1150]: 2026/09/24 11:50:08 INFO Received sign narinfos request method=POST path=/api/pushes/77/sign7160worker1 # [ 177.583373] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.875Z level=INFO msg="Uploading 1 narinfos"7161lb # [ 177.576242] niks3-server[1150]: 2026/09/24 11:50:08 INFO Signed narinfos id=77 count=17162lb # [ 177.584494] niks3-server[1150]: 2026/09/24 11:50:08 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.ls7163lb # [ 177.594505] niks3-server[1150]: 2026/09/24 11:50:08 INFO Received complete push request method=POST path=/api/pushes/77/complete7164worker1 # [ 177.610416] nix-grpc-daemon[1847]: time=2026-09-24T11:50:07.902Z level=INFO msg="Upload complete. (175ms)"7165lb # [ 177.605790] niks3-server[1150]: 2026/09/24 11:50:08 INFO Registered completed upload object_key=6cs4w0lyx4kldp39y5k29p7c0234cffw.narinfo7166worker1 # [ 177.614502] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:07Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=6247167worker1 # [ 177.620943] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:07Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187168worker1 # [ 177.631878] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7169worker1 # [ 177.640943] nix-daemon[867]: reaped child process 2033, status = succeeded7170worker1 # [ 177.646191] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7171worker1 # [ 178.881688] nix-daemon[867]: reaped child process 2042, status = succeeded7172worker1 # [ 178.962086] nix-daemon[867]: accepted connection from pid 2058, user nix-grpc-daemon (trusted)7173worker1 # [ 178.973853] nix-daemon[867]: reaped child process 2061, status = succeeded7174lb # [ 178.974414] niks3-server[1150]: 2026/09/24 11:50:09 INFO Received push request method=POST path=/api/pushes7175worker1 # [ 178.991901] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.283Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7176worker1 # [ 178.994491] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.285Z level=INFO msg="Uploading xxjh88rg0bhdcm1c6b12clv7l5hfbaa9-farm-b (176B)"7177lb # [ 179.009763] niks3-server[1150]: 2026/09/24 11:50:09 INFO Registered completed upload object_key=nar/1a8ws2jbn6nqpggi64yfgxvlwny3nffvyw6svm59is55hih5ppq4.nar.zst7178lb # [ 179.026645] niks3-server[1150]: 2026/09/24 11:50:09 INFO Registered completed upload object_key=log/6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv7179lb # [ 179.034956] niks3-server[1150]: 2026/09/24 11:50:09 INFO Received sign narinfos request method=POST path=/api/pushes/78/sign7180lb # [ 179.039967] niks3-server[1150]: 2026/09/24 11:50:09 INFO Signed narinfos id=78 count=17181worker1 # [ 179.052764] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.344Z level=INFO msg="Uploading 1 narinfos"7182lb # [ 179.050115] niks3-server[1150]: 2026/09/24 11:50:09 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.ls7183lb # [ 179.061179] niks3-server[1150]: 2026/09/24 11:50:09 INFO Received complete push request method=POST path=/api/pushes/78/complete7184worker1 # [ 179.072700] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.364Z level=INFO msg="Upload complete. (219ms)"7185lb # [ 179.066857] niks3-server[1150]: 2026/09/24 11:50:09 INFO Registered completed upload object_key=xxjh88rg0bhdcm1c6b12clv7l5hfbaa9.narinfo7186worker1 # [ 179.078728] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:09Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=6cs4w0lyx4kldp39y5k29p7c0234cffw-farm-b.drv assign_id=15 outputs=17187worker2 # [ 179.072113] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:09Z level=debug event=assigned drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv worker=192.168.1.3:50051 clients=17188worker1 # [ 179.087404] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187189worker2 # [ 179.078392] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:09Z level=debug event=want client=6 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv cached=07190worker1 # [ 179.098968] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:09Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555187191worker1 # [ 179.107381] nix-daemon[867]: reaped child process 2040, status = succeeded7192worker1 # [ 179.109601] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.401Z level=INFO msg="All 1 paths already cached"7193worker1 # [ 179.114729] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:09Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555187194worker1 # [ 179.122072] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7195worker1 # [ 179.206797] nix-daemon[867]: accepted connection from pid 2065, user nix-grpc-daemon (trusted)7196worker1 # [ 179.219974] nix-daemon[867]: reaped child process 2068, status = succeeded7197lb # [ 179.218733] niks3-server[1150]: 2026/09/24 11:50:09 INFO Received push request method=POST path=/api/pushes7198worker1 # [ 179.235570] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.527Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7199worker1 # [ 179.237463] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.529Z level=INFO msg="Uploading 8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv (624B)"7200lb # [ 179.253166] niks3-server[1150]: 2026/09/24 11:50:09 INFO Registered completed upload object_key=nar/1dh50s9lbv3zd84hw5q9axrhfi7ms38cl3gyrb9gas00gni4siz2.nar.zst7201lb # [ 179.262313] niks3-server[1150]: 2026/09/24 11:50:09 INFO Received sign narinfos request method=POST path=/api/pushes/79/sign7202lb # [ 179.271645] niks3-server[1150]: 2026/09/24 11:50:09 INFO Signed narinfos id=79 count=17203worker1 # [ 179.282726] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.574Z level=INFO msg="Uploading 1 narinfos"7204lb # [ 179.279669] niks3-server[1150]: 2026/09/24 11:50:09 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.ls7205lb # [ 179.292144] niks3-server[1150]: 2026/09/24 11:50:09 INFO Received complete push request method=POST path=/api/pushes/79/complete7206lb # [ 179.299907] niks3-server[1150]: 2026/09/24 11:50:09 INFO Registered completed upload object_key=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz.narinfo7207worker1 # [ 179.311039] nix-grpc-daemon[1847]: time=2026-09-24T11:50:09.602Z level=INFO msg="Upload complete. (184ms)"7208worker1 # [ 179.315935] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:09Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=6247209worker1 # [ 179.322332] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:09Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187210worker1 # [ 179.333224] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7211worker1 # [ 179.342403] nix-daemon[867]: reaped child process 2063, status = succeeded7212worker1 # [ 179.349453] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7213worker1 # [ 180.187607] nix-daemon[867]: reaped child process 2072, status = succeeded7214worker1 # [ 180.255332] nix-daemon[867]: accepted connection from pid 2088, user nix-grpc-daemon (trusted)7215worker1 # [ 180.267555] nix-daemon[867]: reaped child process 2091, status = succeeded7216lb # [ 180.268986] niks3-server[1150]: 2026/09/24 11:50:10 INFO Received push request method=POST path=/api/pushes7217worker1 # [ 180.286394] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.578Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7218worker1 # [ 180.287961] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.579Z level=INFO msg="Uploading wha0h9jdcpcy0srjm92hg7pyknwdqp6h-farm-b (176B)"7219lb # [ 180.316172] niks3-server[1150]: 2026/09/24 11:50:10 INFO Received sign narinfos request method=POST path=/api/pushes/80/sign7220lb # [ 180.323700] niks3-server[1150]: 2026/09/24 11:50:10 INFO Signed narinfos id=80 count=17221worker1 # [ 180.335883] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.627Z level=INFO msg="Uploading 1 narinfos"7222lb # [ 180.336663] niks3-server[1150]: 2026/09/24 11:50:10 INFO Registered completed upload object_key=nar/1xpjwixqgm25ifjqv3ldg9cbq2njxf26272sngvq3zhy6zr1mvwg.nar.zst7223lb # [ 180.342905] niks3-server[1150]: 2026/09/24 11:50:10 INFO Registered completed upload object_key=log/8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv7224lb # [ 180.346925] niks3-server[1150]: 2026/09/24 11:50:10 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.ls7225lb # [ 180.356901] niks3-server[1150]: 2026/09/24 11:50:10 INFO Received complete push request method=POST path=/api/pushes/80/complete7226lb # [ 180.364511] niks3-server[1150]: 2026/09/24 11:50:10 INFO Registered completed upload object_key=wha0h9jdcpcy0srjm92hg7pyknwdqp6h.narinfo7227worker1 # [ 180.371652] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.663Z level=INFO msg="Upload complete. (215ms)"7228worker1 # [ 180.377802] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:10Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=8pwrbica5d7v9bivkz2fiy3z2fjlj9dz-farm-b.drv assign_id=16 outputs=17229worker2 # [ 180.367895] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:10Z level=debug event=assigned drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv worker=192.168.1.3:50051 clients=17230worker1 # [ 180.386457] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187231worker2 # [ 180.377986] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:10Z level=debug event=want client=6 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv cached=07232worker1 # [ 180.399521] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:10Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555187233worker1 # [ 180.405495] nix-daemon[867]: reaped child process 2070, status = succeeded7234worker1 # [ 180.410935] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.702Z level=INFO msg="All 2 paths already cached"7235worker1 # [ 180.418206] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:10Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555187236worker1 # [ 180.425198] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7237worker1 # [ 180.510437] nix-daemon[867]: accepted connection from pid 2095, user nix-grpc-daemon (trusted)7238worker1 # [ 180.522838] nix-daemon[867]: reaped child process 2098, status = succeeded7239lb # [ 180.523593] niks3-server[1150]: 2026/09/24 11:50:10 INFO Received push request method=POST path=/api/pushes7240worker1 # [ 180.541239] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.833Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7241worker1 # [ 180.542938] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.834Z level=INFO msg="Uploading 2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv (752B)"7242lb # [ 180.561697] niks3-server[1150]: 2026/09/24 11:50:11 INFO Received sign narinfos request method=POST path=/api/pushes/81/sign7243lb # [ 180.567583] niks3-server[1150]: 2026/09/24 11:50:11 INFO Signed narinfos id=81 count=17244worker1 # [ 180.579640] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.871Z level=INFO msg="Uploading 1 narinfos"7245lb # [ 180.577887] niks3-server[1150]: 2026/09/24 11:50:11 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.ls7246lb # [ 180.585133] niks3-server[1150]: 2026/09/24 11:50:11 INFO Registered completed upload object_key=nar/10gk7wwjbmc835pm215qvrdyjshy6nblydlws97ya3kcny5wz30c.nar.zst7247lb # [ 180.597539] niks3-server[1150]: 2026/09/24 11:50:11 INFO Received complete push request method=POST path=/api/pushes/81/complete7248lb # [ 180.611639] niks3-server[1150]: 2026/09/24 11:50:11 INFO Registered completed upload object_key=2gb1j4yk6j00fr2sscj3nqixv67k9b9g.narinfo7249worker1 # [ 180.618239] nix-grpc-daemon[1847]: time=2026-09-24T11:50:10.910Z level=INFO msg="Upload complete. (187ms)"7250worker1 # [ 180.623919] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:10Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=7527251worker1 # [ 180.630620] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:10Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187252worker1 # [ 180.644117] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7253worker1 # [ 180.652375] nix-daemon[867]: reaped child process 2093, status = succeeded7254worker1 # [ 180.659185] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7255worker1 # [ 181.893542] nix-daemon[867]: reaped child process 2102, status = succeeded7256worker1 # [ 181.978103] nix-daemon[867]: accepted connection from pid 2118, user nix-grpc-daemon (trusted)7257worker1 # [ 181.992445] nix-daemon[867]: reaped child process 2121, status = succeeded7258lb # [ 181.992187] niks3-server[1150]: 2026/09/24 11:50:12 INFO Received push request method=POST path=/api/pushes7259worker1 # [ 182.008775] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.299Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7260worker1 # [ 182.010286] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.299Z level=INFO msg="Uploading 78nhq5a8myvxam7sxfszi5ihz2rlq2gs-farm-top (232B)"7261lb # [ 182.030306] niks3-server[1150]: 2026/09/24 11:50:12 INFO Registered completed upload object_key=log/2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv7262lb # [ 182.032276] niks3-server[1150]: 2026/09/24 11:50:12 INFO Received sign narinfos request method=POST path=/api/pushes/82/sign7263worker1 # [ 182.046356] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.338Z level=INFO msg="Uploading 1 narinfos"7264lb # [ 182.038613] niks3-server[1150]: 2026/09/24 11:50:12 INFO Signed narinfos id=82 count=17265lb # [ 182.044541] niks3-server[1150]: 2026/09/24 11:50:12 INFO Registered completed upload object_key=nar/1gsz81ckdr6lfkhhqmxw3bg6xqqnlz2wx6dzhpycascv9pq13fg7.nar.zst7266lb # [ 182.050868] niks3-server[1150]: 2026/09/24 11:50:12 INFO Received complete push request method=POST path=/api/pushes/82/complete7267lb # [ 182.054262] niks3-server[1150]: 2026/09/24 11:50:12 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.ls7268worker1 # [ 182.065995] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.357Z level=INFO msg="Upload complete. (199ms)"7269lb # [ 182.060937] niks3-server[1150]: 2026/09/24 11:50:12 INFO Registered completed upload object_key=78nhq5a8myvxam7sxfszi5ihz2rlq2gs.narinfo7270worker2 # [ 182.060219] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:12Z level=debug event=assigned drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv worker=192.168.1.3:50051 clients=17271worker1 # [ 182.070880] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:12Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=2gb1j4yk6j00fr2sscj3nqixv67k9b9g-farm-top.drv assign_id=17 outputs=17272worker1 # [ 182.078323] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187273worker1 # [ 182.089176] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:12Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555187274worker1 # [ 182.091360] nix-daemon[867]: reaped child process 2100, status = succeeded7275worker1 # [ 182.095532] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.387Z level=INFO msg="All 2 paths already cached"7276worker1 # [ 182.099806] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:12Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555187277worker1 # [ 182.103565] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7278worker1 # [ 182.187847] nix-daemon[867]: accepted connection from pid 2125, user nix-grpc-daemon (trusted)7279worker1 # [ 182.200594] nix-daemon[867]: reaped child process 2128, status = succeeded7280lb # [ 182.200542] niks3-server[1150]: 2026/09/24 11:50:12 INFO Received push request method=POST path=/api/pushes7281worker1 # [ 182.217974] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.509Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7282worker1 # [ 182.219952] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.511Z level=INFO msg="Uploading yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv (752B)"7283lb # [ 182.231245] niks3-server[1150]: 2026/09/24 11:50:12 INFO Received sign narinfos request method=POST path=/api/pushes/83/sign7284worker1 # [ 182.242587] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.534Z level=INFO msg="Uploading 1 narinfos"7285lb # [ 182.235408] niks3-server[1150]: 2026/09/24 11:50:12 INFO Signed narinfos id=83 count=17286lb # [ 182.244269] niks3-server[1150]: 2026/09/24 11:50:12 INFO Registered completed upload object_key=nar/1ibq19fcqq0l9xim9ly5lzg1pk7qh90qsbpya453vnh8p619zvbs.nar.zst7287lb # [ 182.251957] niks3-server[1150]: 2026/09/24 11:50:12 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.ls7288lb # [ 182.255059] niks3-server[1150]: 2026/09/24 11:50:12 INFO Received complete push request method=POST path=/api/pushes/83/complete7289worker1 # [ 182.268959] nix-grpc-daemon[1847]: time=2026-09-24T11:50:12.560Z level=INFO msg="Upload complete. (160ms)"7290lb # [ 182.262277] niks3-server[1150]: 2026/09/24 11:50:12 INFO Registered completed upload object_key=yjsciba0gpd99p30v1win6wnqyd4v7b6.narinfo7291worker1 # [ 182.271099] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:12Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=7527292worker1 # [ 182.274250] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:12Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187293worker1 # [ 182.281810] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7294worker1 # [ 182.283211] nix-daemon[867]: reaped child process 2123, status = succeeded7295worker1 # [ 182.288187] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7296worker1 # [ 183.116651] nix-daemon[867]: reaped child process 2132, status = succeeded7297worker1 # [ 183.205408] nix-daemon[867]: accepted connection from pid 2148, user nix-grpc-daemon (trusted)7298worker1 # [ 183.216462] nix-daemon[867]: reaped child process 2151, status = succeeded7299lb # [ 183.218323] niks3-server[1150]: 2026/09/24 11:50:13 INFO Received push request method=POST path=/api/pushes7300worker1 # [ 183.236703] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.528Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7301worker1 # [ 183.238371] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.530Z level=INFO msg="Uploading qc1853riggbxq0nbn4qjdymnhx5pncg8-farm-top (232B)"7302lb # [ 183.257909] niks3-server[1150]: 2026/09/24 11:50:13 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.ls7303lb # [ 183.266977] niks3-server[1150]: 2026/09/24 11:50:13 INFO Received sign narinfos request method=POST path=/api/pushes/84/sign7304lb # [ 183.276411] niks3-server[1150]: 2026/09/24 11:50:13 INFO Signed narinfos id=84 count=17305worker1 # [ 183.283811] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.575Z level=INFO msg="Uploading 1 narinfos"7306lb # [ 183.280382] niks3-server[1150]: 2026/09/24 11:50:13 INFO Registered completed upload object_key=nar/1k6m6033nbrw39zmvqlrbvmg39ygyn710za2hl2fkpsxfx40hl13.nar.zst7307lb # [ 183.283351] niks3-server[1150]: 2026/09/24 11:50:13 INFO Registered completed upload object_key=log/yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv7308lb # [ 183.288607] niks3-server[1150]: 2026/09/24 11:50:13 INFO Received complete push request method=POST path=/api/pushes/84/complete7309lb # [ 183.293711] niks3-server[1150]: 2026/09/24 11:50:13 INFO Registered completed upload object_key=qc1853riggbxq0nbn4qjdymnhx5pncg8.narinfo7310worker1 # [ 183.304744] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.596Z level=INFO msg="Upload complete. (208ms)"7311worker1 # [ 183.310616] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:13Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=yjsciba0gpd99p30v1win6wnqyd4v7b6-farm-top.drv assign_id=18 outputs=17312worker2 # [ 183.300282] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:13Z level=debug event=assigned drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv worker=192.168.1.3:50051 clients=17313worker1 # [ 183.316968] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187314worker1 # [ 183.328289] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:13Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:555187315worker1 # [ 183.330805] nix-daemon[867]: reaped child process 2130, status = succeeded7316worker1 # [ 183.335082] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.627Z level=INFO msg="All 2 paths already cached"7317worker1 # [ 183.339356] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:13Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:555187318worker1 # [ 183.344151] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7319worker1 # [ 183.429368] nix-daemon[867]: accepted connection from pid 2155, user nix-grpc-daemon (trusted)7320worker1 # [ 183.442975] nix-daemon[867]: reaped child process 2158, status = succeeded7321lb # [ 183.441097] niks3-server[1150]: 2026/09/24 11:50:13 INFO Received push request method=POST path=/api/pushes7322worker1 # [ 183.459344] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.751Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7323worker1 # [ 183.461326] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.752Z level=INFO msg="Uploading yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv (752B)"7324lb # [ 183.474851] niks3-server[1150]: 2026/09/24 11:50:13 INFO Received sign narinfos request method=POST path=/api/pushes/85/sign7325worker1 # [ 183.485820] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.777Z level=INFO msg="Uploading 1 narinfos"7326lb # [ 183.478107] niks3-server[1150]: 2026/09/24 11:50:13 INFO Signed narinfos id=85 count=17327lb # [ 183.486301] niks3-server[1150]: 2026/09/24 11:50:13 INFO Registered completed upload object_key=nar/1msrgzhylalg20dw693shbnzf48k8pvzp85llwgrnc970rnrliy7.nar.zst7328lb # [ 183.496068] niks3-server[1150]: 2026/09/24 11:50:13 INFO Received complete push request method=POST path=/api/pushes/85/complete7329worker1 # [ 183.507636] nix-grpc-daemon[1847]: time=2026-09-24T11:50:13.799Z level=INFO msg="Upload complete. (158ms)"7330lb # [ 183.501126] niks3-server[1150]: 2026/09/24 11:50:13 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.ls7331worker1 # [ 183.509926] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:13Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=0 paths=1 nar_bytes_in=7527332lb # [ 183.505082] niks3-server[1150]: 2026/09/24 11:50:13 INFO Registered completed upload object_key=yym31x3l626nfpm249qapx1iw7y15mqb.narinfo7333worker1 # [ 183.515400] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:13Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:555187334worker1 # [ 183.519911] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7335worker1 # [ 183.522260] nix-daemon[867]: reaped child process 2153, status = succeeded7336worker1 # [ 183.527320] nix-daemon[867]: accepted connection from pid 1787, user nix-grpc-daemon (trusted)7337worker1 # [ 184.361496] nix-daemon[867]: reaped child process 2162, status = succeeded7338worker1 # [ 184.441517] nix-daemon[867]: accepted connection from pid 2178, user nix-grpc-daemon (trusted)7339worker1 # [ 184.454982] nix-daemon[867]: reaped child process 2181, status = succeeded7340lb # [ 184.454671] niks3-server[1150]: 2026/09/24 11:50:14 INFO Received push request method=POST path=/api/pushes7341worker1 # [ 184.471486] nix-grpc-daemon[1847]: time=2026-09-24T11:50:14.763Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7342worker1 # [ 184.472921] nix-grpc-daemon[1847]: time=2026-09-24T11:50:14.764Z level=INFO msg="Uploading 0y5v4dgfslb2x47r2h6j0zjvpzg6frg6-farm-top (232B)"7343lb # [ 184.495967] niks3-server[1150]: 2026/09/24 11:50:14 INFO Registered completed upload object_key=log/yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv7344lb # [ 184.508591] niks3-server[1150]: 2026/09/24 11:50:14 INFO Received sign narinfos request method=POST path=/api/pushes/86/sign7345lb # [ 184.513428] niks3-server[1150]: 2026/09/24 11:50:14 INFO Signed narinfos id=86 count=17346worker1 # [ 184.524256] nix-grpc-daemon[1847]: time=2026-09-24T11:50:14.815Z level=INFO msg="Uploading 1 narinfos"7347lb # [ 184.519828] niks3-server[1150]: 2026/09/24 11:50:14 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.ls7348lb # [ 184.532239] niks3-server[1150]: 2026/09/24 11:50:14 INFO Registered completed upload object_key=nar/1h7kcfflb8hb0ylxshiprrirnrs580cwzkz50cz6il8jag61wy50.nar.zst7349lb # [ 184.536379] niks3-server[1150]: 2026/09/24 11:50:14 INFO Received complete push request method=POST path=/api/pushes/86/complete7350lb # [ 184.542130] niks3-server[1150]: 2026/09/24 11:50:14 INFO Registered completed upload object_key=0y5v4dgfslb2x47r2h6j0zjvpzg6frg6.narinfo7351worker1 # [ 184.547972] nix-grpc-daemon[1847]: time=2026-09-24T11:50:14.839Z level=INFO msg="Upload complete. (213ms)"7352worker1 # [ 184.554562] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:14Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:55518 duration_s=1 drv=yym31x3l626nfpm249qapx1iw7y15mqb-farm-top.drv assign_id=19 outputs=17353worker1 # [ 184.569148] nix-daemon[867]: reaped child process 2160, status = succeeded7354client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto --expr 'map (tag: import /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix { inherit tag; features = ["vip"]; }) ["f1" "f2" "f3"]' --impure 2>&1, in 13.90 seconds)7355worker1: must succeed: curl -sf http://127.0.0.1:9464/metrics7356worker1: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.04 seconds)7357client: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]'7358client # [ 184.651788] systemd[1]: Started [systemd-run] /run/current-system/sw/bin/nix build -L --store "grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30" --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features "[\"gpu\"]".7359client # Running as unit: gpu.service; invocation ID: 0ab955a1a0ad4092bccaffacef6d8f727360client: (finished: must succeed: systemd-run --unit gpu nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key&restart-grace=30' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag f5 --arg features '["gpu"]', in 0.06 seconds)7361lb: must succeed: curl -sf localhost:9901/clusters7362lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7363lb: must succeed: curl -sf localhost:9901/clusters7364lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.02 seconds)7365worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7366worker2 # [ 184.812389] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:15Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582627367client # [ 184.791910] nix[1600]: don't know how to build these paths:7368client # [ 184.792845] nix[1600]: /nix/store/ynb3m2rijlmsagnc1dpj672gknpqrkcv-farm-top.drv7369worker2 # [ 184.820430] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:15Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582627370worker2 # [ 184.836615] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:15Z level=debug event=want client=7 drv=z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv cached=07371client # [ 184.814615] nix[1600]: warning: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}, waiting up to 30s for a worker7372worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.07 seconds)7373lb # [ 185.419791] envoy[913]: conn peer=[2001:db8:1::1]:53910 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=27107 tx=12322 ms=137257374lb: must succeed: curl -sf localhost:9901/clusters7375lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7376lb: must succeed: curl -sf localhost:9901/clusters7377lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7378worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7379worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7380client: waiting for success: systemctl show -p Result gpu | grep -q exit-code7381client # [ 214.817780] nix[1600]: z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv: no connected, non-draining worker for system 'x86_64-linux' offers features {gpu}, the workers offer {benchmark,big-parallel,kvm,nixos-test,vip}7382client # [ 214.825181] nix[1600]: error: dependency 'z5cr5v9sb2xcx4fmgc48vvwyl2m8rzzs-farm-a.drv' failed7383client # [ 214.857199] systemd[1]: gpu.service: Main process exited, code=exited, status=1/FAILURE7384client # [ 214.860945] systemd[1]: gpu.service: Failed with result 'exit-code'.7385client # [ 214.865624] systemd[1]: gpu.service: Consumed 154ms CPU time over 30.210s wall clock time, 37.6M memory peak, 5.6K incoming IP traffic, 5.3K outgoing IP traffic.7386client: (finished: waiting for success: systemctl show -p Result gpu | grep -q exit-code, in 29.44 seconds)7387client: must succeed: journalctl -u gpu -o cat7388lb # [ 215.422833] envoy[913]: conn peer=[2001:db8:1::1]:56090 tls=TLSv1.3 subject="CN=ci-1" sni=- flags=- tls_fail="-" rx=929 tx=596 ms=300507389client: (finished: must succeed: journalctl -u gpu -o cat, in 0.02 seconds)7390lb: must succeed: curl -sf localhost:9901/clusters7391lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.05 seconds)7392lb: must succeed: curl -sf localhost:9901/clusters7393lb: (finished: must succeed: curl -sf localhost:9901/clusters, in 0.03 seconds)7394worker2: must succeed: curl -sf http://127.0.0.1:9464/metrics7395worker2: (finished: must succeed: curl -sf http://127.0.0.1:9464/metrics, in 0.05 seconds)7396(finished: subtest: requiredSystemFeatures: placed on the worker that has them, refused when none does, in 44.87 seconds)7397subtest: low disk drains a worker and builds go to the other7398worker1: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill7399worker1: (finished: must succeed: fallocate -l $(( $(df --output=avail -B1 /nix/store | tail -1) - 100*1024*1024 )) /nix/.rw-store/fill, in 0.91 seconds)7400worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free'7401worker1 # [ 217.171204] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:47Z level=info event=unhealthy reason=min_free7402worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=unhealthy reason=min_free', in 1.08 seconds)7403client: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&27404client # don't know how to build these paths:7405client # /nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7406worker2 # [ 217.688881] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:47Z level=debug event=rpc_start method=QueryMissing cn=ci-1 peer=ipv4:192.168.1.2:582627407worker2 # [ 217.695121] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582627408worker2 # [ 217.718287] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:47Z level=debug event=want client=8 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv cached=07409worker2 # [ 217.723114] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:47Z level=debug event=assigned drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv worker=192.168.1.4:50051 clients=17410worker2 # [ 217.729558] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:47Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582627411worker2 # [ 217.742060] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:47Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582627412client # copying 1 paths to 'lb:50051'...7413client # copying path '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' to 'lb:50051'...7414worker2 # [ 217.755111] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:47Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582627415worker2 # [ 217.766960] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7416worker2 # [ 217.869402] nix-daemon[857]: accepted connection from pid 2080, user nix-grpc-daemon (trusted)7417worker2 # [ 217.882934] nix-daemon[857]: reaped child process 2083, status = succeeded7418lb # [ 217.895615] niks3-server[1150]: 2026/09/24 11:50:48 INFO Received push request method=POST path=/api/pushes7419worker2 # [ 217.903193] nix-grpc-daemon[1750]: time=2026-09-24T11:50:48.140Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7420worker2 # [ 217.905277] nix-grpc-daemon[1750]: time=2026-09-24T11:50:48.142Z level=INFO msg="Uploading a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv (512B)"7421lb # [ 217.937206] niks3-server[1150]: 2026/09/24 11:50:48 INFO Received sign narinfos request method=POST path=/api/pushes/87/sign7422lb # [ 217.944621] niks3-server[1150]: 2026/09/24 11:50:48 INFO Signed narinfos id=87 count=17423worker2 # [ 217.943564] nix-grpc-daemon[1750]: time=2026-09-24T11:50:48.181Z level=INFO msg="Uploading 1 narinfos"7424lb # [ 217.953804] niks3-server[1150]: 2026/09/24 11:50:48 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.ls7425lb # [ 217.959571] niks3-server[1150]: 2026/09/24 11:50:48 INFO Registered completed upload object_key=nar/1qxv3rw2cwpq06kampqz63l69ci673kh4xz41xmflqybqycfk573.nar.zst7426lb # [ 217.973425] niks3-server[1150]: 2026/09/24 11:50:48 INFO Received complete push request method=POST path=/api/pushes/87/complete7427worker2 # [ 217.974720] nix-grpc-daemon[1750]: time=2026-09-24T11:50:48.212Z level=INFO msg="Upload complete. (196ms)"7428lb # [ 217.982153] niks3-server[1150]: 2026/09/24 11:50:48 INFO Registered completed upload object_key=a33ymyck2p14d34j7jl4zc8zvry6cysy.narinfo7429worker2 # [ 217.979919] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:48Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=5127430worker2 # [ 217.986328] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:48Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582627431worker2 # [ 217.997977] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7432worker2 # [ 218.008253] nix-daemon[857]: reaped child process 2078, status = succeeded7433worker2 # [ 218.015220] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7434client # building '/nix/store/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv' on lb:50051...7435client # farm-a> worker2: building a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7436client # farm-a> disabling binary cache 'https://cache.nixos.org' for 60 seconds7437client # farm-a> LOG-farm-a-drain7438worker2 # [ 219.249477] nix-daemon[857]: reaped child process 2087, status = succeeded7439worker2 # [ 219.338949] nix-daemon[857]: accepted connection from pid 2103, user nix-grpc-daemon (trusted)7440worker2 # [ 219.350237] nix-daemon[857]: reaped child process 2106, status = succeeded7441lb # [ 219.365319] niks3-server[1150]: 2026/09/24 11:50:49 INFO Received push request method=POST path=/api/pushes7442worker2 # [ 219.374387] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.611Z level=INFO msg="Uploading 1 paths to lb (0 already cached)"7443worker2 # [ 219.377051] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.613Z level=INFO msg="Uploading rjdc0hf9mcchhia9m3zkqy1q586i3z3r-farm-a (128B)"7444lb # [ 219.417723] niks3-server[1150]: 2026/09/24 11:50:49 INFO Registered completed upload object_key=nar/1bf9z7y0ikx07b4zjag42zyrv2pq1ycw4461s579kfpmni9kz4dw.nar.zst7445lb # [ 219.422604] niks3-server[1150]: 2026/09/24 11:50:49 INFO Registered completed upload object_key=log/a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv7446lb # [ 219.428116] niks3-server[1150]: 2026/09/24 11:50:49 INFO Received sign narinfos request method=POST path=/api/pushes/88/sign7447lb # [ 219.432853] niks3-server[1150]: 2026/09/24 11:50:49 INFO Signed narinfos id=88 count=17448worker2 # [ 219.429685] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.667Z level=INFO msg="Uploading 1 narinfos"7449lb # [ 219.436274] niks3-server[1150]: 2026/09/24 11:50:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.ls7450lb # [ 219.442227] niks3-server[1150]: 2026/09/24 11:50:49 INFO Received complete push request method=POST path=/api/pushes/88/complete7451lb # [ 219.449167] niks3-server[1150]: 2026/09/24 11:50:49 INFO Registered completed upload object_key=rjdc0hf9mcchhia9m3zkqy1q586i3z3r.narinfo7452worker2 # [ 219.449201] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.686Z level=INFO msg="Upload complete. (226ms)"7453worker2 # [ 219.458262] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:49Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=1 drv=a33ymyck2p14d34j7jl4zc8zvry6cysy-farm-a.drv assign_id=20 outputs=17454worker2 # [ 219.468356] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:49Z level=debug event=want client=8 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv cached=07455worker2 # [ 219.472843] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:49Z level=debug event=assigned drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv worker=192.168.1.4:50051 clients=17456worker2 # [ 219.476332] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582627457worker2 # [ 219.486361] nix-daemon[857]: reaped child process 2085, status = succeeded7458worker2 # [ 219.487813] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:49Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582627459worker2 # [ 219.494550] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.732Z level=INFO msg="All 1 paths already cached"7460client # copying 1 paths to 'lb:50051'...7461worker2 # [ 219.499838] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:49Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582627462client # copying path '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' to 'lb:50051'...7463worker2 # [ 219.504635] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7464worker2 # [ 219.591308] nix-daemon[857]: accepted connection from pid 2112, user nix-grpc-daemon (trusted)7465worker2 # [ 219.603703] nix-daemon[857]: reaped child process 2115, status = succeeded7466lb # [ 219.615439] niks3-server[1150]: 2026/09/24 11:50:50 INFO Received push request method=POST path=/api/pushes7467worker2 # [ 219.617319] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.855Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7468worker2 # [ 219.619137] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.856Z level=INFO msg="Uploading xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv (624B)"7469lb # [ 219.632909] niks3-server[1150]: 2026/09/24 11:50:50 INFO Received sign narinfos request method=POST path=/api/pushes/89/sign7470lb # [ 219.635658] niks3-server[1150]: 2026/09/24 11:50:50 INFO Signed narinfos id=89 count=17471worker2 # [ 219.632191] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.869Z level=INFO msg="Uploading 1 narinfos"7472lb # [ 219.637944] niks3-server[1150]: 2026/09/24 11:50:50 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.ls7473lb # [ 219.641080] niks3-server[1150]: 2026/09/24 11:50:50 INFO Registered completed upload object_key=nar/06skgfsii1iysx6lv8hy20iyys0q28pmz13rs51634x6bgrlvs7k.nar.zst7474lb # [ 219.646478] niks3-server[1150]: 2026/09/24 11:50:50 INFO Received complete push request method=POST path=/api/pushes/89/complete7475worker2 # [ 219.647552] nix-grpc-daemon[1750]: time=2026-09-24T11:50:49.885Z level=INFO msg="Upload complete. (137ms)"7476lb # [ 219.652160] niks3-server[1150]: 2026/09/24 11:50:50 INFO Registered completed upload object_key=xihm5w957gm8bh55zxlc6dnczbbl387q.narinfo7477worker2 # [ 219.649559] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:49Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=6247478worker2 # [ 219.653243] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:49Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582627479worker2 # [ 219.660246] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7480worker2 # [ 219.662314] nix-daemon[857]: reaped child process 2110, status = succeeded7481worker2 # [ 219.666555] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7482client # building '/nix/store/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv' on lb:50051...7483client # farm-b> worker2: building xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7484client # farm-b> disabling binary cache 'https://cache.nixos.org' for 60 seconds7485client # farm-b> LOG-farm-b-drain7486worker2 # [ 220.909624] nix-daemon[857]: reaped child process 2119, status = succeeded7487worker2 # [ 220.986443] nix-daemon[857]: accepted connection from pid 2135, user nix-grpc-daemon (trusted)7488worker2 # [ 221.000880] nix-daemon[857]: reaped child process 2138, status = succeeded7489lb # [ 221.010916] niks3-server[1150]: 2026/09/24 11:50:51 INFO Received push request method=POST path=/api/pushes7490worker2 # [ 221.016346] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.253Z level=INFO msg="Uploading 1 paths to lb (1 already cached)"7491worker2 # [ 221.018201] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.255Z level=INFO msg="Uploading pmvxn7ymjj8pdn6gb65j5kqb45n38x0p-farm-b (176B)"7492lb # [ 221.057251] niks3-server[1150]: 2026/09/24 11:50:51 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.ls7493lb # [ 221.064995] niks3-server[1150]: 2026/09/24 11:50:51 INFO Received sign narinfos request method=POST path=/api/pushes/90/sign7494lb # [ 221.071658] niks3-server[1150]: 2026/09/24 11:50:51 INFO Signed narinfos id=90 count=17495worker2 # [ 221.067964] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.305Z level=INFO msg="Uploading 1 narinfos"7496lb # [ 221.076608] niks3-server[1150]: 2026/09/24 11:50:51 INFO Registered completed upload object_key=log/xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv7497lb # [ 221.081676] niks3-server[1150]: 2026/09/24 11:50:51 INFO Registered completed upload object_key=nar/17gyl0zgiivck81rhv539x50216ksc3nsf369ismwd4j1zwsdqrb.nar.zst7498lb # [ 221.092384] niks3-server[1150]: 2026/09/24 11:50:51 INFO Received complete push request method=POST path=/api/pushes/90/complete7499lb # [ 221.099709] niks3-server[1150]: 2026/09/24 11:50:51 INFO Registered completed upload object_key=pmvxn7ymjj8pdn6gb65j5kqb45n38x0p.narinfo7500worker2 # [ 221.094501] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.332Z level=INFO msg="Upload complete. (220ms)"7501worker2 # [ 221.101900] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:51Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=1 drv=xihm5w957gm8bh55zxlc6dnczbbl387q-farm-b.drv assign_id=21 outputs=17502worker2 # [ 221.114562] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:51Z level=debug event=want client=8 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv cached=07503worker2 # [ 221.118690] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:51Z level=debug event=assigned drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv worker=192.168.1.4:50051 clients=17504worker2 # [ 221.122319] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582627505worker2 # [ 221.132700] nix-daemon[857]: reaped child process 2117, status = succeeded7506worker2 # [ 221.135139] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:51Z level=debug event=rpc_start method=QueryValidPaths cn=ci-1 peer=ipv4:192.168.1.2:582627507worker2 # [ 221.144085] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.381Z level=INFO msg="All 2 paths already cached"7508client # copying 1 paths to 'lb:50051'...7509worker2 # [ 221.151356] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:51Z level=debug event=rpc_start method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:582627510client # copying path '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' to 'lb:50051'...7511worker2 # [ 221.156511] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7512worker2 # [ 221.245222] nix-daemon[857]: accepted connection from pid 2144, user nix-grpc-daemon (trusted)7513worker2 # [ 221.258177] nix-daemon[857]: reaped child process 2147, status = succeeded7514lb # [ 221.270366] niks3-server[1150]: 2026/09/24 11:50:51 INFO Received push request method=POST path=/api/pushes7515worker2 # [ 221.275286] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.513Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7516worker2 # [ 221.277275] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.514Z level=INFO msg="Uploading q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv (752B)"7517lb # [ 221.302166] niks3-server[1150]: 2026/09/24 11:50:51 INFO Received sign narinfos request method=POST path=/api/pushes/91/sign7518lb # [ 221.312122] niks3-server[1150]: 2026/09/24 11:50:51 INFO Signed narinfos id=91 count=17519worker2 # [ 221.311471] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.549Z level=INFO msg="Uploading 1 narinfos"7520lb # [ 221.321489] niks3-server[1150]: 2026/09/24 11:50:51 INFO Registered completed upload object_key=nar/1m4vpncprrwddjpk4qry79vzd9g39s9pjs6874i2aarpm7ax0ykq.nar.zst7521lb # [ 221.326778] niks3-server[1150]: 2026/09/24 11:50:51 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.ls7522lb # [ 221.341104] niks3-server[1150]: 2026/09/24 11:50:51 INFO Received complete push request method=POST path=/api/pushes/91/complete7523worker2 # [ 221.342868] nix-grpc-daemon[1750]: time=2026-09-24T11:50:51.580Z level=INFO msg="Upload complete. (180ms)"7524lb # [ 221.348026] niks3-server[1150]: 2026/09/24 11:50:51 INFO Registered completed upload object_key=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s.narinfo7525worker2 # [ 221.346670] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:51Z level=info event=rpc method=AddMultipleToStore cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=0 paths=1 nar_bytes_in=7527526worker2 # [ 221.351405] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:51Z level=debug event=rpc_start method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:582627527worker2 # [ 221.361106] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7528worker2 # [ 221.369556] nix-daemon[857]: reaped child process 2142, status = succeeded7529worker2 # [ 221.374287] nix-daemon[857]: accepted connection from pid 1670, user nix-grpc-daemon (trusted)7530client # building '/nix/store/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv' on lb:50051...7531client # farm-top> worker2: building q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7532client # farm-top> disabling binary cache 'https://cache.nixos.org' for 60 seconds7533client # farm-top> LOG-farm-top-drain7534worker2 # [ 222.588278] nix-daemon[857]: reaped child process 2151, status = succeeded7535worker2 # [ 222.665366] nix-daemon[857]: accepted connection from pid 2167, user nix-grpc-daemon (trusted)7536worker2 # [ 222.677392] nix-daemon[857]: reaped child process 2170, status = succeeded7537lb # [ 222.691492] niks3-server[1150]: 2026/09/24 11:50:53 INFO Received push request method=POST path=/api/pushes7538worker2 # [ 222.699276] nix-grpc-daemon[1750]: time=2026-09-24T11:50:52.936Z level=INFO msg="Uploading 1 paths to lb (2 already cached)"7539worker2 # [ 222.700621] nix-grpc-daemon[1750]: time=2026-09-24T11:50:52.936Z level=INFO msg="Uploading hgmfp1yw6m31jn6cpma19ikl6dy1b17r-farm-top (232B)"7540lb # [ 222.738897] niks3-server[1150]: 2026/09/24 11:50:53 INFO Received sign narinfos request method=POST path=/api/pushes/92/sign7541lb # [ 222.749378] niks3-server[1150]: 2026/09/24 11:50:53 INFO Signed narinfos id=92 count=17542worker2 # [ 222.748515] nix-grpc-daemon[1750]: time=2026-09-24T11:50:52.986Z level=INFO msg="Uploading 1 narinfos"7543lb # [ 222.755894] niks3-server[1150]: 2026/09/24 11:50:53 INFO Registered completed upload object_key=log/q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv7544lb # [ 222.761338] niks3-server[1150]: 2026/09/24 11:50:53 INFO Registered completed upload object_key=nar/1xin0j0xlljjqnd952hnxy8yw45d9j9qpj7m241vv6cckfq8kk4p.nar.zst7545lb # [ 222.765487] niks3-server[1150]: 2026/09/24 11:50:53 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.ls7546lb # [ 222.774697] niks3-server[1150]: 2026/09/24 11:50:53 INFO Received complete push request method=POST path=/api/pushes/92/complete7547lb # [ 222.781368] niks3-server[1150]: 2026/09/24 11:50:53 INFO Registered completed upload object_key=hgmfp1yw6m31jn6cpma19ikl6dy1b17r.narinfo7548worker2 # [ 222.776791] nix-grpc-daemon[1750]: time=2026-09-24T11:50:53.014Z level=INFO msg="Upload complete. (195ms)"7549worker2 # [ 222.783535] nix-grpc-daemon[1670]: ts=2026-09-24T11:50:53Z level=info event=rpc method=BuildDerivation cn=ci-1 peer=ipv4:192.168.1.2:58262 duration_s=1 drv=q6a73jmqx0c0v5nlki9f1fnmcls5gx5s-farm-top.drv assign_id=22 outputs=17550worker2 # [ 222.798623] nix-daemon[857]: reaped child process 2149, status = succeeded7551client: (finished: must succeed: nix build -L --store 'grpc://lb:50051?client-cert=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.pem&client-key=/nix/store/pp86g1qg7pz3xz5b0jg8c12b7nz5nsbd-farm-certs/ci.key' --eval-store auto -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain >&2, in 5.28 seconds)7552client: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath7553client: (finished: must succeed: nix eval --raw -f /nix/store/1v3figrsha1j7bs9k2jhjd61g006bb3c-job.nix --argstr tag drain outPath, in 0.11 seconds)7554worker1: must succeed: rm /nix/.rw-store/fill7555worker1: (finished: must succeed: rm /nix/.rw-store/fill, in 0.12 seconds)7556worker1: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free'7557worker1 # [ 223.172543] nix-grpc-daemon[1787]: ts=2026-09-24T11:50:53Z level=info event=healthy reason=min_free7558worker1: (finished: waiting for success: journalctl -u nix-grpc-daemon -o cat | grep -q 'event=healthy reason=min_free', in 1.07 seconds)7559(finished: subtest: low disk drains a worker and builds go to the other, in 8.62 seconds)7560(finished: run the VM test script, in 225.07 seconds)7561test script finished in 225.26s7562cleanup7563kill QemuMachine (pid 49)7564client # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7565client # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7566client # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7567client # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7568kill QemuMachine (pid 50)7569lb # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7570lb # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7571lb # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7572lb # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7573kill QemuMachine (pid 52)7574worker1 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7575worker1 # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7576worker1 # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7577worker1 # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7578kill QemuMachine (pid 51)7579worker2 # qemu-system-x86_64: terminating on signal 15 from pid 39 (/nix/store/d64q19q1xjdwfhqx6czvrjgrhq0n3lcc-python3-3.14.7/bin/python3.14)7580worker2 # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7581worker2 # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7582worker2 # [2026-09-24T11:50:56Z INFO virtiofsd] Client disconnected, shutting down7583(finished: cleanup, in 1.01 seconds)7584additionally exposed symbols:7585 client, lb, worker1, worker2,7586 vlan1,7587 start_all, test_script, machines, machines_qemu, machines_nspawn, vlans, driver, log, os, create_machine, subtest, run_tests, join_all, retry, serial_stdout_off, serial_stdout_on, polling_condition, BaseMachine, QemuMachine, NspawnMachine, t, debug, dump_machine_ssh